Task Success Rate

Task success rate is the share of attempted tasks an automated system completed correctly, end to end. It is the headline number for anything agentic, and almost every disagreement about it is really a disagreement about the denominator and about who decides what counts as correct. The denominator is where most of the inflation happens. A rate computed over the tasks the system chose to attempt excludes everything it declined, abandoned, or never recognised as a task, and those are exactly the hard cases. A rate computed over everything that arrived is a much less flattering number and a much more useful one. If both are reported, the gap between them is itself informative: it is the share of the problem the system currently does not touch. Who judges is the second question. Self-assessment is the weakest option, because a system that misunderstood the task will usually also misjudge whether it completed it. Checkable criteria are better where the task allows them — a test that passes, a record that reconciles, a value that matches a known answer. Human review is expensive and therefore usually sampled, which is fine as long as the sample is random rather than the cases somebody happened to look at. Partial completion has to be handled deliberately or it will be handled accidentally. A task that got most of the way and stopped cleanly is a different outcome from one that finished with a wrong result, and both are different from one that stopped halfway having already changed something. Collapsing all three into failure loses the distinction that matters operationally; counting the first as success flatters the system. The rate is a trend, not a fact about tomorrow. It moves when the model changes, when the prompt changes, when a tool starts returning different data, and when the incoming mix of requests shifts — and the last of those causes more unexplained drops than the first three. A success rate is only interpretable next to the distribution of tasks it was measured on.