Evaluating agents

Success rate is the most common metric: did the agent finish the task, yes or no. Necessary on its own, but incomplete. A single bit treats two very different runs the same: a run that fails one step short of the goal and a run that falls apart on the first step both score zero.

AgentBoard criticises exactly this and proposes progress rate for intermediate state. WebArena shows the best GPT-4 agent finishing only a small fraction of tasks; one number hides where things broke. So evaluation has to be opened into layers.

The single-bit stamp on the left and the multi-metric scorecard on the right describe the same run. One says "done / not done"; the other shows where the run stopped.

The right way to measure task success is not to require the agent's steps to match a reference path. WebArena makes this explicit: what matters is functional correctness, whether the resulting world (web state, database) satisfies the user goal. The same goal can be reached by many valid paths.

SWE-bench applies this most strictly: does the generated patch resolve a real GitHub issue? The answer comes from applying the patch and running the repository tests. fail-to-pass tests must pass, prior tests must not break. Here the metric checks execution, not text.

So the validator checks the outcome, not the path. Below, two agents reach the same goal state through different-length steps; the validator accepts both, because it looks at the final state.

Task success tells you the end state but not the journey. AgentBoard fills this gap with progress rate: it tracks the trajectory's best-matching intermediate state toward the goal, reaching 1 when the task is complete. This separates a run that nearly finishes from one that never starts.

To do this the authors define subgoals by hand and validate progress rate against human ratings; the reported Pearson correlation is above 0.95 across tasks. It has a cost: progress rate depends on hand-annotated subgoals, and the authors note current LLMs cannot generate those subgoals accurately on their own.

Below, two runs move along the same subgoal track. Neither finishes (success = 0), but one reaches 0.9 and the other stalls at 0.2. Success rate equates them; progress rate separates them.

An agent can give the right answer through the wrong path: picking the wrong tool, passing invalid arguments, inventing an API that does not exist. Tool correctness measures this as its own layer. ToolBench (ToolEval) evaluates tool selection, argument validity, execution, hallucination rate, and solution-path quality over 16,000+ real APIs.

tau-bench goes further: it has the agent talk to a simulated user, programmatic APIs, and domain policy documents, then compares the final database state with the expected goal. It checks whether the tool produced the correct side effect, not whether the answer sounds plausible. Model-backed evaluators (like ToolEval) scale, but a deterministic validator is preferred when the side effect can be checked directly.

Below, a tool call passes through four gates: right tool, valid arguments, successful execution, correct side effect. A hallucinated tool is rejected at the first gate.

In production it is not enough for an agent to "pass once"; it has to pass every time. The classic pass@k is optimistic: if at least one of k attempts passes, it counts as success. For a user-facing system this is misleading, because one lucky run hides a bad experience.

tau-bench instead foregrounds pass^k: all k independent attempts must pass. The reported result is striking: function-calling GPT-4o goes from ~61% pass^1 on tau-retail to ~25% pass^8. Consistency is much harder than a single success.

Below, eight independent attempts resolve in sequence. pass@1 asks "did at least one pass?" and turns green; pass^8 asks "did all of them pass?" and a single failure turns it red.

Cost is not just tokens. WebArena counts Docker reset and LLM inference time per task as a non-negligible but small evaluation cost. Repeat-based metrics (pass^k) multiply the number of attempts by k, which is both time and money.

METR proposes a different axis: the 50% task-completion time horizon, the human-time equivalent of tasks an agent can complete with 50% probability. For o3 this horizon is about 110 minutes, and over the studied period the horizon roughly doubles every 207 days. Because it ties success to human time rather than raw task count, it is a strong frame for cost and effort.

Below, the components of an evaluation budget stack up: tokens, inference time, environment reset, and repeated trials.

Many agent benchmarks share one uncomfortable fact: the gap between humans and agents is still large. On WebArena humans score 78% while the best GPT-4 agent scores 14%; on GAIA humans score 92% and GPT-4 with plugins 15%. METR positions agents directly against human time. Human baselines make the word "good" concrete.

When automated pass/fail finishes, the work is not done. The SWE-bench authors note that even model patches passing execution tests can be less comprehensive, efficient, or readable than human solutions. Where maintainability, policy compliance, safety, and subjective quality matter, human review is still needed. The right setup combines deterministic checks with sampled human review.

Below, agent and human success sit side by side on two benchmarks; on the right, outputs that pass automated checks drop into a review queue.

No single metric describes an agent. A solid evaluation is layered like a scorecard: task success (functional correctness) first, then step quality (progress rate), tool correctness, reliability across repeats (pass^k), cost (time and runtime), and human review on top.

Reading these layers together turns "is the agent good?" into "good at which layer, by how much, at what cost?" A good evaluation tells you not what the model achieved, but where the system is reliable.

A good agent is not one that passes a single test; it is a system that can show how reliable it is at every layer.

So before asking "is this agent good?", ask: which layer, with which benchmark, over how many repeats, and at what cost am I measuring?