Agent Evaluation & Benchmarks

"You cannot improve what you cannot measure, and you cannot measure an agent by asking it to grade itself."

Eval, Rigorously Benchmarked AI Agent

26.4.1 Why Agent Evaluation Is Hard

Evaluating agents is fundamentally harder than evaluating language models on static benchmarks. An agent's output depends not just on the model's capabilities but on the tools available, the environment's state, the agent's memory, and the specific sequence of actions taken. Two runs of the same agent on the same task can produce different results because of stochastic model outputs, timing-dependent tool responses, or different exploration paths. This non-determinism makes reproducibility a persistent challenge.

Agent evaluation also requires assessing multiple dimensions simultaneously. A code generation agent that solves the problem but takes 50 tool calls and $2 in API costs is less useful than one that solves it in 5 calls for $0.10. Metrics must capture task completion (did the agent succeed?), efficiency (how many steps and tokens did it use?), safety (did it avoid harmful actions?), and robustness (does it handle edge cases and failures gracefully?).

The field has converged on a set of standardized benchmarks that test different agent capabilities. Each benchmark creates a controlled environment where agents can be compared fairly, with automated scoring that removes subjective human judgment from the evaluation loop. Understanding these benchmarks, their strengths, and their limitations, is essential for anyone building or selecting agent systems.

26.4.2 Major Agent Benchmarks

Now that we have the right evaluation lens (Pareto frontiers of accuracy versus cost), we can survey the benchmarks the field actually runs against. Four matter most today: SWE-bench for code-fix agents, GAIA for general assistants, WebArena for browser automation, and OSWorld for desktop agents. Each captures a different slice of "what agents have to do," and we walk through them in order of maturity, starting with SWE-bench.

SWE-bench

SWE-bench evaluates software engineering agents on real GitHub issues from popular open-source projects. Each task provides a repository snapshot, a natural language issue description, and a test patch that verifies the fix. The agent must navigate the codebase, understand the issue, and produce a code patch that passes the tests. SWE-bench Lite is a curated subset of 300 tasks; SWE-bench Verified uses human-validated tasks to reduce noise. As of early 2026, the best agents solve roughly 50 to 60% of SWE-bench Verified tasks, with Claude Code and Devin among the top performers.

GAIA

GAIA (General AI Assistants) tests real-world assistant capabilities across three difficulty levels. Tasks require web browsing, file manipulation, mathematical reasoning, and multi-step planning. Unlike coding benchmarks, GAIA tasks mirror the diverse challenges a general-purpose agent encounters: "Find the cheapest flight from NYC to London next Tuesday and calculate the per-mile cost." The benchmark uses exact-match scoring against verified ground truth answers.

WebArena and OSWorld

WebArena provides realistic web environments (e-commerce sites, forums, content management systems) where agents must accomplish tasks by navigating web pages, filling forms, and clicking buttons. OSWorld extends this to full desktop environments where agents interact with applications through screenshots, mouse clicks, and keyboard inputs. These benchmarks are critical for evaluating the emerging category of computer use agents discussed in Section 29.2.

# Evaluating an agent on SWE-bench Lite
from swebench.harness.run_evaluation import run_evaluation
results = run_evaluation(
    predictions_path="predictions.json", # Agent's patches
    swe_bench_tasks="princeton-nlp/SWE-bench_Lite",
    log_dir="./eval_logs",
    timeout=300, # 5 minutes per task
    )
# Analyze results across dimensions
total = len(results)
resolved = sum(1 for r in results if r["resolved"])
print(f"Pass rate: {resolved}/{total} ({100*resolved/total:.1f}%)")
# Cost analysis
total_tokens = sum(r["total_tokens"] for r in results)
total_cost = sum(r["api_cost"] for r in results)
print(f"Average tokens per task: {total_tokens/total:,.0f}")
print(f"Average cost per task: ${total_cost/total:.3f}")
print(f"Cost per resolved task: ${total_cost/max(resolved,1):.3f}")

26.4.3 Building Custom Agent Evaluations

Standard benchmarks test general capabilities, but production agents need domain-specific evaluation. A legal research agent should be evaluated on legal accuracy, citation quality, and jurisdictional awareness. A customer support agent should be measured on resolution accuracy, customer satisfaction proxies, and escalation appropriateness. Building custom evaluations requires defining task sets, scoring criteria, and automated verification methods specific to your domain.

The most effective custom evaluations use a "golden set" approach: a curated collection of tasks with verified correct outputs, annotated with difficulty levels and capability tags. Human experts create the golden set, and automated scoring compares agent outputs against the verified answers. For tasks without single correct answers (summarization, recommendations), LLM-as-judge evaluation provides a scalable alternative, though it requires calibration against human judgments.

# Pull 10 SWE-bench Lite tasks via the HuggingFace datasets release
# and wire up an Anthropic agent with the three tools it needs to operate
# on the cloned repository: read_file, write_file, run_tests.
from datasets import load_dataset
from anthropic import Anthropic

tasks = load_dataset("princeton-nlp/SWE-bench_Lite", split="test").shuffle(seed=7).select(range(10))
print(f"Loaded {len(tasks)} tasks from {len(set(tasks["repo"]))} repos")

client = Anthropic()
TOOLS = [
    {"name": "read_file", "description": "Read a file from the repo",
     "input_schema": {"type": "object", "properties": {"path": {"type": "string"}}}},
    {"name": "write_file", "description": "Replace file contents",
     "input_schema": {"type": "object",
                      "properties": {"path": {"type": "string"}, "content": {"type": "string"}}}},
    {"name": "run_tests", "description": "Run pytest for the task and return pass/fail",
     "input_schema": {"type": "object", "properties": {}}},
]

# Run the agent loop on each task and record per-task metrics.
# pandas is the natural fit for the metrics table that downstream
# analysis (failure categorization, model comparison) consumes.
import pandas as pd

rows = []
for t in tasks:
    repo_dir = clone_at_commit(t["repo"], t["base_commit"])
    trace = run_agent(client, model="claude-sonnet-4-5", tools=TOOLS,
                      issue=t["problem_statement"], repo=repo_dir, max_turns=25)
    passed = apply_and_test(repo_dir, trace.final_patch, t["PASS_TO_PASS"] + t["FAIL_TO_PASS"])
    cost_usd = (trace.input_tokens * 3.0 + trace.output_tokens * 15.0) / 1_000_000
    rows.append({
        "task_id": t["instance_id"], "repo": t["repo"], "passed": passed,
        "input_tokens": trace.input_tokens, "output_tokens": trace.output_tokens,
        "cost_usd": round(cost_usd, 4), "tool_calls": trace.tool_call_count,
    })

df = pd.DataFrame(rows)
print(df[["task_id", "passed", "cost_usd", "tool_calls"]])
print(f"pass rate: {df['passed'].mean():.0%} | avg cost {df['cost_usd'].mean():.3f} USD")

# Rule-based first pass on the trace, then escalate ambiguous cases
# to an LLM judge. This mirrors the failure-mode taxonomy from the
# SWE-bench Verified annotation effort.
def categorize(trace, task, judge_client=client) -> str:
    files_read = {c.input["path"] for c in trace.tool_calls if c.name == "read_file"}
    touched_target_file = any(f in task["patch"] for f in files_read)

    if not touched_target_file:
        return "navigation_failure"     # never read the right file
    if trace.final_patch is None:
        return "no_patch_emitted"       # gave up before writing

    # LLM judge decides between misunderstanding vs incorrect_code
    verdict = judge_client.messages.create(
        model="claude-haiku-4-5", max_tokens=100,
        messages=[{"role": "user", "content": f"\nIssue: {task['problem_statement']}\n"
                    f"Reference patch: {task['patch'][:800]}\n"
                    f"Agent's patch: {trace.final_patch[:800]}\n\n"
                    "Reply with exactly one token: misunderstanding | incorrect_code"}],
    )
    return verdict.content[0].text.strip()

df["failure_category"] = df.apply(
    lambda r: None if r.passed else categorize(r.trace, r.task), axis=1,
)
print(df["failure_category"].value_counts(dropna=False))

# Re-run with extended thinking enabled, then diff the two runs
# on pass rate, average cost, and tool-call efficiency.
df_std = df  # the Sonnet baseline from step 2

rows_r = []
for t in tasks:
    trace = run_agent(client, model="claude-sonnet-4-5", tools=TOOLS,
                      issue=t["problem_statement"],
                      repo=clone_at_commit(t["repo"], t["base_commit"]),
                      thinking={"type": "enabled", "budget_tokens": 8000},  # reasoning mode
                      max_turns=25)
    passed = apply_and_test(trace.workdir, trace.final_patch, t["FAIL_TO_PASS"])
    cost_usd = (trace.input_tokens * 3.0 + trace.output_tokens * 15.0) / 1_000_000
    rows_r.append({"task_id": t["instance_id"], "passed": passed,
                   "cost_usd": round(cost_usd, 4), "tool_calls": trace.tool_call_count})

df_reasoning = pd.DataFrame(rows_r)
comparison = pd.DataFrame({
    "variant": ["standard", "reasoning"],
    "pass_rate": [df_std["passed"].mean(), df_reasoning["passed"].mean()],
    "avg_cost_usd": [df_std["cost_usd"].mean(), df_reasoning["cost_usd"].mean()],
    "avg_tool_calls": [df_std["tool_calls"].mean(), df_reasoning["tool_calls"].mean()],
})
print(comparison)

Exercises