Agent Evaluation & Benchmarks

"You cannot improve what you cannot measure, and you cannot measure an agent by asking it to grade itself."

Eval, Rigorously Benchmarked AI Agent

1. Why Agent Evaluation Is Hard

Evaluating agents is fundamentally harder than evaluating language models on static benchmarks. An agent's output depends not just on the model's capabilities but on the tools available, the environment's state, the agent's memory, and the specific sequence of actions taken. Two runs of the same agent on the same task can produce different results because of stochastic model outputs, timing-dependent tool responses, or different exploration paths. This non-determinism makes reproducibility a persistent challenge.

Agent evaluation also requires assessing multiple dimensions simultaneously. A code generation agent that solves the problem but takes 50 tool calls and $2 in API costs is less useful than one that solves it in 5 calls for $0.10. Metrics must capture task completion (did the agent succeed?), efficiency (how many steps and tokens did it use?), safety (did it avoid harmful actions?), and robustness (does it handle edge cases and failures gracefully?).

The field has converged on a set of standardized benchmarks that test different agent capabilities. Each benchmark creates a controlled environment where agents can be compared fairly, with automated scoring that removes subjective human judgment from the evaluation loop. Understanding these benchmarks, their strengths, and their limitations, is essential for anyone building or selecting agent systems.

2. Major Agent Benchmarks

SWE-bench

SWE-bench evaluates software engineering agents on real GitHub issues from popular open-source projects. Each task provides a repository snapshot, a natural language issue description, and a test patch that verifies the fix. The agent must navigate the codebase, understand the issue, and produce a code patch that passes the tests. SWE-bench Lite is a curated subset of 300 tasks; SWE-bench Verified uses human-validated tasks to reduce noise. As of early 2026, the best agents solve roughly 50 to 60% of SWE-bench Verified tasks, with Claude Code and Devin among the top performers.

GAIA

GAIA (General AI Assistants) tests real-world assistant capabilities across three difficulty levels. Tasks require web browsing, file manipulation, mathematical reasoning, and multi-step planning. Unlike coding benchmarks, GAIA tasks mirror the diverse challenges a general-purpose agent encounters: "Find the cheapest flight from NYC to London next Tuesday and calculate the per-mile cost." The benchmark uses exact-match scoring against verified ground truth answers.

WebArena and OSWorld

WebArena provides realistic web environments (e-commerce sites, forums, content management systems) where agents must accomplish tasks by navigating web pages, filling forms, and clicking buttons. OSWorld extends this to full desktop environments where agents interact with applications through screenshots, mouse clicks, and keyboard inputs. These benchmarks are critical for evaluating the emerging category of computer use agents discussed in Section 25.3.

# Evaluating an agent on SWE-bench Lite
from swebench.harness.run_evaluation import run_evaluation

results = run_evaluation(
 predictions_path="predictions.json", # Agent's patches
 swe_bench_tasks="princeton-nlp/SWE-bench_Lite",
 log_dir="./eval_logs",
 timeout=300, # 5 minutes per task
)

# Analyze results across dimensions
total = len(results)
resolved = sum(1 for r in results if r["resolved"])
print(f"Pass rate: {resolved}/{total} ({100*resolved/total:.1f}%)")

# Cost analysis
total_tokens = sum(r["total_tokens"] for r in results)
total_cost = sum(r["api_cost"] for r in results)
print(f"Average tokens per task: {total_tokens/total:,.0f}")
print(f"Average cost per task: ${total_{cost}/total:.3f}")
print(f"Cost per resolved task: ${total_cost/max(resolved,1):.3f}")

3. Building Custom Agent Evaluations

Standard benchmarks test general capabilities, but production agents need domain-specific evaluation. A legal research agent should be evaluated on legal accuracy, citation quality, and jurisdictional awareness. A customer support agent should be measured on resolution accuracy, customer satisfaction proxies, and escalation appropriateness. Building custom evaluations requires defining task sets, scoring criteria, and automated verification methods specific to your domain.

The most effective custom evaluations use a "golden set" approach: a curated collection of tasks with verified correct outputs, annotated with difficulty levels and capability tags. Human experts create the golden set, and automated scoring compares agent outputs against the verified answers. For tasks without single correct answers (summarization, recommendations), LLM-as-judge evaluation provides a scalable alternative, though it requires calibration against human judgments.

Lab: Evaluate an Agent on SWE-bench Lite

pip install swebench openai tiktoken pandas

# TODO: Load SWE-bench Lite tasks and configure the agent
# from swebench import get_tasks
# tasks = get_tasks("lite", limit=10)

# TODO: Loop over tasks, run agent, record metrics in a DataFrame

# TODO: Analyze agent logs for each failure and assign a category

# TODO: Run with reasoning model, then create a comparison table

# Complete solution outline for SWE-bench evaluation
# This lab is primarily a measurement/evaluation exercise.
# The key deliverable is the metrics table and failure analysis,
# not a single script. See the SWE-bench documentation for
# harness setup: https://github.com/princeton-nlp/SWE-bench

Exercises