> ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. --- title: "Benchmarks & Leaderboards" url: https://daily.dev/agentic-ai-hub/benchmarks-leaderboards/ description: "The single most useful mental model here is that benchmarks tell you what a model can do on someone else's task." lastUpdated: "2026-07-22" --- > The single most useful mental model here is that **benchmarks tell you what a model *can* do on someone else's task. Only your own eval tells you what it does on *yours*.** Read benchmarks to shortlist, then test on your workload ([LLMOps: Evals, Observability & Guardrails](/agentic-ai-hub/llmops-evals-observability-guardrails/)). ## The benchmarks worth knowing (and their catch) **LMArena / Chatbot Arena.** *Human preference*: users vote between two anonymous answers, and the votes become **Elo** ratings (the chess rating system, where beating a stronger opponent raises your score more). **Good for:** a real-world read on open-ended chat quality and which model people actually prefer. **Catch:** rewards length and formatting (use the style-controlled board), has no ground truth, and small rank gaps are noise. → [lmarena.ai](https://lmarena.ai/) **MMLU / MMLU-Pro.** *Broad multiple-choice knowledge* across dozens of subjects. **Good for:** a rough general-knowledge floor. **Catch:** the original is **saturated** (frontier models cluster in the low 90s) and contaminated, so use MMLU-Pro (harder, 10 options). → [MMLU-Pro board](https://artificialanalysis.ai/evaluations/mmlu-pro) **GPQA Diamond.** *Expert science reasoning*, 198 PhD-level "Google-proof" questions. **Good for:** telling genuine scientific reasoning apart from recall. **Catch:** approaching saturation, and with only 198 items a few questions swing the score. **SWE-bench Verified.** *Real coding*: fix actual GitHub issues so the project's hidden tests pass (quote the 500-item verified subset). **Good for:** the most credible single "can it fix real bugs" number. **Catch:** heavily **scaffold/harness-dependent** (the same model varies widely by agent), Python-centric, and older instances risk contamination (see **SWE-bench Pro**). → [swebench.com](https://www.swebench.com/verified.html) **Terminal-Bench.** *Agentic CLI competence*: multi-step tasks in a real sandboxed shell. **Good for:** how well an agent actually drives a terminal, close to real agent work. **Catch:** small task count, and the harness matters as much as the model. → [tbench.ai](https://www.tbench.ai/) **τ²-bench (tau-bench).** *Tool use across multi-turn conversations* against a stateful database under policy rules. **Good for:** agent *reliability*, since its **pass^k** metric (solved on all k runs) exposes brittleness that single runs hide. **Catch:** narrow domains, and results depend on the simulated user. → [github](https://github.com/sierra-research/tau2-bench) **AIME / FrontierMath.** *Math reasoning.* **Good for:** hard, checkable step-by-step reasoning. **FrontierMath** (Epoch AI) keeps its problems private and still has real headroom. **Catch:** AIME problems leak every year, so always check which exam year is quoted. → [FrontierMath](https://epoch.ai/frontiermath) **Humanity's Last Exam (HLE).** *Frontier knowledge*, ~2,500 expert questions built to stay hard. **Good for:** separating the very top models (scores are still only in the low-mid 20s%). **Catch:** pure knowledge and reasoning, no agentic ability or tool use. **HumanEval.** *Legacy code-gen* (164 Python functions). **Good for:** almost nothing now, it's saturated (96-98%) and contaminated. Use **LiveCodeBench** instead (problems released after training cutoffs). Also worth knowing: **GDPval** (professional real-work tasks), **Aider Polyglot** (multi-language edits), **GAIA** (general assistant), and **SimpleBench** (common-sense traps where humans beat models). ## Live leaderboards to bookmark - [**Artificial Analysis**](https://artificialanalysis.ai/leaderboards/models). Intelligence index + live speed, latency, price, context in one view. *Best for cost/performance trade-offs.* - [**LMArena**](https://lmarena.ai/). Human-preference Elo with category/style boards. *Best for subjective chat quality.* - [**llm-stats.com**](https://llm-stats.com/). 300+ models, composite score with price baked in. *Best for fast head-to-head selection.* - [**LM Council**](https://lmcouncil.ai/benchmarks). Aggregates ~18 *independently-run* benchmarks. *Best for avoiding vendor self-reports.* - [**Vals AI**](https://www.vals.ai/benchmarks). Industry-specific evals (legal, finance, healthcare). *Best for domain buyers.* - [**OpenRouter Rankings**](https://openrouter.ai/rankings). Real token-usage by model/app across a large routing marketplace. *Best for "what devs actually ship."* - [**Vercel AI Gateway leaderboards**](https://vercel.com/ai-gateway/leaderboards/models). Models ranked by real usage through Vercel's gateway. *Best for the app-builder / TypeScript perspective.* - [**Scale SEAL / Scale Labs**](https://labs.scale.com/leaderboard). Private, contamination-resistant expert evals (incl. SWE-bench Pro). *Best for trust when public benchmarks are gamed.* - [**Vellum LLM Leaderboard**](https://www.vellum.ai/llm-leaderboard). Clean side-by-side benchmark + price/context tables. *Best for a quick reference sheet.* - [**SWE-bench**](https://www.swebench.com/) / [**Terminal-Bench**](https://www.tbench.ai/leaderboard/terminal-bench/2.0). The canonical coding-agent boards. **Task-specific arenas** (LMArena spin-offs and others, when you care about one skill): [**WebDev Arena**](https://arena.ai/blog/webdev-arena/) (web-app building), **Copilot/Code Arena** (in-editor coding), **Design Arena** (UI/design generation), **Search Arena** (retrieval-grounded answers), **Vision/Text-to-Image Arena** (multimodal). Each isolates one capability the general boards blur together. ## Why leaderboards mislead (keep these in mind) - **Contamination.** Public benchmarks leak into training data, so a high score can be memorization. Prefer contamination-resistant or refreshed evals. - **Saturation.** Once models cluster near the ceiling, the gaps between them are noise. - **Style and gaming.** Preference boards reward length and formatting, and vendors can tune to a benchmark's format. - **Harness dependence.** Agentic scores depend as much on the scaffolding as the model, so compare same-harness numbers. *([Papers with Code](https://hyper.ai/en/news/42900) shut down in 2025, so don't rely on it for current SOTA.)*