Evaluating models
Why benchmark numbers lie less than you fear but more than you hope — contamination, benchmarks vs. reality, and human eval.
We can now build an assistant. The remaining question decides everything commercially and scientifically: is it good? Evaluation looks like a solved problem — leaderboard numbers exist — but each number hides a caveat. Reading those caveats is a skill worth 10 minutes.
Start here
A benchmark is a fixed exam. Models improve by studying — sometimes by studying the answer key. The score measures both capability and exposure, and nothing on the sheet tells you which you got.
The five lenses
One model, five benchmarks — same checkpoint, five different stories. Click each for its caveat.
Multiple-choice: format-coachable. Widely cited, easy to game via training data overlap.
Scores are illustrative; the per-benchmark caveats are the lesson. Always ask what a benchmark measures — and whether the model saw its answers during training.
Open the explorer and compare. The patterns generalize to every leaderboard you will ever read:
- Knowledge MCQs (MMLU-style) — cheap, broad, famous. But multiple-choice is format-coachable, and famous question sets leak into training data. A high score means "prepared for this test" at minimum.
- Contamination-resistant reasoning (GPQA, fresh sets) — questions designed post-cutoff or built to resist lookup. Track these when they exist.
- Coding (HumanEval → SWE-bench) — code has ground truth (does the test pass?), so evaluation can be automatic and verifiable. The catch: classic problem sets circulated online; verified, real issue-based variants are the honest signal.
- Human preference (Chatbot Arena) — real users vote on real prompts. Measures what people prefer, which includes style and length biases — a vote, not a correctness check.
- Agentic (SWE-bench Verified) — end-to-end real tasks with checkable outcomes. Hardest to fake, most predictive of job performance, most expensive to run.
Contamination: the quiet killer
Contamination = test questions (or near-paraphrases) appearing in the training corpus. It's not necessarily cheating — scraping the web is the task — but it means the benchmark stops measuring what it measures. Responses to contamination that the field uses:
- Held-out freshness — tests built after the model's data cutoff.
- Canary strings — unique markers benchmarks embed so labs can detect and exclude leaked copies.
- Private evals — labs keep unreleased test sets for exactly this reason; if you only see published benchmarks, assume some leakage.
Beyond leaderboards: how products are actually evaluated
Leaderboards are marketing; shipping teams also use:
- Error analysis — reading failures by hand. Slow, irreplaceable.
- LLM-as-judge — a strong model grades outputs against a rubric. Scales well; inherits judge biases (verbosity, self-preference).
- Regression suites — fixed prompt sets with expected properties, run on every model change, like unit tests for behavior.
- Task success, not vibes — did the user's problem get solved? Lowest signal, highest truth.
Careful
No single number ranks models. Capability is multi-dimensional (knowledge, reasoning, coding, safety, latency, cost), benchmarks proxy parts of it imperfectly, and contamination inflates silently. Read every leaderboard as "measured on this test", never as truth.
Post-training complete
You can now trace the full pipeline: pretraining (knowledge) → SFT (manners) → preference tuning (taste) → evaluation (verification). Next chapter: the frontier — what practitioners actually do with these models, starting with prompting-and-reasoning. And after that, the interpretability chapter opens the box on everything above.