Evaluating models
Why benchmark numbers lie less than you fear but more than you hope — contamination, benchmarks vs. reality, and human eval.
We can now build an assistant. The remaining question decides everything commercially and scientifically: is it good? Evaluation looks like a solved problem — leaderboard numbers exist — but each number hides a caveat. Reading those caveats is a skill worth 10 minutes.
Start here
A benchmark is a fixed exam. Models improve by studying — sometimes by studying the answer key. The score measures both capability and exposure, and nothing on the sheet tells you which you got.
The five lenses
One model, five benchmarks — same checkpoint, five different stories. Click each for its caveat.
Multiple-choice: format-coachable. Widely cited, easy to game via training data overlap.
Scores are illustrative; the per-benchmark caveats are the lesson. Always ask what a benchmark measures — and whether the model saw its answers during training.
Open the explorer and compare. The patterns generalize to every leaderboard you will ever read:
- Knowledge MCQs (MMLU-style) — cheap, broad, famous. But multiple-choice is format-coachable, and famous question sets leak into training data. A high score means "prepared for this test" at minimum.
- Contamination-resistant reasoning (GPQA, fresh sets) — questions designed post-cutoff or built to resist lookup. Track these when they exist.
- Coding (HumanEval → SWE-bench) — code has ground truth (does the test pass?), so evaluation can be automatic and verifiable. The catch: classic problem sets circulated online; verified, real issue-based variants are the honest signal.
- Human preference (Chatbot Arena) — real users vote on real prompts. Measures what people prefer, which includes style and length biases — a vote, not a correctness check.
- Agentic (SWE-bench Verified) — end-to-end real tasks with checkable outcomes. Hardest to fake, most predictive of job performance, most expensive to run.
Contamination: the quiet killer
Contamination = test questions (or near-paraphrases) appearing in the training corpus. It's not necessarily cheating — scraping the web is the task — but it means the benchmark stops measuring what it measures. Responses to contamination that the field uses:
- Held-out freshness — tests built after the model's data cutoff.
- Canary strings — unique markers benchmarks embed so labs can detect and exclude leaked copies.
- Private evals — labs keep unreleased test sets for exactly this reason; if you only see published benchmarks, assume some leakage.
Beyond leaderboards: how products are actually evaluated
Leaderboards are marketing; shipping teams also use:
- Error analysis — reading failures by hand. Slow, irreplaceable.
- LLM-as-judge — a strong model grades outputs against a rubric. Scales well; inherits judge biases (verbosity, self-preference).
- Regression suites — fixed prompt sets with expected properties, run on every model change, like unit tests for behavior.
- Task success, not vibes — did the user's problem get solved? Lowest signal, highest truth.
Careful
No single number ranks models. Capability is multi-dimensional (knowledge, reasoning, coding, safety, latency, cost), benchmarks proxy parts of it imperfectly, and contamination inflates silently. Read every leaderboard as "measured on this test", never as truth.
Check yourself
Eduspheria wiki · Large Language Models, Post-training
0 / 4 answered
Post-training complete
You can now trace the full pipeline: pretraining (knowledge) → SFT (manners) → preference tuning (taste) → evaluation (verification). Next chapter: the frontier — what practitioners actually do with these models, starting with prompting-and-reasoning. And after that, the interpretability chapter opens the box on everything above.