Agents and tools
From answering to acting: the ReAct loop, tool choice, and everything that breaks when a model takes actions.
Retrieval gives the model documents. Tools give it capabilities: calculators, databases, APIs, file systems, other models. The moment output includes structured tool calls that a runtime executes and feeds back, you have an agent — a model in a loop with the world.
Start here
An agent is a loop: the model reasons about what to do, emits a structured call, the runtime executes it, the result goes back into context, repeat. The intelligence is still next-token prediction — what changed is that actions close the loop.
The loop, step by step
Agent loop — user: "What's a 15% tip on $84.50?"
step 0/7ReAct pattern: reason → act → observe → repeat. Illustrative trace; real agents choose tools the same way — by reading tool descriptions in the prompt and emitting structured calls.
The pattern is called ReAct (reason + act + observe). Every serious agent framework is a variation on it. Three design facts it hides:
- Tool choice is prompt-driven. The runtime lists tools with descriptions; the model picks by reading them. A vague description is a bug — the model will mis-select a tool exactly as often as the description misleads it.
- Errors are feedback, not failures. A tool error entering the context lets the model retry with adjusted arguments. Robust agents treat errors as signal; brittle ones crash.
- The loop is the cost. Every iteration is a full forward pass over a growing context. A 20-step agent on a 200k-token context pays for that context on every step — latency and cost scale with loop length, not with question length.
Why agents are hard: the error surface compounds
A chat error is one bad message. An agent error is a bad action with state — and the loop's economics make it worse. Toggle a verifier onto the critical step:
An 8-step agent task — toggle a verifier on the critical step
total tokens: 9,000 vs 7,200 clean
cost scales with loop length, not question length
Every step re-reads the whole transcript (the loop-cost lesson: attention over a growing context), and an unchecked early error compounds — the waste bars grow downstream of the bad step. The verifier costs one extra step but resets the trajectory. Token figures are round-number arithmetic on illustrative steps; the compounding shape is the real phenomenon.
- Compounding drift — a wrong intermediate result poisons every later step (that's why the trace verifies money math with a tool).
- Permission and safety — a tool that can send email can send the wrong email. Serious systems give read-only tools liberally and mutating tools narrowly, with confirmation gates — the same principle as database grants.
- Termination — when to stop looping is itself a decision the model makes; runaway loops are the agent equivalent of infinite recursion, and budget caps are the guardrail.
- Non-determinism — the same prompt can trace differently twice; testing an agent means testing trajectories, not answers.
Careful
The honest current state: agents excel at long, mechanical, checkable flows (code with tests, data pipelines, drafts with review) and remain unreliable at open-ended judgment-heavy ones. Human-in-the-loop checkpoints are an architectural feature, not a temporary crutch.
The stack around the loop
Production agents add scaffolding around the loop: planning (the model writes an explicit plan before acting), memory (summaries of past steps persisted outside the context window), sub-agents (delegating subtasks to fresh contexts), and guardrails (schema validation on every tool call, spend caps, allowlists). Most agent failures in the wild are scaffolding failures, not model failures.
Next: making a 100B model behave like your model — adaptation without retraining, via LoRA.