Agents and tools
From answering to acting: the ReAct loop, tool choice, and everything that breaks when a model takes actions.
Retrieval gives the model documents. Tools give it capabilities: calculators, databases, APIs, file systems, other models. The moment output includes structured tool calls that a runtime executes and feeds back, you have an agent — a model in a loop with the world.
Start here
An agent is a loop: the model reasons about what to do, emits a structured call, the runtime executes it, the result goes back into context, repeat. The intelligence is still next-token prediction — what changed is that actions close the loop.
The loop, step by step
Agent loop — user: "What's a 15% tip on $84.50?"
step 0/7ReAct pattern: reason → act → observe → repeat. Illustrative trace; real agents choose tools the same way — by reading tool descriptions in the prompt and emitting structured calls.
The pattern is called ReAct (reason + act + observe). Every serious agent framework is a variation on it. Three design facts it hides:
- Tool choice is prompt-driven. The runtime lists tools with descriptions; the model picks by reading them. A vague description is a bug — the model will mis-select a tool exactly as often as the description misleads it.
- Errors are feedback, not failures. A tool error entering the context lets the model retry with adjusted arguments. Robust agents treat errors as signal; brittle ones crash.
- The loop is the cost. Every iteration is a full forward pass over a growing context. A 20-step agent on a 200k-token context pays for that context on every step — latency and cost scale with loop length, not with question length.
Why agents are hard: the error surface compounds
A chat error is one bad message. An agent error is a bad action with state — and the loop's economics make it worse. Toggle a verifier onto the critical step:
An 8-step agent task — toggle a verifier on the critical step
total tokens: 9,000 vs 7,200 clean
cost scales with loop length, not question length
Every step re-reads the whole transcript (the loop-cost lesson: attention over a growing context), and an unchecked early error compounds — the waste bars grow downstream of the bad step. The verifier costs one extra step but resets the trajectory. Token figures are round-number arithmetic on illustrative steps; the compounding shape is the real phenomenon.
- Compounding drift — a wrong intermediate result poisons every later step (that's why the trace verifies money math with a tool).
- Permission and safety — a tool that can send email can send the wrong email. Serious systems give read-only tools liberally and mutating tools narrowly, with confirmation gates — the same principle as database grants.
- Termination — when to stop looping is itself a decision the model makes; runaway loops are the agent equivalent of infinite recursion, and budget caps are the guardrail.
- Non-determinism — the same prompt can trace differently twice; testing an agent means testing trajectories, not answers.
Careful
The honest current state: agents excel at long, mechanical, checkable flows (code with tests, data pipelines, drafts with review) and remain unreliable at open-ended judgment-heavy ones. Human-in-the-loop checkpoints are an architectural feature, not a temporary crutch.
The stack around the loop
Production agents add scaffolding around the loop: planning (the model writes an explicit plan before acting), memory (summaries of past steps persisted outside the context window), sub-agents (delegating subtasks to fresh contexts), and guardrails (schema validation on every tool call, spend caps, allowlists). Most agent failures in the wild are scaffolding failures, not model failures.
Check yourself
Eduspheria wiki · Large Language Models, Frontier
0 / 4 answered
Next: making a 100B model behave like your model — adaptation without retraining, via LoRA.