Routing and decision layers
A fast, typed decision model can gate and route work before any expensive reasoning happens — the System One / System Two split as infrastructure.
Not every request needs a reasoning model. Most are easy, high-volume, and decidable by a glance: which team owns this ticket, is this message a refund request, should this go to a human. A decision layer — a fast, typed classifier — can absorb those before the expensive agent ever wakes up, and escalate only the genuinely uncertain remainder.
Start here
Routing is the cheapest agentic win there is. A gate that answers the easy 80% in milliseconds leaves the reasoning model for the hard 20% — lower cost, lower latency, and a smaller, cleaner context for the part that actually needs thought.
Route cheaply, escalate only what is uncertain
auto-handled
5/8
escalated
3
est. cost / batch
$0.0305
avg latency
0.97s
This is the System One / System Two split from the LLM domain, applied as infrastructure: a fast, typed decision layer absorbs the high-volume easy cases and hands the genuinely hard ones to a reasoning model. Lower the threshold and you escalate more — safer, slower, dearer. Figures are illustrative.
Slide the confidence threshold and watch the cost and latency move as more or fewer cases get escalated.
The pattern
- Classify — a fast model returns a typed decision with a probability: route, intent, or a yes/no gate.
- Gate on confidence — above the threshold, act on the decision; below it, escalate to the reasoning agent or a human.
- Act — the route decides which tools, prompts, or agents handle the request downstream.
Why a decision model
This is the System One / System Two split from the LLM domain, used as plumbing. A decision model returns typed values and calibrated probabilities instead of text — no chain of thought, no parsing, no fabricated prose — which makes it fast, cheap, and a natural fit for high-frequency classification. The reasoning model still handles anything open-ended.
Careful
The threshold is a product decision, not a technical default. Set it too low and the fast layer guesses on cases it should have escalated (confident mistakes); too high and you have rebuilt the expensive path with extra steps. Calibration matters: a 0.94 should be right about 94% of the time, and you should measure that rather than trust the vendor's number.
Check yourself
Eduspheria wiki · Agentic AI, Multi-agent & orchestration
0 / 4 answered
Next: what happens when agents hand work to each other directly.