Observability and tracing
Every run is a trace, every step a span. Tracing turns an incident into a query — and connects behaviour to the bill.
You cannot operate what you cannot see. An agent in production produces a stream of decisions, tool calls, and tokens; observability is the practice of recording them so that a failure is something you can query rather than reconstruct from memory. Tracing is the core of it: every run is a trace, every step a span with a start, a duration, and a cost.
Start here
When an agent does something strange, the question is never "why is the model like that?" — it is "what did it actually do?". A trace answers that: the exact thought, the exact tool call with its arguments, the exact observation, and the tokens each cost. Without it you are debugging by vibes.
One run, traced — click a span to inspect it
step 3 · llm reason
3.6s · 4100 tokens · 44% of wall-clock
The trace turns an incident into a query: which step is slow, which burns the tokens, where the loop went wrong. Token counts are what connect a trace to a bill — the reason observability and cost control are the same tool. Durations and counts are illustrative.
Click spans to see where the wall-clock time and the tokens went.
What to record
- Every step — thought, tool name, arguments, observation, tokens, latency. The full trajectory, not just the final answer.
- Correlation ids — a trace id per run, so a user complaint maps to the exact sequence that produced it.
- Cost, per span — token counts are what connect behaviour to the bill, which is why observability and cost control are one tool.
- Inputs and versions — the prompt, model, and tool versions, because a silent version bump is a behaviour change.
Why it pays
- Diagnosis — the slow step, the loop, the tool that returned garbage: visible instead of guessed.
- Cost attribution — which feature, user, or step spends the budget.
- Regression detection — compare traces before and after a change to see what moved.
- Coordination failures — in multi-agent systems, a trace is the only way to see a dropped handoff or a duplicated subtask.
Careful
Instrument from day one. Adding tracing after an incident means the incident you most need to understand is the one you have no data for, and the standard conventions above exist so your traces compose with the rest of your stack instead of being a bespoke silo. Observability is not a debugging tool you reach for later — it is how you know the agent is working at all.
Check yourself
Eduspheria wiki · Agentic AI, Evaluation & observability
0 / 4 answered
That closes the evaluation chapter, and the full first pass of this domain: foundations, planning, memory, orchestration, evaluation, and safety. An agent you can build, bound, measure, and trust.