Skip to slide
Chapter 15 · Observability and Evaluation
148 / 191

CHAPTER 15 · Observability and Evaluation · 2 / 7

What to capture: the trace

The unit of agent observability is the trace: the complete record of one turn. A good trace includes:

  • Inputs: the assembled context (system prompt version, the messages, the available tools), the user, the model and settings.
  • Each model call: the request, the response, token counts (input/output/reasoning), latency, and finish reason.
  • Each tool call: name, arguments, result (or error), and duration.
  • The loop shape: how many iterations, in what order.
  • Outputs: the final answer, emitted structured data (citations), and any artifacts produced.
  • Outcome: success/failure, and any error details.

The event timeline you already build for streaming and persistence (Chapters 8, 9) is most of this for free: it's a structured, ordered record of the turn. Observability often means also routing that timeline (plus the model-call metadata) to a tracing system where you can search, aggregate, and inspect it.

← → arrow keys work too