CHAPTER 18 · Reference Architecture and a Design Checklist · 12 / 15
Observability & quality
- Do you capture per-turn traces (context, model calls, tokens, tool calls, outcome)? (Ch 15)
- Is there an eval set run on every prompt/tool/model change, with deterministic checks where possible? (Ch 15)
- Do you watch production signals (feedback, accept/reject, failures) and feed cases back into evals? (Ch 15)