Skip to slide
Chapter 5 · Prompt Caching and Context Management
29 / 142

CHAPTER 05 · Prompt Caching and Context Management · 2 / 8

What breaks the cache

Because hits need exact prefix matches, a handful of innocent-looking actions cause expensive cache misses:

  • Changing the tools list mid-conversation.
  • Switching the model (this changes the model-specific instructions near the front of the prompt).
  • Changing the sandbox configuration, approval mode, or working directory.
  • Editing your AGENTS.md mid-session (it invalidates the largest cached prefix).

The Codex team learned this the hard way: an early bug enumerated MCP tools in an inconsistent order, which silently broke caching on every turn. The fix for necessary mid-session changes is clever and worth stealing: instead of editing an earlier message (which would change the prefix and bust the cache), append a new message at the end describing the change. For example, if the working directory changes, Codex inserts a fresh <environment_context> message at the tail rather than rewriting the original. The prefix stays intact; the cache survives.

A practical habit falls right out of this: lock your AGENTS.md before a long session. Changing it mid-run throws away your biggest cache discount.

← → arrow keys work too