CHAPTER 16 · Scaling and Infrastructure · 7 / 10
Caching tiers
Introduce caching where it pays:
- External-API cache (a table or a fast cache like Redis) for slow/rate-limited upstream calls.
- Hot-data cache for frequently-read, rarely-changed data.
- Provider prompt caching by keeping a stable context prefix (Chapter 14).
Add these as evidence (from observability, Chapter 15) shows a hotspot, not preemptively.