CHAPTER 08 · Streaming and Real-Time UX · 9 / 10
Backpressure and connection management
A few operational notes:
- Disable buffering at every hop (the framework, and reverse proxies via the appropriate header), or events will be batched and the "real-time" feel disappears.
- Heartbeats (periodic comments) keep idle connections alive through proxies that time out quiet streams.
- Clean up server-side resources when the client disconnects mid-stream (abort the model call if possible) so an abandoned turn doesn't keep burning tokens.