CHAPTER 14 · Cost, Latency, and Model Tiering · 6 / 13
Reduce loop iterations
Fewer iterations means fewer calls means less cost and latency:
- Batching tools. "Fetch these N documents" in one call avoids N serial iterations. "Extract all columns for this document in one call" instead of one call per cell.
- Good tool design and descriptions so the model picks the right tool first time instead of fumbling through several.
- Cheap discovery tools (list/find) so the model targets what it needs instead of reading everything and iterating.