Skip to slide
Chapter 14 · Cost, Latency, and Model Tiering
138 / 191

CHAPTER 14 · Cost, Latency, and Model Tiering · 6 / 13

Reduce loop iterations

Fewer iterations means fewer calls means less cost and latency:

  • Batching tools. "Fetch these N documents" in one call avoids N serial iterations. "Extract all columns for this document in one call" instead of one call per cell.
  • Good tool design and descriptions so the model picks the right tool first time instead of fumbling through several.
  • Cheap discovery tools (list/find) so the model targets what it needs instead of reading everything and iterating.
← → arrow keys work too