Skip to slide
Chapter 3 · Tool Design
21 / 191

CHAPTER 03 · Tool Design · 5 / 11

Batching

When the model commonly needs N of something, give it a tool that does N at once rather than forcing N separate calls. "Fetch these documents" (a list) beats N× "fetch this document." Batching cuts round-trips, which cuts latency and loop iterations. Cap the batch size in the schema to prevent abuse.

← → arrow keys work too