Parallel Tool Calls and Streaming with Tools
Three independent weather lookups serialized is three round trips. Run them in parallel and total time collapses to the slowest single call. Every frontier provider now emits multiple tool calls in a single turn. The payoff is real; the plumbing is subtle. This lesson walks both halves: the parallel fan-out and the streamed-argument reassembly, with emphasis on the id-correlation trap. Explain why paralleltoolcalls: true exists and when to disable it. Correlate streamed argument chunks to the right tool-call id during parallel fan-out. Reassemble partial arguments strings into complete JSON without parsing early. Run a three-city weather benchmark that demonstrates sequential vs parallel latency. Without parallel calls, an agent answering "what is the weather in Bengaluru, Tokyo, and Zurich" does this: Three LLM round trips, each of which also pays the executor latency. Roughly 4x the ideal wall-clock time. With parallel calls: One LLM round trip. Executor time is the maximum of the three, not the sum. Production benchmarks on OpenAI, Anthropic, and Gemini show 60 to 70 percent wall-clock reduction on fan-out workloads. The price is correlation complexity. When the three calls complete out of order, your results must carry the matching toolcallid so the model can line them up. When results stream, you must assemble partial argument fragments into complete JSON before executing. Gemini 3 added unique ids in…
Parallel Tool Calls and Streaming with Tools: Three independent weather lookups serialized is three round trips. Run them in parallel and total time collapses…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.