Tool Calling and Structured Outputs

Declare tools, read calls, return results, run calls in parallel, stream arguments, and force JSON that validates.

Concepts and terms
Tasks and commands
Annotated code
Diagrams
Defaults and numbers
Troubleshooting
Questions and tips

Function call tasks#

Task	Code
Declare a tool, OpenAI	`{"type": "function", "function": {...}}`
Declare a tool, Anthropic	`{"name": ..., "input_schema": {...}}`
Let the model decide	`tool_choice="auto"`
Require a tool call	`tool_choice="required"`
Force one tool	`tool_choice={"type":"function", ...}`
Read OpenAI calls	`message.tool_calls`
Find an Anthropic call	`b.type == "tool_use"`
Return an Anthropic result	`{"type": "tool_result", "tool_use_id": ...}`

From Lesson 11.09

Structured output tasks#

Task	Code
Valid JSON, no schema	`{ type: "json_object" }`
JSON that fits a schema	`{type: "json_schema", strict: true}`
Parse into a Pydantic model	`response_format=Product`
Read the typed result	`message.parsed`
Retry on validation errors	`instructor.from_openai(OpenAI())`

From Lesson 11.03, Lesson 13.04

The tool loop#

From Lesson 13.01, Lesson 11.09

One tool call#

From Lesson 13.01, 13.01 code/main.py, Lesson 11.09

OpenAI round trip#

response = client.chat.completions.create(

model="gpt-4o", messages=messages,

tools=tools, tool_choice="auto",model picks tool or text

)

tool_call = response.choices[0].message.tool_calls[0]

args = json.loads(tool_call.function.arguments)arguments are a string

result = get_weather(**args)your code executes

messages += [

response.choices[0].message,keep the assistant turn

{"role": "tool", "tool_call_id": tool_call.id,same id as the call

"content": json.dumps(result)},

]

final = client.chat.completions.create(model="gpt-4o", messages=messages)

From Lesson 11.09

Shapes by provider#

Schema field: parameters, input_schema, Gemini parameters.
Call: tool_calls[], a tool_use block, a functionCall part.
Arguments: OpenAI sends a string. The others send an object.
Result link: tool_call_id, tool_use_id, functionResponse id.
Forbid tools: tool_choice: "none", {type: "none"}, mode: "NONE".
Strict: strict: true, always on, responseSchema.

From Lesson 13.02

Parallel and streaming calls#

Enable: parallel_tool_calls: true is the OpenAI default.
Serial: Turn it off when calls depend on each other.
Correlate: Each result carries its call id. Order is free.
OpenAI stream: Join chunks per index until finish_reason = "tool_calls".
Anthropic stream: input_json_delta chunks until content_block_stop.
Parse late: Partial JSON raises. Parse at the end signal.
Latency: 400, 600, 800 ms: 1800 ms serial, 800 ms parallel.

From Lesson 13.03, 13.03 code/main.py

Schema design rules#

Name: snake_case, verb then noun: get_weather. No arguments in the name.
Description: "Use when {condition}. Do not use for {close-but-wrong-cases}."
Enums: Put every closed set in an enum.
Atomic tools: Split a tool when action has more than three values.
Strict mode: Every property in required, additionalProperties: false, no $ref.

From Lesson 13.05, Lesson 13.04

Validate, then repair or refuse#

def process_model_output(raw: str, schema: dict) -> ParsedResult:

if raw.startswith("__REFUSAL__"):model declined: no retry

return ParsedResult("refusal", raw.removeprefix("__REFUSAL__").strip(), [])

try:

parsed = json.loads(raw)

except json.JSONDecodeError as e:failure 1: not JSON

return ParsedResult("parse_error", None, [ValidationError("$", str(e))])

errs = validate(schema, parsed)list of path and message

if errs:

return ParsedResult("violation", parsed, errs)inject errs, retry

return ParsedResult("ok", parsed, [])

From 13.04 code/main.py, Lesson 13.04

Limits and defaults#

Setting	Default	Note
OpenAI tools per request	128	Hard limit
Anthropic tools per request	64	Hard limit
OpenAI schema depth	5	Strict mode
OpenAI argument string	8192 bytes	Maximum size
Description length	1024 characters	Longer text is truncated
`MAX_TURNS`	5	Lesson loop bound
Repair retries	3x	More means a bad schema
`timeout_ms`	30_000	Per tool record

From Lesson 13.02, Lesson 13.05, 13.01 code/main.py, Lesson 13.04, 19.21 code/main.py

Troubleshooting#

Symptom	Cause	Fix
`json.loads` raises mid-stream	Arguments are partial JSON	Parse after the end-of-call signal
400 at request time, strict mode	Strict schema rule broken	All in `required`, no `$ref`
Wrong tool picked	Two descriptions overlap	Add a "Do not use for" line
Value outside the `enum`	Prompt-based output	Use constrained decoding
`refusal` in the response	Input does not fit the schema	Handle it as a typed outcome
Extra fields in `input`	Anthropic schema is advisory	Validate on the server
Schema errors retried	Errors are deterministic	Fix the arguments, no retry
Results on the wrong call	Matched by name, not id	Return each result with its id

From Lesson 13.03, Lesson 13.04, Lesson 13.05, Lesson 13.02, Lesson 11.03, Lesson 19.23

Questions and answers#

Structured output or call?: Output is final data. A call asks to act.
Why not prompt for JSON?: It fails 5 to 15 percent of the time.
Why a call id?: Parallel results return out of order.
Can a schema catch a wrong value?: No. Type right, value invented.

From Lesson 11.09, Lesson 13.01, Lesson 13.04, Lesson 11.03

Tips#

Return errors as results: {"error": true, "code": "CITY_NOT_FOUND"}.
Write errors for the model: "'city' is required."
Report errors as JSON pointer paths: /user/email.
Make key fields required. The model writes null.
Expose only the functions the user needs.

From Lesson 11.09, Lesson 13.05, Lesson 11.03, Lesson 19.21