8.3

Cost: the framework against a plain SDK loop

create_agent adds about 5 ms per model call and no tokens to a plain loop, while Deep Agents, long plain threads, and a failing trace endpoint cost far more.

Your team has a plain loop on the openai client that works. Someone asks what LangChain would cost before you move to it. The answer has several parts: time per call, tokens, request bytes, import time, disk, checkpoints, and traces.

When you finish this section, you can say what each layer of the stack adds to a run, and choose where to spend it.

The cost sheet

We measured each row of this table on one machine, and the row names the machine and the count. The Mac is an Apple M1 Max with 10 cores and 64 GiB, shared with other work. The x86 server guest has 32 vCPU and 48 GiB. The machines differ, so compare numbers inside a row, never across rows. Raw runs

CostPlain loopLangChainMachine and countSection
time beyond the model in one run of three Qwen3 4B calls25.1 ms (1.3% of the run)37.6 ms with create_agent (2.3%), 15.6 ms more in the same roundthe Mac, 40 runs eachthis section
prompt tokens of the three calls of that run, on Qwen3 4B261, 300, 340the samethe Mac, 40 runs eachthis section
request bytes of those three calls, on Qwen3 4B679, 936, 1,199696, 948, 1,206the Mac, 40 runs eachthis section
prompt tokens of a one-word answer, on Qwen3 8B2828 with create_agent, 2,279 with create_deep_agentthe Mac, 3 runs each7.4
imports, with bytecode written0.329 s for openai0.831 s for create_agent and ChatOpenAIthe Mac, 10 runs each3.1
install14 packages, 26,356 KiB44 packages, 54,312 KiBthe Mac, one install each3.1
one checkpoint write on PostgresSaver, median of the last 10 of 5,000 stepsnone148.65 ms plain, 1.13 ms with DeltaChannelthe x86 server guest, one run5.5
time outside the model of one Qwen3 4B agent run with tracing on, against a local recordernone26.3 ms against 17.7 ms with tracing off, 0.5% of the runthe Mac, 60 interleaved runs each7.6
one middleware hook, each time it runsnone183 to 190 µs node hook, 45 to 50 µs wrap hook, 0.2 to 0.5% of a real Qwen3 4B runthe x86 server guest under load 14 to 25, 20 runs per cellthis section

The rest of this section gives the recordings behind the first three rows and the last row, which no other section shows in full. Section 9.7 prices a whole support conversation from measured tokens: $0.61 for 1,000 conversations on Claude Haiku 5.5.

Time per run

The task of the recording is one question that needs two tools, add and then multiply, and three model calls. Eight programs ran it against Qwen3 4B on Ollama, five on the /v1 path and three on the native path. A proxy timed each model call. Every run gave the right answer, and each program called the same tools with the same arguments.

Time outside the model: the wall time of a run minus the time of its model calls at the proxy. It holds the HTTP client and the proxy hop too, so a plain loop also has some.

  • plain openai loop: the openai client, a list of messages, and a loop over the tool calls.
  • ChatOpenAI.bind_tools loop: the same loop, with the model of langchain-openai in place of the client.
  • create_agent: the agent with ChatOpenAI and the two tools.
  • create_agent with InMemorySaver: the same agent with a checkpointer and a thread.
  • hand-built StateGraph: a model node, a ToolNode, and tools_condition, the same graph as create_agent written by hand.
  • native programs: a plain ollama loop, create_agent with ChatOllama, and the hand-built StateGraph with ChatOllama, all on /api/chat.
Fig. 8.4time beyond the model in one run of three Qwen3 4B callschart
Time beyond the model in one run of three Qwen3 4B calls A dot chart of the milliseconds outside the model in each run on the Mac, 40 runs per program, against Qwen3 4B on Ollama through a proxy that timed each model call. Medians: plain openai loop 25.11 ms, ChatOpenAI bind_tools loop 29.32 ms, create_agent 37.62 ms, create_agent with InMemorySaver 37.64 ms, hand-built StateGraph 36.71 ms. 0 20 40 60 80 milliseconds outside the model 25.11 ms plain openai loop 29.32 ms ChatOpenAI bind_tools loop 37.62 ms create_agent 37.64 ms create_agent InMemorySaver 36.71 ms hand-built StateGraph Each dot is one run of three model calls and two tool calls, 40 per program. The black line is the median. Warm-up rounds are left out.
In a real run of 1.8 s, create_agent spent 37.6 ms outside the model against 25.1 ms for a plain openai loop, and a hand-built StateGraph 36.7 ms. Each dot is one run on the Mac, 40 per program, in shuffled rounds, and the black line is the median.

We measured 40 runs of each program on the Mac, in 40 shuffled rounds after 2 warm-up rounds. Figure 8.4 shows each run. Every run gave 20 after add(2, 3) and multiply(5, 4). The median time outside the model was 25.1 ms for the plain loop and 29.3 ms for the bind_tools loop. It was 37.6 ms for create_agent, 37.6 ms with InMemorySaver, and 36.7 ms for the hand-built StateGraph. Against the plain loop in the same round, they added 6.6, 15.6, 12.2, and 11.5 ms. Raw runs

On the native path, the plain ollama loop spent 11.7 ms outside the model. create_agent with ChatOllama spent 29.0 ms, 13.0 ms more, and the StateGraph 22.2 ms, 7.8 ms more.

Per model call, the agent layer adds about 4 to 5 ms, and 3 ms of CPU. On the /v1 path, the time outside the model was 1.3 to 2.3% of a run of 1.7 to 2.0 s. The model time moved from 1.2 s to 3.7 s per run with the other load on the machine. So compare the time outside the model, not the wall time.

Tokens and request bytes

A framework could add tokens: a system prompt of its own, extra tool fields, or a schema. create_agent adds none. We measured the usage that Ollama returned for each call, through a proxy that recorded each request, 40 runs per program on the Mac. Every run of a program had the same three calls:

CallPlain loop: bytes, prompt tokens, completion tokenscreate_agent: bytes, prompt tokens, completion tokens
1679, 261, 25696, 261, 25
2936, 300, 25948, 300, 25
31,199, 340, 31,206, 340, 3

The tokens were the same for both programs, call by call. The request of create_agent was 7 to 17 bytes longer on each call.

We captured the raw request bodies of both programs at the proxy, 5 runs each, and compared them call by call. Every byte of the difference is in this ledger:

outputthe bytes that create_agent adds to each requestFull log
### request 1: a_plain_openai (679 B) vs c_create_agent (696 B), round 1, raw bodies as sent
…
  +15 B  "stream":false added
  +2 B  "temperature":0 -> "temperature":0.0
  ledger sum +17 B; measured +17 B
### request 2: a_plain_openai (936 B) vs c_create_agent (948 B), round 1, raw bodies as sent
…
  +15 B  "stream":false added
  +2 B  "temperature":0 -> "temperature":0.0
  +2 B  assistant turn 1: "content":"" -> "content":null
  +3 B  assistant turn 1: arguments {"a":2,"b":3} -> {"a": 2, "b": 3}
  -10 B  assistant turn 1: "index":0 dropped
  ledger sum +12 B; measured +12 B
### request 3: a_plain_openai (1199 B) vs c_create_agent (1206 B), round 1, raw bodies as sent
…
  +2 B  assistant turn 2: "content":"" -> "content":null
  +3 B  assistant turn 2: arguments {"a":5,"b":4} -> {"a": 5, "b": 4}
  -10 B  assistant turn 2: "index":0 dropped
  ledger sum +7 B; measured +7 B
Cut to the ledger of each request. The other bytes are the same keys and values in another order.

ChatOpenAI adds "stream": false and writes the temperature as a float, 17 bytes on every call. For each earlier tool call in the history, it sends "content": null where the plain loop echoed the empty string of Ollama. It writes the arguments again with spaces, and it leaves out the "index": 0 that the plain loop copied from model_dump(). That is 5 bytes fewer per turn. The tool schemas are equal as JSON but not as bytes: create_agent writes "type": "object" after "required", and "content" before "role" in each message.

The headers differ in two fields. The LangChain programs send x-stainless-raw-response: true, because ChatOpenAI calls the client through with_raw_response. In the recorded requests they also had no x-stainless-read-timeout, which the plain loop sent.

The Deep Agents harness is the exception. On Qwen3 8B, create_deep_agent sent 2,075 to 2,251 more prompt tokens on every call than a plain loop. That was 81.4 times the tokens of a one-word answer, 10.0 times for two tool calls, and 9.1 times for a file task. Section 7.4 and figure 7.8 show each call. Pay those tokens only when you use the planning, the files, or the subagents.

The task suite of section 1.2 gives the same picture at scale. The deep agent sent 3.6 to 4.3 times the prompt tokens of the plain loop. On Claude Haiku 5.5 that was about 3,570 more per call. Prompt caching changes the bill. On Claude the deep agent cost 1.16 to 1.24 times the plain loop, and 3.10 to 3.14 times at list price with no cache.

Imports and disk

The cost of a process start matters for a CLI tool, a test run, and a serverless handler. Section 3.1 gives the import times with figure 3.2. With bytecode already written, the imports for create_agent and ChatOpenAI took 0.831 s at the median, against 0.329 s for import openai. On the first start with no bytecode, they took 2.64 s against 1.064 s. The install is about twice the size: 54,312 KiB against 26,356 KiB.

Checkpoints

A checkpointer writes the whole state at each step, unless a channel saves only the change. Section 5.5 gives the recording with figure 5.12. On PostgresSaver with a plain message list, the median of the last 10 writes was 2.88 ms at 10 steps. At 5,000 steps it was 148.65 ms.

One thread of 5,000 steps held 7,289 MB. With DeltaChannel, the write stayed near 1.1 ms, and the thread held 81 MB with a snapshot every 100 steps. Summarize long threads, and measure the step count of your own threads.

Tracing

Tracing adds a cost to each run and a cost at exit. Section 7.6 gives both with figure 7.13. Against a local recorder that answered at once, tracing added 7.4 ms to a 1.6 s agent run on Qwen3 4B. In six of seven processes, the first traced run spent 83 to 243 ms outside the model. A failing endpoint held the exit for 10 to 17 s, and a hanging endpoint kept the process alive.

Section 9.4 traces a service with its own OpenTelemetry spans. With LANGSMITH_TRACING false or unset, a recorder got no request.

Middleware hooks

Each middleware hook is one more node of the graph or one more wrapper around a call, as section 4.5 shows. A node hook, such as before_model, adds a step of the loop each time it runs. A wrap hook, such as wrap_model_call, adds a function call and no node.

We measured the cost on the x86 server guest with Qwen3 4B on Ollama. Framework time is wall time minus time inside the model call. A run of two model calls and one tool call took 2.61 s, of which 5.88 ms was framework. Cells had 1, 10, or 20 middlewares, 20 timed runs each in shuffled rounds. Every run gave the right answer, and the middlewares did nothing. The cost per hook call is the slope of the median over the middleware count, divided by the calls of that hook in one run. Raw runs

HookCalls in one runCost of each callFramework time with 20 middlewaresGraph nodes with 20
before_model2183 µs13.2 ms24
after_model2190 µs13.5 ms24
before_agent1187 µs9.6 ms24
wrap_model_call245 µs7.7 ms4
wrap_tool_call150 µs6.9 ms4

The framework share of the run was 0.2% with no middleware and 0.5% with 20 node hooks. The load average was 14 to 25, because Ollama used every core. So read the shares and the ratio of node hooks to wrap hooks, not the microseconds.

We measured nine setups of built-in middleware in the same way on the x86 server guest with Qwen3 4B, 20 runs per cell. Seven held one middleware, one held five PIIMiddleware instances, and one held a stack of five. None of them acted: no retry, no limit reached, no summary, no interrupt. Raw runs

Middleware, as configuredAdded to a runGraph nodes
ToolRetryMiddleware()0.14 ms4
ModelRetryMiddleware()0.15 ms4
SummarizationMiddleware with a trigger that never fires0.36 ms5
ToolCallLimitMiddleware(run_limit=1000)0.75 ms5
HumanInTheLoopMiddleware with no interrupt for the tool that ran0.86 ms5
ModelCallLimitMiddleware(run_limit=1000)1.30 ms6
PIIMiddleware("email", strategy="redact")1.36 ms6
five PIIMiddleware instances, one per type4.77 ms14
a stack of summarization, PII, a model call limit, tool retry, and review3.39 ms10

The cost follows the node count. The retries wrap calls and add under 0.2 ms, and each middleware with a node adds 0.4 to 1.4 ms. The five PII instances added 4.8 ms to a 2.64 s run of Qwen3 4B, 0.4%. So choose middleware for what it does, and keep node hooks out of a loop that makes thousands of fast calls.

LangChain (Python): langchain, langchain-core, langgraph, deepagents, langsmith 1.5.0 · c2f4036 · 2026-11-11 · Edition 2026.11