R.10

How this manual was made

This page records the pin, the two machines, the lab method, each experiment with its machine and its count, and what the lab folder holds.

The pin

The manual pins langchain 1.5.0, released on 2026-11-11 at commit c2f4036e. The same pin holds langchain-core 1.8.0, langgraph 1.2.22, deepagents 0.7.38, and langsmith 0.14.17. Sources and conflicts gives the commit of each package. We read the code and the docs and checked every fact on 2026-11-14.

The packages of the stack move fast, and their version numbers are not aligned. A statement in this manual is about the pin only. Section 8.5 lists what differs in older code and older pages.

Four kinds of evidence

Each fact carries one of four labels, and the sources page ranks them.

The code: a class, a default, or an error text read in the source at the tag. A paragraph says when we also ran it.

A capture: a listing from the capture kit, which drives the real stack. Where a capture needs a model, the model is Qwen3 4B Instruct on Ollama on the x86 server guest. The file names the model, its settings, the machine, and the date.

We measured: a number from one of our lab runs. The paragraph names the machine and the number of runs.

The documentation: a statement from the docs at commit 04d4d4a. We use it for paths that this edition did not run, and the text says so.

The two machines

Name in the manualMachineSoftware
the Macan Apple M1 Max laptop with 10 cores and 64 GiB, shared with other workmacOS 26.2, Python 3.13.12, Ollama 0.31.1, PostgreSQL 14.21
the x86 server guesta KVM guest with 32 vCPU and 48 GiB on a shared server with an AMD EPYC 7452. Its disk was in the memory of the serverUbuntu 24.04, Python 3.13.18, PostgreSQL 16.15, Redis 7.0.15, Docker 29.1.3, Ollama 0.31.1 on CPU

The two machines differ, so no section sets a number from one beside a number from the other. Every time from the Mac is an upper bound for that machine, because other work ran on it during every timed run. The disk of the guest was in memory, so its write times are lower bounds for a real disk.

The lab method

  • Real models of the lab. The local models were Qwen3 0.6B, 1.7B, 4B, 8B, 14B, and 30B-A3B, and gpt-oss 20B, on Ollama, with nomic-embed-text for embeddings. Qwen3 4B on the x86 server guest ran the failure drill, the runtime traces, and the T02 agent matrix. It also ran T05, T07, T08, T11, T17, T18, and T19, and the reruns of T10, T15, and T16. The other model runs of the lab ran on the Mac. The capture kit is apart from the lab, and it recorded on the x86 server guest.
  • The context on the guest. The reruns of T10, T15, and T16 set a context of 8192 tokens. The other runs on the guest set none, so Ollama used its default context. During T02, ollama ps showed 4096. The prompts of those runs were short.
  • Hosted models on a budget. Claude Haiku 5.5 (claude-haiku-5-5) and Claude Sonnet 5.5 (claude-sonnet-5-5) ran in the bench, through the Anthropic API from the Mac. A shared ledger recorded every call and its cost, and stopped the runs at a fixed cap. The bench spent $0.93 in 2,589 calls. Haiku ran the full suite, and Sonnet ran the 30-task subset. The bench method gives the settings. Haiku also ran the 20 evals of section 9.6 once, in 64 calls for $0.011353, with a ledger of their own. It ran the 60 file-rule attacks of T22 in 134 calls for $0.02136, and the ledger rows are in T22-sec-fs-permissions-mac/.
  • Stand-ins for the stack alone. T03, T10, T15, and T16 on the guest used a model stand-in with a fixed delay, or no delay. Those times hold the cost of the stack and the database only. T10, T15, and T16 then ran again with Qwen3 4B, and sections 5.2, 5.5, and 8.1 show only those rows. T03 keeps its stand-in, because it measures checkpoint writes with no model.
  • Local recorders, proxies, and relays. A recorder on 127.0.0.1 stood in for a LangSmith endpoint. For the streaming runs, a proxy in front of Ollama stamped each line it passed on. So the lab could time each token on one clock. For the retry runs, a relay in front of Ollama dropped or held each connection.
  • A new process for each step. The crash, approval, and assistant runs started each step in a new Python process, so each resume was a restart.
  • Warm-up runs left out. A file marks its warm-up rows, and the summary leaves them out. The raw files keep them.
  • One LangSmith account and no license. T21 used a new LangSmith account on the free plan, from the Mac, against the hosted API. The kept files replace its ids with labels. We used no Agent Server license. Section 8.8 lists what that left out.
  • Small samples. Many cases ran once or three times. Each paragraph gives its count, and a result from one run is an example, not a rate.

The experiments

FolderMachineQuestionnMain resultSections
T01the MacWhat does create_agent cost against a plain openai loop?40 shuffled runs of each of 8 programs on Qwen3 4B, 10 per import37.6 ms outside the model at the median against 25.1 ms, and the same tokens in each request3.1, 4.1, 8.3
T02the MacDoes a side effect run twice after a crash and a resume?450 trials at five kill points, 30 of them never killedtwice in 80 of 150 graph trials, and in 0 of 150 with a key. Its agent rows used a scripted model, and T02 x86 replaces them5.3
T02 x86the x86 server guestDoes a charge run twice after a crash, with a real model in the agent?1,290 trials on Qwen3 4B: 20 per cell with no key and with a tool_call_id key, 5 per cell with an order keytwice in 331 of 560 killed trials with no key, 120 of 560 with tool_call_id, all in exit, and 0 of 140 with the order5.3, 2.3
T03the x86 server guestWhat does a checkpoint cost as a thread grows?3 runs at 10 to 1,000 steps, 1 at 5,000a plain Postgres thread held 292.58 MB at 1,000 steps5.5
T04the MacHow fast does each stream API deliver a token?24 runs per API on Qwen3 4B for a text reply and for a tool call, and 5 per loop on Qwen3 8B with reasoninga median first-token delay of 1.28 to 3.46 ms for a text reply on eight of nine APIs5.1
T05the x86 server guestHow many attempts do the retry layers make?3 runs per cell, Qwen3 4B9 attempts with both layers on ChatOpenAI, 1 with ChatOllama alone, and a timed-out tool wrote 3 ledger rows3.2, 4.7
T06the MacHow often does each structured output strategy fail?600 runs on four Qwen3 models, and 75 strategy runs on five Ollama modelsToolStrategy failed in 15% to 45% of runs per model, and no chat model on Ollama rejected ProviderStrategy4.2
T07the x86 server guestWhich tool runs after each review decision?25 model cases with Qwen3 4B, 3 runs each. G1 and G2 have no model and ran on the Macbatched mode can switch tools and skip a review, and per-call mode refused both6.4, 6.5
T08the x86 server guestDo parallel tool calls run at the same time?100 ToolNode cells with 5 runs each, and 12 agent cells with 3 each, Qwen3 4B16 sync calls of 200 ms took 264 ms with no cap, and ToolNode under ainvoke ignored max_concurrency3.4, 4.1
T09the MacHow many prompt tokens does each harness send?27 runs on Qwen3 8B, and 9 request variants counted 3 times eachthe prompt and tool schemas of a default deep agent were 57.6% smaller at 0.7.38 than at 0.6.12, by one tokenizer7.4
T10the x86 server guestWhat does a long thread do with and without a summary, on a real model?1 run per variant, four variants, 200 turns, Qwen3 4Bwithout a summary the median turn grew from 4.7 s to 11.2 s until the context filled at turn 88, then turns took 5 to 10 minutes and lost early facts. A summary call took 44 to 85 s at the median, and two of three triggers lost facts5.5
T11the x86 server guestWhat does each middleware hook cost?20 timed runs per cell, Qwen3 4B0.2% to 0.5% of a run, and a node hook call cost about 4 times a wrap hook call8.3
T12the MacWhat does each MCP failure do to a run?1 run per case, 50 timed calls per transporta tool error became an error ToolMessage, and a server crash ended the run7.3
T13the MacWhat does the serializer rebuild, and what does a loader accept?29 values, and 36 loader cases, 26 with the defaults and 10 strict, 1 run each20 values round-tripped, and 5 cases built the lab class5.6, 8.4
T14the MacDo namespaces keep the users of a store apart?8 users on three stores, 1 run per query24 of 24 searches stayed inside one user5.4, 8.4
T15the x86 server guestWhat do langgraph dev and langgraph up do?1 run per setting with the model-free graphs, and 104 load runs, 40 floor runs, and 4 recursion runs with Qwen3 4B1 job per worker by default, 0.6 to 1 s added per run, and 0.22 runs per second with 1 or 10 jobs8.1
T16the x86 server guestHow many threads can one AsyncPostgresSaver serve with a real model?2 runs per cell, 1 to 64 threads, Qwen3 4B0.42 to 0.44 model calls per second in every cell from 4 threads up, aput p50 under 5 ms, and the same throughput with one connection and a pool of 205.2
T17the Mac and the x86 server guestDo the recursion limits and the replays work as the docs say?1 run per graph case on the Mac, 3 per agent case with Qwen3 4B on the guesta graph stopped at 10007 steps, and an agent made 13 model calls at a limit of 251.3, 6.6, 6.8
TRCthe MacWhat does tracing cost, and what happens when the endpoint fails?60 interleaved runs per condition on Qwen3 4B, 50 runs per endpoint state26.3 ms outside the model with tracing on, against 17.7 ms off, and an exit held 10 to 17 s by a failing endpoint7.6
T18the x86 server guestWhat does evaluate(upload_results=False) send with the default client?5 processes per condition, five conditions, a Qwen3 4B agentone GET /info in every process, and the 7 runs of an agent target in three multipart posts7.7
T19the x86 server guestWhat does the pytest plugin of langsmith send?3 processes per condition, nine conditions, a Qwen3 4B agent testwith tracking on and no environment, each run contacted the default host. With tracking and tracing off, nothing left. With tracking off and tracing on, every process sent GET /info and three POST /runs/multipart7.7
T20the MacWhat do elicitation, an error with structured content, and progress do through langchain.mcp?3 runs per case, 27 cases, with no model: the agent cases use a scripted fake chat modelone answered question ran the tool 3 times, and an error result lost its structured content. These are behaviours of the server and the adapter7.3
T21the Mac, against the hosted LangSmith APIWhat do tracing, evaluate, the pytest plugin, the prompt hub, and langgraph dev do on a real account?30 traced and 30 untraced runs of a Qwen3 4B agent, 3 local runs, 10 examples per evaluate mode, 8 pytest processes, 5 server runstracing added 5.9 ms at the median, and upload_results=False stored 70 agent runs with no root7.6, 7.7, 8.1
T22the Mac, and the Anthropic API from the MacWhich file calls does a rule list of Deep Agents stop?8 rule lists and 28 calls, one run per cell with a scripted caller and no model, and 20 adversarial prompts under 3 rule lists, one run per cell, on Qwen3 8B and on Claude Haiku 5.522 calls ran that a docs rule list was written to block, all on dotfiles. With the dot-aware list, no run changed the disk or leaked a secret, in 40 of 40 runs8.4
T23the MacDoes a constant StoreBackend namespace give one user's memory file to another user?10 sessions for each of 4 setups on each of 2 stores, Qwen3 8B, and one run of the raw store probes on each storethe docs factory gave Bob Alice's code in 20 of 20 sessions, and a factory per user in 0 of 205.4, 8.4
B2the Mac, and the Anthropic API from the MacDo a harness profile, the eval tool, or a rubric grader change the score of a deep agent?profiles on 99 and 30 tasks on Qwen3 4B and 8B, the eval tool on 39 tasks on four models, a grader on 25 tasks, one run per taskprofile B took Qwen3 4B from 2 to 14 of 20 file tasks and left its total at 55 of 99, no model gained from eval, and the grader said satisfied on 12 failed runs7.4, 4.6
MAthe Mac, and the Anthropic API from the MacWhich of five multi-agent patterns passes the suite, and at what cost?30 tasks, one run per task, on Haiku and Qwen3 14B, and the Claude Agent SDK against a deep agent on HaikuHaiku passed 30 of 30 in four patterns and subagents cost 2.9 times the single agent, Qwen3 14B passed 15 to 22 of 306.7, 1.2
sql-agentthe MacDoes a SQL agent answer and write only with approval?15 guard cases and 5 questions, 1 run on Qwen3 4B and 1 on Qwen3 8Bboth models answered 5 of 5 right, and each refused delete changed no row4.9
retrieval-agentthe MacDoes a retrieval agent cite its sources?5 questions, 1 run per modelSection 7.2 gives each answer7.2
assistantthe MacDoes an assistant survive a crash after an approved email?2 runs of seven steps, one per durability modethe reviewer approved twice, and the key kept the outbox at one email5.7
deep-agentthe MacWhat does a deep agent with a subagent send?1 run on Qwen3 8B24,300 prompt tokens in 7 model calls7.5
failure-drillthe x86 server guestWhat does each documented failure look like?3 runs of seven cases with Qwen3 4Beach run took 101 to 103 s8.7
chat-ui-macthe MacWhat does Agent Chat UI send to Agent Server, and what does a typed message do to a pending approval?15 runs of Qwen3 4B in one headless browser session: approve, edit, and reject once each, a typed message over an approval once in each mode, and 3 plain questions for the first tokenthe typed message closed the approval as a success in 2 of 2 trials, and the model said that it sent the email. A per-call approval showed no decision control8.2
internalsthe x86 server guestWhat does the runtime do in each superstep, save, resume, and stream?traces on Qwen3 4B, 50 crash trials per cell in two runs, 30 cost runs per cell, 100 round trips of growth, and 5 runs per loop on Qwen3 8Ba task that just returned ran again in 97 of 100 sync and 95 of 100 async trials, and the framework took under 0.5% of a run2.1 to 2.6
benchthe Mac and the Anthropic APIDoes the harness change what an agent gets right?99 tasks on Qwen3 4B, Qwen3 8B, and Claude Haiku 5.5, and 30 on four more models, one run per taskthe plain loop, create_agent, and StateGraph gave identical runs on 60 of 60 local tasks, and Claude passed every task as a plain loop and as a deep agent1.2, 4.3, 4.4, 7.4
prodapp-macthe Mac, and the Anthropic API from the MacWhat does a FastAPI service on create_agent with Postgres do with real models, crashes, faults, and load?27 tests in 3 runs on Qwen3 4B and 3 on Qwen3 14B, and the 20 evals once on Claude Haiku 5.5. Also 10 crash trials per resume form, 5 trials per disconnect and drain case, and one load run per cell. The fix runs: the 6 refund evals in 3 runs on each of the two Qwen models, the crash test in 3 runs on each, 3 crash trials on Qwen3 4B, and the 50-user load cell oncethe approval was pending again after 20 of 20 kills, the key kept one refund, and 1, 2, and 4 workers gave the same throughput. After the fix, the 0-cent row was gone in 6 of 6 runs, and the heartbeat left 0 of 200 requests timed out9.1 to 9.7
last-two-macthe MacWhat does ToolStrategy do with no tools, and which bytes does create_agent add?40 runs on Qwen3 4B, 10 on Qwen3 0.6B, and 5 request captures per programa 400 from Ollama in 40 of 40 runs on Qwen3 4B, and 7 to 17 bytes per request, each one in a ledger4.2, 8.3

The names follow the plan of the research, with no gap. T01 to T17 are the planned tests, and TRC is the tracing test. T18 to T20 are later runs for sections 7.3 and 7.7, and T21 is the account run for sections 7.6, 7.7, and 8.1. T22 and T23 are the runs of the Deep Agents file rules and store namespaces for sections 5.4 and 8.4. chat-ui-mac is the run of Agent Chat UI for section 8.2.

The five demo folders are the worked examples of sections 4.9, 5.7, 7.2, 7.5, and 8.7. internals, bench, T02 x86, and last-two-mac are the runs of the second pass, which put a real model in every result. prodapp-mac is the production service of Part 9.

What is in the lab folder

The lab folder holds one folder for each row of the table above, and the folder summary. 13 rows also keep an older Mac folder: T01, T02, T04, T05, T06, T07, T08, T11, T17, T18, T19, TRC, and the failure drill. The lab README marks which of its rows the x86 runs replaced. The rows of T10, T15, and T16 keep their older stand-in folder beside a second-pass folder that ends in -real.

  • Raw recordings. Each experiment folder keeps the files of its run, unchanged, as JSON lines, logs, and text. A long log is kept as an excerpt whose lines are exact.
  • Scripts. Many folders keep the lab scripts that made them, and the lab README names them. The demo folders hold the output of run.sh in capture/demos/.
  • Summaries. capture/run.py computes one summary for each data set into summary/. Each chart reads one or two of the 48 summary files, or the summary of the bench, and each measured number comes from one. The production service keeps ten more summaries in its own folder, and capture/labsum.py rebuilds them from its raw runs.
  • A list of changes. The README of the lab folder names every file that we cut, left out, or changed, with the reason.

No file holds a key, a token, or the name of a person. The only host names are 127.0.0.1 and the public hosts that recorded requests and error texts name, such as api.smith.langchain.com. The addresses and names in the test inputs are made up.

The capture kit

The capture kit drives the stack in one Python process, with no keys. It writes 56 files to capture/out/, and every listing in the manual is cut from one of them.

  • 19 model-free files. They call no model. They cover the versions, the messages, the tool schemas, the graph structure, and the Mermaid text of every graph the manual draws. They also cover durability, interrupts, reducers, the functional API, subgraphs, error handlers, and the recursion limit. They give the same bytes on every run.
  • 36 recorded files and one sidecar. 34 of them call ChatOllama on qwen3:4b-instruct-2507-q4_K_M, with temperature 0, seed 42, num_ctx 16384, and num_predict 512. The store, retrieval, and retrieval agent files call OllamaEmbeddings on nomic-embed-text, and two of them call no chat model. We recorded them on the x86 server guest, on CPU, on 2026-11-14. The first line under each title names the models, the settings, the machine, and the date.

No scenario uses a fake chat model. A failure comes from a tool that raises, from a port where nothing listens, or from a relay that drops connections. The decision middleware asks Qwen3 4B through a small adapter of the kit. No scenario uses a fake embedding. The store and retrieval scenarios embed with nomic-embed-text on the Ollama server of the guest.

bash
cd manuals/langchain-101
python3 capture/run.py --model-free
python3 capture/run.py --check
python3 capture/run.py --check --live

The first command writes the model-free files and the lab summaries, offline. A plain python3 capture/run.py writes every file and needs Ollama.

The check runs the model-free scenarios again and compares each file byte for byte. For each recorded file, it checks the model line and the strings that the manual relies on. These are message types, tool names, and error texts. It then computes the summaries again and compares them. It prints one skipped line for the recorded scenarios and one for the lab runs. With --live, it records the model scenarios again into a temporary folder and checks the same structure.

Record it again on your machine

  1. Create the virtual environment of the kit from capture/requirements.txt with Python 3.13.
  2. Install Ollama, pull the Qwen3 models, and start PostgreSQL for the experiments that use them.
  3. Run the script of one experiment, from a folder that keeps one. Each script writes the files of its own folder.
  4. Run python3 capture/run.py from the manual directory to compute the summaries again.

Your times will differ from ours, and so can the words of the model. The message types, the error texts, the checkpoints, and the order of events must not. If one of them differs at the pin, a claim of this manual depends on our machines.

LangChain (Python): langchain, langchain-core, langgraph, deepagents, langsmith 1.5.0 · c2f4036 · 2026-11-11 · Edition 2026.11