5.5

Context engineering and summarization

A thread grows on every turn, so each model call gets longer and each checkpoint larger, until you trim or summarize the messages and store them as deltas.

Your assistant feels quick in the first hour. After a few dozen turns on one thread, every answer takes longer, the provider bill grows, and the database grows faster than the bill. Nothing failed. The agent sends the whole thread to the model on every call, and the checkpointer saves the whole thread on every step.

When you finish this section, you can choose what the model receives, set a summarization trigger, and predict how a long thread grows.

What the model receives

Context engineering: the choice of what the model receives on each call: which messages, which tools, and which instructions.

Context window: the most tokens that a model accepts in one call, input and output together.

Section 5.2 showed that each call gets every message of the thread. The short-term memory page names three ways to keep the list short: trim it, delete messages, and summarize it. Each one must keep the rule of section 3.4: one ToolMessage answers each tool call, and no ToolMessage stands alone.

trim_messages

trim_messages: a function in langchain_core.messages that keeps the first or the last messages of a list up to a token budget. It changes the list that you pass to a model, and it leaves the state as it was.

The kit trims the five messages of section 3.3 to a budget of three. The token counter is len, so each message counts as one token:

jsontrim_messages keeps the last three messagesFull log
max_tokens=3, token_counter=len:
[
  {
    "type": "SystemMessage",
    "content": "You are terse."
  },
  {
    "type": "ToolMessage",
    "content": "5",
    "name": "add",
    "tool_call_id": "call_1",
    "status": "success"
  },
  {
    "type": "AIMessage",
    "content": "The sum is 5.",
…
]
The usage numbers are cut.

The result keeps the ToolMessage of call_1 and drops the AI message that made the call. Most providers reject such a request. With start_on="human", the trim drops messages until it finds a human message. The last two messages hold none, so only the system message was left. Trim on whole turns, and check the first message after the system message.

SummarizationMiddleware

Summary message: a human message that SummarizationMiddleware writes in place of old messages. It starts with Here is a summary of the conversation to date: and carries lc_source summarization.

SummarizationMiddleware calls a second model to summarize the old messages when a trigger fires, and keeps the newest ones. The kit sets trigger=("messages", 6) and keep=("messages", 2), and sends seven messages:

jsonthe state after the summaryFull log
  {
    "type": "HumanMessage",
    "content": "Here is a summary of the conversation to date:\n\n## SESSION INTENT  \nCalculate the sum of two numbers provided by the user.…",
    "id": "<uuid-1>",
    "additional_kwargs": {
      "lc_source": "summarization"
    }
  },
  {
    "type": "AIMessage",
    "content": "The sum is 9.",
…
  {
    "type": "HumanMessage",
    "content": "Now 1 + 1?",
The summary is cut after its first section. The ids of the last two messages are cut.

The agent model then got three messages: the summary, the last answer, and the new question. It called add with 1 and 1, and answered The sum is 2..

The summary is a model output, so it can be wrong. The summarizer saw only the five old messages. It wrote that 4 + 5 got no answer, and listed an add call for it that never ran. The answer The sum is 9. was in the two kept messages, out of its sight. The middleware replaced the messages in the state, so later turns start from the summary.

With no trigger, the middleware never summarizes. The built-in middleware page says the same. In the kit, the model got all seven messages. Set the trigger. The default keep is ("messages", 20). Part 4 covers ContextEditingMiddleware, which clears old tool results in place of a summary.

A long thread on a real model

We measured long threads on the x86 server guest, 32 vCPU and 48 GiB, with create_agent, PostgresSaver on PostgreSQL 16.15, and default durability. The model was Qwen3 4B on Ollama on CPU, temperature 0, seed 42, with a context of 8192 tokens. The summary model was the same model. Raw runs

Each variant was one run of 200 turns on one thread. Turns 1 to 5 gave five facts, and the rest cycled through a plain answer, one add call, and two parallel add calls. Every 25 turns, five recall questions about the facts ran on a fork of the thread, so they never entered it. The script gave each variant about 70 minutes.

VariantSummariesTurn p50 without / with a summary callLargest promptAdd turns rightRecall of the 5 factsThread size
no summarization07.4 s, and 308 to 621 s from turn 888,191 tokens, the full context59 of 595 of 5 to turn 75, then 0 of 4 after turn 94, and the fifth got HTTP 40020.52 MB at turn 90
("messages", 60), keep 7135.5 s / 55.2 s2,146 tokens126 of 1305 of 5 at all 8 probes12.42 MB at turn 200
("tokens", 6000), keep 1,500 tokens47.5 s / 108.4 s5,985 tokens130 of 1305 of 5 to turn 50, 0 of 5 from turn 7538.00 MB at turn 200
("messages", 20), keep 1395.4 s / 57.3 s1,219 tokens103 of 130: the sums were right in 129, and the tool was skipped in 265 of 5 to turn 75, then 4 of 5 without the code word6.40 MB at turn 200

Figure 5.11 shows the median of each block of 25 turns. Without summarization, the median turn grew from 4.7 s in turns 1 to 25 to 11.2 s in turns 51 to 75. At turn 88 the prompt reached the context of 8,192 tokens. From then on, each model call took about 300 s on this CPU, nearly all of it to read the prompt. The turns took 308 to 621 s, and the run stopped at its time limit after turn 94.

Ollama cut the oldest part of the prompt without an error. The answers stayed right. But four recall questions on the last checkpoint all said that the model did not know. The fifth failed with HTTP 400, request (8192 tokens) exceeds the available context size (8192 tokens).

With a trigger, the median turn stayed between 5.4 and 7.5 s. The median turn with a summary call took 55 to 108 s, because the summary is one more model call over the old messages. The summary call alone took 44 to 85 s at the median. LangChain, LangGraph, and Postgres took 18.9 to 24.7 ms of a turn at the median.

A summary can lose facts. The 6,000-token trigger summarized first at turn 65. From turn 75 on, the model knew none of the five facts, in 6 of 6 probes. The 20-message trigger lost the code word from turn 100 on. The 60-message trigger kept all five in all 8 probes. Check what your summary keeps before you trust a trigger.

The tool messages stayed paired in every case. 0 of 1,143 model requests had a ToolMessage without its tool call (Qwen3 4B, x86 server guest). Issue #38352 reports such orphans after a summary, and they did not happen in our runs.

Fig. 5.11turn time over 200 turns on Qwen3 4B, with and without summarizationchart
Turn time over 200 turns on Qwen3 4B, with and without summarization A dot chart of the median turn time of each block of 25 turns, on a log scale from 1 to 1,000 seconds, for one run per variant on the x86 server guest. Without summarization the median grows from 4.7 s to 18.6 s, and the run stops after turn 94. Open rings mark its turns 88 to 94, after the prompt reached 8,191 tokens, which took 308 to 621 s each. With a trigger, every block median stays between 4.4 and 9.2 s over 200 turns. A shaded band behind each line spans the 10th to the 90th percentile of the turns in each block, and its top holds the turns with a summary call. 1 3 10 30 100 300 1000 0 25 50 75 100 125 150 175 200 turn number median turn, seconds no summarization, turns 88 to 94 context full, run stopped 6,000 tokens, keep 1,500 20 messages, keep 1 60 messages, keep 7 One run per variant on PostgresSaver with Qwen3 4B on CPU. Each dot is the median of 25 turns, and each open ring is one turn. The band spans the 10th to the 90th percentile of each block.
Without summarization a turn takes 5 to 10 minutes once the context fills at turn 88, and with a trigger block medians stay at 4.4 to 9.2 s. Each dot is the median turn time of 25 turns in one run, one run per variant. Each open ring is one turn of the run without summarization after its prompt filled the context. The shaded band spans the 10th to the 90th percentile of the same 25 turns. The y axis is on a log scale.

The thread still grew with a summary. Every old checkpoint stays in Postgres, so storage grew in a straight line. With the 60-message trigger, it grew from 5.98 MB at turn 100 to 12.42 MB at turn 200. With the 6,000-token trigger, it grew from 17.58 MB to 38.00 MB. A summary limits what the model gets. It does not limit what the checkpointer keeps.

Checkpoint growth, and DeltaChannel

A plain message channel saves the whole list in every checkpoint. A thread of n steps then stores about n squared over two messages.

DeltaChannel: a channel type in langgraph.channels.delta that saves only the new writes of each step, plus a full snapshot. The snapshot comes after snapshot_frequency updates to the channel, 100 by default, or after 5,000 steps, whichever is first. It is beta at the pin.

The lab graph declares its message channel in three ways:

pythona plain channel and two DeltaChannel settingsFull log
    if mode == "plain":
        ann = add_messages
    elif mode == "delta100":
        ann = DeltaChannel(_messages_delta_reducer, snapshot_frequency=100)
    else:
        ann = DeltaChannel(_messages_delta_reducer, snapshot_frequency=1000)

    class S(TypedDict):
        messages: Annotated[list, ann]
        n: int

_messages_delta_reducer is the delta form of add_messages in langgraph.graph.message. Its name starts with _, so it is private at the pin.

We measured checkpoint cost on the x86 server guest with durability="sync", one 400-character message per step, and one thread per run. We ran 3 runs at 10, 100, and 1,000 steps and 1 run at 5,000, for InMemorySaver, SqliteSaver, and PostgresSaver. On Postgres, a plain thread held 0.05 MB at 10 steps and 3.08 MB at 100. It held 292.58 MB at 1,000 steps and 7,289.37 MB at 5,000. Raw runs

With DeltaChannel and a snapshot every 100 steps, the 5,000-step thread held 81.2 MB. With a snapshot every 1,000, it held 15.7 MB. From 1,000 steps on, the plain thread had the same size on the three savers, within 0.05 percent. Figure 5.12 plots every run.

Fig. 5.12storage of one thread by steps, plain and DeltaChannelchart
Storage of one thread by steps, plain and DeltaChannel A log-log dot chart of the storage of one thread in MB against steps, 10 to 5,000, on PostgresSaver on the x86 server guest. The plain channel stores 0.05, 3.08, 292.58, and 7289.37 MB. DeltaChannel with a snapshot every 100 steps stores 81.2 MB at 5,000 steps, and every 1,000 steps stores 15.7 MB. 0.01 0.1 1 10 100 1k 10k 10 100 1k 5k steps MB in one thread plain, add_messages DeltaChannel, every 100 DeltaChannel, every 1,000 Each dot is one run, durability sync, one 400-character message per step.
A plain message channel stores 7,289 MB after 5,000 steps, and DeltaChannel stores 81 MB with snapshots every 100 steps and 16 MB every 1,000. Both axes are on a log scale. Each dot is one run on PostgresSaver, three runs at 10 to 1,000 steps and one at 5,000.

Write time grew with the plain thread too. The median put of the last 10 steps on Postgres went from 2.88 ms at 10 steps to 148.65 ms at 5,000. With either DeltaChannel setting it stayed between 1.02 and 1.18 ms.

Reads went the other way. At 1,000 steps on Postgres, get_state took 29.28 to 31.8 ms on the plain thread. With DeltaChannel it took 43.24 to 65.61 ms, best of three reads in each run. A delta channel rebuilds the list from its writes since the last snapshot. Section 2.6 found the same direction on a real agent on SqliteSaver, at a smaller scale. After 100 round trips, a read took 2.76 ms with DeltaChannel against 2.59 ms plain.

The pregel page lists two rules for DeltaChannel. Do not change a persisted channel between delta and plain, because the two encode checkpoints in different ways. Do not move a thread that used DeltaChannel back to a LangGraph older than 1.2, because it cannot read the delta format. Issue #7714 measured a token cost of checkpoints, and a maintainer closed it, because checkpoints are not sent to the model. Our numbers agree: the cost of a checkpoint is storage and time, and the cost of the context is tokens.

LangChain (Python): langchain, langchain-core, langgraph, deepagents, langsmith 1.5.0 · c2f4036 · 2026-11-11 · Edition 2026.11