3.5

The system prompt

create_agent puts the system prompt first in every model request and never stores it in the state, so a new prompt applies to the next call of every thread.

You save a conversation with a checkpointer, change the system prompt in your code, and resume the thread. The model follows the new prompt. You look for the old prompt in the saved state, and it is not there.

When you finish this section, you can say where the system prompt lives during a run and set it from runtime context. You can also find the prompt hub at the pin.

A string or a SystemMessage

System prompt: the instructions that go first in every model request of an agent.

The agents page says that the parameter takes a string or a SystemMessage, and that a prompt built at run time needs middleware. create_agent turns a string into a SystemMessage once, when it builds the graph.

Not in the state

The model node adds the system message to the request on each call. It does not write it to messages. Section 1.3 showed both halves of this. The model got system: You add numbers. first on both calls, and the final state held four messages with no SystemMessage:

jsonwhat the model received on the second callFull log
    "call": 2,
    "messages": [
      "system: You add numbers.",
      "human: What is 2 + 3?",
      "ai: calls add",
      "tool: 5"
    ],
Cut to the message list of call 2.

Figure 3.7 draws the request that the model node builds. Three things follow from it.

Fig. 3.7how the model node builds each requestflow
How the model node builds each request Two rows, one for each model call of the run in 06-agent-run.txt. On the left of each row is state.messages before the call, in teal: 1 message before call 1 and 3 before call 2. In the middle is the request the chat model received: the violet system prompt "You add numbers." first, dashed because it is not in the state, then the same messages from the state. A plum arrow carries the request to the chat model. A teal arrow carries the reply back, and the model node appends it to state.messages: a tool call to add after call 1, and "2 + 3 = 5." after call 2. Between the rows the tools node appends the ToolMessage. The state ends with 4 messages and no system message. STATE.MESSAGES THE REQUEST CHAT MODEL 1 human: What is 2 + 3? system: You add numbers. human: What is 2 + 3? call 1 tools: add 2 messages appended: ai: tool_calls add ai: tool_calls add 2 human: What is 2 + 3? ai: tool_calls add tool: 5 system: You add numbers. human: What is 2 + 3? ai: tool_calls add tool: 5 call 2 tools: add 4 messages appended: ai: 2 + 3 = 5. ai: 2 + 3 = 5. then the tools node appends "tool: 5", and call 2 sees it. After the run state.messages holds 4 messages. The system prompt is not one of them. the system_prompt of create_agent: sent first on every call, never saved
Each model request is the system prompt from the agent plus the messages from the state, and only the model reply goes back into the state. Read each row from left to right, one row for each model call. The dashed violet line is the system prompt, sent first and never saved. The teal lines are the saved state. The plum arrow is the model call, and the teal arrow appends only its reply.
  • A checkpoint holds no prompt. A thread resumed by new code uses the new prompt from its next model call.
  • Two agents can share one thread with two prompts. Each one sends its own prompt with the shared messages.
  • The prompt costs input tokens on every call, because it is sent every time. Long prompts multiply with the number of steps.

A SystemMessage that you put in the input messages is different. It is a message, so add_messages stores it in the state, and every later call sends it again.

A prompt from runtime context

The dynamic_prompt decorator turns a function of the model request into middleware that sets the system prompt for each call. The kit reads the user name from the runtime context:

pythona dynamic promptFull log
    @dynamic_prompt
    def by_user(request):
        return f"You are helping {request.runtime.context['user']}."
…
    out = create_agent(m, middleware=[by_user]).invoke(ask("hi, who are you helping?"), context={"user": "Ada"})
Cut from the scenario. The model is Qwen3 4B on Ollama.

The model got the prompt with the name filled in:

jsonthe request with a dynamic promptFull log
    "messages": [
      "system: You are helping Ada.",
      "human: hi, who are you helping?"
    ]
The last section of the file, cut to the messages.

The model did not use the name. It answered Hi! I'm here to help you with any questions or tasks you might have. and more. A prompt sets what the model reads, not what it says.

For more control, a wrap_model_call middleware can read and replace the prompt of each request. The custom middleware page says to read and change the prompt through the system_message field of ModelRequest. Part 4 explains the middleware hooks and their order.

The prompt hub

The LangChain docs at commit 04d4d4a have no page on prompt templates for Python agents. The prompt hub moved to LangSmith. Its page says that the hub used to be the langchainhub package, which is now deprecated. The same page still shows from langchain_classic import hub, which is the legacy path that section 3.1 lists. Use the langsmith client to push and pull prompts, and pass the text to create_agent as a string.

LangChain (Python): langchain, langchain-core, langgraph, deepagents, langsmith 1.5.0 · c2f4036 · 2026-11-11 · Edition 2026.11