Read the loop, what the caller gets at the turn cap

L2WorkingCode reading8 minCodingDebugging

RolesAgent Systems EngineeringAI Evaluation and Reliability

The question

Read this run method from the toy agent loop. A caller shows the returned string to the user as the answer. What is wrong with the contract, and how would you change it?

python

def run(self, user_message: str) -> str:
    self.history.append(Turn(kind="user", content=user_message))
    for step in range(self.max_turns):
        reply = self.llm.respond(self.history)
        if reply["kind"] == "finish":
            self.history.append(Turn(kind="final", content=reply["content"]))
            return reply["content"]
        thought = reply.get("thought", "")
        self.history.append(Turn(kind="thought", content=thought))
        call = ToolCall(name=reply["action"], args=reply.get("args", {}))
        observation = self.tools.dispatch(call)
        self.history.append(Turn(kind="action", content=call.name, tool_call=call, observation=observation))
    self.history.append(Turn(kind="final", content="budget exhausted"))
    return "budget exhausted"

Answer it first, aloud or on paper. Then open the answer.

The answer

Show short answer

The method returns a plain string in both cases, so the caller cannot tell a real answer from a run that hit the cap. The text budget exhausted goes to the user as if the model said it. The fix is to return a typed result with a reason, or to yield control the way the harness contract does, so that the caller decides whether to extend the budget or stop.

Show full answer

What the code does

The loop runs at most max_turns iterations. Each iteration asks the model for one reply. A finish reply ends the run and returns its content. Any other reply is an action: the loop records the thought, dispatches the tool, and appends the observation. When the loop runs out of iterations, it appends a final turn with the text budget exhausted and returns that text.

The three problems

One return type for two outcomes. A finished run and a capped run both return str. The caller has no field to test. Any code that shows the result to the user shows the cap message as an answer.
The cap is a dead end. The partial state is in self.history, but nothing tells the caller that the run can continue. The harness contract treats the budget as a yield: it emits budget.warn, fires on_budget_exceeded, and returns a pull request with a reason. The caller inspects the state and calls resume.
A reply with no action and no finish is a crash. The code reads reply["action"] for every non-finish reply. A model that answers in prose without a tool call raises a KeyError. The lesson lists a turn with no tool calls as a normal stop condition.

The change

Return a small result object with the final text, a reason such as finished, budget or no_tool_call, and the turn count. Keep the history on the object so that the caller can resume or hand off. Treat a reply without an action as a stop, not as an error. Keep the budget as a cap on this call, and let the caller decide what to do after it.

Follow-ups

L2WorkingWhere is the observation formatter in this code?
Show expected answer
inside tools.dispatch, which returns an error string for an unknown tool, bad arguments, or an exception, so the model always gets a string back.
L2WorkingA tool raises an exception. What does the model see on the next turn?
Show expected answer
an observation that starts with error: and carries the exception type and text, which lets it correct the call.
L3SeniorHow does the harness know which state to return to after a budget pause?
Show expected answer
it records the previous state before it pauses, resets the counters on resume, and re-enters executing or reflecting from that record.
L3SeniorWhat property of the harness makes a saved trace replayable without a model?
Show expected answer
the state machine is deterministic, so the same event log re-enters the same state.

Wrong answers

Show the wrong answers that interviewers hear

“Return an empty string at the cap, and the caller will handle it.”
An empty string is still a string, and the caller still has no reason code to test.
“Raise an exception at the cap.”
The partial history is useful, and an exception forces the caller to restart from zero instead of resuming.
“Set the maximum turns high enough that the cap is never reached.”
The cap exists for the run that does not converge, and a high cap only makes that run longer.

Taught in

The agent loop lesson ships this loop, and the harness loop contract replaces the string with a pull point that the caller resumes.