Context Engineering and Agent Memory

Budget the window, order the context, page memory in and out, fuse memory stores and persist agent state.

The context window budget#

Lost in the middle: 85-90% accuracy with the document first or last. 60-70% at position 10 of 20.
Low zone: positions 40-70% of the context lose 10-20% accuracy.
Curated beats dumped: a 10K token context often beats a dumped 100K token context.
Tool cost: 50-200 tokens per tool. 50 tools at 150 tokens is 7,500 tokens.
Tool pruning: cuts tool tokens by 60-80%.
History: 50 turns at 200 tokens is 10,000 tokens.
Summary: 100 tokens replace 10 turns that took 2,000 tokens.
Token estimate: word count times 1.3 in count_tokens.

budget = ContextBudget(max_tokens=128000, generation_reserve=4000)

budget.allocate("system_prompt", self.system_prompt, max_tokens=1000)

tools, tool_tokens = select_tools(query, token_budget=2000)

budget.allocate("tools", tool_text, max_tokens=2000)

budget.allocate("retrieved_context", doc_text, max_tokens=3000)

budget.allocate("conversation_history", history_text, max_tokens=5000)

budget.allocate("user_query", query, max_tokens=500)query goes last

print(budget.report())tokens and percent per component

Command	What it does
`core_memory_append(section, text)`	Writes to a pinned section of the prompt
`core_memory_replace(section, old, new)`	Edits a pinned section
`archival_memory_insert(text)`	Writes to the searchable external store
`archival_memory_search(query, top_k)`	Pages facts back into the next turn
`conversation_search(query)`	Scans past turns, evicted turns too
`block_append(label, text)`	Adds text to a memory block
`block_replace(label, old, new)`	Edits a memory block
`block_summarize(label)`	Condenses a block near its limit

Setting	Default	Note
`max_tokens`	128000	Minus `generation_reserve` 4000
`max_history_tokens`	5000	Summarize old turns above it
`token_budget`	2000	Cap for tool definitions
Relevance `threshold`	0.05	The docs version uses 0.1
`max_messages`	4	MemGPT main context. Older messages go to `evicted`
Block `limit`	300	Characters. `near_limit` fires at 0.8
`w_relevance`	0.6	`w_importance` 0.2, `w_recency` 0.2
`recency_halflife_s`	86400.0	Recency is 0.5 per half-life

Bigger window, no memory?: 128k-window baselines still miss long-horizon facts that a 4k-window agent with external memory catches.
Why sleep-time compute?: Primary turns do not wait. The sleep agent can use a stronger, slower model.
Why three stores?: A single store is wrong for two of three query classes.
How does fusion rank?: A weighted sum of relevance, importance and recency. Weights change per product.
What goes in repo memory?: Task id, touched files, assumptions, blockers, next action. Not raw chat transcripts.

Factor	MemGPT two tiers	Letta blocks	Mem0 hybrid	Repo memory
Stores	Main context plus archival	Core, recall, archival	Vector, KV, graph	JSON files under a schema
Write	Memory tool call	Block tools and sleep-time rewrites	`add(text, user_id, metadata)` to all three	Validate, then atomic write
Read	`archival_memory_search`	Core blocks stay in the prompt	`search(query, user_id)` fuses scores	Load and validate
Stale facts	Consolidation, invalidation	Sleep-time agent invalidates	Old edge marked invalid, not deleted	`schema_version` refuses unknown files
Main risk	Memory rot, poisoning	Block bloat, silent drift	Embedding drift, graph explosion	Partial writes
Use it when	Data is larger than the window	Some facts must stay in the prompt	Queries mix similarity, lookup, relations	State must outlive the chat

Task	Code
Create a block	`create(label, description, limit=300)`
Test a block for summary	`block.near_limit()`
Consolidate off the critical path	`sleep.run(contradictions=[...])`
Soft delete a stale fact	`self.archival.invalidate(record.rid)`
Write to all three stores	`mem.add(text, user_id=..., kv_triples=...)`
Fused, scoped search	`mem.search(query, user_id=..., top_k=3)`
Persist state safely	`atomic_write(path, content)`
Refuse a bad write	`mgr.commit(state)` raises `SchemaError`

Command	What it does
`python3 scripts/project_test.py memory-server --init <dir>`	Creates your workspace. A fresh starter fails
`python3 scripts/project_test.py memory-server --all --path <dir> --strict`	Grades all four stages
`node cli.ts --data-dir memory-data --put samples/memory.json`	Appends revision 1 of `cache-policy`
`node cli.ts --data-dir memory-data --query "cache policy"`	Searches the `docs` namespace
`node cli.ts --data-dir memory-data --put samples/memory-updated.json --revision 1`	Appends revision 2. Revision 0 again is a conflict
`node cli.ts --data-dir memory-data --history cache-policy --html history.html`	Prints all revisions and writes a comparison table
`node cli.ts --data-dir memory-data --serve --port 8788`	Serves REST and MCP. Set MEMORY_TOKEN first

Symptom	Cause	Fix
Model ignores a fact in the middle	Lost in the middle	Key facts first, query last. Repeat the key point at the end
Tools fill the window	Every tool sent on every call	Classify intent and prune tools
Retrieval drowns in stale facts	Memory rot	Consolidate and invalidate in a sleep pass
Agent obeys an old injected note	Memory poisoning	Treat retrieved memory as untrusted text
Agent recalls but cannot cite	Citation loss	Store session ID and turn ID with each write
Block hits its cap	Block bloat	Summarize before the write that crosses the cap
One user sees another user's facts	Scopes mixed	Pick one scope for each write
Session resumes on corrupt state	Non-atomic write	Temp file, `fsync`, `os.replace`