Cheat sheets

Autonomous Agents and Permissions

Run an agent unattended with limits. Permission modes, budgets, kill switches, durable execution, approvals, rollback.

  • Concepts and terms
  • Tasks and commands
  • Annotated code
  • Diagrams
  • Defaults and numbers
  • Troubleshooting
  • Questions and tips

Source

Operator tasks#

TaskCode
Review a plan firstpermission mode plan
Auto-approve file edits onlypermission mode acceptEdits
Cap spend and loop turnsmax_budget_usd, max_turns
Limit the tool setallowed_tools / disallowed_tools
Kill the agent containerkubectl delete pod
Trip on repeated callsCircuitBreaker(threshold=5)
Replay after a crashworkflow(EventLog(<path>), <query>)
Make an approval retry-safek = p.key()

From Lesson 15.10, Lesson 15.13, Lesson 15.14, 15.14 code/main.py, 15.12 code/main.py, 15.15 code/main.py

Control layers#

actionspendbreachcallsreadsapprovepersistAgent loopturns, tool callsPermission modeplan tobypassPermissionsCost governorstack of capsKill switchagent cannot writeitCircuit breakerclosed, open,half-openCanary tokenaccess is the alarmProposal storepropose, commit,verifyEvent logkeyed by thread_idactionspendbreachcallsreadsapprovepersistAgent loopturns, tool callsPermission modeplan tobypassPermissionsCost governorstack of capsKill switchagent cannot write itCircuit breakerclosed, open,half-openCanary tokenaccess is the alarmProposal storepropose, commit,verifyEvent logkeyed by thread_id

From Lesson 15.10, Lesson 15.13, Lesson 15.14, Lesson 15.15, Lesson 15.12

Propose, commit, verify#

1ProposeStore intent, lineage, blast radius, rollback.Status waiting.2SurfaceA person reviews the metadata, not the agent.3Approvechecklist_approve needs all three checks. Statusapproved.4PreconditionState still matches the approval. Elseaborted-precondition.5CommitPersist the intent first, then run the sideeffect.6VerifyRe-read the target. A failure runs the rollback.7RetryA terminal status returns the stored result. Nosecond run.

From Lesson 15.15, 15.15 code/main.py, Lesson 15.16, 15.16 code/main.py

Durable activity#

def activity(name: str):
def deco(fn):
@functools.wraps(fn)
def wrapper(log: EventLog, *args):
hit = log.lookup(name, args)done event in log?
if hit:
return hit["result"]replay, no re-run
log.append({"name": name, "args": list(args), "status": "started"})
result = fn(*args)the real side effect
log.append({"name": name, "args": list(args),
"status": "done", "result": result})log output
return result
return wrapper
return deco

From 15.12 code/main.py

Governor and breaker#

gov = Governor(max_turns=200, max_budget_usd=50.0)session caps
if gov.enable_iter_cap and turn >= gov.max_turns:
run.stopped_by = "max_turns"iteration cap
if gov.enable_session_cap and run.dollars >= gov.max_budget_usd:
run.stopped_by = "max_budget_usd"dollar cap
if gov.enable_velocity and velocity_exceeded(run, gov, now_min):
run.stopped_by = "velocity_limit"rolling spend rate
if kill_switch["enabled"]:
breakexternal off-button
allowed = breaker.record(f"{a.kind}:{a.payload}")
if not allowed:
break5 identical calls opened it

From 15.13 code/main.py, 15.14 code/main.py

Permission modes#

plan
User approves the plan. Each action is reviewed first.
default
"Manual". Prompts for shell, destructive and network actions.
acceptEdits
File writes auto-approve. Shell and network still prompt.
auto
A classifier model reviews each action. Blocks escalation.
dontAsk
Never prompts. Unapproved actions are denied.
bypassPermissions
Approves everything. Throwaway containers only.

From Lesson 15.10

Which cap catches what#

Runaway loop
Caught by the velocity limit.
Slow leak
About 2x expected work. Caught by the daily cap.
Bad release
New version uses 5x tokens. Caught by weekly or monthly cap.
Legitimate surge
Caught by the hour or day cap with a clear log.
Slow adversarial drift
Beats statistical detectors. Hard limits catch it.
Composed safe actions
No detector catches them.

From Lesson 15.13, Lesson 15.14

Defaults in the simulators#

SettingDefaultNote
max_tokens_per_request10_000Per-call completion cap
max_turns200Loop iterations per session
max_budget_usd50.0Session dollars, aborts on breach
velocity_usd_per_min5.0Rolling spend rate cut-off
velocity_window_min10.0Window for the rate
monthly_cap_usd500.0Catches slow leaks late
threshold5Identical calls that open the breaker
Half-open probesdefault 11 to 3 after the cool-down

From 15.13 code/main.py, 15.14 code/main.py, Lesson 15.14

Troubleshooting#

SymptomCauseFix
Crash re-runs all tool callsNaive while True loopActivities plus an event log
Replay divergesWall clock or random in workflowRegister it as a side effect
Approved action runs twiceNo idempotency keyOne key per proposal
Overdraft after resumeIdempotency check onlyAdd a precondition check
"Committed" but no side effectStatus saved, action lostVerify by re-read, then roll back
Approvals with no real reviewPlain Approve buttonChallenge-and-response checklist
Bill tripled after a new toolNo per-tool capPer-tool cap, daily-growth alert
Agent turns off its canaryDetector in the agent's reachKeep it where agent cannot write

From Lesson 15.12, Lesson 15.13, Lesson 15.14, Lesson 15.15, Lesson 15.16

Glossary#

HITL
Human-in-the-loop. A person approves before the action.
METR horizon
Task length a model completes at 50% reliability.
Denial of Wallet
Agent loop that spends with no cap to stop it.
thread_id
Key that scopes durable state for one session.
Lease
Short claim that a worker runs a run. It expires on crash.
Blast radius
Scope of effect if the action goes wrong.
SAGA
Compensating transaction that neutralizes the original action.

From Lesson 15.01, Lesson 15.12, Lesson 15.13, Lesson 15.15, Lesson 15.16

Q&A#

99% per step, 70 steps?
The agent still fails half the time.
Why a stack of caps?
One monthly cap fires after the wallet is gone.
Why persist every transition?
Replay lands anywhere and lease recovery is precise.
When skip durable execution?
Short runs with no human input, or read-only work.
What does auto miss?
Subtle injection and harmful sequences of safe actions.

From Lesson 15.01, Lesson 15.13, Lesson 15.12, Lesson 15.16, Lesson 15.10

Tips#

  • Never run auto without max_budget_usd.
  • Check the kill switch on every consequential action.
  • Re-enable a kill switch only by an explicit human action.
  • Give every new tool its own cap and alert.
  • Always HITL for irreversible writes. Never for reads.
  • Rehearse each rollback end to end at least once.

From Lesson 15.13, Lesson 15.14, Lesson 15.15, Lesson 15.16, Lesson 15.12