Coding Agents: Landscape and Capstones

Pick a scaffold, set the budgets and gates, run the capstone agents and the three coding-agent projects.

JSON tool calls vs CodeAct#

Factor	JSON tool calls	CodeAct
One action is	One structured JSON payload	A whole Python program
Audit	Easy, one action per turn	Harder, wide blast radius
Composition	Limited	Loops, chains tools, catches errors
Safety	Validator on every call	Needs a hardened sandbox
Toy blast radius	1 file per action	Every failing file at once
Dominant in	Managed services	Open platforms

Factor	Terminal agent (01)	Issue-to-PR (16)	Migration (09)
Trigger	`agent run <repo> "<task>"`	Issue label `@agent fix this`	`migrate <repo> --target java17`
Sandbox	E2B or Daytona, git worktree	Daytona or E2B per task	Daytona per branch
Turn cap	50 turns	30 agent turns	20 agent turns
Cost cap	$5 per task	$20 per PR, 5 PRs per repo per day	$8 per repo
Other cap	200k context	30 min wall-clock	30 min per repo
Gate before PR	Tests pass	Full CI in sandbox, coverage delta	Tests pass, coverage not down 2%
On failure	`Stop` hook writes a trace bundle	CI red: halt, no PR	Failure class plus repro

Use JSON tool calls when: each action needs an explicit validator and an easy audit.
Use CodeAct when: one action must chain tools over many files. Harden the sandbox first.
Use a terminal agent when: the work is CLI in, pull request out, with a live plan.
Use issue-to-PR when: the UX is a GitHub label. No interactive loop, no terminal.
Use a migration agent when: the test suite is the ground truth across many repos.
Use deterministic recipes first: they catch 70 to 80% of mechanical migration fixes.

SWE-bench Verified: 4% to 80.9% in under three years. 500 human-curated tasks.
Scaffold delta: Claude Sonnet 4.5 got 43.2% on SWE-agent v1 and 59.8% on Cline.
Easy tail: 161 of 500 Verified tasks need only a 1 or 2 line change.
SWE-bench Pro: 10+ line changes. Frontier systems score 23 to 59%.
Verifier loop: a 10+ point delta on SWE-bench.
Live-SWE-agent: 79.2% on SWE-bench Verified with Opus 4.5.

Which coding agent is best?: Wrong question. Measure reliability on your own task distribution.
Why does the toy CodeAct take fewer turns?: One snippet rewrites every failing file at once.
Why no assert for a safety gate?: python -O strips asserts.

Task	Code
Compare the toy scaffolds (L09)	`code/main.py`
Plan, act, observe loop (01)	`python main.py`
TS harness demo plus offline eval	`npm start`
Simulate 50 migrations (09)	`python main.py`
Dispatch 20 issues over 3 repos (16)	`python main.py`
Fix the fizz fixture (29)	`python3 code/main.py`
Test the end-to-end demo	`python3 -m pytest code/tests/ -v`

Factor	tiny-coding-agent	rust-agent-shell	pr-review-reporter
Builds	Bounded repair loop with exact patches	Rust stdin/stdout action loop	Anchored patch review report
Language	Python 3.10+	Rust plus a Python adapter	TypeScript on Node 22.18+, Python
Input	Workspace plus `proposals.json`	Workspace plus `requests.jsonl`	Unified diff, stdin or `--repo`
Output	`repair-trace.json`	JSONL events per request id	`review.json`, `review.html`, SARIF
Budget	`--max-steps` default 6	`--limit` default 50	Diff at most one megabyte
Safety	Path containment, one test command	No OS shell, 16 KiB reads	HTML escaping, exact quote gate
End states	`completed`, `failed`, `budget_exhausted`	`ok`, `error`, `rejected`	Accepted or rejected findings

Command	What it does
`project_test.py <project> --init learning-artifacts/<project>`	Creates a learner workspace
`project_test.py <project> --stage 1 --path <workspace>`	Grades one stage
`project_test.py <project> --all --solution --strict`	Checks the reference
`cli.py <workspace> <proposals> --copy-to <dir> --out <trace>`	Repairs a disposable copy
`client.py <workspace> <requests> --limit 8 --out <file>`	Rust shell through the adapter
`node cli.ts --diff samples/change.diff --sarif review.sarif`	Writes JSON, HTML and SARIF
`--repo /path --base REF --head REF`	Read-only Git diff mode
`--candidates recorded-findings.json`	Validates another reviewer's findings

chain = GateChain(gates=(

WhitelistGate(allowed=frozenset({"read_file", "write_file", "run_tests"})),allow-set

RegexGate.from_strings(items=(r"\brm\s+-rf\b", r"\bsudo\b", r"^/etc/")),refused patterns

BudgetGate(max_tokens=budget),observation budget

))

outcome = self.chain.evaluate(call, ctx)stops at the first deny

sandbox = Sandbox(project_root=repo, timeout_seconds=10.0)denylist plus path jail

Symptom	Cause	Fix
Loop crashes on 8MB of ripgrep matches	Untruncated tool output	Cap each call at 4k tokens
`rm -rf` reaches the sandbox	No `PreToolUse` guard	`destructive_guard` sets `blocked`
High Verified score, weak real results	Easy tail, 161 of 500 tasks	Score a Pro-like subset of your backlog
`budget_exhausted`	30 min, $8 or 20 turns used	File the repo with the current diff
`coverage_regression`	Coverage dropped more than 2%	File under the class, no PR
`policy_denied`	`can("force_push")` returns False	Branch protection, app not in bypass list
`patch must match exactly once`	Old text missing or not unique	Use one unique exact old string
`path escapes root`	Path leaves the canonical root	Pass a relative path inside the root