Critic Loop
A critic that returns "looks good" the first time is broken. A critic that always returns "needs work" is broken. The interesting critic is the one that converges, and you have to engineer convergence. Score a paper draft across five fixed dimensions: clarity, novelty, evidence, methodology, related-work. Apply each round's critique as a structured revision diff rather than a freeform rewrite. Detect convergence by comparing scores across rounds; stop on plateau, target met, or budget exhausted. Cap rounds with a max-iteration budget so a non-converging critic does not run forever. Emit a per-round trace so the dashboard or the next stage can render the score trajectory. A freeform critic is a model that returns a paragraph of suggestions. The next round's revision treats the paragraph as ambient context. Whether the rewrite addresses the criticism is unverifiable because the criticism never had structure. Five dimensions give the harness a contract. The score is a vector. The harness watches each dimension across rounds. A revision that raises clarity but tanks evidence is a regression on evidence, and the convergence check sees it. A model-only critic cannot offer that guarantee. Every suggestion carries the dimension it improves, the section it targets, and an edit instruction the reviser can apply. The reviser is also a callable. The lesson ships a deterministic reviser that interprets…
Critic Loop: A critic that returns "looks good" the first time is broken. A critic that always returns "needs work" is broken. The interesting critic is the…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.