Turn Every Agent Correction into a System Improvement
A correction that lives only in chat fixes one run. A correction promoted into a test, boundary, example, or tool improves every later run. Convert agent corrections into durable controls. Place each control at the earliest layer that can prevent recurrence. Deduplicate repeated lessons with stable fingerprints. Retire controls that no longer protect a real risk. When you tell an agent “do not edit that file,” you have learned that the scope boundary was not executable. When you say “this output shape is wrong,” you have learned that an example or test was missing. When setup fails again, you have learned that environment knowledge belongs in automation. Treat the correction as an observation about the work system, not as a prompt-writing failure. Use this order: Earlier controls are cheaper. A type that prevents an invalid state is stronger than a review comment that catches it later. A focused test is stronger than a paragraph asking the agent to remember. Capture: symptom;. root cause;. consequence;. recurrence count;. chosen control;. verification for the control;. owner;. date to review or retire it. Do not promote every one-off preference. Promote a correction when recurrence or consequence justifies permanent complexity. “The agent edited README” is a symptom. Possible causes include: the task allowed the repository root;. docs were implicitly considered safe;. the plan bundled implementation…
Turn Every Agent Correction into a System Improvement: A correction that lives only in chat fixes one run. A correction promoted into a test, boundary,…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.