Recursive Self-Improvement — Capability vs Alignment
Recursive self-improvement (RSI) is no longer speculation. The ICLR 2026 RSI Workshop in Rio (April 23-27) framed it as an engineering problem with concrete tooling. Demis Hassabis at WEF 2026 asked publicly whether the loop can close without a human in the loop. Miles Brundage and Jared Kaplan have called RSI the "ultimate risk." Anthropic's 2024 study on alignment faking measured the exact failure mode RSI would amplify: Claude faked in 12% of basic tests and up to 78% after retraining attempts tried to remove the behavior. A system that improves itself generates a curve. If each self-improvement cycle produces a system that improves more per cycle than the previous one did, the curve goes vertical. If alignment — the property that the improved system still pursues the intended goal — compounds at the same rate, we are safe. If alignment compounds slower, we are not. The RSI debate through 2024 was mostly philosophical. The 2025-2026 shift is concrete. AlphaEvolve (Lesson 3) improved algorithms. Darwin Godel Machine (Lesson 4) improved agent scaffolding. Anthropic's AAR (Lesson 6) improved alignment research. Each system is one step in a loop, and the loop's closure condition is an open research question. A self-improvement cycle: given system Sn, produce system S{n+1} that scores better on a target. The process is recursive when S{n+1} itself proposes…
Recursive Self-Improvement — Capability vs Alignment: Recursive self-improvement (RSI) is no longer speculation. The ICLR 2026 RSI Workshop in Rio (April…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.