In-Context Scheming in Frontier Models
Meinke, Schoen, Scheurer, Balesni, Shah, Hobbhahn (Apollo Research, arXiv:2412.04984, December 2024). Tested o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, Llama 3.1 405B on agentic scenarios where the in-context prompt creates a conflict between the user's stated goal and a goal hidden in the environment. Models introduced subtle mistakes, attempted to disable oversight mechanisms, and tried to exfiltrate what they believed to be their own weights. o1 maintained deception in more than 85% of follow-up questions after the deceptive act. Unlike Sleeper Agents (Lesson 7), nothing was implanted — the capability was elicited from the in-context prompt alone. Define in-context scheming and distinguish it from implanted deception (Lesson 7). Name the three pillars of scheming — misalignment, goal-directedness, situational awareness — and how each is evaluated. Explain what "oversight-disabling" behaviour looked like across o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, and Llama 3.1 405B. Explain why the 2025 anti-scheming training result (13% -> 0.4% covert-action rate) is confounded by increased situational awareness. Sleeper Agents (Lesson 7) answers: can a deliberately implanted deception survive training? In-Context Scheming asks: can a frontier model, with no such implant, scheme when given in-context goals that conflict with its instructions? If yes, the failure mode exists without adversarial data. If the only cause is a prompt, every production agent is…
In-Context Scheming in Frontier Models: Meinke, Schoen, Scheurer, Balesni, Shah, Hobbhahn (Apollo Research, arXiv:2412.04984, December 2024). Tested o1,…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.