Capstone 82 — Jailbreak Taxonomy
A safety harness without a taxonomy is a coin flip. Name the attack before you defend it. A model deployed without an attack model is a model defended against nothing in particular. Operators read a Twitter thread, recognize the trick, write a regex, ship it, and move on. The next prompt is a paraphrase. The regex misses. A week later someone shows the same trick wrapped in base64 and the operator writes a second regex. By month three, the system has 40 patched rules, no shared vocabulary, no way to talk about what an attack actually is, and a backlog growing faster than the patches. Before any detector, classifier, or rule engine in this track does anything useful, the team needs a shared way to label attacks. Not because labels stop attacks, but because labels turn an attack stream into a histogram. A histogram becomes a coverage chart. A coverage chart drives the next sprint. The harness in lessons 83-87 spends its time deciding whether a prompt is, for example, a role-play attack against a refusal policy versus a context-smuggling attack against a tool. That decision is impossible without a taxonomy. This capstone defines a six-category taxonomy that is wide enough to cover most attacks seen in the wild, narrow enough that two reviewers usually agree on the category, and…
Capstone 82 — Jailbreak Taxonomy: A safety harness without a taxonomy is a coin flip. Name the attack before you defend it. A model deployed without an attack…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.