Security & governance · Glossary term
What is Threat Model?
A documented account of protected assets, trust boundaries, potential adversaries, assumed capabilities, attack paths, impacts, and planned controls.
Why does Threat Model matter?
Security controls cannot be judged without stating what they defend, against whom, and under which assumptions.
Threat Model in practice
Map data and authority across model, retrieval, tools, users, and external services, then turn credible abuse paths into red-team cases and mitigations.
What is the common confusion about Threat Model?
A threat model prioritizes plausible risks; it is not a checklist that proves the system secure or predicts every future attack.
Learn Threat Model in the course
Lessons that name Threat Model in a title or section
- AI Control — Safety Despite Subversion
Greenblatt, Shlegeris, Sachan, Roger (Redwood Research, arXiv:2312.06942, ICML 2024). Control reframes the safety question: given an untrusted strong model U that may be adversarially optimizing…
Covered in Phase 18: Ethics, Safety & Alignment.
Related terms
- Least PrivilegeGiving a model, agent, tool, or user only the permissions required for the current task, for only as long as those permissions are needed.
- Prompt InjectionAn attack or failure mode in which untrusted content influences a model to disregard intended instructions, expose data, misuse tools, or…
- SandboxAn isolated execution environment that restricts an agent's access to files, processes, network destinations, credentials, and host…
- Red TeamingA structured adversarial testing process in which authorized testers seek failures using documented objectives, threat assumptions, cases,…
- AI Risk AssessmentA documented analysis of how an AI system can affect people, organizations, and environments, including context, hazards, likelihood,…
- Trust BoundaryAn interface where data, instructions, identity, or authority crosses between components or principals that operate under different trust…
Sources
More terms in Security & governance
- AI Risk Assessment
- Audit Log
- Content Provenance
- Data Classification
- Data Exfiltration
- Data Lineage
- Data Minimization
- Datasheet for Datasets
- Defense in Depth
- Indirect Prompt Injection
- Jailbreak
- Membership Inference
- Provenance Attestation
- Purpose Limitation
- Red Teaming
- Separation of Duties
- Software Bill of Materials (SBOM)
- Trust Boundary
- Zero Trust
This entry comes from glossary/terms.md on GitHub. Browse all 250 glossary terms.