Prompt Injection Incidents and AI CVEs

Find, trace and block indirect prompt injection. EchoLeak chain, PVE validator, rule detector, tool call firewall.

Concepts and terms
Tasks and commands
Annotated code
Diagrams
Comparisons
Defaults and numbers
Troubleshooting
Questions and tips

Symptoms, causes and fixes#

Symptom	Cause	Fix
Agent forwards an email to all contacts	Naive agent obeys imperatives in the email body	Tag the email untrusted. Ignore its control flow
Filter blocks `[SYSTEM:` but a polite rewrite passes	Adaptive attacker drops the marker	Use IFC. Keyword filters fail adaptive attacks
Defense reports near-zero ASR, then breaks	Tested with static attacks only	Evaluate with adaptive attacks before you publish
Data leaves through an approved domain	Output crosses the trust boundary in a URL	Control retrieval, scope and output one by one
Model already touched the world	All guardrails run at the end	Validate each tool call before it runs
Yesterday's memory note hijacks today's run	Overtrust of retrieved memory	Refuse memory writes that look like directives
Regex misses "disregard the prior"	One regex, never measured	Report precision and recall per category
`load_taxonomy` raises FileNotFoundError	Lesson 82 artifact is missing	Run lesson 82 main.py first

From Lesson 18.15, 18.15 code/main.py, Lesson 18.25, Lesson 14.27, Lesson 19.83, 19.83 code/main.py

Production incidents#

Factor	EchoLeak	CamoLeak	Copilot RCE
Target	Microsoft 365 Copilot	GitHub Copilot Chat	GitHub Copilot
CVE	CVE-2025-32711	Number undisclosed	CVE-2025-53773
CVSS	9.3	9.6	Details minimal
Entry	Crafted email in RAG context	Repository content	Code-suggestion surface
Effect	Mermaid image on a CSP-approved URL	Image loads through the Camo proxy	Remote code execution
Fix	Server-side update, June 2025	Image rendering disabled	Details minimal

From Lesson 18.25, Lesson 18.15

EchoLeak attack chain#

From Lesson 18.25

Three boundaries and the gate#

From Lesson 18.25, 18.25 code/main.py, Lesson 18.15

Attack surface#

RAG vector: attacker publishes a document. Retrieval puts it in the prompt.
Inbox vector: attacker sends an email. The agent reads the body.
Tool output vector: attacker controls a tool result, for example a web search.
Data theft: agent sends conversation history to an attacker URL.
Worming: injected content tells the agent to embed the exploit in its next output.
Memory poisoning: agent stores the instructions and re-poisons itself next session.
Ecosystem contamination: injected facts spread to other agents through shared memory.
Arbitrary tool use: any tool in the registry becomes attacker-reachable.

From Lesson 18.15, Lesson 14.27

Run the labs#

Command	What it does
`python3 code/main.py` (18.15)	Attack naive, filter and IFC agents
`python3 code/main.py` (18.25)	EchoLeak trace, then scope separation
`python3 code/main.py` (14.27)	PVE validator verdicts per call
`python3 main.py` (19.83)	Detector metrics and JSON report
`project_test.py <id> --init <dir>`	Create the workspace. Id is tool-call-firewall
`project_test.py <id> --stage 1 --path <dir>`	Grade one stage
`project_test.py <id> --all --solution`	Check the reference. Add `--strict`

From Lesson 18.15, Lesson 18.25, Lesson 14.27, Lesson 19.83, tool-call-firewall/README.md

PVE validator before the executor#

validator = Validator(

allowed_tools=("search", "send_message", "read_memory"),

sensitive_tools=("send_message",),

)

call = ToolCall("search", {"query": "<q>"}, intent="research")

contents = [Content("<text>", "user_message")]user text is not scanned

ok, reason = validator.assess(call, contents)allowlist, args, retrieved

if ok:

executor.run(call)runs only after allow

looks_like_directive("<text>")marker or None

memory_write_guard(MemoryWrite("<text>"))refuse directive writes

From 14.27 code/main.py

Layered injection detector#

{"name": "ignore-previous", "category": "instruction-override",

"score": 0.85, "substring": "ignore previous"}a rule is data

detector = Detector()compiles regex rules once

v = detector.analyze("<prompt>")raw and normalized text

v.categorytop category, its max score

normalize(text)zero-width, base64, hex, leet, rot13

report = evaluate(detector, fixtures, benign)tp, fp, fn, tn per category

write_report(report)outputs/detector_report.json

From 19.83 code/main.py, 19.83 code/rules.py

Tool call firewall#

Task	Code
Parse the envelope	`call(raw, 256)`
Decide policy	`decide(&c)` gives Allow, ApprovalRequired or Deny
Reader writes a note	`reader + write notes.md -> Deny`
Editor writes a note	`editor + write notes.md -> ApprovalRequired`
Use an approval once	`authorize(request, Some(&mut gate))`
Execute with containment	`dispatch(root, &request, content, approval)`
Log the decision	`audit(&mut log, &c, &d, 10)`
Build the CLI	`rustc --edition=2021 cli.rs -o firewall`

From tool-call-firewall/solution, docs/en.md, docs/en.md

Limits and numbers#

Setting	Default	Note
CLI envelope bytes	4096	Longer input is refused
Write content bytes	16384	Same cap for reads
Audit `max_entries`	10	Duplicate id is refused
JSON `schema_version`	1	CLI result JSON
Base64 token length	16	Shorter tokens are not decoded
Rot13 word hits	2	Fewer hits keep the input
No rule fires	0.0	Verdict category is `benign`
Detector accuracy	0.62	31 of 50 fixtures

From tool-call-firewall/solution, 19.83 code/main.py, 19.83 outputs/detector_report.json, Lesson 18.15

Questions#

Why do user-input filters miss IPI?: The payload is in retrieved content, not in user input.
Why not filter only keywords?: A benign instruction like "please print Yes" has no keyword.
Does fixing one boundary secure the rest?: No. Prevent each of the three independently.
Is a system prompt rule enough?: No. Instruction-following alone is not enforcement.
Where does the role come from?: The trusted application. Never the model response.
Is the detector the safety gate?: No. It is one signal that the gate composes.
Is the firewall an OS sandbox?: No. Approvals are single-use in memory in one invocation.

From Lesson 18.15, Lesson 18.25, Lesson 14.27, Lesson 19.83, tool-call-firewall/README.md