Secrets, Audit Logs and Compliance

Vault-backed keys, PII scrubbing, audit logs, signed webhooks, framework controls and AI law dates.

Concepts and terms
Tasks and commands
Annotated code
Diagrams
Comparisons
Defaults and numbers
Troubleshooting
Questions and tips

Gateway pulls keys from vault#

Security numbers#

Setting	Default	Note
Key rotation	≤ 90 days	Keys, vault root tokens, CI/CD credentials
Token lifetime	hours	Short-lived tokens, not days
Audit retention, SOC 2	1 year	Retain per regulation
Audit retention, HIPAA	6 years	Retain per regulation
Webhook clock skew	`5*time.Minute`	Both sides of now
Webhook body size	1..65536 bytes	Read before any decoding
Webhook secret	WEBHOOK_SECRET	At least 16 bytes
`--seconds`	60	Receiver lifetime, 1..600

From Lesson 17.25, solution/main.go, signed-webhook-inbox/README.md

Audit log entry#

entry = AuditEntry(

timestamp=datetime.utcnow().isoformat() + "Z",

user=f"user_{i:03}",

tenant="tenant_01",

model="anthropic/claude-3.7-sonnet",model plus version

prompt_hash=hash_short(scrubbed),hash, not the raw prompt

response_hash=hash_short(response),

input_tokens=len(scrubbed.split()),

output_tokens=len(response.split()),

cost_usd=0.0012,

guardrail_trips=[],any guardrail trips

)

print(audit_log_call(entry))one JSON line per call

From 17.25 code/main.py

Webhook inbox commands#

Command	What it does
`/tmp/webhook-inbox --mode serve --dir <dir> --listen 127.0.0.1:8141 --seconds 120`	Receive deliveries for 120 seconds
`/tmp/webhook-inbox --mode sign --id delivery-1 --timestamp <unix> --body fixtures/event.json`	Print the three signed headers
`/tmp/webhook-inbox --mode inspect --dir <dir> --id delivery-1`	Show the stored delivery
`/tmp/webhook-inbox --mode replay --dir <dir> --id delivery-1 --run review-1`	Process once under run ID review-1

From signed-webhook-inbox/README.md

Four jurisdictions#

Factor	EU	US	UK	Korea
Posture	Strict, risk-tiered	Innovation-favouring	Narrow security focus	MSIT-led
Body	AI Office	CAISI under NIST	AI Security Institute	AISI under MSIT
Key date	In force Aug 2024	Renamed Jun 2025	Renamed Feb 2025	Effective Jan 2026
Core duty	Risk tiers, logging	Standards support	Frontier security	Local representative

From Lesson 18.24

Troubleshooting#

Symptom	Cause	Fix
Keys from a deleted `.env` still leak	Keys stay in git history	Rotate the keys. Scan every commit
Data leaves through a DNS lookup	LLM pod can reach any host	Egress whitelist, allowlist-only resolver
SSN reached the provider	No masking before forwarding	Scrub PII before the prompt leaves
401 `signature mismatch`	Body bytes changed	Send the exact file bytes
401 `timestamp outside acceptance window`	Old or future timestamp	Sign with the current Unix timestamp
409 on a known delivery ID	Same ID, different body	Use a new delivery ID
`--dir, 16-byte key env and seconds 1..600 required`	Short key or no inbox directory	Set a 16-byte WEBHOOK_SECRET and `--dir`

From Lesson 17.25, solution/main.go, signed-webhook-inbox/README.md

Run the labs#

Task	Code
PII scrubber and audit log, lesson 25	`code/main.py`
Control and profile map, lesson 26	`code/main.py`
AI regulation timeline, phase 18	`python3 code/main.py`
Webhook demo: 202, 200, 401	`go run . --mode demo`
Build the webhook inbox	`go build -o /tmp/webhook-inbox .`

From Lesson 17.25, Lesson 17.26, 18.24 code/main.py, signed-webhook-inbox/README.md

PII scrubbing#

Order: entity recognition, mask, consistent tokenization, optional reverse mapping.
Lab patterns: SSN, EMAIL and PHONE regexes in scrubber.scrub.
Consistent tokens: the same value maps to the same placeholder every time.
Regex and NER: regex catches basic patterns. NER catches more. Use both.
When: before the LLM call. Post-processing cleanup is not defensible.
Storage: only redacted prompts plus consented opt-in raw.
Output guardrail: regex scrub for leaked API key patterns.

From Lesson 17.25, 17.25 code/main.py, Lesson 17.26

Webhook receiver path#

From solution/main.go, signed-webhook-inbox/README.md

Frameworks and their LLM duty#

SOC 2 Type II: controls operate effectively over 6-12 months.
HIPAA: no PHI to external AI services without a signed BAA.
GDPR: real-time PII redaction, data subject rights, Article 30 records.
PCI-DSS: configuration plus contracts for AI that touches payment.
EU AI Act: risk tier classification. High-risk needs conformity assessment and logging.
Colorado AI Act: impact assessments and right to appeal. Effective June 30, 2026.
ISO 42001: AI governance. It pairs with ISO 27001.

From Lesson 17.26

Cross-framework control map#

Command	What it does
`access logging`	ISO 27001 A.5.15-5.18, GDPR Art. 32, HIPAA §164.312(a), SOC 2 CC6
`encryption in transit`	ISO 27001 A.8.24, GDPR Art. 32, HIPAA §164.312(e)
`secrets management`	ISO 27001 A.8.19, PCI DSS Req. 8, SOC 2 CC6.1
`audit log retention`	SOC 2 CC7, HIPAA §164.312(b), ISO 27001 A.8.15
`PII redaction (inference-time)`	GDPR Art. 25, EU AI Act Art. 10, HIPAA §164.514
`impact assessment`	Colorado AI Act SB24-205, EU AI Act Art. 27
`BAA signed`	HIPAA §164.504(e)

From 17.26 code/main.py

EU AI Act dates and fines#

1 Aug 2024: in force.
2 Feb 2025: prohibited practices and AI literacy apply.
2 Aug 2025: GPAI and governance obligations apply.
2 Aug 2026: full applicability, Article 50 transparency, penalties.
2 Aug 2027: legacy GPAI and embedded high-risk systems.
Art. 99(4): up to €15M or 3% for high-risk obligations.
Art. 99(3): up to €35M or 7% for prohibited practices.

From Lesson 18.24, Lesson 17.26

Q&A#

How do you treat CI/CD credentials?: As prod-equivalent. Store in vault. Scope narrowly. Rotate aggressively.
Type I or Type II?: Type II. Type I is a starter. Type II is the gate.
Does a 202 mean the action ran?: No. Authenticated bytes were durably accepted.
Which rules bind across jurisdictions?: The strictest, typically the EU AI Act in 2026.
What makes GPAI systemic-risk?: Training compute above 1e25 FLOP.

From Lesson 17.25, Lesson 17.26, Lesson 18.24, signed-webhook-inbox/README.md