Why it matters
NVIDIA's AI Red Team distills recurring pre-production findings into three concrete failure classes: prompt-injected model output reaching exec or eval and causing code execution; RAG stores that lose source permissions or accept attacker-writable content; and active Markdown or HTML that turns model output into a browser-based data-exfiltration channel.
My takeaway: Map model intent to an allowlist of safe functions instead of executing generated code; preserve document-level authorization and separate externally writable RAG sources; and constrain rendered output with URL sanitization, full-link disclosure, restrictive image CSP, or disabled active content. Regression-test all three paths with attacker-controlled retrieved content.