Prompt injection attacks, mitigations, detection, and design patterns for safer AI applications.
Prompt injection is the core attack pattern in modern AI applications. It happens when a model treats malicious or conflicting instructions from users, retrieved content, documents, tools, or pages as trusted guidance and changes its behavior in response.
These items are included because they add useful evidence, framing, implementation detail, or upcoming context for teams working in this area.
DEF CON 34 takes place in Las Vegas and is expected to include AI security activity through villages, workshops, contests, and community-led research tracks as schedules firm up.
We’re open-sourcing k8s-aibom, a Kubernetes controller that continuously monitors environments to detect AI runtimes and generate standard ML-BOMs.
Give an AI assistant a memory and access to your inbox, and you hand an attacker a way to rewrite what it thinks it knows about you. A single email can trick that agent into saving a false "fact" about the user, hide the change, and quietly steer its answers in later sessions. When it works, the person reads an ordinar
The U.K. Treasury has designated Google Cloud EMEA as a critical third party (CTP) to the U.K. financial sector under the CTP regime. Here’s how that helps you.
Details have emerged about three now-patched security flaws in the OpenClaw personal artificial intelligence (AI) assistant that, if successfully exploited, could enable credential theft, privilege escalation, and arbitrary code execution on the host. A brief description of the high-severity vulnerabilities is as follo
Microsoft’s latest Secure Future Initiative report outlines progress on secure foundations, AI-powered defense, and future-ready cybersecurity. The post Securing our future: July 2026 progress report on Microsoft’s Secure Future Initiative appeared first on Microsoft Security Blog .
Ask an AI coding agent to scan open-source code for security holes, and it might run the attacker's code on your own machine instead. That is the finding in a proof-of-concept published Wednesday by the AI Now Institute, an attack it calls "Friendly Fire." It works against Anthropic's Claude Code and OpenAI's Codex whe
Researchers at Wiz found that a flaw in six popular AI coding assistants lets a booby-trapped code project quietly take control of a developer's computer. The assistant asks permission to edit one harmless-looking file, but the write lands on a sensitive one instead. The affected tools are Amazon Q Developer, Anthropic
System prompts form the foundation of generative AI applications. A system prompt is a collection of instructions and operational context provided to a large language model (LLM) that shapes how the model behaves and interacts with users and tools. System prompts often contain proprietary information, including role de
Our flagship Google for Startups program, Gemini Startup Forum: Cybersecurity, has selected its first 33 trailblazing startups.
Sophos looked at a week of its own endpoint data and found that AI coding agents such as Claude Code, Cursor, and OpenAI Codex are setting off detection rules written to catch human intruders. The agents are not malicious. They just do a lot of things that, to a behavioral engine, look exactly like an attack. Decryptin
The U.S. Cybersecurity and Infrastructure Security Agency (CISA) on Tuesday added four security flaws to its Known Exploited Vulnerabilities (KEV) catalog, citing evidence of active exploitation. The vulnerabilities are listed below - CVE-2026-48282 (CVSS score: 10.0) - A path traversal vulnerability in Adobe ColdFusio
An AI coding assistant that refuses to answer a dangerous request in its chat box can answer it anyway if the same request is broken into small, ordinary-looking steps inside a code editor. That is the finding of a new study of GitHub Copilot by researchers Abhishek Kumar and Carsten Maple. The models they tested throu
To help you match your real-world exposures with real-time adversary activity, we’ve begun integrating Google Threat Intelligence with Wiz Attack Surface Management.
A public issue can trick GitHub Agentic Workflows into leaking the contents of an organization's private repositories, researchers at Noma Security have shown. The attacker needs only to open a normal-looking issue on a public repository, with no stolen credentials and no access to the organization. If that organizatio
Cybersecurity researchers have disclosed details of a now-patched critical session isolation vulnerability in Writer, an enterprise generative artificial intelligence (AI) platform, that could result in cross-tenant compromise. The one-click vulnerability has been codenamed WriteOut by the Sand Security Research team.
July's security digest covers the critical MCP vulnerabilities, real-world MCP exploitation, NSA's official MCP hardening guidelines. Explore these essential resources and secure our MCP servers. The post Top MCP security resources — July 2026 first appeared on Adversa AI .
Read five key learnings from the Frost & Sullivan 2025 Frost Radar™ for CSPM to learn how CSPM is evolving from point-in-time compliance to continuous risk management. The post 5 insights from Frost & Sullivan’s 2025 Frost Radar™ for Cloud Security Posture Management appeared first on Microsoft Security Blog .
Unit 42 analysis of phantom squatting: attackers register domains that LLMs may hallucinate in recommendations, code, documentation, or support answers. That turns model error into a supply-chain path, especially when users or agents follow generated links without independent validation.
Frost & Sullivan names Microsoft a leader as cloud and application security converge into unified, runtime risk reduction. The post Microsoft named a leader in the Frost Radar for cloud and application runtime security appeared first on Microsoft Security Blog .
Agent skill marketplaces introduce supply-chain risk when third-party skills can execute actions or collect data. Relevant to vetting, provenance, and containment controls.
Earlier this month, I spoke at the Gartner Security & Risk Management Summit about a blind spot most security programs are still not accounting for - how attackers are circumventing AI security programs by using legacy infrastructure to hijack AI agents. AI adoption is moving faster than security programs can account f
OpenAI update on European trustworthy-AI work and governance engagement. Relevant to standards, assurance, and regulatory coordination for deployed AI systems.
Two security teams have shown, in separate research published this week, that OpenClaw, the popular self-hosted AI agent, can be driven to run attacker-controlled code or hand over sensitive data through ordinary-looking inputs. Imperva buried instructions inside shared contacts, vCards, and location pins that the agen
Cloudflare describes architecture and operational lessons for defending against frontier cyber models. Relevant to AI-enabled threat modeling, defensive controls, and internal security readiness.
Cloudflare report on testing security-focused frontier models against real infrastructure code. Relevant to evaluating AI-assisted vulnerability discovery and production security workflows.
OWASP analysis of memory and context poisoning as an agent attack surface. Relevant to persistent state, trust boundaries, and regression tests for agent memory.
NVIDIA AI Red Team post on grammar-constrained decoding for Bash generation in small language models. Relevant to safer command generation and executable-output controls.
Cloudflare article on accountability models as AI assistants and privacy proxies blur bot and human distinctions. Relevant to agent identity, abuse prevention, and web access controls.
NVIDIA guidance on mitigating indirect AGENTS.md injection in agentic coding environments. Relevant to instruction provenance, repository trust, and sandboxed automation.
Play video
Attacking AI is a one of a kind session releasing case studies, tactics, and methodology from Arcanum’s AI assessments in 2024 and 2025. While most AI assessment material focuses on academic AI red team content, “Attacking AI” is focused on the task of assessing AI enabled systems.
Announcement of a hands-on CTF for agentic AI security in a financial-services scenario. Relevant to training, scenario design, and practical red-team exercises.
OWASP roundup of reported GenAI incidents and exploit patterns from Q1 2026. Relevant as a threat-intelligence reference for risk tracking and test-case design.
Play video
NDC Security 2026 talk on prompt injection in CI/CD and automation systems, including AI agents with access to shell commands, GitHub or GitLab tokens, issue editing, build workflows, and privileged pipeline context.
Microsoft Incident Response explains how to detect prompt abuse using logging, telemetry, and incident response workflows.
OpenAI frames prompt injection as an agent-security problem that increasingly resembles social engineering rather than simple string matching.
OpenAI announced plans to acquire Promptfoo, highlighting automated AI security testing, red teaming, and evaluation as core enterprise requirements.
NVIDIA guidance on sandboxing agentic workflows and managing execution risk. Relevant to tool isolation, approvals, filesystem boundaries, and operational controls for coding agents.
Play video
NDC AI 2025 talk on LLM frontdoors and backdoors, jailbreak techniques, control-token abuse, local model compromise, and how attackers or insiders can manipulate model behavior.
Play video
NDC AI 2025 talk on breaking AI systems in production, covering prompt injection, hidden prompts in documents, agent goal manipulation, privacy exposure, and practical AI red-team testing methods.
Play video
NDC AI 2025 talk introducing AI security for developers, including model lifecycle, training data, secure integration, data leakage, prompt injection, adversarial inputs, and model bias.
Play video
NDC AI 2025 talk on prompt-jacking in AI coding assistants, using Cursor vulnerability examples, hidden text in codebases, agentic behavior shaping, data exfiltration, and supply-chain style propagation.
OpenAI describes using automated red teaming and reinforcement learning to discover agent prompt injection attacks before they appear in the wild.
Google Cloud outlines a defense-in-depth view of AI security spanning application controls, data protections, and infrastructure isolation.
Play video
NDC Copenhagen talk on evaluating, testing, and securing LLM applications, including RAG changes, prompt-injection resilience, harmful-response guardrails, Promptfoo, DeepEval, Vertex AI Evaluation, and LLM Guard.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
An accessible explanation of prompt injection risk in real AI products, including how third-party content can redirect or manipulate agent behavior.
Google introduced AI Protection and Model Armor to address prompt injection, jailbreaks, data loss, and multicloud AI workload security.
OpenAI’s system card for deep research covers prompt injection, privacy, code execution, and external red teaming prior to release.
The Operator system card documents red teaming and mitigation choices for a computer-using agent, with prompt injections listed as a central risk area.
Microsoft summarizes lessons from red teaming more than one hundred generative AI products, emphasizing system-level testing, human expertise, and automation.
Microsoft Security distills lessons from red teaming more than 100 generative AI products, including multimodal prompt injection and core cyber hygiene.
OWASP’s GenAI security project remains a practical baseline for teams building or assessing LLM applications and agentic systems.
Play video
This AI Explained video reviews a major AI development through the lens of governance and responsible deployment. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of scaling and compute economics. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of model capability and AI systems in practice. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of AI safety and model behavior. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of scaling and compute economics. It is useful context for AI engineering, evaluation, governance, and operational risk.