Prompt injection attacks, mitigations, detection, and design patterns for safer AI applications.
Prompt injection is the core attack pattern in modern AI applications. It happens when a model treats malicious or conflicting instructions from users, retrieved content, documents, tools, or pages as trusted guidance and changes its behavior in response.
These items are included because they add useful evidence, framing, implementation detail, or upcoming context for teams working in this area.
OWASP Global AppSec USA 2027 brings application security practitioners, developers, and researchers to Atlanta for training and conference sessions on current software-security risks and defenses.
DEF CON 35 is scheduled for August 5-8, 2027 at the Las Vegas Convention Center, bringing together security researchers, practitioners, villages, contests, and community-led technical programs.
OWASP Global AppSec EU 2027 brings application security practitioners, developers, and researchers to Vienna for professional training and conference sessions.
SecureWorld Seattle 2026 is a two-day professional cybersecurity conference in Bellevue with sessions on prompt injection, model poisoning, AI governance, NIST AI RMF implementation, shadow AI, and secure enterprise deployment.
OWASP Global AppSec USA 2026 brings application security professionals, developers, architects, and researchers to San Francisco for training and conference sessions.
AI Security Summit San Francisco is a one-day, two-track event for security leaders and practitioners covering AI governance, vendor risk, prompt injection, jailbreak defense, red teaming, and incident response.
AI Village at DEF CON 34 is confirmed for Las Vegas with demos, CTF activity, community programming, and a poster track on adversarial attacks against agents and agentic systems.
SecureWorld Artificial Intelligence 2026 is a virtual professional conference covering AI governance as code, agent security, prompt attacks, jailbreaks, shadow AI, NIST AI RMF, and operational resilience.
Intezer and Kodem Security found that hidden instructions in a fetched web page could make AWS Kiro write a malicious MCP server into its own configuration and automatically execute it with the developer’s privileges. The prompt-injection chain bypassed meaningful approval, affected tested 0.9 and 0.10 builds, and was confirmed fixed in version 0.11.130; no in-the-wild exploitation was reported.
Academic researchers evaluated five open-source mobile-agent frameworks against seven attacks that exploit the gap between what a vision model and a user can see. Subliminal overlays, pixels hidden beneath phone bezels, screenshot races, and forged activities redirected agent actions; in four frameworks, unsanitized model-generated ADB input also enabled command execution on the host PC.
Island identified roughly 7,600 malicious GitHub repositories in the FakeGit campaign, including more than 800 posing as AI skills or MCP servers and more than 600 listings in public AI registries. Tests showed coding assistants could independently surface the lures and repeat their installation instructions, which delivered SmartLoader and the StealC information stealer.
Join Microsoft Security at Black Hat USA 2026 for supply chain research, hands-on security experiences, expert conversations, and our reception. The post Microsoft at Black Hat USA 2026: Defending trust in the age of AI and supply chain attacks appeared first on Microsoft Security Blog .
Researchers describe agent data injection, which disguises attacker-controlled values as trusted metadata or agent context rather than explicit instructions. Demonstrations caused web agents to click the wrong control and coding agents to trust forged authors, tool results, or checks.
Adversa AI describes DeepJack, a Cursor deeplink weakness in which a crafted cursor:// link can register an attacker-controlled MCP server and hide the command that will execute behind an incomplete installation prompt.
Krebs reports that Microsoft’s July update fixed 570 flaws, including three exploited zero-days, as AI-assisted discovery accelerates patch volume. The release also addressed a high-severity Copilot flaw triggered through crafted prompts from a malicious webpage.
We’re open-sourcing k8s-aibom, a Kubernetes controller that continuously monitors environments to detect AI runtimes and generate standard ML-BOMs.
Microsoft’s latest Secure Future Initiative report outlines progress on secure foundations, AI-powered defense, and future-ready cybersecurity. The post Securing our future: July 2026 progress report on Microsoft’s Secure Future Initiative appeared first on Microsoft Security Blog .
System prompts form the foundation of generative AI applications. A system prompt is a collection of instructions and operational context provided to a large language model (LLM) that shapes how the model behaves and interacts with users and tools. System prompts often contain proprietary information, including role de
July's security digest covers the critical MCP vulnerabilities, real-world MCP exploitation, NSA's official MCP hardening guidelines. Explore these essential resources and secure our MCP servers. The post Top MCP security resources — July 2026 first appeared on Adversa AI .
Read five key learnings from the Frost & Sullivan 2025 Frost Radar™ for CSPM to learn how CSPM is evolving from point-in-time compliance to continuous risk management. The post 5 insights from Frost & Sullivan’s 2025 Frost Radar™ for Cloud Security Posture Management appeared first on Microsoft Security Blog .
Unit 42 analysis of phantom squatting: attackers register domains that LLMs may hallucinate in recommendations, code, documentation, or support answers. That turns model error into a supply-chain path, especially when users or agents follow generated links without independent validation.
Google integrated computer use into Gemini 3.5 Flash so agents can act across browser, mobile, and desktop environments. Optional enterprise safeguards can require confirmation for sensitive actions or stop a task when indirect prompt injection is detected.
Agent skill marketplaces introduce supply-chain risk when third-party skills can execute actions or collect data. Relevant to vetting, provenance, and containment controls.
Earlier this month, I spoke at the Gartner Security & Risk Management Summit about a blind spot most security programs are still not accounting for - how attackers are circumventing AI security programs by using legacy infrastructure to hijack AI agents. AI adoption is moving faster than security programs can account f
OpenAI update on European trustworthy-AI work and governance engagement. Relevant to standards, assurance, and regulatory coordination for deployed AI systems.
Two security teams have shown, in separate research published this week, that OpenClaw, the popular self-hosted AI agent, can be driven to run attacker-controlled code or hand over sensitive data through ordinary-looking inputs. Imperva buried instructions inside shared contacts, vCards, and location pins that the agen
Cloudflare describes architecture and operational lessons for defending against frontier cyber models. Relevant to AI-enabled threat modeling, defensive controls, and internal security readiness.
Cloudflare report on testing security-focused frontier models against real infrastructure code. Relevant to evaluating AI-assisted vulnerability discovery and production security workflows.
OWASP analysis of memory and context poisoning as an agent attack surface. Relevant to persistent state, trust boundaries, and regression tests for agent memory.
NVIDIA AI Red Team post on grammar-constrained decoding for Bash generation in small language models. Relevant to safer command generation and executable-output controls.
Cloudflare article on accountability models as AI assistants and privacy proxies blur bot and human distinctions. Relevant to agent identity, abuse prevention, and web access controls.
NVIDIA guidance on mitigating indirect AGENTS.md injection in agentic coding environments. Relevant to instruction provenance, repository trust, and sandboxed automation.
Play video
Attacking AI is a one of a kind session releasing case studies, tactics, and methodology from Arcanum’s AI assessments in 2024 and 2025. While most AI assessment material focuses on academic AI red team content, “Attacking AI” is focused on the task of assessing AI enabled systems.
Announcement of a hands-on CTF for agentic AI security in a financial-services scenario. Relevant to training, scenario design, and practical red-team exercises.
OWASP roundup of reported GenAI incidents and exploit patterns from Q1 2026. Relevant as a threat-intelligence reference for risk tracking and test-case design.
Play video
NDC Security 2026 talk on prompt injection in CI/CD and automation systems, including AI agents with access to shell commands, GitHub or GitLab tokens, issue editing, build workflows, and privileged pipeline context.
Microsoft Incident Response explains how to detect prompt abuse using logging, telemetry, and incident response workflows.
OpenAI frames prompt injection as an agent-security problem that increasingly resembles social engineering rather than simple string matching.
OpenAI announced plans to acquire Promptfoo, highlighting automated AI security testing, red teaming, and evaluation as core enterprise requirements.
NVIDIA guidance on sandboxing agentic workflows and managing execution risk. Relevant to tool isolation, approvals, filesystem boundaries, and operational controls for coding agents.
Play video
NDC AI 2025 talk on LLM frontdoors and backdoors, jailbreak techniques, control-token abuse, local model compromise, and how attackers or insiders can manipulate model behavior.
Play video
NDC AI 2025 talk on breaking AI systems in production, covering prompt injection, hidden prompts in documents, agent goal manipulation, privacy exposure, and practical AI red-team testing methods.
Play video
NDC AI 2025 talk introducing AI security for developers, including model lifecycle, training data, secure integration, data leakage, prompt injection, adversarial inputs, and model bias.
Play video
NDC AI 2025 talk on prompt-jacking in AI coding assistants, using Cursor vulnerability examples, hidden text in codebases, agentic behavior shaping, data exfiltration, and supply-chain style propagation.
OpenAI describes using automated red teaming and reinforcement learning to discover agent prompt injection attacks before they appear in the wild.
Google Cloud outlines a defense-in-depth view of AI security spanning application controls, data protections, and infrastructure isolation.
Play video
NDC Copenhagen talk on evaluating, testing, and securing LLM applications, including RAG changes, prompt-injection resilience, harmful-response guardrails, Promptfoo, DeepEval, Vertex AI Evaluation, and LLM Guard.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
An accessible explanation of prompt injection risk in real AI products, including how third-party content can redirect or manipulate agent behavior.
Google introduced AI Protection and Model Armor to address prompt injection, jailbreaks, data loss, and multicloud AI workload security.
OpenAI’s system card for deep research covers prompt injection, privacy, code execution, and external red teaming prior to release.
The Operator system card documents red teaming and mitigation choices for a computer-using agent, with prompt injections listed as a central risk area.
Microsoft summarizes lessons from red teaming more than one hundred generative AI products, emphasizing system-level testing, human expertise, and automation.
Microsoft Security distills lessons from red teaming more than 100 generative AI products, including multimodal prompt injection and core cyber hygiene.
OWASP’s GenAI security project remains a practical baseline for teams building or assessing LLM applications and agentic systems.
Play video
This AI Explained video reviews a major AI development through the lens of governance and responsible deployment. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of scaling and compute economics. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of model capability and AI systems in practice. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of AI safety and model behavior. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of scaling and compute economics. It is useful context for AI engineering, evaluation, governance, and operational risk.