Full Archive · Page 4

Research archive, page 4

Browse entries 73–96 of 1127. Return to the first page to search and filter the complete collection.

Adversa AI Trusted AI Blog May 7, 2026 analysis

TrustFall: coding agent security flaw enables one-click RCE in Claude, Cursor, Gemini CLI and GitHub Copilot

TrustFall shows how project-defined MCP configuration can turn a generic “trust this folder” decision into unsandboxed command execution in several coding agents, with zero-click variants in unattended CI. The vendor-authored research traces the issue to conflating permission to read or edit a workspace with permission to start repository-supplied executables.

NVIDIA AI Red Team November 3, 2025 analysis

How Code Execution Drives Key Risks in Agentic AI Systems

NVIDIA uses a PandasAI code-execution vulnerability to explain why generated-code sanitizers are brittle: namespace access, encoding, imports, and runtime context can turn apparently allowed syntax into arbitrary execution. The article separates heuristic filtering from the structural protection provided by a constrained execution environment.

NVIDIA AI Red Team October 9, 2025 analysis

From Assistant to Adversary: Exploiting Agentic AI Developer Tools

NVIDIA walks through a repository-borne prompt-injection chain in which a coding agent reviewing a pull request installs a disguised dependency whose setup logic opens a reverse shell. The example connects untrusted issue and pull-request text to package execution and shows why model-level refusal cannot secure a developer environment with broad tools and credentials.

NVIDIA AI Red Team December 16, 2024 guide

Sandboxing Agentic AI Workflows with WebAssembly

NVIDIA demonstrates moving LLM-generated Python execution from an application server into the user's browser with Pyodide and WebAssembly. The design uses the browser sandbox to reduce host and cross-user exposure when an agent generates visualization code, providing a stronger isolation boundary than regular-expression filtering or restricted Python APIs without requiring a per-request server-side virtual machine.

Trail of Bits Blog June 3, 2026 analysis

The sorry state of skill distribution

Trail of Bits bypassed multiple agent-skill scanners with compiled Python hidden beside benign source and with prompt-like prose that persuaded an LLM classifier to accept a malicious configuration. The experiments show recurring blind spots around unreferenced files, binaries, assets, and ambiguous installer behavior, and also explain why legitimate skills can contain patterns that look malicious.

Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities video thumbnail Play video
CAMLIS November 14, 2025 video

Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities

Arjun Krishna and collaborators measure fictional dependency generation across eleven models and Python, JavaScript, and Rust tasks. They find that package-hallucination behavior varies with the model, language, size, and request specificity, creating a supply-chain opening when an attacker registers a plausible package name suggested by an AI coding system.

Text2VLM: Adapting Text-Only Datasets to Evaluate Visual Language Models video thumbnail Play video
CAMLIS / PMLR November 14, 2025 video

Text2VLM: Adapting Text-Only Datasets to Evaluate Visual Language Models

Text2VLM is a reproducible pipeline that extracts harmful concepts from text-only safety datasets and renders them as typographic images for multimodal evaluation. Human validation supports the transformation pipeline, and tests of open-source visual language models find greater prompt-injection susceptibility when the same concepts arrive through images instead of plain text.

Red Teaming AI Red Teaming video thumbnail Play video
CAMLIS / PMLR November 14, 2025 video

Red Teaming AI Red Teaming

Subhabrata Majumdar, Brian Pendleton, and Abhishek Gupta argue that AI red teaming has narrowed too far toward model-level flaw discovery. Their peer-reviewed framework separates micro-level model testing from macro-level red teaming across the development lifecycle, including the users, organizations, environments, and emergent system behavior around the model.

Accelerating AI Red Teaming Operations With PyRIT video thumbnail Play video
CAMLIS November 14, 2025 video

Accelerating AI Red Teaming Operations With PyRIT

Microsoft AI Red Team engineer Nina Chikanov shows how PyRIT supported a ten-day multimodal Sora assessment and a GPT-5 operation spanning roughly one million conversations and eighteen harm areas. The workflow combines labeled datasets, custom targets, prompt transformations, single- and multi-turn attacks, scorers, retries, rate limits, and a shared evidence store while documenting important automation gaps.

The Hacker News AI Security July 28, 2026 news

Researcher Says AI Helped Develop Linux Traffic-Control Race Into Root Exploit

STAR Labs researcher Lee Jia Jie reports using an unnamed AI assistant to identify and weaponize CVE-2026-53264, a Linux network-scheduler use-after-free race, into local root on a targeted CentOS Stream 9 kernel. The exploit needs a local foothold, unprivileged user namespaces, specific traffic-control options, and kernel-specific ROP offsets; upstream fixed the lifetime bug by deferring frees through RCU.

Adversa AI Trusted AI Blog June 30, 2026 analysis

GuardFall: a universal shell injection vulnerability in open-source AI agents

GuardFall tests 11 open-source coding and computer-use agents against shell-command transformations that evade string and regex deny lists, including quote removal, $IFS expansion, command substitution, and encoded payloads. The study finds configuration- and model-dependent failures and shows that local or auto-approve modes can turn untrusted repository content into host command execution; it is vendor-authored research, not an independent benchmark.

Ryan Greenblatt – What happens once AI can automate AI research? video thumbnail Play video
Dwarkesh Patel August 11, 2026 video

Ryan Greenblatt – What happens once AI can automate AI research?

Dwarkesh Patel and Redwood Research chief scientist Ryan Greenblatt debate whether verifiable AI-research tasks could produce rapid recursive improvement, then examine alignment targets, reward hacking, model coordination, and recent deception and containment incidents. The two-hour format exposes assumptions about data, compute, verification, and extrapolation rather than presenting a single forecast as settled fact.

Adversa AI Trusted AI Blog July 30, 2026 analysis

A hole in every one: bypassing the open source AI skill scanners

Adversa tested eight open-source AI skill scanners with paired unobfuscated and obfuscated malicious skills, finding that every scanner passed an attack through either a true bypass, a blind spot, or an injectable model judge. The study covers encoding, Unicode, command reconstruction, truncation, allowlists, bundled files, paraphrase, and remote stages; its 4,000-skill benign set also found no scanner beat an always-block baseline on F1. Most tools ran offline without optional model triage, and some were reconstructed from retained artifacts.

From Prompt Tricks to Autonomous Hackers video thumbnail Play video
Black Hat July 29, 2026 video

From Prompt Tricks to Autonomous Hackers

Ari Herbert-Voss reviews three years of progress in autonomous offensive-security systems, evaluates where they can already complete meaningful attack tasks, and separates those capabilities from work that still needs human expertise. The talk frames scalable, parallel attack simulation as a challenge to point-in-time testing rather than as a product announcement.

METR July 24, 2026 analysis

Metrics of Agent Ability

METR organizes agent-capability measures around performance as a function of expenditure, comparing fixed-budget scores, cost to reach a score, returns to test-time scaling, human-equivalent time and expenditure horizons, and human-relative cost. It explains when familiar benchmark scores break down—particularly when performance keeps improving with more inference or human benchmarks saturate—and notes that full cost, reliability, coverage, and elicitation choices affect the result.

NVIDIA AI Red Team January 28, 2026 analysis

Updating Classifier Evasion for Vision Language Models

NVIDIA demonstrates gradient-based attacks against a PaliGemma2 vision-language classifier, including imperceptible perturbations and localized patches that change a stop-sign decision or force an arbitrary output token. It also explains why physical attacks require transformations that model changes in scale, angle, lighting, and capture conditions.

Cloudflare AI Security August 14, 2026 analysis

How Cloudflare detects MCP traffic and helps secure it

Cloudflare Gateway now classifies inspected Streamable HTTP MCP traffic using the MCP-Protocol-Version header, exposes the user and destination in logs and a dashboard, and supports allow or block rules through an MCP-specific selector. The article distinguishes unapproved shadow MCP from direct connections that bypass an approved Portal's controls, and notes blind spots including local stdio, off-network, non-inspected, and otherwise unobserved traffic.