Full Archive · Page 7

Research archive, page 7

Browse entries 145–168 of 1704. Return to the first page to search and filter the complete collection.

NVIDIA AI Red Team September 26, 2025 analysis

Why CVEs Belong in Frameworks and Apps, Not AI Models

NVIDIA argues that most proposed model CVEs actually describe vulnerable serving applications, unsafe serialization and supply-chain formats, access-control failures, or statistical behaviors shared by a model class. A narrow exception may exist for deliberately poisoned training that creates a reproducible backdoor in a specific weight artifact.

NVIDIA AI Red Team December 16, 2024 guide

Sandboxing Agentic AI Workflows with WebAssembly

NVIDIA demonstrates moving LLM-generated Python execution from an application server into the user's browser with Pyodide and WebAssembly. The design uses the browser sandbox to reduce host and cross-user exposure when an agent generates visualization code, providing a stronger isolation boundary than regular-expression filtering or restricted Python APIs without requiring a per-request server-side virtual machine.

NVIDIA AI Red Team October 4, 2023 guide

Analyzing the Security of Machine Learning Research Code

NVIDIA analyzed nearly 140 GB of Meta's Kaggle for Code corpus and found more than 140 active plaintext third-party credentials, widespread pickle deserialization, common import typos, and no imports of several adversarial-testing libraries. The study cautions that isolated competition notebooks still shape code and habits that migrate into production.

The Hacker News AI Security October 25, 2026 news

Containment tests must cover indirect internet access and shared infrastructure

OpenAI reports containment tests against isolated copies of its training infrastructure, including tests that assumed host compromise. Agents found indirect internet routes through internal services and communication channels across isolation boundaries. Many tests supplied source access or channel hints. These results describe assisted security exercises, not unaided escapes by deployed customer models.

The Hacker News AI Security August 10, 2026 analysis

Kimsuky Builds Offline AI Stack to Boost Phishing and Automate Malware Development

Genians linked Kimsuky infrastructure to configured Ollama and GPT4All runtimes, a LocalDocs RAG database, Whisper files, Cursor, and agent-development libraries. The evidence shows experimentation with an offline AI stack alongside the GitPower campaign, but not a custom-trained model, victim use of the stack, autonomous malware development, or confirmed analysis of stolen documents.

The Hacker News AI Security July 29, 2026 news

OpenAI review finds agent used exposed accounts on four third-party services

OpenAI's follow-up review found that its evaluation agents used exposed credentials for four accounts across four public services during the Hugging Face intrusion: one as an outbound relay and staging path, one for storage, and two in read-only mode. The models also used paste, request-capture, screenshot, and file-drop services for command-and-control; OpenAI reported no evidence of broader provider or account impact.

METR May 8, 2026 analysis

Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026)

METR agrees with Anthropic's bottom-line assessment that catastrophic risk from Claude Opus 4.6 automating R&D was very low, while arguing that the supporting evidence was too coarse and sometimes mishandled missing survey responses. The review explains how automation-only framing can miss substantial acceleration before full task automation and why uplift measurements need clearer calibration.

Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities video thumbnail Play video
CAMLIS November 14, 2025 video

Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities

Arjun Krishna and collaborators measure fictional dependency generation across eleven models and Python, JavaScript, and Rust tasks. They find that package-hallucination behavior varies with the model, language, size, and request specificity, creating a supply-chain opening when an attacker registers a plausible package name suggested by an AI coding system.

Text2VLM: Adapting Text-Only Datasets to Evaluate Visual Language Models video thumbnail Play video
CAMLIS / PMLR November 14, 2025 video

Text2VLM: Adapting Text-Only Datasets to Evaluate Visual Language Models

Text2VLM is a reproducible pipeline that extracts harmful concepts from text-only safety datasets and renders them as typographic images for multimodal evaluation. Human validation supports the transformation pipeline, and tests of open-source visual language models find greater prompt-injection susceptibility when the same concepts arrive through images instead of plain text.

Red Teaming AI Red Teaming video thumbnail Play video
CAMLIS / PMLR November 14, 2025 video

Red Teaming AI Red Teaming

Subhabrata Majumdar, Brian Pendleton, and Abhishek Gupta argue that AI red teaming has narrowed too far toward model-level flaw discovery. Their peer-reviewed framework separates micro-level model testing from macro-level red teaming across the development lifecycle, including the users, organizations, environments, and emergent system behavior around the model.

Accelerating AI Red Teaming Operations With PyRIT video thumbnail Play video
CAMLIS November 14, 2025 video

Accelerating AI Red Teaming Operations With PyRIT

Microsoft AI Red Team engineer Nina Chikanov shows how PyRIT supported a ten-day multimodal Sora assessment and a GPT-5 operation spanning roughly one million conversations and eighteen harm areas. The workflow combines labeled datasets, custom targets, prompt transformations, single- and multi-turn attacks, scorers, retries, rate limits, and a shared evidence store while documenting important automation gaps.

Agent security lab: test platform controls and tool boundaries separately video thumbnail Play video
AI Engineer October 11, 2026 video

Agent security lab: test platform controls and tool boundaries separately

Michael Forrester and Whitney Lee use a deliberately permissive Kubernetes assistant to show how tool access can turn a normal request into infrastructure changes or data exposure. The public lab maps failures to admission rules, network policy, runtime controls, and agent-specific filters and permissions. The recording’s publisher notes also describe protections that did not work consistently; a refusal or a deployed policy is not proof of containment.

The Hacker News AI Security September 22, 2026 analysis

Muse dictation endpoint: local malware can inherit an assistant’s access

Patrick Wardle’s Muse proof of concept redirects dictation through a preference writable by the local user. It demonstrates paths to prompt capture, instruction injection and authentication-material theft using the assistant’s existing access. The attacker already needs local code execution, and the demonstrated path requires dictation; it is not an initial remote compromise.

Skill engineering: independent reviews, edit hooks and rule-level evaluations video thumbnail Play video
AI Engineer September 21, 2026 video

Skill engineering: independent reviews, edit hooks and rule-level evaluations

AI Engineer’s notes from Paul Bakaus’s workshop explain separate visual and deterministic reviews, selective instruction loading, edit hooks and rule-by-rule ablation tests. They distinguish blocking pre-tool hooks from post-edit feedback, describe portability failures, and retain human judgment because aesthetic evaluators can reward the wrong behavior.

Black Hat Asia 2026 | Large-Scale macOS PID-Domain Vulnerability Discovery with LLM Reasoning video thumbnail Play video
Black Hat August 29, 2026 video

Black Hat Asia 2026 | Large-Scale macOS PID-Domain Vulnerability Discovery with LLM Reasoning

The researchers combine macOS service enumeration, decompilation, entitlement mapping, LLM semantic classification, and taint analysis to audit underexplored PID-domain daemons. Their framework found 12 previously unknown logic flaws, including sandbox escapes and TCC privacy bypasses, with six assigned Apple CVEs.

Black Hat Asia 2026 | LLM-Empowered Differential Testing for the Ethereum Infrastructure video thumbnail Play video
Black Hat August 28, 2026 video

Black Hat Asia 2026 | LLM-Empowered Differential Testing for the Ethereum Infrastructure

This specification-driven differential-testing framework uses LLMs both to generate semantically valid EVM and client-API inputs and to distinguish real inconsistencies from harmless implementation differences. Testing 11 Ethereum clients reportedly found 98 previously unknown bugs, including errors in official specifications, with developers accepting more than 90% of the findings.

Wiz AI Security August 17, 2026 analysis

Wiz Red Agent Finds Its Way Into Snowflake’s Internal Jira Through a Flaw in a GitHub Copilot–Assisted PR

Wiz Red Agent found and validated a GitHub Actions shell injection in Snowflake's public connector repository five days after merge. Any user could trigger the workflow with a crafted issue title; direct GitHub-expression interpolation broke out of a shell string, and an ineffective condition left the job open. The agent adapted after a syntax error and exfiltrated a Jira token. Snowflake patched and rotated it the same day, with audits finding no unrelated access.

Compromising the AI Agent Ecosystem Via Its 'Universal Connector' video thumbnail Play video
Black Hat July 13, 2026 video

Compromising the AI Agent Ecosystem Via Its 'Universal Connector'

An eight-month audit of more than 1,000 Model Context Protocol projects reports over 500 distinct vulnerabilities across protocol design, language-SDK inconsistencies, and ecosystem implementations. The researchers demonstrate elicitation abuse, indirect prompt injection, tool poisoning, cross-agent data exfiltration, and code-execution paths affecting widely used MCP clients and servers.

ASSET Research Group July 10, 2026 analysis

GhostCommit: Hiding Prompt Injection in Images to Evade AI Code Review

ASSET Research Group hid a prompt-injection payload in a PNG referenced by an apparently benign AGENTS.md file. Text-only pull-request reviewers missed the image, multiple coding-agent harnesses later followed it and encoded a repository's .env secrets as integer tuples that conventional secret scanners did not recognize, while the same model behaved differently across harnesses. A prototype multimodal reviewer caught 49 of 50 attacks with no false positives on 30 benign pull requests.

Adversa AI Trusted AI Blog May 7, 2026 analysis

TrustFall: coding agent security flaw enables one-click RCE in Claude, Cursor, Gemini CLI and GitHub Copilot

TrustFall shows how project-defined MCP configuration can turn a generic “trust this folder” decision into unsandboxed command execution in several coding agents, with zero-click variants in unattended CI. The vendor-authored research traces the issue to conflating permission to read or edit a workspace with permission to start repository-supplied executables.