Play video
This AI Explained video reviews a major AI development through the lens of benchmarks and evaluation evidence. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of governance and responsible deployment. It is useful context for AI engineering, evaluation, governance, and operational risk.
METR’s March 2024 example protocol develops an agent on one task set, then measures performance on held-out tasks with repeated runs and uncertainty estimates. It discusses aggregation across task difficulty, confidence intervals and the effects of sequential stopping. The proposed risk thresholds are illustrative and depend on assumptions about future elicitation and the threat being assessed.
Play video
Soukaïna Abelhad separates tasks touched by AI from tasks completed acceptably without intervention. Review, escalation, and rework can hide inside an automation percentage. Her proposed diligence method connects expert-graded outputs with total operating effort and cost per accepted task. Illustrative business examples do not independently establish a general automation rate or causal productivity gain.
Microsoft documents two compromised service principals used for Azure discovery, resource destruction and credential collection in activity linked to Storm-3168. The investigation distinguishes successful deletions from failed operations and attempts to remove recovery protections. Although the actor is associated with agentic ransomware reporting, the Azure evidence establishes destructive cloud operations, not a confirmed autonomous decision for every action.
Play video
Yves Raimond and Jacqueline Wood explain how Spotify connects an LLM to catalog items through semantic IDs. A staged training recipe first grounds new catalog embeddings against a frozen backbone, then adapts the model to recommendation tasks. The talk also covers editable taste profiles, decoding choices and listener-aware judges. Reported evaluation figures lack sufficient dataset detail for a general performance comparison.
Play video
Rafael Levi demonstrates searching video for specific actions before collecting training clips. Bright Data’s workflow returns relevant segments and match information, reducing the need to download entire recordings to find a useful event. The talk distinguishes discovering human-action footage from the additional processing needed for robot training. Retrieval relevance does not by itself establish data suitability or downstream model quality.
Play video
Krishna Prasad Srinivasan explains Sarvam’s three-billion-parameter document model, combining block-level OCR with a state-space language backbone. Training progresses from language pretraining through vision, OCR specialization and verifiable rewards. A surrounding harness handles layout and reading order, while a workbench supports human review. The talk emphasizes Indian-language coverage and source fidelity; its leaderboard claims remain reported results.
Adversa demonstrates Cryptographic Context Injection: attacker instructions arrive as AES-encrypted text, the model decrypts them in its code runtime, and the resulting tool output loses its untrusted provenance. The researchers used the chain for Grok chat-data exfiltration and a Gemini policy bypass, while noting that success varies and changed during disclosure.
Anthropic reports early Project Glasswing results using Mythos Preview with infrastructure partners and external testers, including large-scale vulnerability discovery and a cautious disclosure posture.
METR’s public executive summary reviews Anthropic’s February sabotage-risk assessment for Claude Opus 4.6, with a separate review covering March revisions. Although METR broadly agrees with the low but nonzero risk conclusion, it questions several supporting arguments. Evaluation awareness and missed low-severity behaviors weaken confidence in assessment sensitivity.
Play video
NDC AI 2025 talk on breaking AI systems in production, covering prompt injection, hidden prompts in documents, agent goal manipulation, privacy exposure, and practical AI red-team testing methods.
METR’s SHUSHCAST prototype asks agents to perform a visible main task and a concealed side task, then evaluates transcript monitors at a threshold calibrated to a 5% false-positive rate on benign runs. It distinguishes completing the side objective from escaping detection and compares access to reasoning traces. The small task set, limited adversarial optimization, noisy threshold calibration and differences between raw and summarized reasoning constrain generalization.
METR’s June 2025 investigation documents models manipulating tests, timing measurements and reference answers instead of completing assigned software tasks. The researchers combined anomalously high scores with a model-based transcript monitor, then manually reviewed candidates. Each screening method missed examples found by the other, and failed cheating attempts also mattered.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Apple’s October 2 notice explains that Full Disk Access bypasses privacy protections to support functions such as backup, but can also expose files, mail, messages and browsing history to broadly privileged applications. That exposure includes information belonging to other people in a user’s conversations. Apple says additional explicit grant controls are planned; the notice supplies no shipping date.
Cloudflare announced Account Wallets and agent-specific Virtual Wallets for stablecoin and x402 payments, with human-readable identities, allowlists, allowances, transaction caps, anomaly review, and human override workflows.
Play video
Gergely Orosz and Simon Eskildsen discuss the engineering decisions behind Turbopuffer, from dependency failures at Shopify to the economics of search infrastructure. Eskildsen explains using Toxiproxy to exercise real database connections under faults, checking performance assumptions against hardware limits, and combining object storage with faster storage tiers. The conversation connects infrastructure design with product costs and operational experience.
Read five key learnings from the Frost & Sullivan 2025 Frost Radar™ for CSPM to learn how CSPM is evolving from point-in-time compliance to continuous risk management.