Browse entries 385–408 of 1704. Return to the first page to search and filter the complete collection.
Play video
Morgan Suo uses a document-research task to explain why faster token generation can still produce slower work. Source retrieval, retries, missed evidence, human corrections, and tail latency affect completion. The talk compares separate model and serving tests; these do not establish a speedup on the research task itself. Its useful output is a task-level acceptance procedure.
Play video
Denys Linkov describes duplicate, mixed-patient, and missing records that complicate claims analysis. The linked MLCR benchmark tests synthetic medical files and scores accuracy, completeness, and conciseness separately. Its public easier cases differ from the private harder leaderboard set. Conditional accuracy excludes some failed responses, so it must be read alongside overall pass rate and coverage.
Play video
Langfuse’s banking-assistant workshop turns expert trace reviews into failure categories, evaluators, and prompt-change checks. A masking demo redacts a card number, but official documentation limits each hook’s coverage and warns that other exporters can receive unmodified spans. The demo triggers CI after assigning a production label, so its passing checks alone do not establish a safe release sequence.
Play video
Nick Nisi and Zack Proser demonstrate an app-specific CLI that gives agents browser state, storage checks, and verification results. Their snack-waitlist example can build successfully while losing entries on reload or claiming a confirmed seat it never granted. The session also exposes stale evidence and verifier problems. Passing a health check or a model judge is only evidence for the behavior actually checked.
Play video
The Managed Agents research-desk workshop turns a requested deliverable into an explicit rubric and an iterative evaluation loop. Anthropic’s documentation describes a grader with a separate context window, per-criterion feedback, and events for inspecting the result. Separating the grader’s context can reduce direct influence from the worker’s reasoning, but a satisfied rubric still does not establish that every fact or source in the artifact is correct.
Play video
Hrishi Olickel describes evolving extraction schemas around the decisions a dataset must support. Designer, worker, and optimizer stages propose definitions, process records, and revise failures. Typed outputs and model agreement help organize checks, but shared errors in dates, signs, or labels can survive them. The private pipeline illustrates a workflow; its cost and performance claims are not independently verified.
Play video
Kevin Orellana describes using a coding agent through an MCP wrapper to exercise sandbox and browser workflows, storing results in a structured database. Reported failures include encoding, file handling, and resource exhaustion. The useful testing pattern separates interface defects from execution limits and preserves a reproducible case for each finding. An agent-generated test count is not evidence of coverage, and a passing test does not prove containment.
Play video
Nnenna Ndukwe uses a small FastAPI payments app to connect task planning, repository rules, behavior tests, static checks, and pull-request review. The public SignalPay workshop supplies planning templates, local verification commands, CI gates, and a process for resolving review findings. A passing local run is one checkpoint; changes made during remediation require verification again. The example is a training application, not a production payments design.
Play video
Du’an Lightfoot and Khaja Omer provide vLLM notebooks for measuring streaming latency, prefill and decode, KV-cache use, quantization, speculative decoding, and concurrent requests. Their live workshop includes an optimization that slows the tested configuration down. The reusable method is to establish a baseline, change one serving parameter, and measure quality and latency alongside throughput. Running the lab requires a compatible GPU-backed endpoint.
Play video
Dhruv Nathawani builds search-oriented training examples from structured seeds, generated questions, and tool-use trajectories. NVIDIA Data Designer’s public code and documentation expose dependency-aware generation, MCP tool access, previews, and deterministic or model-based validators. The workshop highlights a central limitation: a plausible answer or an approving model judge can still be wrong, particularly when the source facts have changed.
Play video
Zixuan Li introduces GLM-5.2 through its coding and agentic capabilities, adjustable thinking budget and open-weight deployment options. He separates the model from Z Code, a coding harness that also accepts other models. The talk explains the roles of local inference, domain fine-tuning and ecosystem tooling; its benchmark placements are Z.ai’s reported comparisons with incomplete evaluation conditions.
Play video
Kevin Hou describes Antigravity’s move toward model-led teams that create specialist subagents, listen for events and generate task-specific interfaces. Examples include an operating-system kernel demonstration and parallel investigations of evaluation differences. The reported costs and completion times belong to particular demonstrations; generated hypotheses and working applications still require independent review.
Play video
Erina Karati’s publisher notes use a simulated game village to expose memory failures: agents can retain a topic while losing its source, certainty or implications for later plans. The proposed evaluation records observations, memory writes, retrievals and belief changes across complete scenarios. It freezes the harness and evaluator while searching a limited policy space. A corrected rumor example is illustrative; the talk does not claim repeated evidence of general improvement.
Play video
Daksh Gupta compares likely agent-written and human-written pull requests using Greptile’s enterprise review data. Authorship is inferred from metadata, while reverts, flagged issues and review rounds serve as quality proxies. He reports broadly similar aggregate results with different failure patterns, then describes validating related code and running applications in sandboxes. The observational comparisons do not control all differences in task selection or difficulty.
Play video
Jason Ma’s publisher notes describe using a video-based progress model to find robot failures, then collecting human demonstrations of recovery and fine-tuning the policy. Napkin folding exposes both bad grasps and quality failures that a nominal task-completion measure can miss. The reported 24-hour success rate belongs to a particular folding evaluation, with no sample size supplied in the notes. Transfer to a new site and recovery from unfamiliar mistakes require separate evidence.
Play video
Armen Aghajanyan describes Perceptron AI’s effort to combine perception, reasoning and robot control. Task-dependent token routing focuses computation, while zooming, tiling and revisiting video intervals help gather visual evidence. Joint video and control training aims to reduce dependence on teleoperation. The talk leaves parts of the training objective undisclosed and identifies temporal reliability and severe visual disruption as unresolved challenges.
Play video
Noam Brown discusses agent cooperation, alignment and recursive self-improvement with Dwarkesh Patel. The interview examines why cooperation between agents does not establish alignment with users, and why longer tasks complicate safety evaluation before release.
Play video
Conference talk on secure AI agents, focusing on how tool use, identity, and execution boundaries change when assistants can act across systems.
Google integrated computer use into Gemini 3.5 Flash so agents can act across browser, mobile, and desktop environments. Optional enterprise safeguards can require confirmation for sensitive actions or stop a task when indirect prompt injection is detected.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
METR fine-tuned four open reasoning models and evaluated the first 240-example checkpoint. Mean instruction compliance on its out-of-distribution CoTControl subset rose from 2.9% to 8.8%. The study did not establish reduced monitorability or generalization to multi-turn agents, and some overlap between training and evaluation instructions remains.
Play video
This AI Explained video reviews a major AI development through the lens of governance and responsible deployment. It is useful context for AI engineering, evaluation, governance, and operational risk.
OECD Due Diligence Guidance for Responsible AI helps businesses manage AI risks, meet global standards and build trustworthy AI value chains.
Play video
This AI Explained video reviews a major AI development through the lens of governance and responsible deployment. It is useful context for AI engineering, evaluation, governance, and operational risk.