Play video
AI Engineer session on Real World Development with GitHub Copilot and VS Code, presented by Harald Kirschner, Christopher Harrison. It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on How to build world-class AI products, presented by Sarah Sachs (AI lead @ Notion) & Carlos Esteban (Braintrust). It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on RAG Evaluation Is Broken! Here's Why (And How to Fix It) - Yuval Belfer and Niv Granot. It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on How to build the world's fastest voice bot: Kwindla Hultman Kramer. It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on How Deep Research Works - Mukund Sridhar & Aarush Selvan, Google DeepMind. It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on How to Construct Domain Specific LLM Evaluation Systems: Hamel Husain and Emil Sedgh. It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on Customized, production ready inference with open source models: Dmytro (Dima) Dzhulgakov. It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on Understanding AI Stakes to Break Production Code: Philip Rathle. It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on How to evaluate a model for your use case: Emmanuel Turlay. It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on [Full Workshop from Microsoft] Github Copilot - The World's Most Widely Adopted AI Developer Tool. It adds practical context for how teams are building and operating AI systems in production.
Play video
AI Engineer session on GitHub Copilot: The World's Most Widely Adopted AI Developer Tool. It adds practical context for how teams are building and operating AI systems in production.
Play video
Pedro Lopez explains how command interfaces can help agents act predictably: consistent resource-and-operation names, JSON arguments and results, runtime schema discovery, and explicit errors. Airbyte’s companion CLI documentation makes the sequence concrete: identify the configured connector, describe its schema, then execute a narrowly scoped operation.
Play video
Dominik Kundel describes embedding the Codex harness through app-server rather than rebuilding its agent loop. The integration uses JSON-RPC requests, notifications, threads, and turns; a client must handle progress and server-initiated approval requests as well as final output. Official documentation supports generating protocol types from the installed binary so client and server versions agree.
Play video
Jim Clark’s architecture separates research, fact-checking and publishing into stages with different permissions. Sandboxed agents use permitted tools through a gateway, while credentials remain outside the agent environment; signing authority belongs only to the stage that needs it. The design aims to reduce persistent secrets and limit what a compromised stage can do.
Play video
Melanie Warrick models a human decision as an asynchronous signal to a durable workflow. External model and tool calls belong in activities, while replayable orchestration records progress and waits for approval without keeping a worker occupied. In the demonstration, a worker stops while an approval is pending; the approval is recorded and processing resumes after restart, while other orders can continue.
Play video
Ivan Leo explains the distinction between conversation continuity and a persistent execution environment in Gemini’s managed-agent APIs. An interaction identifier carries conversational state; an environment identifier selects the sandbox and its files. Primary documentation adds explicit environment lifecycle controls and a network proxy that can attach credentials to configured destinations without placing those secrets inside the sandbox.
System prompts form the foundation of generative AI applications. A system prompt is a collection of instructions and operational context provided to a large language model (LLM) that shapes how the model behaves and interacts with users and tools.
Three METR staff spent two hours simulating how their research organization might work with agents capable of roughly 200-hour tasks. The exercise advanced through two hypothetical workdays and exposed choices about prioritization, delegation, context and review. Participants used present-day research needs but assumed future agent capabilities.
A METR exploratory analysis uses 5,305 coding-agent transcripts from seven staff to estimate time saved on AI-assisted tasks. A model estimates counterfactual human effort, while message activity approximates actual human time. The judge was checked against only 34 human estimates, and transcript selection excludes work done without AI. Task substitution, specialization and uncertain success judgments further limit interpretation.
AI could help defenders of critical infrastructure identify the vulnerabilities that attackers might exploit—and close them before they are exploited.
METR’s July 2025 analysis asks whether task-completion horizons transfer beyond its original software evaluations. It reanalyzes nine existing benchmarks, fitting success against human completion time where suitable measurements are available. The relationship is useful in several settings but differs across domains and weakens when proxies such as video length or task fees stand in for human effort.
METR’s August 2024 evaluation update describes expanding its task set, using human completion time to organize difficulty and weighting task families to limit domination by families with many variants. It examines how success changes with additional agent time and where progress levels off under a simple harness. Sparse human baselines and the mixture of easy and hard tasks limit interpretation.
Play video
Leo Platzer and Jeff Koss demonstrate a document-preparation workflow using a fictional legal-operations collection. Classification produces metadata that drives inclusion rules, followed by source inspection and human corrections before retrieval. Sensitivity, duplicate and superseded documents are selection concerns rather than problems that embeddings alone solve. Scheduled synchronization must also account for deleted source files.
Play video
Sujee Maniyam and Dylan Bristot explain why identical open-model weights can behave differently across serving stacks. Their overview covers routing requests toward reusable KV state, offloading that state from GPU memory, separating prefill from decode, and using a small draft model whose tokens a larger model verifies. Quantization and application-specific draft training introduce additional quality and workload dependencies.