Full Archive · Page 37

Research archive, page 37

Browse entries 865–888 of 1704. Return to the first page to search and filter the complete collection.

Agent CLI design: discover schemas, constrain output, and separate credentials video thumbnail Play video
AI Engineer October 9, 2026 video

Agent CLI design: discover schemas, constrain output, and separate credentials

Pedro Lopez explains how command interfaces can help agents act predictably: consistent resource-and-operation names, JSON arguments and results, runtime schema discovery, and explicit errors. Airbyte’s companion CLI documentation makes the sequence concrete: identify the configured connector, describe its schema, then execute a narrowly scoped operation.

Codex app-server integration: track turns, approvals, and terminal events video thumbnail Play video
AI Engineer / OpenAI October 9, 2026 video

Codex app-server integration: track turns, approvals, and terminal events

Dominik Kundel describes embedding the Codex harness through app-server rather than rebuilding its agent loop. The integration uses JSON-RPC requests, notifications, threads, and turns; a client must handle progress and server-initiated approval requests as well as final output. Official documentation supports generating protocol types from the installed binary so client and server versions agree.

Agent sandboxes: keep credentials outside the harness and scope tools by stage video thumbnail Play video
AI Engineer October 6, 2026 video

Agent sandboxes: keep credentials outside the harness and scope tools by stage

Jim Clark’s architecture separates research, fact-checking and publishing into stages with different permissions. Sandboxed agents use permitted tools through a gateway, while credentials remain outside the agent environment; signing authority belongs only to the stage that needs it. The design aims to reduce persistent secrets and limit what a compromised stage can do.

Durable human approval: resume agent workflows after a worker restart video thumbnail Play video
AI Engineer October 4, 2026 video

Durable human approval: resume agent workflows after a worker restart

Melanie Warrick models a human decision as an asynchronous signal to a durable workflow. External model and tool calls belong in activities, while replayable orchestration records progress and waits for approval without keeping a worker occupied. In the demonstration, a worker stops while an approval is pending; the approval is recorded and processing resumes after restart, while other orders can continue.

Managed agent sandboxes: separate conversation state, files, and credentials video thumbnail Play video
AI Engineer October 3, 2026 video

Managed agent sandboxes: separate conversation state, files, and credentials

Ivan Leo explains the distinction between conversation continuity and a persistent execution environment in Gemini’s managed-agent APIs. An interaction identifier carries conversational state; an environment identifier selects the sandbox and its files. Primary documentation adds explicit environment lifecycle controls and a network proxy that can attach credentials to configured destinations without placing those secrets inside the sandbox.

METR March 19, 2026 analysis

Use tabletop simulations to find bottlenecks before expanding agent autonomy

Three METR staff spent two hours simulating how their research organization might work with agents capable of roughly 200-hour tasks. The exercise advanced through two hypothetical workdays and exposed choices about prioritization, delegation, context and review. Participants used present-day research needs but assumed future agent capabilities.

METR February 17, 2026 analysis

Transcript-based time savings are an upper bound, not a productivity experiment

A METR exploratory analysis uses 5,305 coding-agent transcripts from seven staff to estimate time saved on AI-assisted tasks. A model estimates counterfactual human effort, while message activity approximates actual human time. The judge was checked against only 34 human estimates, and transcript selection excludes work done without AI. Task substitution, specialization and uncertain success judgments further limit interpretation.

METR July 14, 2025 analysis

Human-time horizons need validation within each evaluation domain

METR’s July 2025 analysis asks whether task-completion horizons transfer beyond its original software evaluations. It reanalyzes nine existing benchmarks, fitting success against human completion time where suitable measurements are available. The relationship is useful in several settings but differs across domains and weakens when proxies such as video length or task fees stand in for human effort.

METR August 6, 2024 guide

Agent benchmarks: inspect task weighting and budget-dependent failure

METR’s August 2024 evaluation update describes expanding its task set, using human completion time to organize difficulty and weighting task families to limit domination by families with many variants. It examines how success changes with additional agent time and where progress levels off under a simple harness. Sparse human baselines and the mixture of easy and hard tasks limit interpretation.

Prepare retrieval data with explicit inclusion rules and deletion checks video thumbnail Play video
AI Engineer October 5, 2026 video

Prepare retrieval data with explicit inclusion rules and deletion checks

Leo Platzer and Jeff Koss demonstrate a document-preparation workflow using a fictional legal-operations collection. Classification produces metadata that drives inclusion rules, followed by source inspection and human corrections before retrieval. Sensitivity, duplicate and superseded documents are selection concerns rather than problems that embeddings alone solve. Scheduled synchronization must also account for deleted source files.

Open-model serving: evaluate cache locality, decoding and quantization together video thumbnail Play video
AI Engineer October 3, 2026 video

Open-model serving: evaluate cache locality, decoding and quantization together

Sujee Maniyam and Dylan Bristot explain why identical open-model weights can behave differently across serving stacks. Their overview covers routing requests toward reusable KV state, offloading that state from GPU memory, separating prefill from decode, and using a small draft model whose tokens a larger model verifies. Quantization and application-specific draft training introduce additional quality and workload dependencies.