Full Archive · Page 67

Research archive, page 67

Browse entries 1585–1597 of 1597. Return to the first page to search and filter the complete collection.

METR November 22, 2024 analysis

RE-Bench compares research agents and human experts under explicit compute budgets

METR’s November 2024 RE-Bench release provides seven research-engineering environments, human-expert attempts and agent transcripts. Tasks involve objectives such as optimizing kernels, recovering model performance and fitting scaling laws under specified resources. Results depend strongly on time allocation: agents often benefit from selecting the best of many short attempts, while humans benefit from longer continuous work.