Full Archive · Page 73

Research archive, page 73

Browse entries 1729–1738 of 1738. Return to the first page to search and filter the complete collection.

METR November 22, 2024 analysis

RE-Bench compares research agents and human experts under explicit compute budgets

METR’s November 2024 RE-Bench release provides seven research-engineering environments, human-expert attempts and agent transcripts. Tasks involve objectives such as optimizing kernels, recovering model performance and fitting scaling laws under specified resources. Results depend strongly on time allocation: agents often benefit from selecting the best of many short attempts, while humans benefit from longer continuous work.