Why it matters
A METR researcher clarifies that a 50% time horizon relates success to the human labor required for sampled tasks; it is neither agent runtime nor a guarantee that shorter work is safe to delegate. Domain choice, task selection, human timing conventions and curve fitting can materially affect estimates. Sparse long tasks and benchmark saturation widen uncertainty. Extrapolating from this measure to dependable automation or high-reliability work needs evidence about intervention, verification and the actual deployment distribution.
My takeaway: Choose a required success probability and representative tasks before using a horizon estimate. Budget human checking and recovery time, inspect confidence intervals, and avoid using a median-success benchmark as an acceptance gate for critical work.