Why it matters
Drawing on nine years of cross-industry ML deployments, Megan Robertson explains why a statistically accurate model can still fail to deliver in production. The session moves beyond offline performance to scoping, organizational failure modes, monitoring, maintainability, and the operational conditions required for a model to keep producing useful results.
My takeaway: Set production acceptance criteria alongside model metrics: define the decision and owner, validate the data and workflow assumptions, instrument model and business outcomes, and plan retraining, rollback, and maintenance before launch. A strong offline score is evidence about a model, not proof that the surrounding system will succeed.