AI Engineer · August 19, 2026

Shipping AI to a Million Patients Without an A/B Test — Jared Joselowitz, Ufonia

video AI Engineering Model Evaluation Prompt Engineering

Shipping AI to a Million Patients Without an A/B Test — Jared Joselowitz, Ufonia video thumbnail

Why it matters

You cannot A/B test on patients, because randomizing someone into the worse variant is unethical and often illegal. You cannot undo a call once spoken. And a vendor's benchmark number is not a defense at a post incident review.

My takeaway: Shipping AI to a Million Patients Without an A/B Test — Jared Joselowitz, Ufonia is an agent-security signal. The practical read is that autonomy, memory, tool permissions, and third-party integrations are the control surface that needs threat modeling and monitoring.

Keep exploring

More curated notes connected through AI Engineering and Model Evaluation.

OpenAI News · framework

Pacing model development in an era of cyber-critical capabilities

OpenAI says preliminary evidence that Astra may meet its Critical cybersecurity threshold led it to pause frontier reinforcement-learning work for two weeks and keep its largest planned run on hold. New safeguards include stronger workload and network isolation, continuous boundary testing, token-level monitoring that escalates suspicious tool activity, and broader alignment checks for deception, reward hacking, and unauthorized access.

OpenAI News · framework

OpenAI’s Frontier Governance Framework

OpenAI's 22-page Frontier Governance Framework maps its frontier-model processes to California's Transparency in Frontier AI Act and the EU AI Act's general-purpose AI code. It documents lifecycle risk assessment, cyber-offense and other risk tiers, mitigation and residual-risk decisions, critical-incident handling, security risk management, model reporting, external review, responsibility allocation, and change control.

OpenAI News · framework

Cybersecurity in the Intelligence Age

OpenAI proposes a five-pillar strategy for AI-enabled cyber defense: tiered access for trusted defenders, faster government-industry coordination, stronger protection of frontier models and infrastructure, risk-scaled deployment monitoring, and broader defensive support for individuals and small organizations.