Dwarkesh Patel · November 5, 2026

Clocktower Bench: separate deception from detection and control the game harness

Clocktower Bench: separate deception from detection and control the game harness video thumbnail
Why it matters

Dwarkesh Patel's Blood on the Clocktower evaluation uses a social-deduction game to inspect multi-agent coordination, bluffing and evidence assessment. One model fills the good-team seats and another the evil-team seats, with separate conversations and only permitted in-game communication. Each deal is replayed with sides swapped. A scripted storyteller samples legal outcomes using preset weights, reducing discretionary intervention by a human or model judge. The public methodology describes equal reasoning settings, token and speech limits, separate detection and deception ratings, and uncertainty estimated by resampling whole paired games. Private belief probes track how good players' suspicions change without feeding those answers back into play. The original video discusses a fixed study, while the live site has since added newer models, so its current rankings should not be treated as the video's frozen results. Success at permitted bluffing in this game does not establish a model's propensity to deceive real users or its performance against humans.

My takeaway: When adapting this evaluation, pin the harness and model versions, preserve side-swapped pairs, report uncertainty and inspect complete game traces. Test communication, role balance and memory changes as separate ablations; avoid interpreting a small leaderboard gap as a reliable general capability difference.
Keep exploring

More curated notes connected through Model Evaluation and Agent Security.

OpenAI News · framework

Pacing model development in an era of cyber-critical capabilities

OpenAI says preliminary evidence that Astra may meet its Critical cybersecurity threshold led it to pause frontier reinforcement-learning work for two weeks and keep its largest planned run on hold. New safeguards include stronger workload and network isolation, continuous boundary testing, token-level monitoring that escalates suspicious tool activity, and broader alignment checks for deception, reward hacking, and unauthorized access.

OWASP GenAI Security Project · guide

OWASP Top 10 for Agentic Applications for 2026

OWASP's community guide organizes agentic-system risk into ten categories, including goal hijacking, tool misuse, identity and privilege abuse, memory poisoning, insecure inter-agent communication, cascading failures, and rogue-agent behavior. It provides a shared taxonomy and mitigation starting point rather than a certification checklist or evidence that a deployed system is secure.

OpenAI News · framework

A blueprint for democratic governance of frontier AI

OpenAI proposes a three-part U.S. frontier-AI governance model: harmonize emerging state safety laws into a federal baseline, strengthen CAISI as an evaluation and standards institution, and coordinate a broader resilience program. Proposed controls include severe-risk evaluations, transparency reports, independent audits, safety-incident reporting, model-weight security, whistleblower protection, and periodic technical assessments.