Why it matters
Denys Linkov describes duplicate, mixed-patient, and missing records that complicate claims analysis. The linked MLCR benchmark tests synthetic medical files and scores accuracy, completeness, and conciseness separately. Its public easier cases differ from the private harder leaderboard set. Conditional accuracy excludes some failed responses, so it must be read alongside overall pass rate and coverage.
My takeaway: Start with synthetic records and expert-reviewed answers. Vary irrelevant context while fixing the model and configuration; record missed facts separately from invented ones. Inspect empty, overlong, and unscored responses instead of dropping them from reports. Calibrate judges against source documents and human review, and keep public development cases separate from held-out tests.