这篇论文告诉你当前最强模型在临床推理上有多弱：关键问题正确率不到一半，而简单问题却能答得很准。适合想了解模型真实局限的读者。
研究者设计了5个临床场景（麻醉、内科/家庭医学、急诊、产科），共184条MECE评分标准。GPT 5.4、Claude Opus 4.7、Gemini 3.1 Pro的平均rubric通过率分别为0.39、0.47、0.37。最关键的权重5标准通过率仅32.4-41.7%，而低权重（1）标准通过率80-90%。108个关键标准中52%未被任何模型满足。LLM自动评分器在92.8-94.7%的标注上再现专家判断。
A rubric-based controlled comparison of frontier language models on expert-authored clinical reasoning tasks
Multiple-choice medical benchmarks are increasingly saturated, and recent rubric-based evaluations such as HealthBench have shown that open-ended clinical performance is far from solved - its "Hard" subset top score remains 32%. We present a small, deliberately difficult evaluation dataset of five clinician-authored clinical scenarios spanning four specialties (anaesthesia, internal/family medicine, emergency medicine, and obstetrics), each accompanied by an atomic, weighted, MECE rubric (25-62 criteria per task; 184 criteria total) authored from a clinician-drafted golden answer. We evaluate three frontier models: GPT 5.4, Claude Opus 4.7, and Gemini 3.1 Pro. Mean rubric pass rates were 0.47 (Claude), 0.39 (GPT), and 0.37 (Gemini). The central finding is an inversion of clinical priority: the highest-weighted (weight-5, critical) criteria passed at only 32.4-41.7%, while low-stakes weight-1 criteria passed at 80-90%. 56 of 108 critical (weight-5) criteria (52%) were satisfied by no model. Three LLM autoraters reproduced expert met/not-met labels on 92.8-94.7% of 552 graded criteria. We position this as a methods-and-preliminary-findings contribution: the five tasks demonstrate a scalable, defensible pipeline ready to develop into a large-scale benchmark.