MedPhysBench 基准发布：评估大模型医学物理计算与错误检查能力
MedPhysBench: A Benchmark for Evaluating Language Models on Medical Physics Calculations and Error Detection
医学物理计算出新基准了，1200 道题里 Claude 系列算得准但查错翻车，做医疗 AI 的可以看看自己模型的真实水平。
MedPhysBench 面向放射治疗、影像、核医学和辐射防护领域，设置 10 类计算题生成器，参考答案由实现 TG-51 等协议的代码给出。在 1200 题的保留测试集上，12 个模型的计算得分在 33.3% 到 100.0% 之间。五个模型在错误检查中检出至少 96.7% 的植入错误，误报不超过 4%。但 Claude Sonnet 5.5 在 20 道等中心 MU 题中有 19 道漏掉平方反比校正，Claude Opus 5.5 接受了 20 份错误结果中的 6 份。
MedPhysBench: A Benchmark for Evaluating Language Models on Medical Physics Calculations and Error Detection
Medical physicists turn a prescription into a delivered dose and check one another's calculations, because one error in a beam calibration or a monitor-unit (MU) calculation can affect every patient treated with it. MedPhysBench has ten seeded generators of calculation items in radiation therapy, diagnostic imaging, nuclear medicine and radiation protection, up to clinical tasks such as a TG-51 calibration worksheet. Reference answers come from code implementing the governing protocols, and a grader converts units and applies tolerance bands. From the same inputs, separate checking items test error detection, showing a colleague's worksheet or only the colleague's result, half correct and half with one planted error. On a held-out test split of 1,200 items (half for two models), 12 models scored 33.3 to 100.0% on calculation. Of the nine answers graded incorrect for the four best (99.3-100.0%), five contained model errors and four followed NCRP Report 151 on points our items misstate or leave open. Below them, most models did worst on level-3 clinical tasks, and Claude Sonnet 5.5 omitted the inverse-square correction in 19 of 20 isocentric MU items. Five models detected at least 96.7% of mostly large planted errors with at most 4% false alarms; for four of the other seven, checking only the result raised false alarms significantly. In an exploratory targeted test with 20 public cases of an omitted inverse-square correction, the three best models tested caught it in every worksheet, and three weaker models accepted 16 to 18 of 20 planted worksheets, one of them while applying the correction in most of its own calculations. Claude Opus 5.5 accepted 6 of 20 wrong results, although it applied the correction in all 20 of its own. We report the effect of two problems in our shielding items, found after the runs, and release the code, per-item outcomes and targeted-test responses.