Papers

风险控制只保证错误率不保证幅度:选择性胎儿生物测量审计

Risk Control Certifies Rates, Not Magnitudes: An Audit of Selective Fetal Biometry

Why we picked it

研究发现医疗AI的风险证书防得住错误频率防不住极端大错,胎儿头围测量里一张图就翻车,改用解剖学范围约束花0.4个覆盖点就能把最坏误差压到11毫米级。

这项审计研究针对胎儿头围(HC)测量的选择性预测管线,基于 HC18 数据集 999 张标注图像做了两次独立预注册的四路划分,每组重训分割器三次。用 Clopper-Pearson 精确风险控制校准的轮廓完整度阈值给出证书后,管线仍输出 271.3 mm 的错误值,而真实 HC 为 324.6 mm,该候选的完整度得分高达 0.94。在 α=0.10 的证书下,约 150 个答案中的单个离群值不会破坏风险界,另一组划分在不同图像上复现了 233.3 mm 的同类错误。把解剖学下界从 50 mm 提到 55 mm 仅损失 0.4 个覆盖点,就将两次划分的最坏误差降到 11.4 和 11.1 mm;而靠完整度阈值压制同样误差需付出 19.0 和 12.1 个覆盖点。在解剖学定义的 70 mm 下界处,冻结测试的 MAE 为 1.91 和 1.81 mm。

Original · arXiv cs.AI

Risk Control Certifies Rates, Not Magnitudes: An Audit of Selective Fetal Biometry

Prenatal ultrasound biometry must return a head circumference (HC) and refuse when the image cannot support one. Selective prediction trades coverage against error, but its finite-sample guarantees bound the rate of gross errors, not their magnitude. We audit a tool-grounded biometry pipeline on two separately pre-registered four-way splits of the 999 labelled HC18 images, retraining the segmenter three times on each. Calibrating the contour-completeness threshold by exact Clopper-Pearson risk control returns a certificate indistinguishable from a hand-set floor, yet the pipeline still emits a 271.3 mm error: a small, geometrically complete ring fitted to an image whose true HC is 324.6 mm. The accepted candidate scores 0.94 completeness, so the score cannot see the failure, and one outlier among $\approx 150$ answers leaves the certified risk bound ($α= 0.10$) intact. A second, untouched split reproduces the mechanism on a different image (233.3 mm). Constraining the admissibility region---the anatomical range in which the measurand is defined---removes the family instead: raising the lower bound from 50 to 55 mm costs 0.4 coverage points and cuts the worst-case error on both splits to 11.4 and 11.1 mm, whereas suppressing it through the completeness threshold costs 19.0 and 12.1 points. At an anatomically defined 70 mm the frozen tests give 1.91 and 1.81 mm MAE.