Event ·Official first-hand

Decision 与 System One 模型在风湿病学考试题和诊断病例上的评测

研究用两套基准评测 System One 系列模型:扩展版 RheumaMIR(172 道西班牙专科考试题)和 RheumBench(70 个英文诊断病例,含 5-18 个候选诊断)。在考试题上 Jev 准确率达 93.0%(西语),GPT-6 Sol 最高为 99.4%,GPT-6 Luna 为 97.7%。但在诊断病例上准确率整体下降,GPT-6 Sol 为 76.9%,Jev 仅 50.0%,Laya 接近随机水平(随机为 9.6%)。Jev 在病例题上出现过度自信,概率 ≥0.90 的答案正确率只有 84.6%,而 GPT-6 Luna 同样条件下达 95.5%。

Current read

拿考试题和真实诊断病例测了一批医疗 AI 模型,结果反差挺大:考试能上 93%,真病例却掉到 50%,高置信答案也没那么靠谱。

2 sources31° AI heatLast updated 2026/10/29 15:53:05

Evidence

2 sources
Primary sourcearXiv: OpenAI
View source

Conflict check

Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.