Papers

NPCBench 发布:鼻咽癌临床全流程多模态推理基准

NPCBench: A Clinical Apprenticeship Benchmark for Multimodal Care-Pathway Reasoning in Nasopharyngeal Carcinoma

Why we picked it

一个专测鼻咽癌诊疗全流程的医学基准,30 个模型里最强的严格成功率也只有 23%,能看出模型和真实医生差距在哪。

NPCBench 是一个针对鼻咽癌(NPC)的临床学徒式基准,覆盖指南知识、病例规划、纵向随访到开放式会诊。数据集包含 4,777 个组件与问诊条目,以及 100 名患者的 CarePath 队列、417 个评分诊疗任务。评测采用 episode 级评估,要求同一患者的诊断与治疗答案联合正确。在 30 个受测模型中,21 个多模态模型的严格 CarePath 成功率最高仅 23%,显示单任务强表现无法保证全程诊疗正确。

Original · arXiv cs.AI

NPCBench: A Clinical Apprenticeship Benchmark for Multimodal Care-Pathway Reasoning in Nasopharyngeal Carcinoma

Current medical benchmarks typically break down clinical competence into isolated factual questions, single-image tasks, or static case vignettes. These tests score individual answers, but reveal less about whether a model can integrate subspecialty knowledge with multimodal evidence and track a patient's condition throughout a course of care. To close this gap, we introduce NPCBench, a clinical apprenticeship benchmark for nasopharyngeal carcinoma (NPC). NPCBench mirrors specialist training, from guideline and foundational knowledge to case planning, longitudinal review, CarePath integration, and open-ended consultation. It contains 4,777 component and consultation items and a 100-patient CarePath cohort with 417 scored diagnostic and care tasks. The episode-level evaluation tests whether diagnostic and care answers are jointly correct within a care pathway. By linking stepwise tasks to patient-level care trajectories, NPCBench evaluates whether models ground decisions in visual evidence, interpret changes in patient state, and meet clinical requirements in consultation. Across 30 evaluated models, we observe a composition gap: strong performance on individual tasks does not ensure jointly correct diagnostic and care answers for the same patient. Among 21 multimodal models, strict CarePath success reaches at most 23%. Paired controls and physician calibration further examine evidence grounding, longitudinal revision, and consultation judging. Overall, NPCBench broadens the evaluation of medical artificial intelligence beyond static question-answering paradigms to an apprenticeship-oriented assessment of clinical workflows, thereby offering a testbed for evidence-based and temporally coherent reasoning in oncology care.