临床诊断AI评估有新基准了！1089个真实案例、3760张影像，15个模型测试下来发现，再好的模型完全正确诊断率也不高，而且容易犯信息整合错误和视觉幻觉。
ClinMM-Bench是迄今最大的多轮多模态临床诊断评估基准，包含1089个真实临床案例和3760张医学图像，覆盖8个专科。研究使用两级评估框架，对15个代表性多模态大模型进行诊断准确性和推理质量评测。结果显示，专有模型总体准确率最高，但完全正确诊断的比例仍很低。错误分析识别出信息整合失败、知识映射错误、感知错误、过早闭合和视觉幻觉五种典型失败模式。
Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases
Clinical diagnostic evaluation should not only assess whether models can provide correct diagnoses, but also reflect the realities of clinical practice, including progressive disclosure of multimodal information, dynamic updating of diagnostic hypotheses, and continuous refinement of clinical reasoning. However, existing evaluations of multimodal large language models (MLLMs) typically rely on single-turn or isolated tasks, making it difficult to fully capture the complexity of real-world clinical diagnosis. To bridge this gap, we developed ClinMM-Bench, the largest multi-turn multimodal clinical diagnostic evaluation benchmark to date. ClinMM-Bench contains 1,089 challenging real-world clinical cases and 3,760 medical images across eight specialties. We systematically evaluated 15 representative MLLMs using a two-level evaluation framework that assessed both diagnostic accuracy and diagnostic reasoning quality. Results showed that proprietary models achieved the highest overall diagnostic accuracy, but the proportion of completely correct diagnoses remained limited across all models. In terms of diagnostic reasoning quality, current models can identify plausible diagnostic directions but still have considerable limitations in generating reliable diagnostic reasoning. Error analysis further identified five representative failure modes: information synthesis failure, knowledge mapping error, perception error, premature closure, and visual hallucination.