医疗AI开发者终于有了一个能评估LLM临床推理过程的基准——ClinEnv不仅看结果,还看信息收集策略,做临床决策系统的团队值得关注。
ClinEnv 是一个交互式基准测试,用于评估大语言模型在真实住院病例中的临床决策能力。它模拟了医生在不确定性下逐步收集信息并做出不可逆决策的过程,每个病例被自动构建为有序的决策阶段,模型必须主动查询四个专业智能体后才能做出诊断和治疗决策。评估结果显示,最强模型仅达到0.31的决策F1分数,且结果质量与过程质量严重脱钩——模型在恢复出院诊断上表现较好(0.51 F1),但在管理行动上很差(0.17 F1),且会持续发出冗余查询。ClinEnv 揭示了传统结果导向评估无法捕捉的信息获取差距。
ClinEnv: An Interactive Multi-Stage Long Horizon EHR Environment for Agents
Clinical practice is not the selection of an answer from enumerated options: a physician gathers heterogeneous information incrementally and commits to sequential, irreversible decisions under uncertainty. Static benchmarks cannot probe and existing interactive medical benchmarks each compromise on at least one of them. We present ClinEnv, an interactive benchmark that evaluates LLMs as attending physicians over real inpatient admissions under a paradigm we term Longitudinal Inpatient Simulation. Each case is automatically constructed into an ordered sequence of decision stages; at every stage the model must actively query four specialized agents before committing to medications, procedures, and diagnoses. ClinEnv scores both what the model decides, through deterministic ontology-grounded matching, and how it gathers information. Across seven models, the strongest reaches only 0.31 decision F1, and outcome quality is sharply decoupled from process quality. Difficulty concentrates in management decisions and later stages, where models recover discharge diagnoses far more reliably than management actions (0.51 vs. 0.17 F1) and continue to issue redundant queries as cases progress. ClinEnv makes this information-acquisition gap, invisible to outcome-only evaluation, directly measurable.