ResidencyRL:模拟临床环境中的强化学习

ResidencyRL: Reinforcement Learning in Simulated Clinical Environments

精选理由

医疗AI训练新方法ResidencyRL,模拟临床对话,诊断准确率涨7%,漏诊率降31%,专家更看好。

AI 摘要

ResidencyRL通过强化学习在模拟临床环境中训练医疗AI,每个轨迹最多包含60轮对话和8次工具调用。在对抗条件下,该智能体的诊断准确率达到88.0%,比基础模型提升7.0%。漏诊红旗率降低31%,表明能有效减少过早闭合。盲审临床专家在87.6%的对比中更偏好训练后的智能体。在AMIE多访基准的六个临床轴上均超过基础模型。

原文 · arXiv cs.AI

ResidencyRL: Reinforcement Learning in Simulated Clinical Environments

In medical education, physicians convert academic knowledge into clinical expertise through residency: years of training across thousands of encounters, with diverse sources of feedback and progressively greater autonomy. Much of clinical reasoning relies on the patient encounter, a dialogue in which a clinician elicits history, refines diagnostic hypotheses, and decides management under uncertainty. While large language models (LLMs) excel on static medical benchmarks, methods to optimize the full sequence of clinical decisions remain underdeveloped. We present ResidencyRL, a reinforcement learning (RL) method for training clinical artificial intelligence (AI) agents through simulated multi-turn clinical encounters (up to 60 dialogue turns and 8 tool calls per trajectory). ResidencyRL pairs the policy agent with LLM simulators capable of complex, adversarial behaviors, training against a structured reward aligned to diagnostic accuracy, management quality, communication, documentation, and safety. On held-out evaluations, the ResidencyRL agent improves diagnostic accuracy by 7.0% under adversarial conditions (88.0% vs. 81.0%) and reduces missed red flag rates by 31%, demonstrating rigorous mitigation of premature closure. Blinded expert clinicians validated these gains, preferring the trained agent in 87.6% of side-by-side comparisons. The procedural competencies transfer to unseen benchmarks: the agent outperforms the base model across all six clinical axes of the AMIE multi-visit benchmark, and shows consistent directional improvements on AgentClinic and CRAFT-MD. Our findings demonstrate that sequential clinical decision-making can be effectively learned through multi-turn RL in simulation, yielding robust, generalizable capabilities, paving the way towards clinical mastery. Prospective validation with real-world workflows remains necessary to establish clinical utility.