PCQC 训练方法让医疗对话模型学会更精准提问
PCQC: Privileged Counterfactual Question Credit for Multi-Turn Medical Dialogue
训练医疗问诊模型一直只会看最终诊断对不对，这篇给每个问题单独打分，连没问出口的问题也能学，比 GRPO 少问三分之一轮就赢了。
PCQC（Privileged Counterfactual Question Credit）针对医疗对话中 RL 训练只看最终诊断正确率、无法评估单个提问价值的问题，在训练时利用特权患者信息为未执行的备选问题构造答案。冻结的诊断评分器衡量每个问答对支持正确诊断的强度，转化为问题级别的相对信用信号。在四个医疗基准上，PCQC 平均诊断准确率达 63.10%，比 GRPO 和 ATPO 分别高出 4.38 和 4.21 个百分点，且问诊轮次比 GRPO 少 33.1%。
PCQC: Privileged Counterfactual Question Credit for Multi-Turn Medical Dialogue
Large language models (LLMs) have made substantial progress on medical question-answering, yet effective medical dialogue also requires learning to ask questions that uncover relevant patient information. To train such dialogue policies, a common pipeline combines supervised fine-tuning with reinforcement learning (RL) based on final diagnostic correctness. However, this outcome-based supervision does not directly distinguish the contributions of individual questions and provides no question-level feedback for unexecuted alternatives. To address this gap, we introduce PCQC (Privileged Counterfactual Question Credit), which uses privileged patient information during training to learn from questions never asked. During training, PCQC makes alternative questions directly comparable at the same dialogue state by using privileged patient facts to construct their answers. A frozen diagnostic scorer evaluates the diagnostic utility of each resulting question-answer pair by how strongly it supports the correct diagnosis. PCQC turns these comparisons into relative question credit that teaches the policy which questions to favor, directly supervising both executed and unexecuted questions alongside outcome-based RL without requiring complete rollouts for the unexecuted alternatives. Extensive experiments across four medical benchmarks demonstrate that PCQC achieves 63.10% mean diagnostic accuracy, outperforming GRPO and ATPO by 4.38 and 4.21 percentage points, respectively. These gains are achieved with 33.1% fewer inquiry turns than GRPO.
