医疗AI开发者注意了:推理模型在临床文档任务上可能适得其反,做医疗NLP的团队在部署前务必做任务专属评估,别盲目相信推理能力。
一项新研究评估了GPT-5.4、DeepSeek-V4-Flash和Gemma-4-E4B在临床SOAP笔记生成中的表现,发现启用推理能力反而显著降低了GPT-5.4的输出质量。研究使用OMI Health、ACI-Bench和PriMock57三个数据集,通过2x2实验设计控制推理和检索增强生成(RAG)两个因素。结果显示,非推理配置的GPT-5.4整体质量最高,而DeepSeek-V4-Flash在推理配置中表现最佳。同源RAG带来模型依赖的小幅提升,但推理能力不应被假设为能自动改善对保真度敏感的临床文档生成。
When Reasoning Hurts: Source-Aware Evaluation of Frontier LLMs for Clinical SOAP Note Generation
Reasoning-enabled LLMs perform strongly on medical reasoning benchmarks, but it remains unclear whether these gains transfer to structured clinical documentation; we investigate this question using SOAP note generation from clinical dialogue in a source-aware benchmark spanning OMI Health, ACI-Bench, and PriMock57. We evaluate GPT-5.4, DeepSeek-V4-Flash, and Gemma-4-E4B in a controlled 2x2 design that independently toggles provider-native reasoning and same-source retrieval-augmented generation (RAG). Outputs are assessed using seven automatic metrics alongside two reference-aware LLM judges. Both evaluation approaches agree that a non-reasoning GPT-5.4 configuration achieves the highest overall quality, while DeepSeek-V4-Flash performs best among reasoning-enabled configurations. Enabling reasoning significantly degrades GPT-5.4 performance across all three datasets, whereas same-source RAG yields smaller, model-dependent improvements. Overall, the findings indicate that stronger reasoning capability should not be assumed to improve fidelity-sensitive SOAP note generation without dedicated, task-specific evaluation.