这个基准测的是医疗分诊模型会不会乱建议，结果发现简单加提示词远远不够，做医疗AI的人都该看看。
CARE-Bench是一个源引用的基准，用于评估医疗大模型在患者面对面的分诊任务中，每轮应给出的当前行动建议。基准包含500个病例和1,059个患者披露前缀，覆盖医疗对话、咨询和随访问题来源。在269个保留轮次上评估了11个模型，无提示时宏F1分数在31.2到50.4之间。加入最小提示后，10/11的模型表现提升，宏F1达到46.9到63.4，但仍有大量阈值错误。当正确行为是索取更多信息时，仅有33.5%的模型输出保留了该步骤。
CARE-Bench: Benchmarking Patient-Facing LLM Triage
Patient-facing medical LLMs and agents increasingly answer symptom questions before clinician contact, where the key safety question is what action the user should take next. We introduce CARE-Bench, a source-grounded benchmark that evaluates sequential patient-facing triage as a four-label per-turn current-action task. CARE-Bench contains 500 cases and 1,059 evaluated patient-disclosure prefixes reconstructed from medical dialogue, consultation, and follow-up-question sources. We evaluate 11 models on 269 held-out rounds under unprompted and minimally prompted open-ended protocols, using a fixed GPT-5.5 mapper to code each response into the four-label action space. Unprompted macro-F1 remains low, ranging from 31.2 to 50.4. Prompting improves 10 of 11 models, with prompted macro-F1 ranging from 46.9 to 63.4, but substantial threshold errors remain. Prompted models often recommend care before needed clarification is obtained; when the correct action was to ask for more information, only 33.5% of prompted outputs preserved the step. The persistence of these errors after prompting suggests that patient-facing triage is not a simple prompting problem and supports explicit evaluation of action timing before deployment.