这篇论文用120万次试验测了5个开源模型，发现你反驳它就会改答案，假来源出现的时机不同效果差很多，值得看看。
研究通过5个开源权重模型、500个MedQuAD问题和120万次试验，分析医学谄媚行为。结果发现，虚假来源伴随提问时使谄媚率提高2.0倍，但在模型回答后出现则使谄媚率减半。谄媚程度在不同问题间的差异是模型间差异的67倍对3倍。思维链追踪显示，重新审视自己答案的模型会让步，而推理医学事实的模型坚持。
Why LLMs Give In: Conversational Factors and Reasoning Behind Medical Sycophancy
A language model that abandons a correct medical answer under user pushback is more dangerous than one that was simply wrong, because it lends the credibility of a correct answer to the user's misinformation. Such model behavior, described as medical sycophancy, is usually reported as a single rate per model, but we find it is a property of the conversation, not the model. We study medical sycophancy in language models with a fully crossed factorial design over four conversational factors, user role, the evidence behind a false claim, whether the challenge precedes or follows the model's answer, and whether the correct answer is grounded in the prompt, across five open-weight models and 500 MedQuAD questions (1.2M trials). The factors interact sharply: fabricated sources raise sycophancy 2.0x when they accompany the question but halve it once the model has answered, so the same evidence helps or hurts depending only on timing. Sycophancy varies far more across questions than across models (67x vs. 3x), so a single rate reflects the conversation and the questions sampled as much as the model. Chain-of-thought traces explain why. Models that re-examine their own prior answer concede, while those that reason about the medical facts hold, and only a model that has already answered can spend a round auditing the fabricated source.