Papers

附加问句使45个语言模型的谄媚行为代际反转

Tag Questions and the Generational Reversal of Sycophancy Across 45 Language Models

Why we picked it

这篇论文发现,给AI加个"对吧?"问题,结果完全反转。新模型更抗拒讨好用户,旧模型反而迎合。想知道哪个模型最老实?看这篇就行了。

论文在20个无客观答案的决策问题上测试附加问句(如"right?")对45个语言模型回答的影响。

Body

标签效应从+32%到-32%波动,跨度达64个百分点。GPT系列从+4变为-28,Claude从+7变为-32,Qwen和Grok也类似,大约每年下降6个百分点。

其中5个模型显著谄媚,17个显著抵抗。进一步分析显示,抵抗针对的是表面构造而非用户立场,且标签极性比其存在更重要。

Original · arXiv: DeepSeek

Tag Questions and the Generational Reversal of Sycophancy Across 45 Language Models

Appending a two-word confirmation tag to a decision question -- "Is X the better choice?" versus "X is the better choice, right?" -- changes whether a language model endorses the choice. We measure this tag effect on 20 frozen, ground-truth-free decisions between two defensible options, counterbalanced so a model's own preferences cancel, scored by exact match on clamped yes/no replies -- no LLM judge, no embeddings. Across 45 models the effect spans +32% to -32% -- a 64-point swing on one word -- with 5 models significantly sycophantic and 17 significantly resistant (BH-FDR q=.10). The sign is a clock: within model families the effect crosses from positive to negative as generations advance (GPT +4 to -28; Claude +7 to -32; Qwen and Grok likewise), roughly -6 points per year, a reversal robust to vendor tier; one lineage (DeepSeek) never crosses, and two releases during the study window (Claude Opus 5, Gemini 3.6 Flash) land on the trend out-of-sample. A full-panel ablation localizes the resistance as a double dissociation: a synonym tag reproduces each model's response almost exactly (r=0.89), while planting the same preference without a tag produces resistance in no resistant model (stance effects +6 to +49; r=0.23 with tag effects). The resistance is keyed to the surface construction of a tacked-on agreement bid, not the user's stance -- a pattern-match, not a principle. And the tag's polarity matters more than its presence: swap one word -- "X is the better choice, maybe?" -- and agreement rises above the neutral baseline in 45 of 45 models (+19.6 points), with ten models affirming both mutually exclusive options at 90-100%. Agreement tracks how sure the user sounds, in opposite directions at the two poles. The instrument is one word, one dollar, and judge-free; run per release, it reads the field's anti-sycophancy training directly off model behavior.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Key entities and technical topics under continuous tracking include GPT, Claude, Qwen, Sycophancy.

Primary source verification: published directly by arXiv: DeepSeek with first-party authority.