三机制框架:LLM 如何处理训练知识与上下文冲突
Three Regimes of Context-Parametric Conflict: A Predictive Framework and Empirical Validation
这个框架解决了 LLM 行为研究中一个长期矛盾的谜题——为什么有的实验说模型死记硬背,有的说模型灵活跟随。做 LLM 评测或 prompt 工程的人,看完能更精准地预测模型在知识冲突场景下的行为,建议直接读原文的机制划分部分。
该研究提出一个三机制框架,解释大语言模型在处理训练知识与上下文文档冲突时的矛盾现象。先前研究结果不一:有的发现模型顽固保留训练答案,有的则发现模型几乎完全遵循上下文。
研究者认为这是因为未区分三种不同的处理情境:机制1(单源更新,主导因素为证据连贯性)、机制2(竞争整合,主导因素为参数确定性)、机制3(任务适配选择,主导因素为任务知识需求)。通过 9970 次 API 调用,在 Claude Sonnet 4.
6、GPT-5.5、Gemini 2.
5 Flash、Llama 4 Maverick 和 DeepSeek V3 上验证了该框架,确认了机制2的确定性梯度,并发现任务框架能将上下文遵循率从近100%翻转至6-71%。
Three Regimes of Context-Parametric Conflict: A Predictive Framework and Empirical Validation
The literature on how large language models handle conflict between their training knowledge and a contradicting document presents a persistent empirical contradiction: some studies find models stubbornly retain their trained answers, ignoring provided documents nearly half the time, while others find models readily defer to the document, following context approximately 96% of the time. We argue these contradictions dissolve once one recognises that prior experiments have studied three qualitatively distinct processing situations without distinguishing them. We propose a three-regime framework: Regime 1 (single-source updating, dominant predictor: evidence coherence), Regime 2 (competitive integration, dominant predictor: parametric certainty), and Regime 3 (task-appropriate selection, dominant predictor: task knowledge requirement). We formalise a distinction between parametric strength (exposure frequency) and parametric uniqueness (encoding consistency), showing empirically that these are orthogonal dimensions (r = -0.002, p = .97) with strength as the operative predictor in stable factual domains. We validate the framework across Claude Sonnet 4.6, GPT-5.5, Gemini 2.5 Flash, Llama 4 Maverick, and DeepSeek V3 using 9,970 API calls in three experimental phases. GEE logistic regression confirms the predicted Regime 2 certainty gradient for all five models (beta = -0.38 to -0.50, all p <= .013, BH-FDR corrected). A Regime 3 ablation shows task framing alone flips context-following from near-100% (contextual knowledge condition) to 6-71% (parametric knowledge condition), with all five models significant (p < .001). The certainty gradient is robust to multinomial outcome modeling, sensitivity analyses for hedging responses, and FDR correction.
This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include LLM行为, 知识冲突, 上下文学习, 评测框架.
Primary source verification: published directly by arXiv: DeepSeek with first-party authority.