LLM在上下文学习中展现贝叶斯预测能力
Large Language Models Develop Belief State Geometry In-Context
这是篇挺有意思的论文，作者用隐藏马尔可夫模型数据测试了六款开源LLM，发现它们在上下文学习中能近似贝叶斯预测，还做了干预实验验证了这一点。
研究通过隐藏马尔可夫模型（HMM）数据对六款开源LLM进行提示，发现其残差流激活能线性解码出后验分布（信念状态），R²值在0.83-0.99之间。通过干预该子空间，能保持下游预测质量，证明ICL近似最优贝叶斯预测。
Large Language Models Develop Belief State Geometry In-Context
Large language models (LLMs) trained on next-token prediction exhibit remarkable in-context learning (ICL) abilities, yet the representations that support ICL remain poorly understood. We consider such representations in a controlled setting: prompting LLMs with data emitted from hidden Markov models (HMMs) and probing for the corresponding belief state -- the posterior distribution over the HMM's hidden states given the observed token history. Across six open-source LLMs prompted with data from 40 HMMs selected for non-trivial belief structure, we find that belief states are linearly decodable from residual stream activations, with peak probe $R^2$-values from 0.83-0.99 across HMM and LLM combinations, ranging from early to late layers. To establish functional relevance, we intervene directly on the probe-identified subspace via patching and steering, resulting in downstream prediction quality on the order of the untampered model, while controls degrade performance substantially. Together, these results provide representation-level evidence that ICL in open-source LLMs approximates optimal Bayesian prediction over a context-inferred generative model. More broadly, our findings extend prior results linking input-distribution structure to activation geometry: from toy networks trained explicitly on HMM data to production-scale LLMs.
