新研究发现大模型脑子里有张星空图，还能解码出来，挺有意思的。
研究发现，100B参数级别的语言模型在残差流中编码了夜空地图表示，且可从提示“夜空中靠近这个天体的目标是什么”中解码。多数开源模型在该表示上LOO测试显著，解释方差R²达65%-85%，中位角度误差为12°-21°。这是高维弯曲不可约特征流形的首个实例。代码已开源。
Sky sphere representation in language models
We analyze whether language models of size ~100B have a representation of the night sky map that is decodable from their residual stream. We find that most of the considered open-source models do have such a representation, and it often even surfaces to the top principal components on prompts that ask questions like ``what is close to this object in the night sky''. In all but one model this representation showed significant scores in LOO testing, containing up to 65-85% of variance ($R^2$-score) and having median angular error down to $12^\circ-21^\circ$. We verify that our representation is not a simple leak from a correlated flat representation. To our knowledge, this representation is the first example of a curved high-dimensional irreducible feature manifold. Codes used in the paper are published at https://github.com/l3erdnik/Decodable-sky