研究：微调前线性可读的知识才能泛化，惰性事实做不到
Knowledge That Is Linear Before Fine-Tuning Generalizes After It
一篇挺有意思的可解释性论文：模型记住了不代表会用，能不能泛化取决于知识在表征里是不是线性可读的，跨 11 个模型验证了这一点。
arXiv 论文 2610.22300 区分了语言模型中的主动事实与惰性事实：前者能支持超出自身回忆的泛化，后者只能被回忆。方法是把已有关系重新打标签（如 Paris -> Europe 变成 Paris -> crab），在部分主体上微调模型，再测试共享同一原始答案的未见主体（如 Munich -> crab）能否答对。在 11 个语言模型和 70 个关系上，原始关系在预训练表征中的线性可读性强烈预测泛化效果，模型内皮尔逊相关系数达 0.87 到 0.92。控制微调主体数、答案数、原始关系准确率和预训练出现频率后，该关联依然成立，且错误标签映射回原答案后多出现在线性探针的高分答案中。
Knowledge That Is Linear Before Fine-Tuning Generalizes After It
Not all facts a language model can recall support further learning equally well. We distinguish active facts, which support generalization beyond their own recall, from inert facts, which can be recalled but do not support such generalization in a given task. We study this distinction by constructing new relations from existing ones. Take the relation that maps a place to its continent (Paris -> Europe) and assign each of its answers an arbitrary new label (Europe -> crab, Asia -> zebra). The new relation then maps Paris -> crab. We fine-tune the model on the new relation for a subset of subjects, without telling it the original relation or the relabeling rule, and test whether it assigns the corresponding labels to held-out subjects that share an answer in the original relation. Having learned Paris -> crab, does it answer Munich -> crab? Across eleven language models and 70 relations, how linearly readable the original relation is in pretrained representations strongly predicts this generalization. Within each model, the two correlate across relations at Pearson r = 0.87 to 0.92. This association persists when controlling, separately and jointly, for the number of fine-tuning subjects and answers, the model's accuracy on the original relation, and how often the subjects occur in pretraining text. It is also robust to re-randomizing the label assignment and fine-tuning seed, and to replacing the animal labels with labels from thirteen other semantic categories. When we map a fine-tuned model's wrong label back to the original answer it was assigned to, Munich -> zebra becomes Munich -> Asia. Answers recovered this way are more often among the linear probe's top answers for the subject than among the pretrained model's own. Whether a new relation generalizes depends less on whether its facts can be recalled than on how readily their shared structure can be read out linearly.