arXiv 论文提出免训练权重烘焙方法,用拓扑描述符改写 LLM 前馈层
Can AI Mature a Model's Geometry? Training-Free Weight Baking in LLMs
一篇挺有意思的方法论文:不用梯度训练,靠拓扑描述符加岭回归直接改 LLM 前馈层权重。126 次运行验证纯度提升、基准不掉点。做模型编辑或可解释性研究的可以看看这套免训练基线。
arXiv 论文(2611.21310v1)探讨一个免训练的模型编辑方法:由一个分析师模型读取目标模型各层潜空间的拓扑描述符摘要,提出一个变换,再用闭式岭回归把变换直接写入前馈层权重,全程不做梯度训练。
实验用了轻量模型、共 126 次运行,结果显示未见过的词句上类别纯度上升,基准准确率保持在采样误差范围内;分析师模型的效果与一个依赖类别信息的启发式方法持平,但改动幅度更温和。在五模型循环实验中,16 对模型提升了纯度,4 对目标模型保持不变。
这项工作的意义在于给出了一个基线:仅凭潜空间几何和拓扑摘要,不接触原始权重、不做梯度训练,也能组织模型的潜空间。对研究 LLM 引导式免训练模型编辑的团队来说,这是一个可对照的起点。
Can AI Mature a Model's Geometry? Training-Free Weight Baking in LLMs
Can an AI make a trained language model more mature, in the sense of a better organized latent space, by looking only at the geometry of that space? An analyst model reads a fixed-size digest of topological descriptors of the layers of a frozen target and proposes a transformation, which closed-form ridge regression writes into the feed-forward weights without gradient training. Using lightweight models, 126 runs show that category purity rose on unseen words and sentences, benchmark accuracy stayed within sampling error, and the analyst matched a category-aware heuristic with gentler edits. In a five-model round robin approach, 16 pairs raised purity and four left the target unchanged. The results provide a baseline for future work on LLM-guided, training-free model editing without using raw weights.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LLM, model editing, arXiv.
Source attribution: curated from arXiv cs.AI (@Amin Golnari, Saeid Sanei).