WeightCode 提出权重空间编程:不训练直接向网络写入新能力
WeightCode: Installing New Capabilities into Neural Networks via Weight-Space Programming
不用再跑微调了:一个冻结的语言模型直接写程序改权重,把新技能装进训好的网络。ImageNet 上 8.84-17.62 点的增益超过 8 种微调方法,程序还能跨 5 个独立训练的 ViT 复用。做模型定制和持续学习的工程师值得认真评估这条路子。
一篇发表于 arXiv 的论文(编号 2611.25476)提出 Weight-Space Programming(WSP),用可执行程序直接读取、构造或修改神经网络权重,从而在已训练模型中安装新能力。其框架 WeightCode 由一个冻结的语言模型生成权重程序,无需梯度训练,且安装后新能力可独立于该语言模型运行。
实验数据方面:给定语法或 8 个演示样本后,权重程序将未见过的命令组合准确率从 0% 提升到 99.8%,新 pickup 动作成功率从 0% 升至 98.4%,同时保留原有指令与导航能力。在 ImageNet 上,4 个权重程序取得 8.84-17.62 点的净增益且不损失 clean accuracy,在同等样本数、检查次数和保留约束下超过 8 种微调方法中的最优者。
可迁移性是关键卖点:同一组权重程序在 5 个独立训练的 Vision Transformer 上重跑,平均保留 83-110% 的源网络增益,而直接复制固定权重编辑几乎无增益;6 个程序写入已编辑网络后额外带来 5.23-14.50 点对比鲁棒性,并保留此前安装的噪声增益。论文将 WSP 定位为在训练后网络中积累、复用能力的另一条路线,与 LoRA 等参数高效微调形成互补。
WeightCode: Installing New Capabilities into Neural Networks via Weight-Space Programming
Neural networks encode learned capabilities in their weights, yet extending those capabilities with explicit task knowledge remains a fundamental challenge. Here we show that Weight-Space Programming (WSP), in which executable programs read, construct or modify a network's weights directly, can install new capabilities within trained neural networks. WeightCode, a WSP framework, uses a frozen language model to write executable weight programs that install new computations into trained neural networks using task knowledge and their existing weights, without gradient-based training. Once installed, the new capabilities run independently of the language model. Given a grammar or eight demonstrations, weight programs enable previously unseen command compositions (0% to 99.8% exact match) and a new pickup action (0% to 98.4% success), while preserving the original command and navigation capabilities. On ImageNet, four weight programs achieve net gains of 8.84-17.62 points while preserving clean accuracy, outperforming the best of eight fine-tuning methods under matched examples, checks, and retention constraints. In a Vision Transformer, causal interventions show that installed noise-correction gates drive robustness gains, while their selective activation preserves clean accuracy. Rerun on five independently trained Vision Transformers without further language model calls, ten weight programs retain 83-110% of their source-network gains on average, whereas copying their fixed weight edits yields negligible gains. Six programs written for an already edited network add 5.23-14.50 points of contrast robustness while retaining the previously installed noise gain. Together, these results establish WSP as a new route to acquiring and accumulating capabilities in trained neural networks through reusable weight programs.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include WeightCode, Vision Transformer, ImageNet, fine-tuning.
Source attribution: curated from arXiv cs.AI (@Bo Li, Taoyong Cui, Bob Zhang, Mengran Li, Chengyang Zhang, Yang Zhang, Pheng Ann Heng, Wanli Ouyang, Zhenfei Yin, Yingcheng Wu, Ling Yang).