研究揭示 Meta-Learned 网络参数空间存在可组合的加性结构
Linking Rapid Learning to Compositional Structure in Neural Networks: Evidence from Additive Analogies in Meta-Learned Parameter Spaces
这篇论文把 word2vec 时代的加性类比搬到了参数空间：meta-learned 网络适配不同函数的参数向量可以直接加减组合，减出来的网络不用训练就能跑对应函数。做少样本学习或元学习研究的人值得看看机制部分。
一篇 arXiv 论文（编号 2611.17314）研究 meta-learning 训练的神经网络为何能快速学习新函数。作者借鉴词向量领域 king − man + woman ≈ queen 的经典加性类比，把网络适配某个函数后的全部参数拼接成单一向量，当作该函数的"表示"。
实验发现，这些参数向量可以按函数之间的符号关系进行加减运算，得到的新向量所定义的网络无需任何直接训练就能实现预期函数。这说明 meta-learning 把网络的初始点调整到使后续学习沿系统性、功能有意义的轨迹组织。
论文为神经网络中的归纳偏置提供了一个几何层面的解释框架，也解释了 meta-learned 网络少样本快速适应的内部机制。
Linking Rapid Learning to Compositional Structure in Neural Networks: Evidence from Additive Analogies in Meta-Learned Parameter Spaces
Humans can learn rapidly from limited experience, while neural networks typically require extensive training. In this work, we investigate what properties of a neural network can enable it to learn rapidly. We study neural networks that have been optimized via meta-learning, a technique that exposes a network to many related functions to facilitate its subsequent learning of similar functions. Prior work has shown that these meta-learned networks can learn new functions rapidly, but how they do so remains unclear. To investigate their internal structure, we take inspiration from classic work demonstrating additive relationships among the learned vectors that neural networks use to represent words (e.g., "king" minus "man" plus "woman" approximates "queen"). Analogously, we look for additive relationships among a meta-learned network's "representations" of different functions; when a meta-learned network rapidly learns to implement a specific function, all of its adapted parameter values can be concatenated into a single vector that we treat as representing the function. We find that such parameter vectors can be added and subtracted according to symbolic relationships among the functions they represent, yielding new parameter vectors that successfully represent the functions predicted by the corresponding symbolic manipulations: the networks specified by these artificially recombined vectors actually implement the intended functions without any direct training on them. Our results suggest that meta-learning can tune a network's starting point to organize subsequent learning along systematic, functionally meaningful trajectories through the space of possible parameter vectors, providing a geometric basis for realizing inductive biases in neural networks.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include meta-learning, 参数空间, 少样本学习, arXiv.
Source attribution: curated from arXiv cs.LG (@Abigail L. Tenenbaum, R. Thomas McCoy).