论文精选

仅9800参数:研究显示NCA隐藏通道学到可迁移的通用特征

Neural Cellular Automata Learn General Features in their Hidden Channels

精选理由

NCA只用9800个参数就完成少样本迁移:老师只见过数字0到5,学生照样能认没见过的数字。

一篇arXiv论文研究了神经自动机(NCA)隐藏通道的内部动态,并提出把预训练教师模型的隐藏状态注入学生模型的迁移学习机制。在少样本和尺度变化的MNIST基准上,NCA仅用约9800个参数就超过了同量级的循环与前馈架构。对NCA的机制分析显示,隐藏通道通过吸收形态复杂度、收敛到相互正交的状态,把特征提取与统一分类共识解耦。这些通道学到的是尺度不变的拓扑基元而非类别模板,教师模型只用数字0-5训练,学生模型就能在未见过的类别上取得较强的少样本表现。

原文 · arXiv cs.AI

Neural Cellular Automata Learn General Features in their Hidden Channels

Modern deep learning models achieve impressive generalization through over-parameterization, but this paradigm often struggles with overfitting and memorization in few-shot regimes. Neural Cellular Automata (NCAs) offer a highly parameter-efficient alternative, yet research has focused primarily on their output, leaving the role of their internal hidden channels largely unexplored. In this paper, we investigate the internal dynamics of NCA hidden channels and introduce a novel transfer-learning mechanism that injects a pretrained teacher's hidden states into a student model to guide early optimization. Evaluated on few-shot and scale-variant MNIST benchmarks, NCAs outperform comparable recurrent and feed-forward architectures, demonstrating superior generalization with a minimal parameter budget (~9,800 parameters). Mechanistic analysis reveals that the hidden channels decouple feature extraction from uniform classification consensus by absorbing morphological complexity and converging to mutually orthogonal states. Furthermore, we demonstrate that these hidden channels capture general, scale-invariant topological primitives rather than class-specific templates. This allows a student model to achieve strong few-shot performance on unseen classes using features transferred from a teacher trained only on a subset of digits (0-5). Our results highlight the potential of utilizing hidden-state dynamics as a robust, decentralized computational substrate for parameter-efficient transfer learning