Neural Cellular Automata Transfer General Features via Hidden Channels
Neural Cellular Automata Learn General Features in their Hidden Channels
The ~9,800-parameter NCA beats recurrent and feed-forward baselines on few-shot MNIST, a concrete comparison point for practitioners weighing parameter-efficient training against large over-parameterised models.
A new arXiv paper (2609.21870) investigates the internal hidden channels of Neural Cellular Automata (NCAs) and introduces a transfer-learning mechanism that injects a pretrained teacher's hidden states into a student model to guide early optimisation.
On few-shot and scale-variant MNIST benchmarks, the NCAs outperform comparable recurrent and feed-forward architectures with a budget of roughly 9,800 parameters.
Mechanistic analysis on the MNIST runs shows the hidden channels decouple feature extraction from classification consensus by absorbing morphological complexity and converging to mutually orthogonal states.
A student model using features from a teacher trained only on digits 0-5 achieves strong few-shot performance on unseen classes, indicating the channels capture scale-invariant topological primitives rather than class-specific templates.
Neural Cellular Automata Learn General Features in their Hidden Channels
Modern deep learning models achieve impressive generalization through over-parameterization, but this paradigm often struggles with overfitting and memorization in few-shot regimes. Neural Cellular Automata (NCAs) offer a highly parameter-efficient alternative, yet research has focused primarily on their output, leaving the role of their internal hidden channels largely unexplored. In this paper, we investigate the internal dynamics of NCA hidden channels and introduce a novel transfer-learning mechanism that injects a pretrained teacher's hidden states into a student model to guide early optimization. Evaluated on few-shot and scale-variant MNIST benchmarks, NCAs outperform comparable recurrent and feed-forward architectures, demonstrating superior generalization with a minimal parameter budget (~9,800 parameters). Mechanistic analysis reveals that the hidden channels decouple feature extraction from uniform classification consensus by absorbing morphological complexity and converging to mutually orthogonal states. Furthermore, we demonstrate that these hidden channels capture general, scale-invariant topological primitives rather than class-specific templates. This allows a student model to achieve strong few-shot performance on unseen classes using features transferred from a teacher trained only on a subset of digits (0-5). Our results highlight the potential of utilizing hidden-state dynamics as a robust, decentralized computational substrate for parameter-efficient transfer learning
This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include Neural Cellular Automata, MNIST, transfer learning, few-shot learning.
Source attribution: curated from arXiv cs.AI (@Etienne Guichard, Stefano Nichele).