Natural PC：用侧向回路实现局部自然梯度学习，效率超反向传播
Natural Gradient Learning in Predictive Coding through Lateral Circuits
做生物启发学习或替代 BP 训练的人该看看这篇：预测编码加上一个侧向回路，就能隐式算 Fisher 逆矩阵。在 ImageNet 上比 AdamW 训的 BP 省一个数量级算力、两个数量级样本，性能还不掉。
一篇新论文提出 Natural PC，在预测编码（Predictive Coding）框架中加入侧向回路，实现空间和时间上局部、精确的自然梯度学习。侧向回路通过学习神经活动相关性来确定 Fisher 信息矩阵，并隐式计算其逆矩阵，绕开了代价高昂的直接矩阵求逆。
在所有测试的图像重建基准上，Natural PC 在样本效率和计算效率上都超过了现有 PC 方法。最关键的结果是：Natural PC 可以扩展到 ImageNet 规模，以低一个数量级的计算成本、少两个数量级的训练样本，达到与 AdamW 及多种二阶优化器训练的反向传播相当的性能。
这项工作说明预测编码此前被诟病的推理开销问题可以通过优化理论手段缓解，局部自然梯度学习使 PC 在重建任务上首次与 BP 在样本效率和计算成本两个维度同时具备竞争力。
Natural Gradient Learning in Predictive Coding through Lateral Circuits
Predictive coding (PC) trains neural networks through local energy-minimization dynamics without depending on a global error signal that backpropagation (BP) requires. Although PC is a promising alternative algorithm for efficient learning, the computational cost of inference dynamics currently outweighs the resulting learning benefits in practice. In this work, we take inspiration from optimization theory to extract more learning benefit from each inference phase. We introduce Natural PC, which augments standard PC with a lateral circuit, enabling exact and efficient natural gradient learning locally in space and time. Our lateral circuit learns the activity correlations that determine the Fisher information matrix and implicitly computes its inverse, avoiding expensive matrix inversion. Across all tested image reconstruction benchmarks, Natural PC achieves state of the art in terms of sample and compute efficiency compared to existing PC approaches. Most notably, Natural PC scales to ImageNet and matches the performance of BP trained with AdamW and various second-order optimizers at an order of magnitude lower computational cost while also using two orders of magnitude fewer training samples. Our results demonstrate that local natural gradient learning makes PC competitive with BP in both sample efficiency and computational cost in reconstruction tasks.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Predictive Coding, 自然梯度, ImageNet, Fisher信息矩阵.
Source attribution: curated from arXiv cs.AI (@Lhea Beumer, Sander de Haan, Niclas Pokel, Pau Vilimelis Aceituno, Benjamin F. Grewe).