这篇论文揭示了反馈对齐在深层网络中失效的根本原因——低维梯度动力学，并给出了可落地的解决方案（Muon+归一化）。做生物启发学习或替代反向传播研究的团队值得关注，实验方法可以直接复现。
反向传播（BP）因需要反馈权重与前向权重的转置一致而被认为生物上不可信。反馈对齐（FA）使用固定随机反馈权重来规避此问题，但在深层架构中效果不佳。研究发现FA误差的有效秩远低于BP，限制了参数空间的探索。通过使用Muon优化器正交化权重更新和隐藏活动归一化促进激活正交性，可提高FA的有效维度。在CIFAR100和ResNet-18上，这些方法将准确率提升了9个百分点。
Overcoming Rank Collapse in Feedback Alignment
Backpropagation (BP) is widely viewed as biologically implausible, in part because it requires feedback weights to be the transpose of forward weights for error propagation. Interestingly, when training a network with fixed random feedback weights to circumvent this issue, learning aligns the forward weights with the feedback weights, leading the backpropagated error signal to become an approximation of the standard gradient used by BP. This process, called Feedback Alignment (FA), occurs in MLPs and very shallow CNNs but does not scale well to deeper architectures. In this work, we first investigated differences between BP and FA models, trained on CIFAR10, specifically focusing on the effective rank of the signal. We found that the FA error has a considerably lower rank and hence is constrained to a lower-dimensional subspace compared to BP, limiting exploration of the parameter space. Motivated by this observation, we evaluated two mechanisms for increasing the effective dimensionality of FA: Muon, an optimiser that orthogonalises weight updates; and hidden activity normalisation, which promotes activation orthogonality. Across larger architectures and benchmarks, we find that these methods consistently improve over FA baselines, for example, on CIFAR100 with a Resnet-18, accuracy increases by 9 percentage points. Our results identify low-dimensional gradient dynamics as a key obstacle to scaling FA and suggest that inducing higher-dimensional update geometry is a promising route toward scaling alternatives to backpropagation.