持续学习中的可塑性丧失是困扰AI训练的核心难题，本文从理论到实践给出了新解法——做持续学习、强化学习或终身学习的开发者，值得关注这个等距性视角。
深度神经网络在非平稳环境下的持续训练常导致可塑性逐渐丧失，限制进一步学习。本文首次将可塑性与经验神经正切核联系起来，并识别出动态等距性（各层雅可比奇异值接近1）是保持持续学习可塑性的关键机制。作者重新审视了一类几乎处处等距且保持通用Lipschitz函数逼近能力的网络，证明近动态等距性与表达性非线性表示兼容。针对通用架构，提出了高效的等距促进正则化方案，并发现其能重新激活休眠的ReLU单元。基于此，开发了AdamO优化器，将等距正则化与梯度更新解耦。在多个持续学习基准上，该方法一致匹配或超越现有方法。
Preserving Plasticity in Continual Learning via Dynamical Isometry
Continual training of deep neural networks under non-stationarity often leads to a progressive loss of plasticity, eventually limiting further learning. We relate plasticity to the empirical Neural Tangent Kernel, and identify dynamical isometry (the condition that layer-wise Jacobian singular values remain close to one) as a key mechanism for preserving plasticity in continual learning. We revisit a class of networks that are almost-everywhere isometric while remaining universal Lipschitz function approximators, demonstrating that near-dynamical isometry is compatible with expressive nonlinear representations. For general architectures, we propose an efficient isometry-promoting regularization scheme and identify a novel mechanism by which it can reactivate dormant ReLU units. Building on this, we introduce AdamO, an Adam-style adaptive optimizer that decouples isometry regularization from gradient updates, analogous to AdamW. We further reinterpret prior plasticity-preserving approaches through the lens of dynamical isometry, showing that they target only a partial measure of isometry. Across supervised and reinforcement-learning continual-learning benchmarks designed to induce plasticity loss, our methods consistently match or outperform existing approaches.