这篇论文用几何视角重新定义了灾难性遗忘,做持续学习或模型遗忘研究的开发者会看到新方向——遗忘不是删除,而是藏起来了。建议关注其恢复子空间维度的测量方法,可能启发新的抗遗忘算法。
该论文挑战了持续学习中“灾难性遗忘即信息破坏”的传统观点,提出遗忘知识其实仍以紧凑形式存在于模型表征中。研究者在Split CIFAR-100上训练ResNet-18,通过引入“恢复子空间维度”指标发现,尽管表征漂移显著,但恢复遗忘知识所需的子空间维度在整个训练过程中保持稳定(均值8.0)。主角度漂移与可恢复性高度相关(r=-0.862),一个简单几何模型能解释82.2%的可恢复性方差。这些结果支持“稳定恢复流形”假说,表明灾难性遗忘本质上是可访问性和流形对齐问题,而非信息丢失。
The Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual Learning
Catastrophic forgetting is often viewed as the destruction of previously learned knowledge during sequential learning. Building on the Accessibility Collapse framework, we investigate the geometric structure of recoverability in continual learning. Using Split CIFAR-100 and a sequentially trained ResNet-18, we analyze recoverability, representational drift, and recovery complexity across ten tasks. We introduce Recovery Subspace Dimensionality (k_t), a measure of the minimum number of singular directions required to preserve 90 percent of full probe performance. Contrary to our Recoverability Diffusion hypothesis, recovery dimensionality remains stable throughout training (mean k_t = 8.0) despite substantial representational drift. Principal-angle drift strongly predicts recoverability (r = -0.862), and a simple geometric model explains 82.2 percent of recoverability variance. These findings support the Stable Recovery Manifold hypothesis, suggesting that forgotten knowledge remains compactly decodable despite representational reorganization. The results indicate that catastrophic forgetting is primarily an accessibility and manifold-alignment problem rather than information destruction.