这篇论文从理论上搞清楚了持续学习中同质深度网络的收敛性质，比之前只分析线性模型或单任务模型的结果更通用。
该论文刻画了同质模型中的弱正则化持续分类问题，将其视为在任务间隔集上的顺序投影。这一结果推广了此前仅限于单任务深度模型或持续线性模型的分析。研究表明，即使对于数据线性但参数非线性的简单模型，全局收敛一般也会失败。然而，利用非凸投影理论，论文识别了同质深度网络的规则性，保证在随机和循环任务序列下的局部线性收敛。最后，分析扩展到持续回归，统一了同质模型的框架。
Convergence of Continual Learning in Homogeneous Deep Networks
We characterize weakly regularized continual classification in homogeneous models as sequential projections onto task margin sets. This result generalizes prior analyses restricted to either stationary (single-task) deep models or continual linear models. We show that global convergence generally fails, even for simple models linear in data but nonlinear in parameters. Nevertheless, by leveraging results from nonconvex projection theory, we identify regularity properties of homogeneous deep networks that guarantee local linear convergence under random and cyclic task sequences. Finally, we extend our analysis to continual regression, unifying the framework for homogeneous models.