这篇论文发现：渐进适应时任务分得越细，每个任务训练时间反而要按1/N缩放。用MNIST和Yearbook验证了，做持续学习的朋友可以看看。
该论文通过过参数化线性回归模型分析渐进适应的训练时间分配。当任务数N增大且每任务训练时间为s_N时，若Ns_N趋近τ，学习进度收敛为连续曲线。进度在小τ时按Θ(τ)增长，大τ时按Θ(1/τ)下降，因此最优每任务训练时间缩放为s_N^*=Θ(1/N)。实验在逐渐旋转的MNIST和Yearbook时间偏移上验证了该结论。
Optimal Training-Time Scaling in Gradual Adaptation
In gradual adaptation, how should the training time on each task change as the number of intermediate tasks increases? We study this question for overparameterized linear regression tasks that change smoothly and share a zero-loss solution. With $N$ tasks and training time $s_N$ on each, the final learning progress converges to a continuum curve when $Ns_N\toτ$. The limiting progress is $Θ(τ)$ for small $τ$ and $Θ(τ^{-1})$ for large $τ$, so both very short and very long training produce little progress. It follows that optimal per-task training times scale as $s_N^\star=Θ(N^{-1})$, equivalently $Ns_N^\star=Θ(1)$. Experiments on gradually rotated MNIST and a natural Yearbook time shift are consistent with less per-task training as the path is divided more finely.