持续学习的转型：从参数中心到系统级适应
Continual Learning in Transition
这篇论文把持续学习的新玩法梳理清楚了，不再只盯着参数调整，连测试时训练、外部记忆都算进去了。搞AI系统的可以看看。
该论文从When、How、Where三个维度梳理持续学习（CL）的演进，指出其正从参数中心学习转向系统级适应。
How维度涵盖off-policy、on-policy及超越梯度的优化机制；When维度横跨预训练、后训练与推理时阶段；
Where维度区分内部参数更新与外部结构约束。论文系统综述了代表性方法，并讨论了这一范式转变的关键挑战与未来方向。
Continual Learning in Transition
Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.g., training strategies, architectural designs, and weight adaptation. However, emerging paradigms are reshaping the scope of CL beyond this traditional model adaptation view. For instance, on-policy learning broadens the space of update mechanisms; test-time training extends CL from the training phase to inference; and external harness components such as memory, skill libraries, and interaction protocols extend the evolutionary boundaries of model capabilities far beyond the static parameter space. Collectively, these developments indicate a transition from parameter-centric learning toward system-level adaptation. To characterize this transition, we examine the evolution of continual learning through three dimensions: When, How, and Where learning occurs. The How dimension encompasses off-policy, on-policy, and beyond-gradient optimization mechanics. The When dimension captures evolution across pre-training, post-training, and inference-time stages. The Where dimension delineates updates occurring within internal parameters versus external structural constraints. Anchored by this tri-axial framework, we systematically survey representative methods, trace the ongoing transition of continual learning, and discuss the key challenges, broader implications, and future directions arising from this paradigm shift.
This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include continual learning, test-time training.
Source attribution: curated from arXiv cs.LG (@Zhiyan Hou, Dan Zhang, Tao Feng, Liyuan Wang, Wei Li, Xiangzhao Hao, Hongyan An, Junfeng Fang, Haokai Ma, Zhaohui Xu, Haiyun Guo, Jinqiao Wang, Tat-Seng Chua).
