这篇用一个可控环境拆解了AI长程规划的训练机制，发现次优数据会拖累性能，多教师蒸馏能整合不同能力，值得做训练方法的人细读。
论文提出统一多轮环境，系统研究规划能力在预训练与后训练三阶段的获取、塑造与整合。预训练中，显式世界模型通过CoT状态转换建模提升长程泛化，少量长程数据即可实现组合泛化，但次优轨迹因误差累积严重损害性能。后训练中，OPD在低质长程设置下的有效区域比GRPO更广，不同知识的教师蒸馏可能损害学生先验世界模型。多教师在线策略蒸馏（MOPD）通过收敛到共享规划模式整合能力，兼容模式支持跨环境泛化，冲突模式导致干扰。
The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation
Multi-turn long-horizon planning is critical for foundation model agents, yet how to fundamentally improve it remains unclear. Existing models are trained on uncontrollable and opaque Internet data, making it difficult to identify how planning ability is acquired, shaped, and integrated. To address this challenge, we introduce a unified and controlled multi-turn environment that enables precise control. It allows systematically study long-horizon planning across three stages. (1) Planning ability acquisition during pre-training. We study data format, distribution, and quality. Explicit world model construction through CoT state transition modeling yields stronger long-horizon generalization. Atomic skills alone are insufficient for compositional generalization, whereas a litte long-horizon data works. Moreover, suboptimal trajectories severely impair performance because errors amplify over long horizons. (2) Planning ability shaping via GRPO and OPD post-training. Through mutual information, we distinguish general planning patterns from task-specific planning knowledge. For planning patterns, we identify three application regions of post-training: unnecessary, effective, and unsupported. OPD has a broader effective region than GRPO under low-quality and long-horizon settings, as it provides more consistent update directions. For planning knowledge, distilling unseen procedures from a teacher with different knowledge may impair student's prior world modeling without fully establishing new knowledge. (3) Planning ability integration through MOPD post-training. We show that multi-teacher on-policy distillation (MOPD) integrates capabilities by converging to shared planning-pattern across environments. Compatible patterns enable cross-environment generalization, partially shared patterns support continual learning, while completely conflicting patterns cause severe interference.