这篇论文提出了一个很聪明的改进：当学生模型推理跑偏时，让老师模型短暂接管一段，再交回学生继续训练。在Qwen3小模型上效果明显，数学推理准确率提升，训练还更快。
提出Relay-OPD方法，解决同策略蒸馏（OPD）中的前缀失败问题：当学生模型进入错误推理方向后，后续生成偏离正确路径。该方法利用教师-学生在失败前缀上的延续不对称性作为无标签触发信号，让教师短暂接管生成然后交回学生训练。使用Qwen3-4B-Instruct-2507作为教师，Qwen3-0.6B/1.7B-Non-Thinking作为学生，在8个数学推理基准上，Relay-OPD在全部基准中取得最佳或次佳结果，1.7B模型平均超越标准OPD 5.73%，超越最强基线FastOPD 1.49%，且训练轨迹长度减少超过50%。
Pass the Baton: Trajectory-Relayed On-Policy Distillation
On-policy distillation (OPD) grounds token-level supervision in the student's own trajectory, yet suffers from prefix failure: once the student commits to a wrong reasoning direction, all subsequent generation builds on this deviation, producing misdirected continuations that elicit unreliable supervision and waste compute. We identify a teacher-student continuation asymmetry on failed prefixes, where the teacher tends to redirect while the student continues along the original direction, and convert it into a label-free handoff trigger in Relay On-Policy Distillation (Relay-OPD). During training, Relay-OPD constructs relay trajectories by letting the teacher briefly take over at detected trigger points to produce a teacher leg, after which the student resumes and is optimized on the resulting trajectory. A limited relay budget concentrates intervention on critical early positions while limiting departure from the student policy. With a Qwen3-4B-Instruct-2507 teacher and Qwen3-0.6B/1.7B-Non-Thinking students on eight mathematical reasoning benchmarks, Relay-OPD achieves the best or second-best results on every benchmark, outperforming standard OPD by +5.73% and the strongest baseline FastOPD by +1.49% on average for 1.7B, with consistent gains at 0.6B. Training trajectory length is reduced by over 50%.