发了个叫Adaptive FastOPD的蒸馏加速方法，比OPD 15K快一半多，性能还更高。做训练加速的可以看看。
在线蒸馏（OPD）沿学生生成轨迹提供密集教师监督，但长响应会拖慢批量完成。现有加速方法用固定预算或绝对师生一致阈值控制回滚长度，难以反映学习进度。Adaptive FastOPD 仅在当前边界区域学习趋于平稳且视野利用充分时扩展回滚视野，由四个师生信号相对值判断进度。在两个师生对上，Adaptive FastOPD 比 OPD 15K 减少训练时间 49.1%–71.2%，同时取得最高平均性能，且对超参数设置鲁棒。
Adaptive FastOPD: Progress-Aware Rollout Horizon Expansion for Efficient On-Policy Distillation
On-policy distillation (OPD) provides dense teacher supervision along student-generated trajectories, but its online rollout process incurs substantial computational cost, particularly when a few long responses delay batch completion. Existing acceleration methods typically control rollout length using fixed budgets or absolute teacher--student agreement thresholds, which may not reflect learning progress across different models and training stages. We propose Adaptive FastOPD, a progress-aware strategy that expands the rollout horizon only when learning near the current boundary region has plateaued and the current horizon is sufficiently utilized. The former is determined from four teacher--student signals measured relative to their values upon entering each horizon, making expansion responsive to stage-specific progress rather than a predefined step interval or an absolute threshold on the raw agreement signals, while the latter prevents a small number of long responses from triggering increases in rollout cost. Across two teacher--student pairs, Adaptive FastOPD achieves the highest average performance while reducing training time by 49.1--71.2\% relative to OPD 15K, and remains robust across a range of hyperparameter settings.