TESD：用教师进化机制改进 On-Policy 自蒸馏训练
On-Policy Self-Distillation with Teacher Evolution
解决自蒸馏训练到后面不涨分的问题，思路是让教师自己也能被环境反馈训练，做后训练的可以看看。
On-Policy Self-Distillation（OPSD）在序列级后训练中常因自教师能力有限而难以持续涨点。论文提出 Teacher-Evolution Self-Distillation（TESD），让自教师生成 rollout 并依据任务正确性和上下文独立性获取环境反馈。教师导向优化持续增强自教学能力，学生导向优化则依据教师引导可靠性对样本加权降权。论文称在多个数据集和模型规模上，TESD 稳定优于代表性 token 级和序列级方法。
On-Policy Self-Distillation with Teacher Evolution
On-Policy Self-Distillation (OPSD), which aligns a student model's token-level distribution with that of its self-teacher prompted by privileged context, has emerged as a promising direction to mitigate reward sparsity in sequence-level post-training paradigms. However, OPSD often fails to achieve continuous performance gains during training. In this paper, we find that the model's limited self-teaching capability underlies this phenomenon by generating unreliable guidance on certain samples. To address this problem, we propose Teacher-Evolution Self-Distillation (TESD), which enables the self-teacher to generate rollouts and receive environmental feedback based on task correctness and context independence. First, using these rewards as training objectives, we perform teacher-oriented optimization to continuously enhance the model's self-teaching capability during distillation. Second, by estimating teacher guidance reliability from these rewards, we apply adaptive student-oriented optimization to down-weight samples with unreliable guidance. Extensive experiments across multiple datasets and model scales show that TESD consistently outperforms representative token-level and sequence-level methods, highlighting the necessity of improving supervision quality in OPSD.