这篇论文讲了个反直觉的事：蒸馏时师生token一致反而可能有害。TIDE方法在数学推理上把准确率从6.9%拉到20.3%，还让回答短了3.6倍，值得做LLM后训练的人看看。
在线策略蒸馏（OPD）是LLM后训练的核心环节，但存在退化一致性问题：学生通过重复循环获得与教师近乎完美的token一致，但整体回答有缺陷。论文将焦点转向师生不匹配，发现不匹配token分为学生多余和学生缺失两类。提出的TIDE方法采用有界Hellinger整形抑制多余token，并通过教师top-K注入恢复缺失token的概率质量。在Qwen3多个师生对上的数学推理基准中，TIDE优于标准OPD及近期基线，在强不匹配场景下Avg@8从6.9%提升至20.3%，平均响应长度缩短3.6倍。
Mismatch Matters: On-Policy Distillation Beyond Token Agreement
On-policy distillation (OPD) has emerged as a core component of modern LLM post-training pipelines, yet we reveal a failure mode: degenerate agreement, where students exploit repetitive loops to achieve near-perfect token agreement with the teacher despite globally flawed responses. We therefore shift our focus from agreement to teacher-student mismatch, and find that mismatch tokens can be mainly categorized into two types: student-excess tokens and student-deficit tokens. Student-excess tokens are generated by the student but assigned near-zero probability by the teacher; their log-ratio corrections grow unbounded and destabilize the update. Student-deficit tokens, in contrast, are preferred by the teacher but rarely sampled by the student; their absence blocks the transfer of the teacher's reasoning patterns. To tackle these mismatch directions, we propose TIDE (Token-level Independent Deficit-Excess correction), which applies bounded Hellinger shaping to suppress the most severe sampled excesses and an analytic teacher top-$K$ injection to restore deficient probability mass without requiring deficit tokens to be sampled. Across mathematical reasoning benchmarks with multiple Qwen3 teacher-student pairs, TIDE consistently outperforms standard OPD and recent token-selection and reward-shaping baselines. Moreover, the gains of TIDE are more pronounced under strong teacher-student mismatch, where it improves Avg@8 from 6.9% to 20.3%, reduces average response length by a factor of 3.6, and substantially reduces formatting failures. Code is available at https://github.com/yzc-666/TIDE