统一令牌级门控家族用于在线蒸馏：FKL/RKL混合与多通道和偏置系数
A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients
朋友，这是篇论文，讲的是一种新的令牌门控方法，在知识蒸馏任务上表现不错，比之前的方法有提升。
本文提出了一种新的令牌级门控参数化方法，用于在线知识蒸馏。在TweetEval情绪和仇恨任务上，使用Qwen3-32B作为教师模型、Qwen3-4B作为学生模型，该方法在33个可比单元中优于匹配单通道（熵仅/差距仅）1D限制，在26个单元的隔离实验中，动态门控在19个单元中优于有效-KL匹配的静态基线。该参数化主要作为比较短输出分类OPD中令牌门控设计的共享坐标系。
A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients
Per-token gating of forward/reverse KL losses has become a standard technique for on-policy knowledge distillation (OPD), but existing methods such as EOPD (Jin et al., 2026) and ToDi (Jung et al., 2025) each fix a single gating signal and a single gating direction, and the two have never been compared directly. We introduce a four-coefficient parameterization lambda_t = sigma(a * h_t + b * u(x) + c + d * gap_t) in which direction-aligned proxies of EOPD and ToDi appear as one-dimensional (1D) restrictions, and which adds multi-channel composition and an explicit bias as further degrees of freedom. On TweetEval (Barbieri et al., 2020) emotion and hate, with a Qwen3-32B teacher and a Qwen3-4B student, configurations in the full family reach higher accuracy than the matched-magnitude single-channel (entropy-only / gap-only) 1D restrictions in 33 of 36 comparable cells, and a 26-cell mean-match isolation experiment places dynamic gating ahead of effective-KL-matched static baselines in 19 of 26 cells. Because cells share training data, models, and parameter substructure, we report both counts as exploratory aggregate directional evidence rather than as independent hypothesis tests. Targeted three-seed paired replications of the nine headline comparisons singled out by that sweep -- including a third task, offensive -- are directionally consistent, but individually smaller than the single-seed estimates and not significant at n=3. We therefore present the parameterization primarily as a shared coordinate system for comparing per-token gating designs in short-output classification OPD.
