想提升数学推理模型？SKALD 把技能蒸馏进权重，比 GRPO 最高涨 12 分，还不用改推理时的输入。
SKALD 是一种基于策略的自蒸馏框架，使用 Qwen3-Base 模型的两种上下文视图：仅问题的学生和基于抽象技能卡的教师。该方法通过退火指数倾斜目标稳定分布失配，并在验证奖励为正时激活蒸馏。在五个数学基准上，SKALD 在 0.6B、1.7B 和 4B 规模下分别比 GRPO 提升 +2.46、+4.85 和 +12.01 的 avg@8。在 1.7B 规模下，零方差蒸馏恢复 84.7% 的完整增益，且比 FLOP 匹配的 GRPO 高 +4.06。
Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation
Reinforcement learning with verifiable rewards yields no group-relative signal when rollout groups are uniformly correct or uniformly wrong, which account for 63.0-68.0% of groups in our experiments. We propose SKALD (Skill-Anchored Latent Distillation), an on-policy self-distillation framework that uses two context views of the same Qwen3-Base model: a question-only student and a teacher conditioned on an abstract, explicit-answer-filtered skill card. The student is trained on its own prefixes, transferring the skill-induced advantage into shared parameters without privileged input at test time. To stabilize context-induced distribution mismatch, SKALD employs an annealed exponentially tilted objective that downweights teacher-preferred tokens with very low student likelihood; as the tilt vanishes, it converges to teacher cross-entropy and recovers the forward-KL student gradient. An empirical gate activates distillation only when verified rollouts estimate a positive teacher advantage. Across five held-out mathematics benchmarks, SKALD improves overall avg@8 over GRPO by +2.46, +4.85, and +12.01 at 0.6B, 1.7B, and 4B, respectively. At 1.7B, zero-variance-only distillation recovers 84.7% of the full gain, while SKALD remains +4.06 above FLOP-matched GRPO and exceeds contextual skill exposure by +3.77. These results show that abstract skills provide dense supervision where group-relative rewards become uninformative.