Primary sourceLMSYS Org (SGLang)
View sourceEvent ·Single source
Miles 引入 OPD:使 Qwen3.5-35B 推理缩短 3 倍且精度提升
Miles 框架新增 On-Policy Distillation(OPD)作为一等训练原语。在 Qwen3.5-35B-A3B 上,纯 OPD 使推理 token 数从 18.6k 降至 5.5k–6.7k,缩短约 3 倍,同时 held-out DAPO 准确率从 0.8457 升至 0.8945。反向 KL 散度从 0.045 降至 0.010,学生模型收敛至教师。采用稀疏逐位置评分代替密集 O(R²K) 评分,提升效率。支持纯蒸馏或与 GRPO/PPO 奖励结合,计划扩展多教师和 RL 场景。
Current read
Miles 发布了 OPD 方法,让 Qwen3.5 模型推理更快更准,无需任务奖励,比传统蒸馏更高效,适合模型压缩场景。
2 sources52° AI heatLast updated 2026/7/23 14:38:46
Evidence
2 sourcesRelated source 1arXiv: DeepSeek
View sourceConflict check
Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.