这篇论文戳破了蒸馏防御的假象——被动评估下的鲁棒性在自适应攻击面前不堪一击。做模型安全或部署的团队，建议用文中的自适应评估框架重新审视你的防御方案。
本文通过最小最大博弈框架研究模型蒸馏攻击中的部署权衡：模型输出越有用，越容易被模仿。作者提出自适应评估规则，学生可重加权高价值样本；同时设计教师端防御模板，抑制最易被蒸馏的输出。基于示例价值的廉价代理，提出Product-of-Experts (PoE) 防御，仅需前向传播即可结合教师与代理学生。实验表明，自适应评估揭示被动与自适应攻击的巨大差距：在GSM8K和MATH上，自适应学生恢复的能力远超被动评估。PoE在成本远低于现有防御的情况下，鲁棒性差距显著缩小，且保留更高质量的推理轨迹。代码已开源。
The Distillation Game: Adaptive Attacks & Efficient Defenses
Distillation attacks create a deployment trade-off for model providers: the same outputs that make a model more useful can also make it easier to imitate. We study this trade-off through a minimax game between a utility-constrained teacher and an adaptive student. Our framework yields tractable one-sided response rules: an adaptive evaluation rule in which the student reweights high-value examples, and a teacher-side defense template that suppresses outputs most useful for distillation. From a cheap proxy for example value, we derive Product-of-Experts (PoE), a simple forward-pass-only defense that combines the teacher with a proxy student during generation. Empirically, adaptive evaluation reveals a large passive--adaptive gap: on state-of-the-art defenses, adaptive students recover substantially more capability than passive evaluation suggests on GSM8K and MATH. Under this stronger evaluation, the apparent robustness gap between expensive defenses and PoE narrows considerably, while PoE remains substantially cheaper and preserves higher-quality reasoning traces. Overall, our results suggest that strong distillation remains difficult to stop, and that progress on antidistillation should be judged against adaptive students rather than passive ones. Our code is available at: https://github.com/ysfalh/distillation-game.