这篇论文对比了蒸馏不同教师模型的效果，发现推理型教师擅长教写作质量，托管型管线擅长教标签多样性，对选哪种蒸馏方案做端侧部署很有参考价值。
该论文研究将8B推理教师模型deepseek-r1:8b蒸馏为0.6B学生模型Qwen3-0.6B（使用QLoRA，三个随机种子）。学生模型每篇处理时间0.8秒，教师模型需39秒，速度提升约48倍。在摘要质量上，学生模型恢复教师与基线之间58%的差距，比约束解码高16.8分，比少样本提示高4.9分。相同规模的非推理教师（非deepseek-r1）训练的学生表现不优于未经微调的基线，表明摘要提升源于教师的推理特性。推理教师传递写作质量，托管管线传递标签多样性；在22篇短源文章中，指令教师的学生更忠实（74 vs 55篇忠实），而推理系学生易编造内容。
Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment
High-volume structured extraction pays a large model's latency on every item, so distilling the task into a small on-device model is attractive: comparable output at a fraction of the time and cost. We measure what that distillation actually delivers, per sub-task. Each news article is mapped to one JSON object with a short summary and five categorical labels. We distill an 8B reasoning teacher (deepseek-r1:8b) into a 0.6B student (Qwen3-0.6B; QLoRA, three seeds), and add two teacher controls: a same-size non-reasoning teacher and a larger managed pipeline. A blinded, reference-free, three-judge panel scores every arm against the full article, alongside two non-distillation baselines, few-shot prompting and constrained decoding. The student runs at about 0.8 s per article against the teacher's 39 s, and recovers 58% of the base-to-teacher gap on summary quality, beating its primary baseline (constrained decoding) by +16.8 points and few-shot prompting by a secondary +4.9. A same-size non-reasoning teacher trains a student no better than the untuned base, so the summary gain follows from the teacher's reasoning nature rather than its scale. Capabilities then split by teacher: the reasoning teacher transfers writing quality and the managed pipeline transfers label diversity, while a same-size instruction teacher's students stay more grounded on the 22 short, thin-source articles in the 93-item test set (74 versus 55 faithful), where the reasoning-lineage student fabricates. That grounding difference is a consistent ordering rather than a significant aggregate effect, and the subgroup is small, so we report it as a direction. Because no single engine wins every field, the deliverable is a per-field routing map for on-device enrichment.