Reasoning From Scratch round 8 covers distillation with Qwen3-0.6B
Distilling reasoning traces from a larger teacher into a 0.6B-parameter model shows practitioners a far cheaper path than RL training runs, with MATH-500 as the benchmark to verify the gains. Builders training small models on limited compute can reuse the standalone training script and data-preparation pipeline directly. The soft vs. hard and on-policy vs. off-policy distillation framing is a useful mental model when choosing a distillation approach.
