SWE-OPD distills agentic coding from teacher into smaller models
The finding that cached, offline teacher scores work nearly as well as on-policy ones makes agentic distillation affordable for teams without large training clusters, cutting GPU-hours by well over half. Teams fine-tuning small coding agents should treat falling reverse-KL loss with suspicion and watch for repetition rather than trusting the loss curve alone. The demonstrated cross-tokenizer transfer between DeepSeek and GLM checkpoints matters for builders mixing open-source model families in agentic coding pipelines.