CompassOPD Cross-Family On-Policy Distillation
CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
CompassOPD enables better knowledge transfer between different AI model families, potentially improving reasoning capabilities in existing models without full retraining.
CompassOPD improves cross-family on-policy distillation by separating model offset from within-family likelihood shifts.
The method outperforms standard OPD across three student families and multiple teacher families, improving average reasoning accuracy by up to 5.
50 points.For MoE teachers, CompassOPD eliminates the need for separate reference checkpoints while retaining a 3.
43-point gain over OPD.
CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
On-policy distillation (OPD) provides dense token-level supervision on student-generated trajectories. Although OPD performs strongly when teacher and student belong to the same model family, we find that its effectiveness degrades in cross-family settings even after tokenizer alignment, with substantially stronger external teachers offering little additional improvement. To understand this disconnect, we decompose the cross-family OPD signal into two components: an offset between a low-capability teacher-family reference and the student, and the within-family log-likelihood shift from that reference to the strong teacher. Standard OPD transfers both components together, allowing the offset to dominate the update direction and obscure the changes associated with teacher capability improvements. We propose CompassOPD, which removes this offset and transfers the within-family shift, while a frozen student reference anchors updates to the student's initial policy. Thus, both teacher-side and student-side changes are measured within their respective model families. Experiments across three student families and multiple teacher families show that CompassOPD consistently outperforms standard cross-family OPD, improving average reasoning accuracy by up to 5.50 points. For an MoE teacher, we further construct the reference directly from the teacher checkpoint by reducing expert activation, eliminating the need for a separate reference checkpoint while retaining a 3.43-point gain over OPD.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and AI regulation, compliance frameworks, and sovereign compute policy. Key entities and technical topics under continuous tracking include CompassOPD, OPD, distillation.
Source attribution: curated from arXiv cs.LG (@Naibin Gu, Qingyi Si, Chenxu Yang, Chuanyu Qin, Junhao Zhou, Peng Fu, Zheng Lin, Weiping Wang).
