Soft-MTP Fixes Multi-Token Prediction Training via KL Gradient Recovery
The KL gradient-recovery trick recovers exact KL divergence gradients from cross-entropy gradients without vocabulary-sized logit matrices, staying compatible with standard fused kernels — a practical detail most MTP implementations overlook. Teams pre-training models with MTP, including DeepSeek-style architectures, can adopt it as a drop-in change with no parameter or throughput cost. The repo offers a direct starting point for replication.