做LLM微调的团队终于有了一个更系统的设计框架——Target-SFT直接告诉你如何选择目标分布，而不是盲目拟合每个token。做推理模型优化的开发者建议试试，效果在多个数据集上都有提升。
该论文重新审视了监督微调（SFT）的传统做法，指出最大化每个token的似然可能因噪声或与模型先验不一致而效果不佳。作者提出将SFT视为目标分布设计问题，并引入Q-target框架，将监督分解为对观测token的依赖程度和剩余概率的分配方式。基于此，他们提出了Target-SFT方法，直接根据期望的目标分布构建训练目标。在十个推理数据集-模型组合的实验中，Target-SFT一致优于现有方法，展示了基于目标的设计原则的有效性。这项工作为SFT提供了更统一的视角，并开辟了更广阔的搜索空间。
A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design
Supervised fine-tuning (SFT) typically maximizes the likelihood of every token in a demonstrated trajectory. However, an observed token can be non-unique, noisy, or misaligned with the model prior. Strictly fitting toward this one-hot target may be suboptimal, especially when the pretrained model encodes a rich knowledge prior. In this work, we reinterpret SFT as target distribution design: instead of studying only the loss objective, we analyze the token-level target that the loss drives the model to match. We introduce the Q-target framework, which decomposes SFT supervision into two explicit choices: (1) how strongly to rely on the observed token, and (2) how to allocate the remaining probability mass over alternatives. This perspective unifies many existing SFT variants as implicit choices of the target distribution Q. Building on this view, we propose Target-SFT which constructs the training objective directly from the desired target distribution. This method consistently outperforms across the ten reasoning dataset-model settings evaluated, showing the effectiveness of this target-based approach. Overall, our formulation reveals a more fundamental design principle for SFT training and opens a broader search space for SFT objectives.