CreativeInstruct用特殊标记让模型在保持质量的同时更会编故事，人类测评七成认为更有创造力，还能给强化学习带来几个点的提升。
CreativeInstruct是一种可扩展的指令微调方法，通过让模型学习注入[StartCreativity]标记来平衡创造性与后训练质量。该方法引入基于图编辑距离的结构多样性指标，捕捉叙事层面的变化。在叙事生成任务中，CreativeInstruct匹配或超过了多模型基线及蒸馏变体的多样性，且无需牺牲质量。人工评估中，70.3%的评分者认为CreativeInstruct的生成比后训练模型更有创造力。将GRPO应用于CreativeInstruct检查点，在AMC上提升约4%，在MATH上提升约5个百分点。
CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity
While post-training improves the capabilities of large language models (LLMs), it generally lowers their output diversity and creativity, negatively impacting tasks that explicitly require creativity (e.g., story generation) as well as those that require it implicitly, e.g., reinforcement learning (RL). We instead propose CreativeInstruct, a scalable instruction-tuning method that teaches LLMs to balance creative, base-model-like generations with the quality of post-trained models, by learning to inject special [StartCreativity] spans that bias generation toward creativity. Furthermore, we introduce a structural diversity metric based on graph edit distance, which captures narrative level variation missed by purely lexical and semantic metrics. On narrative generation, CreativeInstruct matches or exceeds the diversity of both multi-model baselines and distilled variants of their outputs, without sacrificing quality or requiring multiple models at inference time. These results are mirrored in our human evaluation, where we find that annotators rate CreativeInstruct generations as more creative than the post-trained LLMs' generations in 70.3% of cases. We also show the benefits of creative models as a substrate for RL: GRPO applied to a CreativeInstruct checkpoint improves by ~4% on AMC and ~5% points on MATH over the same training applied to the post-trained checkpoint.