构造逆向思维：提升大模型反向推理能力的训练方法
Construting Reverse Thinking: Developing Large Language Models' Reverse Thingking Ability
让模型学会倒着想：论文构建正反双向数学数据集，配合奖励机制让模型自选推理模式，数学证明准确率提升。
论文指出 GPT-o1、GPT-o3 和 DeepSeek-R1 依赖的长思维链默认单向正向推理，并通过统计不同规模模型在数学数据集上的错误，归纳出解空间覆盖不足、计算失误、未验证假设、忽略约束条件、超出最大响应长度这五类错误原因。为此提出反向推理模式构造方法，搭建了包含易-难两阶段的数学数据集，同时收录正向与反向推理路径。训练上采用两阶段监督微调，配合基于平滑奖励信号的细粒度奖励机制，避免 reward hacking。另设计线性衰减均衡采样策略，保持正反向推理样本平衡，让训练收敛更快更稳。实验显示该方法在数学证明等任务上提升了推理效率与准确率。
Construting Reverse Thinking: Developing Large Language Models' Reverse Thingking Ability
When facing complex problems, humans tend to try various ideas for different issues. Human thinking patterns exhibit remarkable flexibility in adapting to diverse scenarios. GPT-o1, GPT-o3, and DeepSeek-R1 adopt long chain-of-thought models to address complex problems by increasing reasoning depth, which default to a forward reasoning mode. We conducted statistical analysis on the accuracy of different mathematical problem datasets on models of different scales, and found five reasons for errors: Insufficient solution-space coverage, Computational mistakes, Unverified assumptions, Ignoring constraint conditions, Maximum response length limitation. To address the above issues, we proposed a backward reasoning pattern construction method aimed at enhancing the model's reverse thinking ability and dynamic adaptability. First, we constructed an easy-hard two-stage Math dataset for training large models and gradually improving their inference ability at different difficulty levels. The dataset contains forward reasoning paths as well as backward reasoning paths. And a two-stage supervised fine-tuning process is applied to progressively train the model's backward reasoning capability. Furthermore, a fine-grained reward mechanism is developed, employing smoothed reward signals to strengthen the model's ability to autonomously select thinking modes during the reasoning process, thereby avoiding reward hacking. A linear-decay balanced sampling strategy is designed to maintain a balance between forward and backward reasoning path samples during training, enabling the model to converge quickly and stably. Experimental results show that our method significantly improves reasoning efficiency and accuracy in tasks such as mathematical proofs, offering a flexible and efficient reasoning paradigm for solving complex problems.