SRPO框架使LLMs能够自我分析，提高长期推理能力，比传统方法更高效。
SRPO是一种框架，使LLMs能够分析自身轨迹，将错误合成简洁的'反思补丁'，并使用条件教师分数作为密集的token级训练信号。在数学推理和长期代理基准测试中，SRPO实现了最先进的性能，使用Qwen3-8B基模型，在AIME'24上达到73.3%，仅使用8%的训练FLOPs，同时显著提高WebShop（64.7%）、ALFWorld（76.8%）和SWE-Bench-Lite（31.2%）的成功率。
SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning
Self-reflection is a powerful mechanism for credit assignment in human learning, converting sparse outcome feedback into actionable guidance. However, its potential for post-training Large Language Models (LLMs) remains underexplored. We propose Self-Reflective Policy Optimization (SRPO), a framework that internalizes this capability. SRPO enables LLMs to analyze their own completed trajectories, synthesize errors into concise "reflection patches," and use reflection-conditioned teacher scores on student on-policy rollouts as dense token-level training signals. This process effectively transforms sparse terminal supervision into dense, token-level learning signals without requiring external critics, separate reward models, or larger teacher models. We demonstrate that SRPO achieves state-of-the-art performance across mathematical reasoning and long-horizon agentic benchmarks with exceptional data efficiency. Using a Qwen3-8B base model, SRPO attains 73.3% on AIME'24 using only 8% (0.08x) of the training FLOPs required by scaled supervised fine-tuning, while significantly improving success rates on WebShop (64.7%), ALFWorld (76.8%), and SWE-Bench-Lite (31.2%). Code is available at https://github.com/Galleons2029/SRPO