研究：多轮强化学习在 LLM 智能体中真正提升的是什么
What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?
论文把 agentic RL 的收益拆开看了一遍：单步决策没变强，强的是出错后能恢复。还给了个 BudgetMix RL 训练法，做智能体的值得看看数据。
一篇 arXiv 论文用 Pass@(K,T) 指标，从采样广度 K 和交互深度 T 两个维度评估多轮 agentic RL 的收益。结果显示轨迹级 RL 优势依赖充足的轮次预算，预算收紧时收益明显下降；RL 并不普遍提升单步决策质量，强项集中在错误恢复和长程韧性。论文还讨论了过程监督信号的作用与局限，并提出 BudgetMix RL，在训练中动态变化交互预算，实验显示其在多种执行设置下表现更稳健。
What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?
While multi-turn agentic reinforcement learning (RL) excels on long-horizon tasks, its performance gains remain highly sensitive to interaction constraints. Evaluating across sampling breadth $K$ and interaction depth $T$ via $Pass@(K,T)$, we show that trajectory-level RL advantages depend on sufficient turn budgets, diminishing significantly when budgets tighten. Besides, agentic RL does not uniformly elevate single-step decision quality; rather, its strength concentrates in error recovery and long-horizon resilience. To mitigate the deployment risks caused by this budget dependence, we provide a preliminary discussion on the benefits and limitations of process supervision signals and introduce BudgetMix RL as an alternative that varies interaction budgets during training. Experiments demonstrate that BudgetMix RL achieves robust performance across diverse execution settings. Project Page: https://rethinking-agentic-rl.github.io/
