用强化学习训练 steering agent，纠正 LLM 智能体的目标漂移
Learning to Steer LLM Agents and Multi-Agent Systems
论文提出给多智能体系统配一个监工，用 GRPO 训练出能自动纠偏的 steering agent，做 agent 系统的可以看看思路。
LLM 智能体和多智能体系统在长程任务中容易偏离目标，造成资源浪费，这类失败被称为目标漂移。研究提出一个 steering agent，通过低成本 fast path 判断何时介入、高成本 slow path 决定如何介入。训练上采用成对排序目标与 GRPO 结合的强化学习方案，同时优化两条路径。在信息检索类搜索任务上，该方法对单智能体和多智能体系统都能提升性能和效率，且干预能力可泛化到域外任务和未见过的执行器，调整干预频率无需重新训练。
Learning to Steer LLM Agents and Multi-Agent Systems
Large language model (LLM)-based agents and multi-agent systems increasingly tackle complex tasks through long-horizon tool use and inter-agent collaboration. However, these systems are prone to deviating from the overall goal as execution unfolds, which wastes execution resources and leads to undesired outcomes, a failure termed goal drift. In this work, we propose a steering agent that supervises execution on the fly and steers the system back on track when drift occurs. The steering agent observes the execution context, deciding when to intervene with a low-cost fast path and how to intervene with a high-cost slow path. To unlock its steering intelligence, the ability to assess goal drift and generate effective interventions, we jointly train the two paths with reinforcement learning, combining a pairwise ranking objective with group-relative policy optimization (GRPO). Experiments on information-seeking search tasks show that the steering agent provides effective supervision for both single-agent and multi-agent systems, improving their performance and efficiency. The learned steering intelligence generalizes to out-of-domain tasks, transfers to unseen executors, and supports controllable intervention frequency without retraining.