PATS用动态训练支架帮弱策略少犯错，ALFWorld和WebShop上最高提升18.6%，而且部署时没负担。
PATS是一种针对长周期LLM智能体强化学习的策略中心训练范式。它将最近的rollout分组转换为证据卡，并通过任务特定评估调整后续rollout的上下文。在ALFWorld和WebShop上，PATS相比强基线提升最高达18.6%。在7个搜索增强QA基准上，PATS在保持竞争力的同时比基线减少32.1%的提示token。该方法在部署时丢弃训练支架，不增加推理开销。
PATS: Policy-Aware Training Scaffolding for Agentic Reinforcement Learning
In long-horizon LLM agent reinforcement learning, weak policies often repeat similar failures, producing uninformative rollout trajectories and limiting effective policy optimization. Existing skill-centric methods improve exploration by optimizing, filtering, or internalizing reusable skills. However, they remain centered on the skills themselves rather than being designed as adaptive training-time support for the evolving policy. To address this, we propose a policy-centric training paradigm that reframes skills as a dynamic training scaffold. Our framework, Pats, converts rollout groups from the latest policy into evidence cards and uses task-specific evaluation to adjust the context used in subsequent rollouts. Concrete guidance helps weak policies to complete challenging tasks. As policy improves, redundant context is revised or removed to reduce reliance on explicit guidance while preserving useful rollout variation. The policy is optimized with environmental rewards using standard RLVR, and the training scaffold is discarded at deployment. On ALFWorld and WebShop, Pats improves over strong baselines by up to 18.6%. Across seven search-augmented QA benchmarks, it remains competitive while using 32.1% fewer prompt tokens than the baseline.