做智能体强化学习的团队终于有了一个能自动积累可复用策略的框架——ReSkill 让技能创建和策略优化不再打架，直接提升泛化能力，做 RL 和 LLM 智能体的研究者值得细读。
ReSkill 是一种新型的智能体强化学习框架，旨在解决现有方法中技能创建与策略优化脱节的问题。它受 Anthropic 的 Skill Creator 启发，将技能创建嵌入到策略学习循环中，通过断言驱动的技能创建器、组内采样和自适应 Thompson 采样三个机制，实现技能与策略的协同进化。实验表明，ReSkill 在多个领域优于现有方法，尤其在未见任务上表现突出，能自动创建、测试、优化和淘汰技能。该工作为构建可泛化的智能体系统提供了新思路。
ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL
Agentic reinforcement learning (RL) enables LLM agents to improve continuously from environment rewards, yet the resulting policies do not systematically accumulate reusable strategies that generalize across tasks. Modular skills can provide such reusable strategies, yet existing skill-augmented RL methods decouple skill creation from policy optimization, risking adopting skills that conflict with the evolving policy. Inspired by Anthropic's Skill Creator, we introduce ReSkill, an RL-in-the-loop skill creation framework that reconciles skill evolution with policy learning. ReSkill exploits the group-wise structure of GRPO to naturally embed three mechanisms with only marginal additional overhead: (1) an assertion-driven skill creator that diagnoses failures from past experience and proposes conditional, trigger-based skill revisions; (2) within-group rollout sampling that enables controlled comparison of skill versions, capturing which version best supports the policy's ongoing learning; and (3) Thompson Sampling with adaptive discounting to balance exploration and exploitation in skill version selection as the policy evolves. Across several domains, ReSkill consistently outperforms existing memory and skill-based RL methods, with the largest gains on unseen tasks. Analysis of the skill lifecycle shows skills being automatically created, tested, refined, and pruned as the policy improves, demonstrating reconciled skill-policy co-evolution.