这篇论文把黑盒RL搬到agent harness上，用Qwen3-30A3B在OpenClaw和Claude Code上跑，ClawGym-Bench直接涨了10到15个点，还支持多个harness一起训练。
ClawGym II提出一种统一的黑盒强化学习框架，用于通过复杂harness稳定优化通用agent。该框架基于沙箱执行基础设施隔离任务环境，并在模型边界放置代理捕获模型调用，将捕获的调用组织成前缀树以重建多轮轨迹。研究者分别用OpenClaw和Claude Code在ClawGym-Bench上对Qwen3-30A3B进行黑盒RL训练，Pass@1提升9.98和14.81个百分点，且200-400步优化中保持稳定。该方法在JobBench和OfficeQA等更难任务上也有持续增益，并支持混合harness联合训练。
ClawGym II: Exploring Black-Box RL on Agent Harness
Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment. However, reinforcement learning through complex harnesses remains largely unexplored, as scaling such training to long-horizon agent tasks introduces fundamental challenges. In this work, we present a unified black-box RL framework for stable and scalable optimization of general agents through complex harnesses. Concretely, we first build a sandbox-based execution infrastructure that isolates task environments and harnesses within temporary sandboxes for large-scale concurrent rollouts. We then decouple policy optimization from opaque harness execution and place a serving proxy at the model boundary to capture model calls. To reconstruct multi-turn trajectories and improve training efficiency, we organize the captured calls into prefix trees and further adapt both critic-based PPO and critic-free GRPO to optimize over the recovered tree structure. Meanwhile, we maintain training-inference consistency throughout the optimization process. Finally, we introduce mix-harness training, allowing a single model to be jointly optimized by heterogeneous harnesses. With Qwen3-30A3B, black-box RL improves Pass@1 on ClawGym-Bench by 9.98 and 14.81 points through OpenClaw and Claude Code, respectively, while remaining stable over 200-400 optimization steps. Moreover, the framework yields consistent gains on more challenging tasks such as JobBench and OfficeQA. Overall, our framework enables effective, stable, and scalable optimization of general agents through black-box harnesses, supporting unified training across heterogeneous execution systems.