CoRL 2026 RoboEval 挑战赛方案：离线策略强化学习+在线推理调优拿下 0.990 RDS
Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge
一位参赛者完整拆解了他在 CoRL 2026 RoboEval 上拿到 0.990 分的做法，π0.5 策略自当价值函数，AWR 加 RECAP 直接在官方评分上训练，再补上生成的满分演示和失败恢复片段。做机器人 RL 或智能体训练的人能直接借鉴这套循环设计。
作者在 CoRL 2026 RoboEval 挑战赛中，用基于 π0.5 的策略拿到 0.990 RDS（held-out 种子）和 0.987（官方种子）的成绩。该挑战赛包含八个双手操作任务，评分不只看成功率，还对照人类演示评估任务各阶段的安全性、效率和双臂协调程度。
方案分两部分。通用部分沿用作者在 LeHome 挑战赛的思路：让 π0.5 策略同时充当自己的价值函数，用 AWR 和 RECAP 在官方精确得分所计算的优势上做离线策略强化学习，且每次 rollout 同时充当约 20 个推理参数的 Thompson 采样试验。这套方法开箱即用，两天内把 held-out 种子上的 RDS 提升到 0.85，且远未收敛。
比赛专属部分则补充局部探索找不到的数据：完整拿到奖励的生成演示，以及从策略自身失败状态出发的恢复片段。RoboEval 采用分阶段、对照人类演示的评分方式，这意味着单纯的"能完成任务"不足以拿高分，数据合成和失败恢复策略成为拉开差距的关键。
Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge
The CoRL 2026 RoboEval challenge has eight bimanual tasks, and it ranks a policy not only on whether it succeeds but on how: every stage of the task, done safely, efficiently and with both arms in coordination, scored against human demonstrations. My solution has two parts. The generic part, carried over from my LeHome challenge solution, is off-policy RL on a $π_{0.5}$-based policy that is also its own value function. It is trained with AWR and RECAP on advantages of the exact official score, in a loop where every rollout is at the same time a Thompson-sampling trial of about 20 inference parameters. Out of the box, it took the policy to 0.85 RDS on held-out seeds in two days (and it was still very far from convergence). The competition-specific part adds data that local exploration cannot find: generated demonstrations that reach the full reward, and recovery episodes that start from the policy's own failures. The submitted policy reaches 0.990 RDS on held-out seeds and 0.987 on the official seeds.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include π0.5, RoboEval, 强化学习, 机器人操作.
Source attribution: curated from arXiv cs.LG (@Ilia Larchenko).
