做网页自动化或视觉智能体的开发者终于有了一个开源的在线RL训练方案——OpenWebRL 用极少的训练数据就达到了接近专有系统的效果，值得直接拿来试试。
OpenWebRL 是一个开源框架，用于在真实网站上通过在线多轮强化学习训练视觉网页智能体。该框架覆盖完整训练流程，包括可扩展的实时浏览器基础设施、监督初始化、多模态上下文管理、轨迹级成功判断和高效的多轮策略优化。基于此框架训练的 OpenWebRL-4B 模型仅用 0.4K 初始化轨迹和 2.2K 开放 RL 训练任务，就在 Online-Mind2Web 和 DeepShop 基准上分别达到 67.0% 和 64.0% 的成功率，超越了同规模或更大规模的开源智能体，并接近 OpenAI CUA 和 Gemini CUA 等专有系统。这项工作解决了高质量演示数据收集昂贵和静态数据集覆盖有限的问题，为构建更强大、可复现且成本高效的开源网页智能体提供了实用路径。
OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents
Building capable visual web agents requires long-horizon reasoning, precise grounding, and robust interaction with dynamic real-world websites. Despite rapid progress, the strongest systems remain largely proprietary, while open agents still depend heavily on supervised post-training over large collections of curated web trajectories. This dependence creates a major scalability bottleneck: high-quality demonstrations are expensive to collect, and static datasets offer limited coverage of the diverse, ever-changing open web. Although online RL has shown promise for text-based agents, its potential for training visual web agents directly on live websites remains largely underexplored. In this paper, we introduce OpenWebRL, an open framework for training visual web agents with online multi-turn RL on real websites. OpenWebRL covers the full training pipeline, including scalable live-browser infrastructure, supervised initialization, multimodal context management, trajectory-level success judging, and efficient multi-turn policy optimization. Using this framework, we train OpenWebRL-4B, which establishes a new open-source state of the art on challenging live-web benchmarks. With only 0.4K initialization trajectories and 2.2K open-ended RL training tasks, OpenWebRL-4B achieves 67.0% success on Online-Mind2Web and 64.0% on DeepShop, outperforming prior open agents of similar or larger scale and remaining competitive with proprietary systems including OpenAI CUA and Gemini CUA. Beyond strong benchmark performance, we systematically study the key design choices that make online RL effective for visual web agents, and analyze how RL improves agentic reasoning. Overall, our work offers a practical path toward building more capable, reproducible, and cost-efficient open web agents. We will release our training data, models, and code to support future research.