做GUI自动化或智能体开发的团队，这个工作解决了混合动作空间路径选择的痛点——不用手动收集真实工具轨迹也能训练出高效智能体，建议点开看看方法细节。
计算机使用智能体（CUA）在同时使用原子GUI操作（如点击、输入）和高级工具调用（如API文件操作）时，常因无法判断何时切换而导致执行路径次优。为解决这一问题，研究者提出ToolCUA，一种端到端智能体，通过分阶段训练范式学习最优GUI-工具路径选择。其核心包括：利用静态GUI轨迹合成工具库的轨迹缩放流水线、结合单步强化学习的工具引导GUI RFT，以及在线智能体强化学习优化。在OSWorld-MCP基准上，ToolCUA达到46.85%准确率，相比基线提升约66%，并比纯GUI设置提升3.9%，证明了混合动作空间训练的有效性。项目已开源。
ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agents
Computer Use Agents (CUAs) can act through both atomic GUI actions, such as click and type, and high-level tool calls, such as API-based file operations, but this hybrid action space often leaves them uncertain about when to continue with GUI actions or switch to tools, leading to suboptimal execution paths. This difficulty stems from the scarcity of high-quality interleaved GUI-Tool trajectories, the cost and brittleness of collecting real tool trajectories, and the lack of trajectory-level supervision for GUI-Tool path selection. In this paper, we propose ToolCUA, an end-to-end agent designed to learn optimal GUI-Tool path selection through a staged training paradigm. We first introduce an Interleaved GUI-Tool Trajectory Scaling Pipeline that repurposes abundant static GUI trajectories and synthesizes a grounded tool library, enabling diverse GUI-Tool trajectories without manual engineering or real tool-trajectory collection. We then perform Tool-Bootstrapped GUI RFT, combining warmup SFT with single-turn RL to improve decisions at critical GUI-Tool switching points. Finally, we optimize ToolCUA with Online Agentic RL in a high-fidelity GUI-Tool environment, guided by a Tool-Efficient Path Reward that encourages appropriate tool use and shorter execution paths. Experiments on OSWorld-MCP show that ToolCUA achieves 46.85% accuracy, a relative improvement of approximately 66% over the baseline, establishing a new state of the art among models of comparable scale. It also improves by 3.9% over GUI-only settings, demonstrating effective GUI-Tool orchestration. The results further suggest that training in a hybrid action space is a promising paradigm for real-world digital agents. Open-sourced here: https://x-plug.github.io/ToolCUA/