CUA-Universe Hybrid Agent Environment
CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents
The 9B model reduces token usage by 60% in hybrid tasks, offering developers a more efficient approach to computer automation than GUI-only methods.
CUA-Universe is a scalable environment-to-data pipeline that transforms real desktop software into hybrid GUI+CLI environments. The system includes App-Forge for creating reproducible VMs and command-line surfaces, Task-Weave for synthesizing diverse hybrid tasks, and Path-Steer for efficient rollout steering. A 9B model trained on this data improved success rates by 39.3 points on CUA-Verse and 16.8 points on OSWorld benchmarks.
CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents
Computer-use agents have advanced on benchmarks like OSWorld and AndroidWorld, but still act mostly through the GUI, often producing inefficient trajectories. Real-world computer work is hybrid, combining visual-state inspection with precise, high-throughput command-line operations, so capable agents must coordinate both modalities over shared application state. Yet scalable hybrid environments remain scarce because supporting both GUI and CLI over real applications typically requires substantial manual engineering for each application. Existing agents also struggle to use the two interfaces complementarily: CLI-native agents lack visual perception for tasks involving interface state or layout, while GUI-native agents are inefficient for operations better executed through commands. We introduce CUA-Universe, a scalable environment-to-data pipeline that turns real desktop software into hybrid GUI+CLI environments. App-Forge adapts applications into reproducible VMs and command-line surfaces it discovers, wraps, or generates, scaling to 16 applications; Task-Weave synthesizes diverse hybrid tasks of controllable difficulty from reusable operations over seed files; and Path-Steer steers rollouts along efficient hybrid paths and harvests verified trajectories for post-training. Training on this data shifts behavior from inefficient GUI interaction and brittle CLI scripting toward effective GUI+CLI orchestration. Our 9B model improves both success and efficiency on CUA-Verse (Score +39.3 pts; -37% steps, -60% tokens), OSWorld (SR +16.8 pts; -57% steps, -44% tokens), and OSWorld-MCP (Score +7.84 pts; -27% steps, -30% tokens). CUA-Universe provides a scalable path toward more capable and efficient computer-use agents.
