OSWorld-Pro：面向计算机操作智能体的过程化评测基准
OSWorld-Pro: Process-based Evaluation for Computer Use Agents
智能体评测不再只看最终结果，按子目标逐步打分，Claude Opus 5 也只拿 75.7%。
研究者推出 OSWorld-Pro，包含 300 多个任务和 2800 多个子目标，专门评测计算机操作智能体（CUA）的执行过程而非只看最终交付物。该基准基于超过 67000 条人工标注，用经过人工对齐的 LLM-Judges 逐个判定子目标完成情况。测试显示 Claude Opus 5 在 OSWorld-Pro 上仅得 75.7%，低于其在 OSWorld 上的 83.4%。团队还归纳出子目标无关操作、点击失误等过程性失败模式，指出键盘输入出错和 UI 点击出错需要不同的改进策略。
OSWorld-Pro: Process-based Evaluation for Computer Use Agents
Evaluation of Computer-Use Agents (CUAs) is often limited to the final deliverables they create (at the end of hundreds of steps) and assessed with functional verifiers, as seen in OSWorld. However, such evaluation of end-state performance lacks transparency into how and why agents fail in various tasks, obfuscating critical insight for subsequent improvement. For instance, agents that err during keyboard inputs would require a different mitigation strategy from those that fail to precisely provide click-based inputs on the graphical UI. We introduce OSWorld-Pro: a set of over 300 tasks containing over 2800 subgoals to enable the procedural evaluation of CUAs grounded in over 67,000 human annotations. We use robust human-aligned LLM-Judges to evaluate the fulfillment of OSWorld-Pro subgoals and thereby reveal the progress that models make throughout a series of sequentially dependent subgoals. Our findings reveal that OSWorld-Pro is challenging even for state-of-the-art LLMs, with top performers like Claude Opus 5 achieving only 75.7% vs. 83.4% on OSWorld. Furthermore, we identify critical process-focused failure modes of various models (e.g. subgoal-irrelevant actions and click-based mistakes) to provide insights to improve performance and efficiency of CUAs.