ModelsSelected

XLANG NLP Lab 发布 RoboWild 基准:前沿模型在真实环境中的机器人操作表现欠佳

Why we picked it

做 OSWorld 的 XLANG 团队把基准从电脑屏幕搬进了真实房间,38 个场景、4480 个可交互物体、341 项任务。最强的 Claude Fable 5.2 也只有 13.3% 准确率,GPT-6 Astra 更省 token 但只有 11.6%。做具身智能和机器人智能体的团队值得把数据集跑一遍。

XLANG NLP Lab 在 OSWorld 发布两年后推出新基准 RoboWild,测试前沿模型在物理世界中作为机器人使用智能体的泛化能力。

Body

基准包含 38 个真实布置的场景(家庭、商店、车间、实验室),4,480 个带可操作门、旋钮、抽屉的物体,以及跨四种机器人形态的 341 项任务。成绩显示 Claude Fable 5.

2 以 13.3% 的准确率位居第一,GPT-6 Astra 的 token 效率更高但准确率止步于 11.

6%。相比之下,同团队此前的 OSWorld 已见证模型在数字环境计算机使用任务上快速进步,而物理世界泛化仍远未解决。

该基准的数据集与代码已在 Hugging Face 和 GitHub 开源,主页为 robowild.xlang.

ai。

Original · Junyang Lin

robo osworld! cong! XLANG NLP Lab @XLangNLP 🖥️ Two years after we introduced OSWorld, frontier models have rapidly advanced in computer use, completing real workflows across the digital world. 🖥️ 🔜 🤖: Physical interaction is the next frontier, where foundation models expand into robot-use agents in the physical world while retaining their general intelligence. Agents like Claude Fable 5.2 already score 65% on RoboDojo. Problem already largely solved? Not even close. 🚀 Today we introduce RoboWild: Can Frontier Models Generalize to Robot Use in the Wild? 🏠 38 furnished realistic scenes, from homes and stores to workshops and labs 🛠️ 4,480 objects, with operable doors, dials, drawers, and more 🤖 341 diverse tasks across four robot embodiments 📊 Best results: Claude Fable 5.2 reaches the highest accuracy at 13.3% on RoboWild, while GPT-6 Astra is far more token-efficient yet plateaus at 11.6%. No model is close to solving real robot use yet! 🏠 robowild.xlang.ai unSuwNXnOH huggingface.co/datasets/xlang… /KGdCpGI7bB docs.google.com/presentation/d… o/qQEOti2 github.com/xlang-ai/RoboW… co/w9XUvNd4b huggingface.co/datasets/xlang… .co/6ATZVUbmMD 🧵 [1/8]

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include RoboWild, XLANG NLP Lab, Claude Fable 5.2, GPT-6 Astra.

Source attribution: curated from Junyang Lin (@JustinLin610).