PapersSelected

XLANG Lab 发布 RoboWild 基准:前沿模型真实环境机器人使用最好成绩仅 13.3%

Why we picked it

OSWorld 团队搞了个新基准 RoboWild,专测模型在真实家庭、商店里开机器人干活,341 个任务、38 个场景。结果很扎眼:Claude Fable 5.2 才 13.3%,比实验室场景的 65% 掉了一大截。做具身智能和机器人 Agent 的朋友可以拿数据集跑跑看。

XLANG NLP Lab 在 OSWorld 发布两年后推出新基准 RoboWild,测试前沿模型在真实物理环境中的机器人使用能力。基准包含 38 个真实布置的场景,覆盖家庭、商店、工作坊和实验室,共 4,480 个带可操作门、旋钮、抽屉的物体,并设置 341 个任务、横跨 4 种机器人本体。成绩方面,Claude Fable 5.2 以 13.3% 的准确率排第一,GPT-6 Astra 更省 token 但止步于 11.6%。相比之下,团队此前在 RoboDojo 上 Claude Fable 5.2 已达 65%,说明实验室环境与真实野外场景之间仍有巨大差距。该基准的数据集已在 Hugging Face 上开放,代码托管在 GitHub,结论是真实环境下的机器人使用远未被解决。

Body

对开发者和研究者来说,RoboWild 提供了一个比 RoboDojo 更贴近真实部署的评测环境:场景更杂乱、物体交互更多样、任务更接近日常物理工作流。模型在数字世界(OSWorld 类任务)进步很快,但迁移到物理世界时泛化能力明显崩塌,13.3% 与 65% 的落差量化了这一鸿沟。四类机器人本体的设置也意味着模型需要跨硬件泛化,而不只是适配单一形态。

Original · Binyuan Hui

Great work!! XLANG NLP Lab @XLangNLP 🖥️ Two years after we introduced OSWorld, frontier models have rapidly advanced in computer use, completing real workflows across the digital world. 🖥️ 🔜 🤖: Physical interaction is the next frontier, where foundation models expand into robot-use agents in the physical world while retaining their general intelligence. Agents like Claude Fable 5.2 already score 65% on RoboDojo. Problem already largely solved? Not even close. 🚀 Today we introduce RoboWild: Can Frontier Models Generalize to Robot Use in the Wild? 🏠 38 furnished realistic scenes, from homes and stores to workshops and labs 🛠️ 4,480 objects, with operable doors, dials, drawers, and more 🤖 341 diverse tasks across four robot embodiments 📊 Best results: Claude Fable 5.2 reaches the highest accuracy at 13.3% on RoboWild, while GPT-6 Astra is far more token-efficient yet plateaus at 11.6%. No model is close to solving real robot use yet! 🏠 robowild.xlang.ai unSuwNXnOH huggingface.co/datasets/xlang… /KGdCpGI7bB docs.google.com/presentation/d… o/qQEOti2 github.com/xlang-ai/RoboW… co/w9XUvNd4b huggingface.co/datasets/xlang… .co/6ATZVUbmMD 🧵 [1/8]

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include RoboWild, XLANG NLP Lab, Claude Fable 5.2, GPT-6 Astra.

Source attribution: curated from Binyuan Hui (@huybery).