灵巧操作与决策感知的闭环评估是具身智能的硬骨头,做机器人操作或具身智能的团队值得看看这个新基准怎么暴露真实部署中的累积错误。
DexHoldem 是一个基于 ShadowHand 灵巧手系统的真实世界基准测试,围绕德州扑克操作任务设计。它提供了 1470 个遥操作演示、14 种操作原语、标准化物理策略基准和智能体感知基准。实验显示,π0.5 在原始执行上完成率最高(61.2%),而 Opus 4.7 在感知准确率上领先(34.3%),但整体闭环部署中感知与策略错误会累积。该工作旨在评估灵巧操作、智能体感知和具身决策路由的协同能力。
DexHoldem: Playing Texas Hold'em with Dexterous Embodied System
Evaluating embodied systems on real dexterous hardware requires more than isolated primitive skills: an agent must perceive a changing tabletop scene, choose a context-appropriate action, execute it with a dexterous hand, and leave the scene usable for later decisions. We introduce DexHoldem, a real-world system-level benchmark built around Texas Hold'em dexterous manipulation with a ShadowHand. DexHoldem provides 1,470 teleoperated demonstrations across 14 Texas Hold'em manipulation primitives, a standardized physical policy benchmark, and an agentic perception benchmark that tests whether agents can recover the structured game state needed for embodied decision making. On primitive execution, $π_{0.5}$ obtains the highest task completion rate ($61.2\%$), while $π_{0.5}$ and $π_0$ tie on scene-preserving success rate ($47.5\%$). On agentic perception, Opus 4.7 obtains the best strict problem-level accuracy ($34.3\%$), while GPT 5.5 obtains the best average field-wise accuracy ($66.8\%$), exposing a gap between isolated visual sub-capabilities and complete routing-relevant state recovery. Finally, we instantiate the full embodied-agent loop in three case studies, where waiting, recovery dispatches, human-help requests, and repeated primitive execution reveal how perception and policy errors accumulate during closed-loop deployment. DexHoldem therefore evaluates dexterous tabletop execution, agentic perception, and embodied decision routing in a shared physical setting. Project page: https://dexholdem.github.io/Dexholdem/.