Library of Hands：免训练修正世界模型的动作映射，机器人成功率从 28.6% 提到 62.3%
Library of Hands: Fix the Hands, Not the World Model
搞机器人世界模型的应该看看这篇。相机一挪、控制器一换，模型就得重训，这活儿很烦。LoH 完全不训练，靠前后两帧图像反推出实际动作映射，还能把映射存起来复用。成功率直接翻倍，开销几乎为零，工程上很容易试。
一篇 arXiv 论文提出 Library of Hands（LoH），一种免训练方法，用于解决世界模型在相机移动或控制器更换后的失效问题。实验显示，冻结的 planner 配合 LeWM 世界模型在指令旋转 90 度后，成功率从 77.5% 跌至 7.8%；现有修复方案需要重新训练模型或依赖训练数据。
LoH 的做法是保持世界模型冻结，只修正指令到实际动作之间的映射（作者称之为 hand）。它无需额外传感器，通过对比两张相机图像之间的变化，找出最能解释该变化的动作来读取 hand；每个识别出的 hand 都会存入库中，再次遇到相同配置时直接识别而非重新学习。闭环测试中，LoH 将三个仿真任务的成功率从 28.6% 提升到 62.3%，且配置未变时仅损失 0.6 个百分点。
该方法无需改动即可适配另外两个世界模型，并在三个基于真实机器人数据预训练的模型中的两个上有效，且增益随读取质量的提高而增长。对需要在不同机器人本体或相机位姿间迁移部署的团队，这提供了一条避开重训练的低成本路径。
Library of Hands: Fix the Hands, Not the World Model
A world model predicts how a scene changes after an action, and a planner uses it to search for actions that reach a goal. If the camera moves or the controller changes, the same command moves the robot differently: when its commands are turned by 90 degrees, a frozen planner with the LeWM world model that succeeds in 77.5% of episodes succeeds in only 7.8%. Current fixes retrain the model, need training data, or forget a set-up when it returns. We propose Library of Hands (LoH), a training-free method that keeps the world model frozen and corrects only the map from commands to applied actions, which we call the hand. LoH reads the hand through the frozen model, with no extra sensor: it finds the action that best explains the change between two camera images (a reading). It stores every hand it identifies, so a returning set-up is recognised rather than re-learnt. In closed-loop tests, LoH more than doubles success on three simulated tasks, from 28.6% to 62.3%, and loses only 0.6 percentage points when nothing changes. It also works, unchanged, with two other world models and on two of three models pretrained on real robot data, and its gain grows with the quality of its readings.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include 世界模型, LeWM, 机器人, arXiv.
Source attribution: curated from arXiv cs.AI (@Hasaan Ahmad, Qizhen Ying, Yuxuan Sun, Tianjin Huang, Guangming Wang).
