铰接工具操作是机器人灵巧操作的硬骨头,Mana 用动画思路解决了数据生成和迁移难题,做机器人操作或 sim-to-real 的团队可以直接参考其零样本迁移方法。
Mana 提出了一种将灵巧操作视为动画问题的 sim-to-real 框架,解决了铰接工具操作中协调内部自由度与接触交互的难题。该框架通过粗到细的流水线,将程序化生成的关键帧转化为操作轨迹,结合运动规划与强化学习实现零样本迁移。数据生成几乎全自动,每个工具仅需不到一分钟的鼠标点击指定功能属性。在四种不同铰接工具上,Mana 实现了零样本的 sim-to-real 抓取与手内操作,展示了可扩展的灵巧操作方案。
Mana: Dexterous Manipulation of Articulated Tools
Articulated tool manipulation remains a major challenge in dexterous robotics due to the need to coordinate internal degrees of freedom and contact-rich interactions. While prior work has largely focused on rigid objects, articulated tool use remains underexplored because of its physical complexity and the difficulty of learning functional grasping and manipulation policies. We present Mana (Manipulation Animator), a general sim-to-real framework that reinterprets dexterous manipulation as an animation problem. Inspired by computer animation, Mana employs a coarse-to-fine pipeline that transforms procedurally-generated grasp keyframes into manipulation trajectories through motion planning and reinforcement learning. The data generation process is largely automatic, requiring only a few mouse clicks to specify functional affordances (<1 minute per tool). Across four articulated tools spanning different scales and joint types, Mana achieves zero-shot sim-to-real transfer for both grasping and in-hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.