这篇论文教机器人像人类一样用盘子切蛋糕，不受工具原始功能限制。用VLM选工具、SAM定位，零样本效果比现有方法好。
GROW^2提出一种层次化功能定位方法，将物体部件作为抽象层分割语义和几何过程。语义层利用VLM（如GPT-4V）分析任务指令并选择合适工具及关键部件；几何层通过视觉基础模型（如SAM）从单张RGB-D图像定位精确3D区域。在功能预测基准上，GROW^2优于现有基线方法，并实现对开集物体的零样本泛化。仿真和真实机器人工具使用实验均验证了其有效性，例如用盘子代替刀切蛋糕。
GROW$^2$: Grounding Which and Where for Robot Tool Use
Can the robot use a plate to cut a cake if no knife is available? Tool use greatly expands robot capabilities, but to use tools creatively beyond their intended functions, the robot faces the challenge of $\textit{open-world affordance grounding}$: select an open-category object to act as a tool and localize its specific region of action. To this end, we introduce GROW$^2$ (GROunding Which and Where), which leverages object parts as a natural abstraction to split the grounding process hierarchically into semantic and geometric levels, thus bypassing the need for data-heavy, end-to-end training. Semantically, GROW$^2$ harnesses the commonsense reasoning of Vision-Language Models (VLMs) to parse a natural-language task instruction, select a suitable object as the tool, and identify task-relevant parts on the tool and the target object. Geometrically, vision foundation models then ground the selected parts into precise 3D regions from a single RGB-D image. Experiments on established benchmarks show that GROW$^2$ outperforms state-of-the-art baselines on affordance prediction benchmarks. Further, it achieves zero-shot generalization over open-category objects and outperforms baselines in both simulated and real-world robot tool use experiments.