做机器人操控和模仿学习的团队值得关注——傅里叶特征解决了神经网络对低频偏好的固有问题,让你在点云策略上直接获得高精度提升,代码和视频已开源,可以直接上手试。
该研究提出将点云从笛卡尔空间映射到高维傅里叶空间,以解决神经网络在模仿学习中难以学习高频空间特征的问题。在 RoboCasa 和 ManiSkill3 基准测试以及真实机器人实验中,傅里叶特征显著提升了基于点云的策略在精细操控任务上的表现。该方法简单、鲁棒,且适用于多种编码器架构,有望成为点云模仿学习的通用工具。
Fourier Features Let Agents Learn High Precision Policies with Imitation Learning
High-precision robotic manipulation requires fine-grained spatial reasoning that is often difficult to achieve with RGB-only policies due to depth ambiguity and perspective scale issues. Policies that leverage 3D information directly, such as those based on point clouds, offer a stronger geometric prior over purely image-based ones, yet their performance remains highly task-dependent. We hypothesize that this discrepancy may be due to the spectral bias of neural networks towards learning low frequency functions, which especially affects architectures conditioned on slow-moving Cartesian features. We thus propose to map point clouds from Cartesian space into high-dimensional Fourier space, effectively equipping the point cloud encoder with direct access to high-frequency features. We experimentally validate the use of Fourier features on challenging manipulation tasks from the RoboCasa and ManiSkill3 benchmarks and on a real robot setup. Despite their simplicity, we find that Fourier features provide significant benefits across diverse encoder architectures and benchmarks and are robust across hyperparameters. Our results indicate that Fourier features let policies leverage geometric details more effectively than Cartesian features, showing their potential as a general-purpose tool for point cloud-based imitation learning. We provide source code and videos on our project page: https://fourier-il.github.io/fourier-il