DARP 用检索替代全局映射,解决了行为克隆在部署时误差累积的痛点,做机器人学习和模仿学习的开发者可以直接参考其开源代码。
华盛顿大学团队提出 DARP(差异感知检索策略),一种半参数检索式模仿学习方法,通过重用在推理时的训练数据来缓解行为克隆的分布外泛化问题。DARP 不学习全局策略,而是基于专家演示的 k 近邻、对应动作及邻居与查询状态的相对距离向量来预测动作。该方法无需额外数据收集、在线专家反馈或任务特定知识,在连续控制和机器人操作等任务上比标准行为克隆提升 15-46%。代码和演示已开源。
Difference-Aware Retrieval Policies for Imitation Learning
Parametric imitation learning via behavior cloning can suffer from poor generalization to out-of-distribution states due to compounding errors during deployment. We show that reusing the training data during inference via a semi-parametric retrieval-based imitation learning approach can alleviate this challenge. We present Difference-Aware Retrieval Policies for Imitation Learning (DARP), a semi-parametric retrieval-based imitation learning approach that addresses this limitation by reparameterizing the imitation learning problem in terms of local neighborhood structure rather than direct state-to-action mappings. Instead of learning a global policy, DARP trains a model to predict actions based on $k$-nearest neighbors from expert demonstrations, their corresponding actions, and the relative distance vectors between neighbor states and query states. DARP requires no additional assumptions beyond those made for standard behavior cloning -- it does not require additional data collection, online expert feedback, or task-specific knowledge. We demonstrate consistent performance improvements of 15-46% over standard behavior cloning across diverse domains, including continuous control and robotic manipulation, and across different representations, including high-dimensional visual features. Code and demos are available at https://weirdlabuw.github.io/darp-site/.