斯坦福团队发了个新方法，让标注者用自然语言描述偏好，比单一打分更精确，机器人策略能学到更多结构信息。
斯坦福AI实验室提出Freeform Preference Learning方法，让标注者用自然语言描述轨迹的偏好轴（如速度、精度、子任务完成度），而非仅做单一整体偏好选择。该方法学习基于这些轴的奖励函数，能提取更优策略。论文arXiv:2606.32027和博客已公开。
What if the way we collect human feedback in robotics is quietly losing information? If one traject...
What if the way we collect human feedback in robotics is quietly losing information? If one trajectory drops the cutlery and another drops the plate, asking for a single preference hides information behind the choice. Freeform Preference Learning lets annotators describe preference axes in natural language, learns rewards conditioned on those axes, and extracts better policies. Congrats to @marceltornev , @anubhamahajan01 , @AbhijnyaBhat , and @chelseabfinn ! Marcel Torné @marceltornev We should stop optimizing robot policies against a single overall reward. Trajectories differ along many axes, such as speed, precision, and subtask completion, and one can be better on some while worse on others. If we collapse all of that into a single overall axis we lose this structure making the reward ambiguous and harder to optimize. Blog: freeform-pl.github.io/fpl.website/ Paper: arxiv.org/abs/2606.32027 Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet 💬 0 🔄 2 ❤️ 11 👀 2883 📊 2 ⚡