Hallucination in World Models is Predictable and Preventable

精选理由

这篇论文用具体数据和实验证明世界模型的幻觉本质是数据覆盖问题,还给出了实用的检测和缓解方法。

AI 摘要

该论文发现世界模型的幻觉集中在状态-动作空间的低覆盖区域,并提出轻量级数据驱动信号可检测和缓解。研究者发布了MMBench2数据集(427小时、210个任务),并训练了350M参数的世界模型。他们识别了三种幻觉模式(感知、动作边缘化、场景发散),开发了三个预测信号。覆盖率感知采样和基于好奇心的在线数据收集方法使得仅需50条真实轨迹即可微调模型适应全新环境。

原文 · arXiv cs.LG

Modern generative world models render increasingly realistic action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the ground-truth dynamics. We hypothesize that hallucination concentrates in low-coverage regions of the state-action space, where lightweight data-centric signals can both detect it and guide mitigation. To test this, we introduce MMBench2, a 427-hour, 210-task dataset for visual world modeling with ground-truth actions, rewards, and live simulators, and train a 350M-parameter world model on it. We identify three distinct hallucination modes: perceptual, action-marginalized, and scene-diverging -- each anchored to a different stage of the pipeline, and develop three signals that accurately predict where the model will fail. To close coverage gaps at training time, we develop a coverage-aware sampling technique; to close them online, our hallucination predictors serve as curiosity rewards for targeted data collection, yielding a data-efficient finetuning recipe that adapts the pretrained world model to entirely unseen environments with as few as 50 real environment trajectories. Overall, our findings reveal that hallucination in world models is inherently a data coverage issue, and that the same signals used to detect it can also be used for mitigation. An interactive web version of our paper is available at https://www.nicklashansen.com/mmbench2

  • @OpenAIDevs06-23 19:15原文
  • IT之家06-25 00:32原文
  • OpenAI: 官网动态06-25 02:00原文