面向自动驾驶的闭环交互式世界模型,行为差异降低85%
Making the World React Realistically: A Closed-Loop Interactive World Model for Autonomous Driving
这篇论文把背景车策略、3DGS渲染和信号灯模拟拧成了一个闭环,专攻自动驾驶仿真里'行为假、画面糊'的老毛病。比现有世界模型行为差异降85%、PSNR高2dB,做自动驾驶仿真评测的团队和端到端规划研究者很该看看。
一篇arXiv论文提出闭环交互式世界模型,解决现有模型在行为真实性与视觉保真度之间难以兼顾的问题。方法分三步:先对模仿学习得到的背景车策略做多智能体强化学习微调,让它们与端到端自车规划器在闭环中交互,使行为贴近真实日志;
再用交互中暴露的视角对每个3D Gaussian车辆资产做逐高斯诊断,衰减观测不足视角下的颜色,并用生成的高斯补全从未观测到的表面;最后通过反事实微调训练的编辑器,按车辆驾驶意图渲染信号灯。
在长尾闭环场景上,该方法将交互行为与真实数据的差异降低85%,新视角车辆PSNR较SOTA提升2.0dB。
实验还显示信号灯渲染的意义:约四分之一的场景中,背景车的灯光信号会间接影响自车的驾驶行为。
Making the World React Realistically: A Closed-Loop Interactive World Model for Autonomous Driving
World models (WMs) generate driving scenarios beyond recorded logs, enabling the training and evaluation of autonomous driving systems in unseen interactions. However, existing WMs struggle to ensure both behavioral realism and visual fidelity, since they overlook the relationship between vehicle interactions and the ego visual observations. We propose a closed-loop interactive WM that couples behavior modeling and visual rendering. First, imitation-learned background-vehicle (BV) policies are fine-tuned with multi-agent reinforcement learning in the closed loop against an end-to-end ego planner that reacts to the rendered images, so that their interaction behaviors match those in real logs. Second, the viewpoints exposed by these interactions guide a per-Gaussian diagnosis of each 3D Gaussian vehicle asset, which attenuates view-dependent colors on under-observed Gaussians and fills never-observed surfaces with generated Gaussians, while preserving the observed appearance. Third, signal lights are rendered according to the driving intentions of the vehicles, by an editor that learns the light states unobserved for a vehicle through counterfactual fine-tuning with a light-state classifier. On long-tail closed-loop scenarios, our method reduces the divergence from real-world interaction behaviors by 85% and improves novel-view vehicle PSNR by 2.0dB over the state-of-the-art WM. The significance of BV light signaling in WMs is shown, as the rendered signal lights conveying BV driving intentions indirectly impact ego driving behaviors in a quarter of the scenarios.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include world models, autonomous driving, 3D Gaussian Splatting, reinforcement learning.
Source attribution: curated from arXiv cs.AI (@Jia Hu, Yang Chang, Panqu Wang, Zhijun Fu, Chang Liu, Changjian Yu, Jintao Lai).