CARLA-GS:解耦表示、推理和物理模拟的自动驾驶角落情况生成

CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis

精选理由

这篇论文提出CARLA-GS,用多智能体LLM做推理、PID控制确保物理可行,在Waymo数据上生成逼真角落场景,解决了端到端方法时空不一致的痛点。

AI 摘要

CARLA-GS是一个模块化角落情况合成框架,将视觉表示、语义推理和物理执行解耦但保持耦合。它利用多智能体LLM进行场景推理,生成意图级路径点轨迹,并由PID控制器在CARLA中实现运动控制,最终将车辆状态重投影到高斯场景中渲染。在Waymo Open数据集上,CARLA-GS能生成与语义意图一致且物理可行的真实感视频。定量和定性实验证明了其可控性及时空一致性。

原文 · arXiv cs.AI

CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis

Safety evaluation for autonomous driving is dominated by rare, safety-critical interactions, motivating simulators that can deliberately synthesize corner cases with photorealistic observations. Corner-case generation is inherently a multi-source problem spanning visual representation, scene reasoning, and vehicle trajectory generation and control. Prior knowledge- and model-based approaches typically focus on scene or trajectory components in isolation, while diffusion-based methods attempt end-to-end generation but still struggle to ensure spatiotemporal consistency and physical realism. To unify these aspects within a single framework, we propose CARLA-GS, a modular corner-case synthesis pipeline that decouples visual representation, semantic reasoning, and physics-based execution while maintaining tight cross-module coupling. Starting from real driving data, we reconstruct an editable gaussian scene with additional geometry-consistent constraints. A multi-agent LLM then performs scene-level reasoning to identify risky interactions and generate intent-level waypoint trajectories, while the low-level motion control is delegated to CARLA, where a PID controller ensures kinematic and dynamic feasibility. The simulated vehicle states are finally re-projected into the gaussian scene for ego-centric rendering. This design enables high-level semantic reasoning, low-level physically executable motion, and photorealistic corner-case generation within a unified pipeline. Experiments on the Waymo Open Dataset show, both quantitatively and qualitatively, that our framework enables controllable corner-case generation and produces photorealistic, spatiotemporally consistent videos aligned with semantic intent and physically feasible motion.