这篇论文把4D雷达和相机融合做到360度检测+占用预测,用状态推理串起两个任务,还搞了个新数据集和评估协议。
4DR360提出一种4D雷达-相机框架,实现360°全景感知,将语义占用作为持久状态而非终端输出。该方法采用跨模态状态推理范式,通过State-guided BEV Enhancement(SBE)增强帧内BEV表示,Doppler-guided Temporal Fusion(DTF)在更长时间范围保持状态证据。研究扩展了ManTruckScenes数据集,利用卫星图生成占用标签,并与OmniHD-Scenes构成统一的跨数据集检测-占用协议。实验在雷达-相机多任务评估框架下验证了准确性、鲁棒性、消融和效率。
4DR360: State Reasoning for Joint 3D Detection and Occupancy Prediction in 4D Radar-Camera Full-Scene Perception
Reliable autonomous driving requires full-scene perception that couples foreground objects with dense semantic layout. Recently, 4D millimeter-wave radar has emerged as a robust and affordable sensor, yet its sparse returns make radar-camera fusion necessary for comprehensive scene understanding. Existing radar-camera methods mainly optimize detection, while dual-task systems usually decode boxes and occupancy with limited interaction. To address this gap and advance radar-based multi-task learning, we propose \method, a 4D radar-camera framework for 360$^\circ$ full-scene perception, which models semantic occupancy as a persistent scene state rather than a terminal output. \method{} follows a cross-modal state reasoning paradigm, where the occupancy state is modeled and propagated through stages for coarse-to-fine feature aggregation. Specifically, State-guided BEV Enhancement (SBE) strengthens intra-frame BEV representation, while Doppler-guided Temporal Fusion (DTF) preserves state evidence over longer temporal horizons. Beyond the model, we further extend ManTruckScenes with satellite-map-based generated occupancy labels and pair it with OmniHD-Scenes in a unified cross-dataset detection-and-occupancy protocol. The resulting experiments cover accuracy, robustness, ablation, and efficiency under one radar-camera multi-task evaluation framework. Code and labels will be released upon acceptance.