ID-MAE:用多模态掩码扩散自编码器检测驾驶受损状态
Multimodal Masked Diffusion Autoencoder for Modeling Impaired Driving
这套方法很实用:只用 gaze 和轨迹就能推断分心、疲劳这些驾驶受损状态,不用花钱做大规模人工标注。DIVERT 数据集有 103 小时、十种受损类型,做车载安全或驾驶员监控的团队可以直接参考它的多模态自监督思路。
研究人员提出 ID-MAE,一种多模态掩码扩散自编码器,将自监督目标引入驾驶受损预测任务。模型除预测受损标签外,还重建被掩码的视线 gaze 与车辆轨迹数据,通过重建损失提升表征质量。
配套发布的 DIVERT 驾驶模拟器数据集包含 103 小时驾驶数据、109 名受试者,涵盖认知、视觉、情绪三类共十种受损诱导,记录了视线、车辆状态和场景视频。在位置匹配样本对上,ID-MAE 的受损检测 AUC 达到 0.706,不使用重建损失时为 0.590,同时优于现有基线。模型在轨迹预测和视线预测等任务上也表现强劲。
消融实验覆盖输入模态、掩码方案、位置编码、注意力机制和解码器等设计维度,并发现视觉负荷型干扰与认知型干扰在训练和推理中可被区分开。该方法减少了人工标注认知状态的成本,代码和数据将在论文接收后开放。
Multimodal Masked Diffusion Autoencoder for Modeling Impaired Driving
Human cognitive state (e.g., distraction, stress), gaze, and behavior interact with the driving environment in subtle ways that AI could use to assist drivers, but ground-truth labels for human cognitive state levels are costly to obtain at scale. We propose a multimodal masked-diffusion autoencoder, ID-MAE, that adds a self-supervised objective to impairment prediction: alongside predicting the impairment label as a masked token, it reconstructs masked gaze and trajectory. To probe the effects of different modalities and types of impairments on model inference, we introduce DIVERT, a driving-simulator dataset containing 103 hours of driving data (n = 109) with ten impairment inductions spanning cognitive, visual, and emotional states, recorded with gaze, vehicle state, and scene video. We demonstrate how ID-MAE affords significant improvements in impairment inference results on both the DIVERT dataset and existing datasets, and enables strong results in additional tasks, such as trajectory and gaze prediction. For location-matched pairs, our model achieves a paired AUC of 0.706 for impairment detection on DIVERT, compared to 0.590 without reconstruction loss, improving over existing baselines. We probe the design space in terms of input modalities, masking scheme, positional encoding, attention mechanism, and decoding. We also investigate the interaction between various impairments in training and inference, demonstrating that visually demanding distractions are separable from cognitive distractions. Code and data will be available upon acceptance.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include ID-MAE, DIVERT, multimodal, self-supervised learning.
Source attribution: curated from arXiv cs.LG (@John Gideon, Anshul Gupta, Megan Applegate-Kenton, Kayli Battel, Todd J Rowell, Patricio Reyes Gomez, Laporsha Dees, Emily Sumner, Avinash Balachandran, Guy Rosman).