做具身 AI 安全或人机协作的团队,VLESA 提供了一个可落地的实时安全监控方案,能根据上下文判断危险动作,建议直接看论文和代码。
VLESA 是一个面向具身 AI 的安全框架,通过分析第一人称视频实时预测危险动作并触发干预。它解决了“意图依赖的安全”问题——相同动作在不同情境下可能安全或危险。研究团队引入了配对第一人称帧与目标条件安全标注的数据集,并训练了基于 GRPO 的目标条件安全 Q 过滤器,无需重新训练即可评估动作安全性。在 ASIMOV-2.0 基准上,VLESA 在精确帧上实现了更高的干预准确率,GRPO 训练的 Q 过滤器通过目标条件约束解码将动作安全性提升了超过 41 个百分点。代码已开源。
VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring
As AI systems increasingly assist humans in physical tasks, ensuring safety becomes paramount -- physical actions carry immediate and irreversible consequences that digital errors do not. We introduce the Vision-Language Embodied Safety Agent (VLESA), a framework that monitors human activities from egocentric video and triggers real-time safety interventions when dangerous actions are predicted. VLESA addresses intent-dependent safety where identical actions can be safe or dangerous depending on context. A dataset pairing egocentric frames with goal-conditioned safety annotations is introduced, enabling a goal-conditioned safety Q-filter trained via GRPO that evaluates actions with respect to inferred intent without retraining. On top of that, an intent-action prediction agent is proposed to jointly infer goals and predict future actions from video. On the ASIMOV-2.0 benchmark, VLESA achieves higher intervention accuracy at the exact ground-truth frame compared to baselines, while the GRPO-trained Q-filter improves action safety by over 41 percentage points through goal-conditioned constrained decoding. Code is available at https://github.com/HanjiangHu/VLESA.