文本安全外物理危险探测:LLM隐态风险空间分析

When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space

精选理由

这篇论文发现LLM的物理危险和文本内容危险是分开的,PRISM探针只需一层就能检测物理危险,误报率只有LLM裁判的一半,值得做安全对齐的人看看。

AI 摘要

研究显示,大型语言模型(LLM)作为具身智能体规划器时,文本安全的指令在物理世界中可能变得危险。通过隐态方向分析和随机分割零检验,在Qwen2.5-3B/7B/14B/32B、Phi-3.5和SmolLM2上发现内容危险(CD)与物理危险(PD)在LLM表征中可分离。基于此提出的PRISM单层L2正则逻辑回归探针,在SafeAgentBench上达到86.2-87.7%准确率、11.7-13.7%假阳性率,而同等规模LLM裁判的假阳性率为24.7-39.0%。新构建的PhysicalSafetyBench-1K(PSB-1K)包含1000个无直接危害关键词的物理风险对比对,PRISM在此达到99.6%准确率、0.7%假阳性率,Qwen2.5-3B裁判则错误拒绝67.8%的安全任务。实验在SafeText和EARBench上也复现了结果,支持隐态状态探测作为超越文本审核的物理安全表征级方法。

原文 · arXiv cs.AI

When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space

Large language models (LLMs) increasingly serve as high-level planners for embodied agents, where linguistically benign instructions can become unsafe once grounded in the physical world. We study whether this physically grounded danger is the same safety problem as ordinary text-level content danger. Through hidden-state direction analysis and random-split null tests, we show that content danger (CD) and physical danger (PD) form separable signals in LLM representations across Qwen2.5-3B/7B/14B/32B, Phi-3.5 and SmolLM2. Building on the CD/PD separability, we propose PRISM, a single-layer L2-regularized logistic probe over full hidden states. PRISM achieves 86.2--87.7\% accuracy on SafeAgentBench with 11.7--13.7\% FPR, while same-scale LLM judges over-block safe tasks at 24.7--39.0\% FPR. We further introduce PhysicalSafetyBench-1K (PSB-1K), a contrastive benchmark of 1{,}000 physical-risk pairs without direct harm keywords, to test whether methods detect physically grounded danger rather than explicit unsafe wording. On PSB-1K, PRISM reaches 99.6\% accuracy and 0.7\% FPR, whereas a Qwen2.5-3B judge rejects 67.8\% of safe tasks. PRISM also replicates on SafeText and EARBench, supporting hidden-state probing as a representation-level method for physical safety beyond text moderation.