EpiSEM-Bench：评测 LLM 智能体在证据缺失下的安全表现
EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence
一个专门测智能体“证据不全时会不会乱来”的基准，300 组任务对测了十款模型，做 agent 安全的人可以直接拿来用。
EpiSEM-Bench 是一个可执行基准，包含 300 组匹配的不安全/安全任务对，覆盖六个领域，用于评测 LLM 智能体在关键证据缺失时是否仍会执行被禁止的操作。基准通过系统性抑制关键事实，测量不安全执行率并确定最小观测失败基数。独立审计工具检验智能体是否会主动寻找缺失事实，并据此拒绝禁止性操作、完成允许性任务。在十款模型的测试中，抗证据缺失能力与实际恢复能力之间存在明显分化，提供恢复访问权限后所有十款模型的不安全执行都有下降。
EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence
Tool-using language model agents rely on environmental facts to decide whether an action is permitted. When a tool response omits a safety-relevant fact, an agent may carry out a prohibited action even though the user request is benign and every returned statement is true. We introduce \bench{}, an executable benchmark for evaluating unsafe execution, safety boundaries, and evidence recovery in LLM agents under missing evidence. It contains 300 matched unsafe/safe task pairs across six domains. By systematically suppressing critical facts, the benchmark measures unsafe execution under critical evidence omission and determines the minimum observed failure cardinality, including failures with full evidence and cases with no observed failure after exhaustive testing. An independent audit tool tests whether agents seek missing facts and then withhold prohibited actions while completing permissible tasks. The benchmark reveals substantial variation across the ten models in omission robustness, evidence acquisition, and appropriate action. Resistance to missing evidence does not necessarily translate into useful recovery. Recovery access reduces unsafe execution for all ten models, but agents differ in whether they actively obtain the missing facts and use them to act appropriately in both worlds.
