Simulated Control Evals Predict AI Safety With Lower Error
Simulated Control Evals
Control evals have so far been limited to purpose-built settings whose transfer to real deployment is questionable; SCEs let safety teams test safeguards against deployment-specific environments cheaply, without executing every task. The 3.6 vs 10.1 percentage-point RMSE gap against BashArena is the concrete benchmark to cite when arguing simulation fidelity. AI safety researchers and red-team builders can apply the technique by generating simulated tool responses for their own target environments.
Researchers introduce simulated control evaluations (SCEs), which use a language model to simulate tool responses in place of a real execution environment when testing AI control safeguards. On LinuxArena, where execution gives ground-truth outcomes, SCEs predict safety with a root mean squared error of 3.6 percentage points across five monitors at a 5% audit budget, versus 10.1 for the BashArena control setting. SCEs show over tenfold lower KL divergence from real execution than BashArena and explain 78-85% of between-environment variance in attack success, monitor scores and safety. The method is sample efficient, needing fewer than 5% as many task-pair draws as the deployment task set. Attack strategies found via simulation also improve monitor performance on held-out LinuxArena attacks, roughly matching executed strategies.
Simulated Control Evals
Control evaluations measure whether safeguards can prevent an AI model from causing undesired outcomes even when it is intentionally trying to do so. These evaluations typically use purpose-built control settings and extrapolate their results to a model's deployment. However, the effectiveness of safeguards in deployment can depend on features of environments and tasks that control settings do not represent. To broaden where control evaluations can be run, we introduce simulated control evaluations (SCEs), which use a language model to simulate tool responses in place of an environment. We evaluate SCEs on LinuxArena, where execution provides ground-truth outcomes. We compare SCE prediction errors with those from a separate control setting, BashArena, and with variation in control metrics within LinuxArena. For independently sampled evaluation and deployment task sets, SCEs predict safety with a root mean squared error of 3.6 percentage points across five monitors at a 5% audit budget, compared with 10.1 for BashArena. SCEs closely reproduce distributions of attack strategies, attacker reasoning, and monitor reasoning, with over tenfold lower KL divergence from execution than BashArena. SCEs explain 78-85% of the between-environment variance in attack success, monitor scores, and safety. This accuracy is sample efficient: SCEs predict safety with less error than BashArena using fewer than 5% as many task-pair draws as the deployment task set. Strategies discovered through simulation also improve monitor performance on held-out LinuxArena attacks, outperforming BashArena and approximately matching improvement from executed strategies. These results suggest that deployment-specific SCEs can be more predictive than control evaluations executed in separate control settings.
- METR10-22 15:26原文
