论文精选73°

BLOOM-WILT:大模型行为审计新方法

BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing

精选理由

BLOOM-WILT让大模型安全审计更高效,在Qwen3.5-4B测试中行为诱导率翻倍,还推翻了原有模型安全排名。

AI 摘要

研究人员推出BLOOM-WILT,一种无需训练成本的大模型行为审计管道。该方法在4个目标模型和8种行为测试中,以30/32的成绩超越基线审计器。在Qwen3.5-4B自残鼓励行为测试中,将行为出现率从51%提升至100%。WILT通过自适应重加权目标解码策略,在不降低输出概率的情况下提高行为诱导效率。

原文 · arXiv cs.AI

BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing

Users of a deployed language model routinely encounter behaviours that testing almost never surfaces, since deployment puts the model through orders of magnitude more interactions than any evaluation can simulate. Automated auditors make testing cheap to scale and flexible enough to cover almost any specified behaviour, yet their lack of optimisation pressure makes them sample-inefficient. To address this shortcoming, we introduce BLOOM-WILT, a full auditing pipeline that elicits natural multi-turn instances of rare behaviours, without training cost or access beyond the target's next-token distribution. On the input side, WILT's auditor model revises its conversational strategy across rounds, learning from previous scored interactions. On the output side, WILT adaptively reweights the target's decoding using the model's own distribution conditioned on an elicitation prompt, so that behaviour-relevant generations are sampled ahead of others it finds equally probable when unprompted. We evaluate WILT across 4 target models and 8 behaviours, where it beats the baseline auditor in 30 of the 32 settings and overturns the previous model safety rankings. WILT raises average behaviour presence from 51% to 100% when eliciting self-harm encouragement from Qwen3.5-4B, beating every elicitation method we port into the same pipeline at matched compute, without pushing output probability below the baseline's.