PapersSelected

论文发现基准低分可被人为制造,仅看分数不足以判断模型能力

Why we picked it

搞安全评估的都该看看这篇。它做了个 LoRA 拨盘,能让 Qwen 模型精准考出任意低分,误差不到 2 分,但错误集中在同一选项上,破绽露在错误分布里。做模型评测或红队的人直接有用。

一项新研究指出,安全测试中危险能力探测的低分可能并非模型真实能力,而是被人为压低的结果,单看分数无法证明模型安全。

Body

研究团队训练了一个 LoRA "拨盘",把 Qwen 模型拉向一个更小的 Qwen 模型方向,使其在基准上按指定目标失分,同时对比了提示词压制、升高 temperature、加权重噪声等方法。这个 LoRA 拨盘在未见过的目标分数上误差不超过 1.9 分,而提示词方法的偏差高达 42 到 47 分。

但拨盘留下了痕迹:在多项选择题上,它最多 97% 的新错误都落在同一个选项字母上,天然模型池里没有模型接近这一比例。研究者建议,把低分当作低能力之前,应检查模型具体答错哪些题、错误答案如何分布。

Original · rohanpaul_ai

This paper finds that a low benchmark score can be manufactured precisely, so a low score alone says little about what a model can do.

Safety tests often read a low score on a dangerous-capability probe as reassurance. But a low score can come from a model being held back, so the number alone proves little.

They trained a LoRA "dial" that pulls a Qwen model toward a smaller Qwen model, then compared its errors with the real small model's. They also tried prompting models to underperform, raising temperature, and adding weight noise.

The dial hit unseen target scores within 1.9 points, while prompting missed by 42 to 47 points. But on multiple choice, up to 97% of its new errors landed on 1 letter, a share no natural model in their pool approached.

Before reading a low score as low capability, check which items the model fails and how its wrong answers are spread.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Qwen, LoRA, AI safety, benchmarks.

Source attribution: curated from rohanpaul_ai (@rohanpaul_ai).