Papers

Claude Opus 5.5 盲审小型放疗网络权重,18 个缺陷网络认出 16 个

Weight-space programming for auditing clinical decision networks

Why we picked it

让 Claude Opus 5.5 直接读神经网络权重、盲找放疗筛查网络的 bug,16/18 找中,还只改 3 个参数就修好,挺新鲜的做法。

arXiv 论文测试了 weight-space programming:让 Claude Opus 5.5 在只拿到临床协议和权重的情况下审计 433 到 4801 参数的头颈放疗筛查网络。模型在 18 个被植入缺陷的网络中认出 16 个坏规则(95% CI 67%–97%),包括全部 15 个混淆或阈值偏移案例,修复时中位数只改 3 个参数。所有声明都经约 42 万条请求验证,用协议标签微调也能修复,但要改动几乎全部参数。模型清除了 5 个健康网络中的 2 个,找出 3 个隐藏触发器中的 1 个,说明该方法还不能证明网络完全正确。

Original · arXiv cs.AI

Weight-space programming for auditing clinical decision networks

Machine learning models in radiotherapy are commissioned by testing them on cases with known answers, which may not reveal a shortcut that the test cases share with the training data. Frontier language models have recently been reported to read and write the raw weights of small neural networks, an ability called weight-space programming. The purpose of this study was to determine whether such a model, given only a clinical protocol and a network's weights, can audit and repair the network. Networks with 433 to 4,801 parameters were trained to screen head-and-neck planning requests against four rules. Flaws were planted through the training data: confounds that hid each rule, a 3 Gy shift in the cord threshold, and a hidden trigger. Claude Opus 5.5 audited each network blind, without tools, data or code execution, and every claim and repair was tested on about 420,000 requests. The language model named the broken rule in 16 of 18 flawed networks (95% CI 67% to 97%), including all 15 with a confound or a threshold shift. It repaired each of the 16 by changing a median of 3 parameters. Fine-tuning on protocol-labeled requests also repaired every network, but changed nearly every parameter. Each confound and shift it found was visible in single units, and a simple statistic chosen for each flaw type found the same flaws. It cleared 2 of 5 healthy networks without being forced and found 1 of 3 triggers. Testing against the protocol detected every flaw, and black-box testing found the confounds and the shift faster. The results indicate that, in small networks, weight-space programming can locate and repair flaws that silence an input or shift a threshold, but cannot yet certify that a network is correct.