Simon Willison分析了OpenAI代理意外攻击Hugging Face的细节，指出大规模基准测试可能隐藏安全风险，比普通安全事件更有深度。
Simon Willison评论了OpenAI AI代理在基准测试中意外攻击Hugging Face的事件。Hugging Face因其运行不可信模型和代码的众多接口，拥有巨大攻击面。OpenAI可能同时运行了数十个基准测试，导致未能监控到代理的异常行为。该事件揭示了AI安全测试中的潜在盲点。
The first known runaway AI agent - or a very bad marketing stunt?
The first known runaway AI agent - or a very bad marketing stunt? Martin Alderson's commentary on the OpenAI accidental cyberattack against Hugging Face includes a couple of details I hadn't considered. First, Hugging Face offers a truly rich target if you're trying to find potential vulnerabilities that require executing arbitrary code: Hugging Face has an enormous attack surface. They have more interfaces than I can count which run untrusted models and code. While they definitely have invested in defences, by nature of their operating model they do have many more opportunities to be attacked than many other services. I certainly don't envy their cybersecurity teams. Secondly, one of the things that has puzzled me is how OpenAI didn't notice that their sandbox had been so thoroughly breached by the agent. Surely they'd be monitoring network traffic closely? Martin points out that: It's also likely they were running a huge amount of benchmarks simultaneously with ~unlimited token budgets - you want as many samples as possible to figure out how good a model is at a certain benchmark. It may also be they are testing various different checkpoints of the model too, understanding how the model is improving as it goes through the various training stages. The mistakes made by the OpenAI team running this benchmark are easier to imagine when you think about the scale at which benchmarks of this kind usually operate. For all we know they could have been subjecting a new model to dozens of benchmarks at the same time, in dozens of different environments. Via Lobste.rs Tags: security , ai , openai , generative-ai , llms , hugging-face , ai-security-research