研究显示AI安全补丁净值为负

“The expected value of a fully LLM-generated, non-human-reviewed patch is a net-negative by a consid...

“The expected value of a fully LLM-generated, non-human-reviewed patch is a net-negative by a considerable margin.” Hedgie @HedgieMarkets 🦔AI can only fix security vulnerabilities 26% of the time. Researchers at 1Password ran over 6,000 AI-generated patches using Claude and ChatGPT against real vulnerabilities. Half the time the AI failed to fix the original bug. 4.5% of the time it created a brand new vulnerability that didn't exist before. And the researchers found that checking an AI-generated security patch takes more effort than just writing the fix yourself. Their conclusion was blunt. "The expected value of a fully LLM-generated, non-human-reviewed patch is a net-negative by a considerable margin." My Take OpenAI launched a program this summer called "Patch the Planet" where AI finds bugs and generates the fixes. These researchers ran 270 patches against one of those same bugs. Zero clean fixes. Not one. Every patch that fixed the original problem created a new vulnerability in the process. The partner that submitted a fix through OpenAI's program produced what the researchers classified as the worst possible outcome, it didn't fully fix the bug and it introduced a new exploit on top of it. Here's what this means if you don't write code for a living. Companies are using these AI tools to patch the software that runs your bank, your hospital, your phone. The pitch has been "AI finds and fixes security holes faster than humans." This study says the AI fix is four times more likely to be broken than working, and a third of the time it recreates the exact same mistakes human programmers already made. Reviewing the AI's work takes longer than doing it yourself. So the speed advantage disappears the moment you try to verify the output, which most companies won't do because the entire point was to move faster. I think we're going to see major breaches traced back to AI-generated patches that nobody checked, and the companies that shipped them are going to blame the tool instead of the decision to trust it. Hedgie🤗 Study 1password.com/files/resource… SX 🔗 View Quoted Tweet 💬 8 🔄 12 ❤️ 53 👀 3646 📊 12 ⚡