CMU 与微软论文：最强 AI 代码审查 agent 仅能发现 71.9% 已知 bug
CMU 和微软造了个 1800 题的 CRBugBench 测 AI 代码审查，最强的 claude-opus-5 也只抓到七成 bug，私有代码库还更差，别急着裁掉人工审查。
Carnegie Mellon 和 Microsoft 发布论文，构建了包含 1800 个代码审查任务的 CRBugBench，每个 pull request 都包含经人工确认并修复过的真实 bug。表现最好的 claude-opus-5 在前 5 条评论中捕获 71.9% 的 bug，比 5 个月前的 opus-4.6 的 47.7% 提升明显。在一家公司的私有代码库上，成绩最多下降 25.5%，因为那些 bug 更依赖项目上下文。作者结论是当前 AI 还不能替代人类审查者，应与人工审查并用，并向 agent 提供代码库自身知识。
New Carnegie Mellon and Microsoft paper finds that the best AI code reviewer caught 71.9% of known bugs in its top 5 comments, still too few to remove human reviewers.
Agents now write code faster than people can review it, tempting teams to let AI review it too. Older review benchmarks scored AI against human comments, which can be nitpicks or miss real bugs.
They built CRBugBench, 1,800 code review tasks where each pull request introduced a bug that humans confirmed and fixed. The agent reviews the original change in a sandbox, and AI judges check whether its top findings name that bug.
The best agent, claude-opus-5, caught 71.9% of bugs in its top 5 comments, up from 47.7% for opus-4.6 5 months earlier. Performance dropped up to 25.5% on 1 company's private repos, where bugs more often need project context.
Use review agents alongside human reviewers, and give them your repo's own knowledge, especially on large or context-heavy changes.
