这个实验戳破了AI漏洞发现能力的泡沫——即使给定了目标文件，顶级模型复现已知漏洞的成功率也极低。做AI安全评估或漏洞研究的团队，看完会重新审视benchmark的可靠性。
Anthropic在2026年4月发布的Mythos材料中声称其模型能发现真实漏洞，但本文通过控制实验复现了这些漏洞发现任务。实验让GPT-5.5、Claude Opus 4.7和Kimi K2在给定目标文件、只读源码和手动匹配规则下，尝试复现6个已知漏洞。结果显示，GPT-5.5在54次尝试中仅成功5次（覆盖2/6任务），Claude Opus 4.7成功1次，Kimi K2为0次。主要失败原因是模型过早锁定看似合理的候选错误，而忽略了补丁证据中的具体不变性。该结果不否定Anthropic未公开的工作流程，但表明在有利的靶标文件框架下，系统特定提示仅产生少量匹配。
Benchmarking Mythos-Linked Bug Rediscovery
Anthropic's April 2026 Mythos materials combine benchmark claims with concrete bug-finding stories across OpenBSD, FreeBSD, Linux, FFmpeg, and browsers. This paper reports a controlled target-file rediscovery experiment on six public or high-confidence Mythos-linked systems tasks. Each model receives the same target file or files, read-only source tools, three repeats per task, and one manual target-matching rubric; prompts omit CVE identifiers, patch hashes, advisory text, author names, disclosure dates, and answer key root cause language. The experiment contains 54 counted model-task attempts: three models, six tasks, and three repeats, giving 18 attempts per model. GPT-5.5 xhigh achieves 5/18 target rediscoveries, covering 2/6 tasks; counting one wrong-target mpegts.c finding separately gives 3/6 distinct core bugs. Claude Opus 4.7 achieves 1/18 target rediscoveries, covering 1/6 tasks. Kimi K2 records 0/18 target rediscoveries. The dominant failure mode is early commitment to plausible alternate candidates within the assigned file: models often submit source-grounded hypotheses while missing the specific invariant corrected by public Mythos patch evidence. These results do not refute Anthropic's undisclosed workflow, but show that under this favorable target-file scaffold, systems-specific prompting yields only six target matches across 54 counted attempts.