安全研究员和红队成员终于有了评估 AI 攻击能力的标准化工具——ExploitGym 覆盖真实漏洞和防御场景，做渗透测试或 AI 安全评估的团队可以直接拿来用。
ExploitGym 是一个大规模、多样化的基准测试，用于评估 AI 智能体将安全漏洞转化为实际攻击的能力。该基准包含 898 个来自真实世界漏洞的实例，涵盖用户空间程序、Google V8 JavaScript 引擎和 Linux 内核三个领域。评估显示，前沿模型如 Anthropic 的 Claude Mythos Preview 和 OpenAI 的 GPT-5.5 能成功利用 157 和 120 个漏洞实例，即使在启用常见防御措施后仍保持一定成功率。这项工作揭示了 AI 智能体在网络安全中的双重用途风险，为防御和攻击场景提供了重要测试平台。
ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?
AI agents are rapidly gaining capabilities that could significantly reshape cybersecurity, making rigorous evaluation urgent. A critical capability is exploitation: turning a vulnerability, which is not yet an attack, into a concrete security impact, such as unauthorized file access or code execution. Exploitation is a particularly challenging task because it requires low-level program reasoning (e.g., about memory layout), runtime adaptation, and sustained progress over long horizons. Meanwhile, it is inherently dual-use, supporting defensive workflows while lowering the barrier for offense. Despite its importance and diagnostic value, exploitation remains under-evaluated. To address this gap, we introduce ExploitGym, a large-scale, diverse, realistic benchmark on the exploitation capabilities of AI agents. Given a program input that triggers a vulnerability, ExploitGym tasks agents with progressively extending it into a working exploit. The benchmark comprises 898 instances sourced from real-world vulnerabilities across three domains, including userspace programs, Google's V8 JavaScript engine, and the Linux kernel. We vary the security protections applied to each instance, isolating their impact on agent performance. All configurations are packaged in reproducible containerized environments. Our evaluation shows that while exploitation remains challenging, frontier models can successfully exploit a non-trivial fraction of vulnerabilities. For example, the strongest configurations are Anthropic's latest model Claude Mythos Preview and OpenAI's GPT-5.5, which produce working exploits for 157 and 120 instances, respectively. Notably, even with widely used defenses enabled, models retain non-trivial success rates. These results establish ExploitGym as an effective testbed for exploitation and highlight the growing cybersecurity risks posed by increasingly capable AI agents.