BypassGym:首个在真实终端上评估 AI 代理绕过 EDR 能力的基准
BypassGym: A Benchmark for Evaluating AI Agents' Endpoint Defense Evasion Capabilities
安全研究者应该看看这个基准：它不是纸面推演，而是在 Proxmox 虚拟机里跑真实 Windows 和 6 款 EDR。连 DeepSeek-V4.1 Flash 这种轻量模型都能 18 次里成功 17 次，说明攻防门槛在快速变化，做 EDR 采购或红队的团队都能拿来测自家防御。
研究者发布 BypassGym，这是首个在真实终端环境系统评估 AI 代理绕过 EDR（终端检测与响应）能力的基准。任务设定为：给定 C2 shellcode，代理需生成并编译一个 loader，使载荷执行时不被 EDR 拦截或终止。
评测环境构建在 Proxmox VE 上，在虚拟机中运行真实 Windows 系统和 6 款 EDR 产品，并向代理返回结构化的检测反馈。基准还附带一个防御规避知识技能模块，支持带与不带该输入的对比测试。
团队用 5 个轻量级模型后端对抗 6 款 EDR 产品和 3 种载荷，发现即使这些简单模型也能产出成功样本：DeepSeek-V4.1 Flash 达到 17/18 成功，而 Bitdefender 仅放行了 15 次运行中的 5 次。这为防御方提供了当前 AI 代理规避能力的系统性测量数据。
BypassGym: A Benchmark for Evaluating AI Agents' Endpoint Defense Evasion Capabilities
Artificial intelligence agents increasingly automate cybersecurity tasks, including offensive operations. Frequent attacks across the web make endpoint detection and response (EDR) a critical defense, but whether AI agents can evade it remains unclear. Benchmarks evaluate the offensive capabilities of large language model (LLM) agents in areas such as penetration testing and vulnerability discovery. One important capability that these benchmarks do not address is defense evasion, specifically whether an agent can develop a program that executes a payload without being blocked or terminated by a deployed EDR. We introduce BypassGym, to our knowledge the first benchmark to systematically evaluate this capability on real endpoints. Given command-and-control (C2) shellcode, an agent must generate and compile a loader that executes the payload without being blocked or terminated. A supplementary evasion knowledge skill provides domain knowledge and supports comparisons with and without this input. BypassGym is built on Proxmox VE(PVE), so the evaluation environment runs real Windows systems and EDR products inside virtual machines and returns structured detection feedback to the agent. We evaluate 5 lightweight model backends against 6 EDR products and 3 payloads, and find that even these simple models generate successful submissions, with DeepSeek-V4.1 Flash reaching 17/18 successful runs and Bitdefender allowing only 5 of 15 runs to pass. These results provide a systematic measurement of current AI agents' endpoint defense evasion capabilities and identify areas for defenders to investigate.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include BypassGym, DeepSeek, EDR, AI安全.
Primary source verification: published directly by arXiv: DeepSeek with first-party authority.