这篇论文讲EvoHunt,能自动生成安全审计剧本,还能把经验迁移给弱模型,效果比商用产品还好。
EvoHunt是一个在开源仓库上运行的安全审计剧本进化环境。它使用三个智能体驱动循环:审计智能体执行当前剧本,评估器根据真实情况打分,修订者根据失败分析更新剧本。在开源安全公告评估中,EvoHunt使Codex/GPT5.4-xhigh的端到端漏洞利用成功率从1.1%提升至6.2%。GLM5.1演变出的剧本在目标匹配率上达到11.3%,超过OpenAI Codex Security的9.2%。转移实验显示,Qwen3.6-27B借助GLM剧本从2.4%提升至6.5%,Qwen3.6-35B-A3B从1.1%提升至4.6%。
Transferable Self-Evolving Playbooks for Agentic Security Auditing
An LLM agent for vulnerability discovery and validation is more than a model. It combines three components: an LLM for code analysis, an agent harness such as Codex or OpenCode for navigation, tool use, and execution, and an audit playbook, domain-specific procedural knowledge that guides the LLM and harness toward vulnerability discovery. Prior work relies on human-supplied playbooks, including prompt engineering, manual workflows, knowledge bases, and heuristics. This raises two research questions: Acquisition - is human curation necessary, and can playbook creation be automated? Transfer - can an evolved playbook transfer the audit procedure to weaker agents, improving their capability? We present EvoHunt, a playbook evolution environment over open-source repositories for security auditing. Three agents drive the evolution loop: an audit agent rolls out the current playbook and produces findings; an evaluator scores outcomes against ground truth; and a reviser commits updates to the playbook based on failure analysis. The playbook format is unconstrained: starting empty, EvoHunt adds or removes workflows, heuristics, vulnerability knowledge, or domain-specific content. The evolved playbook requires only minor adaptation to run under a different LLM or harness. We evaluate EvoHunt on open-source security advisories. For acquisition, playbook evolution raises end-to-end exploits for Codex/GPT5.4-xhigh 6x, from 1.1% to 6.2%, and the evolved OpenCode/GLM5.1 playbook surpasses OpenAI Codex Security on every metric, with 11.3% vs. 9.2% target-match rate, showing open-source evolution can outperform a dedicated commercial product. For transfer, the GLM-evolved playbook gives the strongest student lift: Qwen3.6-27B improves from 2.4% to 6.5%, Qwen3.6-35B-A3B from 1.1% to 4.6%, and A3B obtains 2.4x more matches than GPT transfer.