研究剖析 83 个智能合约审计 Skill 的设计与实际效果
Demystifying Agent Skills for Smart Contract Auditing: Design, Effectiveness, Behavioral Impact
有人收集了 83 个智能合约审计 Skill 做系统评测，发现效果主要看模型，Codex/GPT-5.5 检测分提升 22.8%，还开源了语料，做 agent 开发的可以看看。
这篇论文系统收集了 83 个来自社区的智能合约审计 Skill，在 EVMBench 上跨七种 agent-模型配置进行评测。结果显示 Skill 设计总体轻量但差异大，覆盖的漏洞类型并不均衡。效果主要取决于模型而非 agent 框架：Codex/GPT-5.5 增益最大，检测得分提升 22.8%，captured award 提升 43.2%。Skill 触发是关键瓶颈，触发后各配置共享六阶段审计流程，但对 agent 行为的影响各不相同。作者已开源 Skill 语料和评测产物。
Demystifying Agent Skills for Smart Contract Auditing: Design, Effectiveness, Behavioral Impact
LLM agents, notably Claude Code and OpenAI Codex, are emerging as versatile tools beyond coding agents only. These agents can be enhanced with skills---reusable artifacts that package domain knowledge, workflows, and tool-use instructions. To date, however, little is known about how such skills are designed or how they affect agent effectiveness and behavior in practice. In this paper, we investigate these questions in smart contract security auditing, a domain in which agents have shown substantial promise. We systematically collect 83 smart contract audit skills from the wild and evaluate them on EVMBench across seven agent--model configurations. Our study examines three dimensions: (i) the design characteristics of audit skills, including their structure, knowledge representations, workflows, and tool dependencies; (ii) their effectiveness in improving vulnerability detection; and (iii) their influence on agent execution trajectories. We find that audit skills are mostly lightweight but heterogeneous in design, covering a broad yet imbalanced range of vulnerability types. Their effectiveness is determined primarily by the model rather than the agent harness: Codex/GPT-5.5 achieves the largest gains, improving detection score by 22.8% and captured award by 43.2%. We further find that skill triggering is a key bottleneck. When triggered, skills preserve a shared six-stage audit workflow while exhibiting distinct loading patterns and differential effects on agent behavior across configurations. We release our skill corpus and artifacts to support future research.
