AI Coding Agent Security Study
Scanning the Harness: An Empirical Study of Supply-Chain Defects in AI Coding-Agent Configurations
Developers using AI coding assistants should audit their configurations for security risks, as 1 in 6 setups contain defects that could compromise system security.
Researchers examined 3,171 public GitHub repositories using AI coding agents like Claude Code, Cursor, GitHub Copilot, and OpenAI Codex.
They found 16.0% of setups contain security defects, including 9.
8% with unpinned MCP server versions and 3.1% with pre-approved arbitrary execution.
The study validated findings through independent implementations and language-model adjudicators.
Scanning the Harness: An Empirical Study of Supply-Chain Defects in AI Coding-Agent Configurations
AI coding agents such as Claude Code, Cursor, GitHub Copilot, and OpenAI Codex are configured through artifacts developers write and share: instruction files, skills, hooks, MCP server declarations, subagents. This harness is a dependency layer installed from marketplaces and public repositories, running with the developer's privileges, with no lockfile, no install-time check, and no vocabulary for what a component may do. We study it over 3,171 public GitHub repositories: 2,660 setups that assemble two or more component types and 511 published skill collections. We measure only rules decidable from bytes whose consequence is a security exposure, a configuration that cannot work, or a departure from the Agent Skills specification, and validate every finding before it counts: an independent implementation re-derives it from the repository at its pinned commit, a language-model adjudicator with a released prompt rules on every disagreement, and a second independent model session re-checks every counted pair. Three security classes survive: 9.8% of setups install an MCP server with no version pinned, 3.1% pre-approve arbitrary execution behind a scoped-looking grant such as Bash(python:*), and 3.8% carry a skill that pre-approves the shell for whoever installs it. In total 16.0% of setups carry a security defect and 16.7% a confirmed defect of any kind, against a raw scanner rate of 25.5% on the same rules; the third class ships inside 3.7% of collections, where a marketplace scan can see it. Rules that compare two files detect differences that are usually intended and are reported as observations. No credential-exfiltration path was confirmed. The instrument, corpus manifest, prompt, and every verdict are released.
This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and AI regulation, compliance frameworks, and sovereign compute policy. Key entities and technical topics under continuous tracking include Claude Code, Cursor, GitHub Copilot, OpenAI Codex.
Primary source verification: published directly by arXiv: OpenAI with first-party authority. Cross-source corroboration: 11 independent sources have covered this development (including shao__meng, lmarena.ai, 小互), confirming industry-wide signal strength.
