论文精选

配置如何影响AI编程工具的构建vs购买决策:研究协议

The Impact of Configuring Agentic AI Coding Tools on Build-vs-Buy Decisions: A Study Protocol

精选理由

AI编程工具的自动决策直接影响软件质量和合规性,做AI开发工具或使用AI编程的团队需要了解如何通过配置引导工具行为,建议点开了解实验设计。

AI 摘要

该研究协议探讨了配置机制如何影响AI编程工具(如Claude Code和OpenAI Codex)在构建功能与导入库之间的决策。这些决策直接影响软件安全、许可合规、性能和可维护性。研究将使用控制实验,操纵不同配置(如无配置、上下文文件、技能、MCP库发现工具和权限控制),测量工具选择的库、是否披露新库以及披露的完整性。九项预注册假设指导实验,最终将发布基准数据集和分析管道。

原文 · arXiv cs.AI

The Impact of Configuring Agentic AI Coding Tools on Build-vs-Buy Decisions: A Study Protocol

Agentic AI coding tools write code with increasing autonomy and in doing so decide when to import a library and when to implement functionality from scratch. These decisions, whether to build functionality from scratch or buy into an external library, hereafter build-versus-buy, carry direct consequences for software security, licensing compliance, performance, and long-term maintainability. Yet no controlled experimental study has examined what governs build-versus-buy decisions in agentic AI coding tools. Configuration mechanisms, i.e., the means by which developers tailor agentic AI coding tool behavior to a project or workflow, are one of the primary means by which practitioners can influence these decisions. However, it is unclear which configuration mechanisms influence build-versus-buy decisions most effectively. We present a pre-registered protocol to study how configuration mechanisms alter build-versus-buy behavior in two popular agentic AI coding tools: Claude Code and OpenAI Codex. We will execute controlled programming tasks drawn from a benchmark of staged projects, each constructed around identifiable build-versus-buy points, and will manipulate the configuration supplied to each tool, ranging from no configuration, through context files with soft preferences and explicit prohibitions, to Skills (instructions that can be autonomously discovered), MCP-enabled library discovery tools, and permission controls, measuring which libraries the tool selects, whether it discloses newly introduced libraries, and whether those disclosures are complete and accurate. Nine pre-registered hypotheses structure the protocol. The resulting benchmark dataset and analysis pipeline will be released as a reusable artifact for evaluating build-versus-buy behavior in agentic AI coding tools.