这篇论文通过实验展示了架构规范格式对编码智能体生成代码质量的影响,特别是TypeScript合同在提升API路由覆盖率方面的显著效果,对于关注模型能力均衡的读者来说值得一读。
基于LLM的编码智能体从高级描述生成完整软件系统,但架构规范格式对生成代码质量的影响尚不清楚。本文通过对比五种信息等效规范格式(非正式散文、Mermaid图、OpenAPI、C4/Structurizr DSL和TypeScript接口合同)和六种模型,发现规范格式与模型能力存在显著交互。在90次多轮智能体试验中,格式对模型能力的影响明显。在最强模型上,格式影响较小;在较弱模型上,格式影响较大。TypeScript合同将API路由覆盖率从最弱模型的33%提升到100%。结构化架构规范作为能力均衡器,其价值与模型强度成反比,对成本优化部署的回报最大。
Architecture as Capability Equalizer for Coding Agents
LLM-based coding agents generate complete software systems from high-level descriptions, yet little is known about how the format of architecture specifications affects the quality of generated code or whether this effect depends on model capability. We present a controlled experiment comparing five informationally equivalent specification formats (informal prose, Mermaid diagrams with constraints and ADRs, OpenAPI, C4/Structurizr DSL, and TypeScript interface contracts with ArchUnit-style rules) across six models from three vendor families (Anthropic Claude, OpenAI GPT, Google Gemini). Across 90 multi-turn agent trials, specification format shows a strong format x model interaction. On the strongest models (Sonnet 4.6, GPT-5), format barely matters (quality spread 0.17-0.92). On weaker models, format produces spreads of 0.83-2.42 points, with code-proximate formats (OpenAPI, TypeScript contracts) recovering most of the capability gap. Mid-tier models can consume more tokens than frontier models for worse output when they enter compilation debugging loops that stronger models avoid. Self-validation rates collapse from 100% (Sonnet) to 0% (Gemini Flash) across the capability spectrum. TypeScript contracts triple API route coverage for the weakest model (33% to 100%). Structured architecture specifications serve as a capability equalizer, with value inversely proportional to model strength and the largest returns for cost-optimized deployments.