想了解AI写代码到底安不安全?这篇论文用具体基准和测试告诉你,目前主流的编码助手都靠不住,必须用迭代重提示才能堵上漏洞。
一项研究评估了5款AI编码助手(如GitHub Copilot、Amazon CodeWhisperer等)在生成认证代码时的安全性,采用静态代码分析加动态渗透测试的双模态评估框架,并映射到NIST SP 800-63B标准。结果发现,基础或通用安全提示下生成的代码普遍缺少暴力破解防护、会话管理和密码处理等关键保护。即使给出一次性NIST提示,合规性虽有提升但仍结构不足。只有通过迭代重提示(Reprompting)迫使模型进行上下文自审计,才能实现完整的深度防御安全架构。研究证实当前AI编码助手无法默认生成安全代码,企业需从一次性提示转向持续、标准驱动的验证流程。
The Illusion of Secure LLM Code: Closing the Security Gap via Iterative Reprompting
Large Language Models (LLMs) are increasingly integrated into software development workflows, yet their ability to autonomously generate secure authentication code remains uncertain. This paper evaluates the security architecture of authentication systems generated by five prominent AI coding assistants through a bi-modal assessment framework combining static code analysis and dynamic penetration testing, mapped to NIST SP 800-63B guidelines. The study examines model behavior across four prompting strategies Basic, Secure, NIST-Based, and Reprompting to reflect varying levels of developer guidance. Empirical results demonstrate that code generated from functional or generically secure prompts consistently omits critical protections, particularly concerning brute-force resistance, session management, and robust password handling. While providing explicit, single-shot NIST context significantly improves compliance, the findings reveal that this remains structurally inadequate. Instead, iterative Reprompting: forcing models into a contextual self-auditing loop is strictly required to achieve a comprehensive, defense-in-depth security architecture. Ultimately, this study proves that current AI coding assistants do not produce secure-by-default applications, dictating that enterprise deployments must transition from single-shot prompt engineering to continuous, standards-driven verification pipelines.