想了解AI审计盲区?这篇综述翻了4259篇文献,发现只有58篇关注系统集成,现有审计忽略了组件交互风险,值得看看。
本综述扫描了4,259篇文献,最终纳入58项AI审计研究。通过反思性主题分析,发现现有审计很少针对系统集成特有风险。研究将系统集成审计分为组件间、系统-环境、多系统三类。这些审计评估兼容性、完整性和监督等集成特质。综述建议AI社区将系统集成作为审计核心策略。
A Chain Is Only as Strong as Its Weakest Link: A Scoping Review of System Integration Audits in AI
As AI systems become increasingly integrated into diverse interfaces and applications, model-centric audits are insufficient to address risks arising from interactions among system components and deployment environments. System integration has long been central to software audits in safety-critical domains such as aerospace. However, its role in AI auditing remains underexplored. Scanning through 4,259 documents, we present a scoping review of AI audits that treat system integration as a core tenet of evaluation (n = 58). Using reflexive thematic analysis, we analyze their elements, actors, enablers, and constraints. We find that the corpus represents an emerging yet still fragmented form of AI auditing: few existing measures target integration-specific risks; large gaps remain in meeting traditional audit expectations; and access to necessary information and resources significantly influences audit design. Nonetheless, integration can be categorized across three sites (inter-component, system-environment, and multi-system), each serving the functions of risk exploration, risk determination, coordination, and procedural regularity. Deviating from other types of evaluations, these audits assess qualities specific to system integration, including compatibility, completeness, and oversight. This review calls on the AI community to prioritize system integration as a core strategy for addressing AI risk, and to develop audit practices capable of capturing failures across components, environments, and systems beyond the reach of component-level evaluation.