ADEPT:统一深度学习测试充分性评估框架

ADEPT: A Unified Framework for Deep Learning Test Adequacy

精选理由

做深度学习测试的人可以省事了,ADEPT 把各种覆盖率度量统一成一个框架,不用再挨个配置那些难搞的原型工具,直接就能跑。

AI 摘要

ADEPT 是一个集成了多种深度学习测试充分性度量的统一框架,覆盖神经元覆盖率、惊喜充分性、输入分布覆盖、边界覆盖以及源级和模型级变异分数。框架提供基于模板的度量接口和扩展点,支持 YAML 配置管理、预处理缓存复用和结构化结果报告。该框架旨在让研究者和开发者无需花费数天或数周配置分散的研究原型,即可复现和应用充分性度量。论文发表于 arXiv,编号 2608.12144v1,并附有演示视频。

原文 · arXiv cs.LG

ADEPT: A Unified Framework for Deep Learning Test Adequacy

Over the past decade, many test adequacy metrics have been proposed for deep learning that characterize test dataset adequacy from different perspectives, e.g., neuron activation behavior, latent feature coverage, decision-boundary exploration, etc. However, these metrics are typically released as independent research prototypes with substantially different installation and preprocessing requirements, execution workflows, and configuration mechanisms. These complications make them quite difficult to reproduce, compare, and adopt in research work and practical deployment alike. In this paper, we present the engineering details of ADEPT, a framework that integrates representative adequacy techniques, including neuron-coverage-based metrics, surprise adequacy, input distribution coverage, boundary coverage, and source- and model-level mutation score, under a consistent execution workflow. ADEPT provides a template-based metric interface with well-defined extension points for integrating new adequacy metrics. Furthermore, it provides YAML-based configuration management, preprocessing-cache reuse, and structured result reporting, making it easy to use in any research and development workflows. ADEPT is designed for researchers and practitioners who wish to reproduce and apply adequacy metrics without spending days or weeks implementing missing tooling or configuring disparate research prototypes. A demo video is available at https://aub.ie/ADEPT_video.