大规模测量Hugging Face模型中AI物料清单的完整性

A Large-Scale Measurement of AI Bill of Materials Completeness in Hugging Face Models

精选理由

这篇论文对Hugging Face上近十万个模型的文档完整性做了大规模分析,发现很多模型缺少许可证、局限性和环境信息,对想了解开源模型供应链透明度的朋友很有参考。

AI 摘要

该研究以Hugging Face上的模型仓库为例,实证调查了AI物料清单(AIBOM)的完整性,涉及约97.5K个AIBOM工件。检查发现生成的AIBOM在所需结构字段上覆盖完整,但模型卡、元数据、负责任使用、环境、局限性等AI特定文档字段普遍缺失或薄弱。论文指出当前模型仓库在许可证、数据集声明、模型家族等可追溯性信息上存在不足,呼吁改进模型卡实践和自动化AIBOM验证。

原文 · arXiv cs.AI

A Large-Scale Measurement of AI Bill of Materials Completeness in Hugging Face Models

Pretrained machine learning (ML) models help developers build ML-intensive software systems without training models from scratch. However, model repositories often provide incomplete machine-readable documentation about model provenance, licenses, datasets, limitations, and external references, creating transparency and governance gaps across the AI supply chain. Artificial Intelligence Bills of Materials (AIBOMs) address these gaps by documenting AI artifacts, including models, metadata, licenses, datasets, model-card information, and external references. Taking public Hugging Face (HF) model repositories as a case study, this paper empirically investigates AIBOM completeness, defined as the extent to which repositories provide AIBOM-relevant information for machine-readable AI supply-chain documentation. We examine approximately 97.5K AIBOM artifacts to assess the extent to which generated AIBOMs: (i) contain required structural and metadata fields, (ii) represent model identity, license, and external-reference information, (iii) capture model-card documentation such as datasets, limitations, safety-risk assessment, and environmental information, and (iv) vary in documentation coverage across repository and artifact characteristics such as task, license availability, dataset declaration, model family, and paper reference. Results indicate that generated AIBOMs provide complete coverage of required AIBOM structure but limited AI-specific documentation completeness. Required fields are fully represented, but model-card, metadata, responsible-use, environmental, limitation, and meaningful-description fields remain weakly represented or missing across generated artifacts. Our findings motivate improved model-card practices, repository-level traceability, and automated AIBOM validation to advance the generation and adoption of more complete AIBOMs.

大规模测量Hugging Face模型中AI物料清单的完整性 · AI 热点