这篇论文用56800篇数据告诉你，AI研究的可重复性在过去十年大幅提升，代码共享从11%涨到64%，而且不是靠强制清单推动的。
该研究分析了2014至2024年间五大顶级AI会议发表的56800篇论文，评估其文档实践。结果显示，代码和数据共享比例从11%增至64%，增长了近六倍。基于文档实践推断的可重复性从28%提升至64%。这些改进在可重复性检查清单引入之前就已开始，反映的是开放科学趋势而非形式要求。
The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers
The reproducibility crisis has directed the AI research community toward improving documentation practices. Several studies have identified methodological issues, and in response, the most impactful venues in the field have introduced reproducibility checklists. We seek to understand whether documentation practices have changed over time by assessing all published papers at five leading AI conferences over the past decade. Seven reproducibility variables were identified, quality-assured and used to analyse 56 800 publications. Our analysis reveals that in the period 2014 to 2024, documentation practices have improved; papers sharing both code and data increased nearly sixfold, from 11% to 64% Building on empirical reproducibility rates from a prior study, we estimate - inferred from documentation practices, not direct testing - that reproducibility increased from 28% in 2014 to 64% in 2024. Improvements in documentation practices predate the introduction of reproducibility checklists, suggesting these changes reflect a broader movement toward open science rather than a direct response to formal requirements.