AI-Assisted Computational Reproducibility on the FABRIC Testbed

精选理由

这篇论文用三个具体案例展示了AI助手如何把复现实验的时间压缩到原来的四分之一,特别适合搞科研复现和测试床工作的朋友看看。

AI 摘要

该论文利用FABRIC国际测试床与LLM编码助手LoomAI,成功复现了BBR系列拥塞控制评估、LAMMPS分子动力学扩展基准测试(CPU-only MPI集群)和应激蛋白稳态基因组学管道三个跨领域案例研究。AI助手在环境搭建、代码适配和调试阶段表现有效,但在缺乏明确工作流的分析阶段需要人类指导。实验表明,AI辅助工作流将复现工作量减少约4至6倍。论文还提出了改进研究测试床上AI辅助可重复性的实用建议。

原文 · arXiv cs.AI

Computational reproducibility remains difficult despite being central to scientific research. In this paper, we show how the international FABRIC testbed, combined with large language model (LLM) coding assistants through LoomAI, can simplify reproducing published experiments across multiple domains. We reproduced three case studies on FABRIC, covering BBR-family congestion-control evaluations, LAMMPS molecular dynamics scaling benchmarks on a CPU-only MPI cluster, and stress protein homeostasis genomics pipelines. Rather than focusing only on matching numerical outputs, we evaluate whether the reproduced experiments support the same scientific conclusions as the original studies. The AI assistant was effective in setting up the environment, adapting code, and debugging, but struggled with the analysis stages that lacked clearly defined workflows, which required human guidance to establish execution order and data dependencies. Across the case studies, the AI-assisted workflow reduced reproduction effort by roughly 4--6 times. We conclude with practical recommendations for improving AI-assisted reproducibility on research testbeds.