论文提出Workflow Cards,把工作流执行记录变成人和AI都能读的摘要,比直接查数据库答案质量高一倍,做AI文档的可以看看。
Workflow Cards是一种新文档形式,将工作流执行的机器可读溯源数据压缩为人类和LLM可读的结构化摘要。论文定义了基于溯源问题集的Workflow Card模板,并评估LLM使用其理解工作流执行的效果。结果显示,Workflow Cards提供了Model Cards和Data Cards缺失的执行级信息,填补了文档空白。与基于schema的查询相比,Workflow Cards在LLM-as-a-Judge和人工评估中答案质量几乎翻倍。
Workflow Cards: Structured Summaries of Workflow Executions Using Provenance Data
Model Cards and Data Cards have demonstrated the value of structured, human-readable documentation for machine learning artifacts, capturing their context, parameters, limitations, and intended use. However, these practices remain focused on static artifacts (the datasets and trained models themselves) while overlooking the workflow executions that produce, transform, and evaluate them. Such executions hold critical details about data preparation, parameter choice, runtime behavior, resource use, and intermediate transformations, precisely where bias, performance variation, and reproducibility gaps tend to originate. To close this gap, we introduce Workflow Cards: structured summaries that condense the machine-readable provenance data of a workflow execution into a form both humans and large language models (LLMs) can read and analyze. This paper has two main parts. First, it defines a Workflow Card template informed by a representative set of provenance questions that surface from the execution-level data missing from Model and Data Cards. Second, it evaluates how effectively LLMs use Workflow Cards to understand workflow executions compared with querying provenance databases through a schema-based interface. Results show that Workflow Cards provide execution-level information absent from existing card types, such as Model Cards and Data Cards, thereby filling an important documentation gap; and that Workflow Cards nearly double answer quality compared with schema-based querying, consistently across LLM-as-a-Judge and human assessments.