论文系统测量了LLM读表格犯数据引用错误的频率,还训练了个4B批评模型能查出这些错,推理准确率提升12%。适合做表格推理的开发者看。
这篇论文首次系统评估了大型语言模型在表格任务中的数据引用错误(DRE),覆盖1.7B到20B参数的各种模型。实验发现所有测试模型均存在DRE。通过基于批评模型的过滤和拒绝采样,答案准确率最高提升12.0%。作者训练了一个轻量级4B参数批评模型,在检测分布内和分布外DRE时平均F1达78.2%,并能有效辅助更大模型推理。
When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors
While large language models (LLMs) perform well on table tasks, they still make data referencing errors (DREs), i.e., incorrectly citing or omitting table values, despite understanding the table structure. Beyond final-answer accuracy, DREs directly compromise the correctness and reliability of intermediate reasoning steps. Yet prior studies have only offered limited, small-scale analyses. In this work, we present the first systematic evaluation of tabular data referencing errors across different models and tasks. Our results show that DREs occur across all tested models (1.7B to 20B parameters). Furthermore, we demonstrate that incorporating data referencing as a critic significantly improves answer accuracy up to 12.0%, through critic-based filtering and rejection sampling. Finally, we trained a lightweight 4B-parameter critic model that achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-distribution DREs, and effectively assists inference for larger models.