Agentic-SQL 新研究:按自主度划分 Text-to-SQL 并给出基准分析

Agentic-SQL Revisited: Autonomy-Based Taxonomy and Empirical Benchmark Analysis for LLM Text-to-SQL

精选理由

做 Text-to-SQL 的可以看看这份排行榜聚合方法,它把不同模型和推理方式按自主度归类,还给出了在 Spider、BIRD 上的实测对比,省得自己瞎比较。

AI 摘要

该论文将 LLM Text-to-SQL 领域的进展重组为排行榜聚合,并沿推理自主度轴划分出受限、上下文、迭代、智能体、推理内化五类生成方式。作者在 Spider 基准上做了案例研究,对比 8B 开源模型与 few-shot DeepSeek V3、GLM-4 基线,发现 Spider 成绩迁移到 BIRD 和 Spider 2.0 时不均衡。研究还指出自主性提升能增强鲁棒性但代价不小,CoT 收益集中在 Hard 和 Extra-Hard 查询上。论文发布了对应自主度轴的 Python 工具,供新方法直接接入排行榜。

原文 · arXiv: DeepSeek

Agentic-SQL Revisited: Autonomy-Based Taxonomy and Empirical Benchmark Analysis for LLM Text-to-SQL

LLM-based Text-to-SQL progress is reported across heterogeneous benchmarks, backbones, and inference protocols, making cross-system comparison fragile. We reframe the field as a leaderboard aggregation: we collect the metrics authors themselves report and organize them along an inference-autonomy axis spanning constrained, in-context, iterative, agentic, and reasoning-internalized generation, with traceable provenance for every cell. To anchor the aggregation empirically, we run a focused case study on Spider, comparing 8B open-source backbones with and without chain-of-thought (CoT) supervision against few-shot DeepSeek~V3 and GLM-4 baselines. Four patterns emerge: Spider gains transfer unevenly to BIRD and Spider~2.0; autonomy buys robustness at non-trivial cost; reasoning internalization sits between answer-only decoding and externally orchestrated agents; and CoT gains concentrate on Hard and Extra-Hard queries. We release a Python harness mirroring the autonomy axis so that future methods can be added directly to the leaderboard.