研究人员发布了TIDE-Bench基准，专门测试LLM在复杂对话SQL中的意图理解能力，发现现有模型在链式识别和意图漂移处理上仍有明显不足。
研究团队推出TIDE-Bench基准，评估大语言模型在链式模糊和意图漂移场景下的对话式Text-to-SQL能力。该基准基于BIRD数据集的514个锚定SQL构建，包含1,542个样本。评估了12个先进LLM，发现链式识别瓶颈不受澄清频率影响，意图漂移识别-解决存在显著差距，且两种失败模式同时激活时会相互重叠。
Evaluating LLMs on Conversational Text-to-SQL under Chain Ambiguity and Intent Drift
Recent advances in large language models (LLMs) have established conversational text-to-SQL as a practical interface between users and databases, often involving multiple turns of clarification and revision. However, existing benchmarks primarily evaluate execution accuracy, leaving the unfolding and shifting of user intent across turns largely uncovered. To address this, we introduce TIDE-Bench, a benchmark for conversational text-to-SQL under chain ambiguity and intent drift evaluation, targeting two recurring patterns: chain ambiguity, where an underspecified question triggers layered clarification with conditional dependencies, and intent drift, where the user retracts and replaces a previously committed request element. Built on 514 anchor SQLs from BIRD, TIDE-Bench comprises 1,542 samples and introduces dedicated metrics for chain identification and drift recognition-resolution beyond execution accuracy. Evaluating 12 advanced LLMs reveals a persistent chain identification bottleneck unaffected by clarification frequency, a wide drift recognition-resolution gap, and overlap between failure modes when jointly activated. The corresponding code of TIDE-Bench is released for further research.