FALCON:低成本生成高复杂度 NL2SQL 合成训练数据
FALCON: A Model and Dataset Agnostic Framework for Synthetic Data Generation for NL2SQL Pairs
做 Text-to-SQL 的同学可以看看 FALCON 这个框架,用小开源模型本地就能造出比现有基准更难的训练数据,还不用调外部 API。
FALCON 是一个面向 NL2SQL 的合成数据生成框架,采用保留字 SQL 种子加 persona 提示词生成结构复杂的查询。它通过基于对齐的过滤机制,把真正错误的样本和复杂但合法的查询区分开,避免过滤掉难题。用紧凑开源模型即可低成本运行,生成数据在 SQL 复杂度和自然语言丰富度上超过现有基准,且模型与数据库无关。按难度分层分析显示,查询越复杂,用 FALCON 数据训练的模型相比基线优势越大。
FALCON: A Model and Dataset Agnostic Framework for Synthetic Data Generation for NL2SQL Pairs
Relational databases are among the most widely deployed forms of structured knowledge, and natural language access to them requires grounding language onto schema entities and relations while handling the ambiguity inherent in how people phrase requests. Existing synthetic NL-to-SQL data generation methods largely ignore this ambiguity and produce oversimplified queries that fail to prepare models for the complexity of real-world structured knowledge access. We present FALCON, a framework that generates realistic, ambiguity-aware NL-to-SQL data matching the complexity of challenging real-world benchmarks, at low cost using compact open models. Our approach combines reserved-word SQL seeding and persona-based prompting to generate structurally complex queries, while alignment-based filtering preserves difficulty by distinguishing genuinely incorrect examples from complex but valid queries. Human evaluation confirms consistent high quality across model sizes, and our generated data exceeds existing benchmarks in both SQL complexity and natural language richness. Difficulty-stratified analysis shows models trained on FALCON data increasingly outperform baseline-trained models as query complexity increases, validating our pipeline's success in generating challenging training data. When combined with a small proportion of existing benchmark data, mixed training recovers performance on simpler queries while preserving these advantages on complex ones. The model- and database-agnostic design enables organizations to generate high-complexity NL-to-SQL training data locally without external APIs.