Text-to-SQL Handbook

NL2SQL Handbook

This is the official repository for [TKDE'25] A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going? and [VLDB'24] The Dawn of Natural Language to SQL: Are We Fully Ready?. From this repository, you can explore the latest advancements in Text-to-SQL research (a.k.a NL2SQL). We provide a comprehensive survey, in-depth research papers, and benchmark evaluations.

A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?

Natural Language to SQL: State of the Art and Open Problems

The Dawn of Natural Language to SQL: Are We Fully Ready?

📧If we missed any interesting work, connect with us.

@article{liu2025survey,
  title={A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?},
  author={Liu, Xinyu and Shen, Shuyu and Li, Boyan and Ma, Peixian and Jiang, Runzhi and Zhang, Yuxin and Fan, Ju and Li, Guoliang and Tang, Nan and Luo, Yuyu},
  journal={IEEE Transactions on Knowledge and Data Engineering},
  year={2025},
  publisher={IEEE}
}

🧭 Text-to-SQL Introduction

Translating users' natural language queries (NL) into SQL queries can significantly reduce barriers to accessing relational databases and support various commercial applications. The performance of Text-to-SQL has been greatly improved with the emergence of language models (LMs). In this context, it is crucial to assess our current position, determine the Text-to-SQL solutions that should be adopted for specific scenarios by practitioners, and identify the research topics that researchers should explore next.

📈 Text-to-SQL Lifecycle

• Model: Text-to-SQL translation techniques that tackle not only NL ambiguity and under-specification, but also properly map NL with database schema and instances;

• Data: From the collection of training data, data synthesis due to training data scarcity, to Text-to-SQL benchmarks;

• Evaluation: Evaluating Text-to-SQL methods from multiple angles using different metrics and granularities;

• Error Analysis: analyzing Text-to-SQL errors to find the root cause and guiding Text-to-SQL models to evolve.

🤔 Where Are We?

We categorize the challenges of Text-to-SQL into five levels, each addressing specific hurdles. The first three levels cover challenges that have been or are currently being addressed, reflecting the progressive development of Text-to-SQL. The fourth level represents the challenges we aim to tackle in the LLMs stage, while the fifth level outlines our vision for Text-to-SQL system in the next five years.

We describe the evolution of Text-to-SQL solutions from the perspective of language models, categorizing it into four stages. For each stage of Text-to-SQL, we analyze the changes in target users and the extent to which challenges are addressed.

🧩 Module-based Text-to-SQL Methods

We summarize the key modules of Text-to-SQL solutions utilizing the language model.

• Pre-processing serves as an enhancement to the model’s inputs in the Text-to-SQL parsing process. You can get more details from this chapter: Pre-Processing
• Text-to-SQL translation methods constitute the core of the Text-to-SQL solution, responsible for converting input natural language queries into SQL queries. You can get more details from this chapter: Text-to-SQL Translation Methods
• Post-processing is a crucial step to refine the generated SQL queries, ensuring they meet user expectations more accurately. You can get more details from this chapter: Post-Processing

📚 Text-to-SQL Survey & Tutorial

1. A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?
2. Natural Language to SQL: State of the Art and Open Problems.
3. Next-generation database interfaces: A survey of LLM-based Text-to-SQL.
4. A Survey on Employing Large Language Models for Text-to-SQL Tasks.
5. Large Language Model Enhanced Text-to-SQL Generation: A Survey.
6. From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems.
7. Natural language interfaces for tabular data querying and visualization: A survey.
8. Natural Language Interfaces for Databases with Deep Learning.
9. A survey on deep learning approaches for text-to-SQL.
10. A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions.
11. Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect.
12. A Deep Dive into Deep Learning Approaches for Text-to-SQL Systems.
13. State of the Art and Open Challenges in Natural Language Interfaces to Data.
14. Natural language to SQL: Where are we today?

📰 Text-to-SQL Paper List

1. DeepEye-SQL: A Software-Engineering-Inspired Text-to-SQL Framework.
2. Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
3. NL2SQL-BUGs: A Benchmark for Detecting Semantic Errors in NL2SQL Translation.
4. EllieSQL: Cost-Efficient Text-to-SQL with Complexity-Aware Routing.
5. The Dawn of Natural Language to SQL: Are We Fully Ready?
6. SQL-Factory: A Multi-Agent Framework for High-Quality and Large-Scale SQL Generation.
7. NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions.
8. Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards.
9. SQL-Exchange: Transforming SQL Queries Across Domains
10. TACO: A Benchmark for Open-Domain Text-to-SQL with Ambiguous and Cross-Database Queries.
11. OpenSQL: Data-Efficient Text-to-SQL for Open-Source LLMs via Synthesized Intermediate Supervision.
12. A Comparative Evaluation of Schema Subseing for LLM-based NL-to-SQL over Large-Schema Databases.
13. SafeQL: Search-based Refinement for Safe and Efficient LLM-based Text-to-SQL.
14. Dial: A Knowledge-Grounded Dialect-Specific NL2SQL System
15. Developing and Benchmarking Verification Algorithms to Improve Text-to-SQL Generation.
16. Bridging NL2SQL for CAN Signal Analytics at NIO: From Externalized Schemas to CTE Pipelines.
17. DIVER: A Robust Text-to-SQL System with Dynamic Interactive Value Linking and Evidence Reasoning
18. Comparison and Analysis of Value Linking in Text-to-SQL systems: [Experiments & Analysis]
19. Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewards
20. PRISM: Navigating Cost–Accuracy Trade-offs for NL2SQL
21. SchemaRAG: A Schema-aware Retrieval-Augmented Generation Framework for Text-to-SQL
22. Reliable Answers for Recurring Questions: Boosting Text-to-SQL Accuracy with Template-Constrained Decoding
23. PLForge: Enhancing Language Models for Natural Language to Procedural Extensions of SQL
24. LogicCat: A Chain-of-Thought Text-to-SQL Benchmark for Complex Reasoning.
25. Schema on the Inside: A Two-Phase Fine-Tuning Method for High-Efficiency Text-to-SQL at Scale.
26. Hexgen-Flow: Optimizing LLM Inference Request Scheduling for Agentic Text-to-SQL.
27. SQLMorph: Query Mutation and Fine-Grained Metrics for Text-to-SQL Evaluation.
28. LEAF-SQL: Level-wise Exploration with Adaptive Fine-graining for Text-to-SQL Skeleton Prediction.
29. An Efficient and Effective Evaluator for Text2SQL Models on Unseen and Unlabeled Data.
30. Boosting Small Language Models for Text-to-SQL with Fine-Grained Execution Feedback and Cost-Efficient Rewards.
31. MM2SQL: A Benchmark and Method for Visually-Grounded SQL Generation.
32. Text2SQL-Flow: A Robust SQL-Aware Data Augmentation Framework for Text-to-SQL.
33. Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generation.
34. CYANSQL: Unlock the Power of NL2SQL via Clustering-based Test-Time Scaling.
35. Beyond Static Pipelines: Learning Dynamic Workflows for Text-to-SQL.
36. ReViSQL: Achieving Human-Level Text-to-SQL.
37. Think2SQL: Blueprinting Reward Density and Advantage Scaling for Effective Text-to-SQL Reasoning.
38. AgentSM: Semantic Memory for Agentic Text-to-SQL.
39. LLM-Based SQL Generation: Prompting, Self-Refinement, and Adaptive Weighted Majority Voting.
40. OptiSQL: Executable SQL Generation from Optical Tokens.
41. Memo-SQL: Structured Decomposition and Experience-Driven Self-Correction for Training-Free NL2SQL.
42. Structure-Guided Large Language Models for Text-to-SQL Generation.
43. Sphinteract: Resolving Ambiguities in NL2SQL Through User Interaction.
44. OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale.
45. EVOSCHEMA: TOWARDS TEXT-TO-SQL ROBUSTNESS AGAINST SCHEMA EVOLUTION.
46. Is Long Context All You Need? Leveraging LLM's Extended Context for NL2SQL.
47. The Power of Constraints in Natural Language to SQL Translation.
48. OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment.
49. Reliable Text-to-SQL with Adaptive Abstention.
50. SNAILS: Schema Naming Assessments for Improved LLM-Based SQL Inference.
51. Automated Validating and Fixing of Text-to-SQL Translation with Execution Consistency.
52. Grounding Natural Language to SQL Translation with Data-Based Self-Explanations.
53. AID-SQL: Adaptive In-Context Learning of Text-to-SQL with Difficulty-Aware Instruction and Retrieval-Augmented Generation.
54. CLEAR: A Parser-Independent Disambiguation Framework for NL2SQL.
55. CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL.
56. Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows.
57. ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL.
58. SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQL.
59. DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph.
60. Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQL.
61. STaR-SQL: Self-Taught Reasoner for Text-to-SQL.
62. SQLGenie: A Practical LLM based System for Reliable and Efficient SQL Generation
63. SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning.
64. Confidence Estimation for Error Detection in Text-to-SQL Systems.
65. SQLord: A Robust Enterprise Text-to-SQL Solution via Reverse Data Generation and Workflow Decomposition.
66. DBCopilot: Scaling Natural Language Querying to Massive Databases.
67. Utilising Large Language Models for Adversarial Attacks in Text-to-SQL: A Perpetrator and Victim Approach.
68. You Only Read Once (YORO): Learning to Internalize Database Knowledge for Text-to-SQL.
69. Boosting Text-to-SQL through Multi-grained Error Identification.
70. Gen-SQL: Efficient Text-to-SQL By Bridging Natural Language Question And Database Schema With Pseudo-Schema.
71. MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL.
72. PARSQL: Enhancing Text-to-SQL through SQL Parsing and Reasoning.
73. UCS-SQL: Uniting Content and Structure for Enhanced Semantic Bridging In Text-to-SQL.
74. SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs.
75. Optimizing Reasoning for Text-to-SQL with Execution Feedback.
76. Knowledge Base Construction for Knowledge-Augmented Text-to-SQL.
77. SQLong: Enhanced NL2SQL for Longer Contexts with LLMs.
78. Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL.
79. Agentar-Scale-SQL: Advancing Text-to-SQL through Orchestrated Test-Time Scaling. .
80. Automatic Metadata Extraction for Text-to-SQL.
81. DeepEye-SQL: A Software-Engineering-Inspired Text-to-SQL Framework.
82. Rethinking Text-to-SOL: Dynamic Multi-turn SOIInteraction for Real-world Database Exploration.
83. MARS-SQL: A MULTI-AGENT REINFORCEMENT LEARNING FRAMEWORK FOR TEXT-TO-SQL.
84. RUBIKSQL: Lifelong Learning Agentic Knowledge Base as an Industrial NL2SQL System.
85. CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning.
86. Cheaper, Better, Faster, Stronger: Robust Text-to-SQL without Chain-of-Thought or Fine-Tuning.
87. SLM-SQL: An Exploration of Small Language Models for Text-to-SQL.
88. Reward-SQL: Boosting Text-to-SQL via Stepwise Reasoning and Process-Supervised Rewards.
89. Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL.
90. Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning.
91. SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs.
92. Think2SQL: Reinforce LLM Reasoning Capabilities for Text2SQL.
93. Distill-C: Enhanced NL2SQL via Distilled Customization with LLMs.
94. OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale.
95. Text2SQL is Not Enough: Unifying AI and Databases with TAG.
96. Automatic database description generation for Text-to-SQL.
97. MCTS-SQL: An Effective Framework for Text-to-SQL with Monte Carlo Tree Search.
98. SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQL.
99. FEATHER-SQL: A Lightweight NL2SQL Framework with Dual-Model Collaboration Paradigm for Small Language Models.
100. FI-NL2PY2SQL: Financial Industry NL2SQL Innovation Model Based on Python and Large Language Model.
101. FGCSQL: A Three-Stage Pipeline for Large Language Model-Driven Chinese Text-to-SQL.
102. Transforming Medical Data Access: The Role and Challenges of Recent Language Models in SQL Query Automation.
103. The Dawn of Natural Language to SQL: Are We Fully Ready?
104. Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation.
105. Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generation.
106. Generating Succinct Descriptions of Database Schemata for Cost-Efficient Prompting of Large Language Models.
107. ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems.
108. CodeS: Towards Building Open-source Language Models for Text-to-SQL.
109. FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysis.
110. PURPLE: Making a Large Language Model a Better SQL Writer.
111. METASQL: A Generate-then-Rank Framework for Natural Language to SQL Translation.
112. Archer: A Human-Labeled Text-to-SQL Dataset with Arithmetic, Commonsense and Hypothetical Reasoning.
113. Synthesizing Text-to-SQL Data from Weak and Strong LLMs.
114. Understanding the Effects of Noise in Text-to-SQL: An Examination of the BIRD-Bench Benchmark.
115. I Need Help! Evaluating LLM’s Ability to Ask for Users’ Support: A Case Study on Text-to-SQL Generation.
116. PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL.
117. Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning.
118. Data-Centric Text-to-SQL with Large Language Models.
119. Research and Practice on Database Interaction Based on Natural Language Processing
120. XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL.
121. Structure Guided Large Language Model for SQL Generation.
122. A Plug-and-Play Natural Language Rewriter for Natural Language to SQL.
123. RSL-SQL: Robust Schema Linking in Text-to-SQL Generation.
     
124. In-Context Reinforcement Learning based Retrieval-Augmented Generation for Text-to-SQL.
125. TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoring.
126. LAIA-SQL: Enhancing Natural Language to SQL Generation in Multi-Table QA via Task Decomposition and Keyword Extraction
127. Research on Large Model Text-to-SQL Optimization Method for Intelligent Interaction in the Field of Construction Safety.
128. SQLh-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging.
129. Grounding Natural Language to SQL Translation with Data-Based Self-Explanations.
130. Towards Optimizing SQL Generation via LLM Routing.
131. E-SQL: Direct Schema Linking via Question Enrichment in Text-to-SQL.
132. DB-GPT: Empowering Database Interactions with Private Large Language Models.
133. The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models.
134. CHESS: Contextual Harnessing for Efficient SQL Synthesis.
135. PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency.
136. CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions.
137. AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queries.
138. Text-to-SQL Calibration: No Need to Ask—Just Rescale Model Probabilities.
139. Few-shot Text-to-SQL Translation using Structure and Content Prompt Learning.
140. CatSQL: Towards Real World Natural Language to SQL Applications.
141. DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction.
142. Data Ambiguity Strikes Back: How Documentation Improves GPT's Text-to-SQL.
143. ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought.
144. Selective Demonstrations for Cross-domain Text-to-SQL.
145. RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL.
146. Graphix-T5: Mixing Pre-trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing.
147. Improving Generalization in Language Model-based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques.
148. G³R: A Graph-Guided Generate-and-Rerank Framework for Complex and Cross-domain Text-to-SQL Generation.
149. Importance of Synthesizing High-quality Data for Text-to-SQL Parsing.
150. Know What I don’t Know: Handling Ambiguous and Unknown Questions for Text-to-SQL.
151. C3: Zero-shot Text-to-SQL with ChatGPT
152. SQLformer: Deep Auto-Regressive Query Graph Generation for Text-to-SQL Translation.

📊 Text-to-SQL Benchmark

We create a timeline of the benchmark's development and mark relevant milestones. You can get more details from this chapter: 📊 Benchmark

🎯 Where Are We Going?

• 🎯Solve Open Text-to-SQL Problem
• 🎯Develop Cost-effective Text-to-SQL Methods
• 🎯Make Text-to-SQL Solutions Trustworthy
• 🎯Text-to-SQL with Ambiguous and Unspecified NL Queries
• 🎯Adaptive Training Data Synthesis

📖 Catalog for Our Survey

You can get more information from our subsection. We introduce representative papers on related concepts:

• Pre-Processing
• Text-to-SQL Translation Methods
• Post-Processing
• Benchmark
• Evaluation
• Error Analysis

💾 Practical Guide for Novice

📊 How to get data:

• We collect Text-to-SQL benchmark features and download links for you. You can get more details from this chapter: Benchmark
• The analysis code for benchmarks is available in the src/dataset_analysis directory. Benchmark analysis reports can be found in the report/ directory.

🛠️ How to build an LLM-based Text-to-SQL model:

• Litgpt Repository Link

  This repository offers access to over 20 high-performance large language models (LLMs) with comprehensive guides for pretraining, fine-tuning, and deploying at scale. It is designed to be beginner-friendly with from-scratch implementations and no complex abstractions.

• LLaMA-Factory Repository Link Unified Efficient Fine-Tuning of 100+ LLMs. Integrating various models with scalable training resources, advanced algorithms, practical tricks, and comprehensive experiment monitoring tools, this setup enables efficient and faster inference through optimized APIs and UIs.

• Fine-tuning and In-Context learning for BIRD-SQL benchmark Repository Link

  A tutorial for both Fine-tuning and In-Context Learning is provided by the BIRD-SQL benchmark.

🔎How to evaluate your model:

We collect NL2SQL evaluation metrics for you. You can get more details from this chapter: Evaluation

• NLSQL360 Repository Link

  NL2SQL360 is a testbed for fine-grained evaluation of NL2SQL solutions. Our testbed integrates existing NL2SQL benchmarks, a repository of NL2SQL models, and various evaluation metrics, which aims to provide an intuitive and user-friendly platform to enable both standard and customized performance evaluations.

• Test-suite-sql-eval Repository Link

  This repo contains a test suite evaluation metric for 11 text-to-SQL tasks. It is now the official metric of Spider, SParC, and CoSQL, and is also now available for Academic, ATIS, Advising, Geography, IMDB, Restaurants, Scholar, and Yelp (building on the amazing work by Catherine and Jonathan).

• BIRD-SQL-Official Repository Link

  It is now the official tool of BIRD-SQL. It is the first tool to propose VES and give an official test suite.

🗺️ Roadmap and Decision Flow

You can get some inspiration from the Roadmap and Decision Flow.

📱 Text-to-SQL Related Applications:

• AI for Database: Agentic AI product for databases — connect any database (PostgreSQL, MySQL, MongoDB, etc.) and talk to it in plain English. Features self-refreshing intelligent dashboards, natural language queries, and automated action workflows that trigger on database changes.
• Chat2DB: AI-driven database tool and SQL client, The hottest GUI client, supporting MySQL, Oracle, PostgreSQL, DB2, SQL Server, DB2, SQLite, H2, ClickHouse, and more.
• DB-GPT: AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents.
• Postgres.new: In-browser Postgres sandbox with AI assistance.
• QueryGPT – Natural Language to SQL Using Generative AI.

📮Connect with Us

Please feel free to contact us if we missed any interesting work.

📧 xliu371[at]connect.hkust-gz.edu.cn