Text-to-SQL: Natural Language Database Querying

Overview

This project develops a Text-to-SQL system that converts natural language questions into executable SQL queries. The focus is on understanding how well a sequence-to-sequence transformer can learn the mapping between conversational inputs and structured query logic while remaining practical for real database settings.

The system is built around a fine-tuned T5 model and emphasizes schema-aware inputs, stable decoding, and evaluation through execution rather than string matching alone.

Key Contributions

• Fine-tuned a T5-small encoder-decoder model for the Text-to-SQL task using schema-aware prompts.
• Built a training workflow incorporating FP16 training, gradient accumulation, and beam search decoding.
• Implemented evaluation using both Exact Match and execution-level metrics to better reflect real query correctness.
• Conducted error analysis identifying recurring issues such as missing operators, redundant joins, and alias inconsistencies.

Approach

Model

The system fine-tunes google-t5/t5-small as a text-to-text model for translating natural language into SQL queries.

Input Design

Each question is paired with a compact schema description and the prompt “translate to SQL,” then tokenized jointly. Sequence lengths are capped at 384 tokens to avoid truncation during training.

Training Setup

• Optimizer: AdamW
• Learning rate: 2.33e-3
• Batch size: 32
• Linear warmup schedule
• Early stopping after 5 epochs
• Beam search with 10 beams

Evaluation

Performance is measured using both SQL string matching and execution-based metrics. The fine-tuned model achieves a test F1 of 76.87%, which provides a more meaningful assessment than exact match given that logically equivalent SQL queries often differ syntactically.

Tech Stack

Python, PyTorch, HuggingFace Transformers

Usage

Train

python train_t5.py --finetune --experiment_name ft_experiment

Evaluate

python evaluate.py \
  --predicted_sql results/t5_ft_ft_experiment_dev.sql \
  --predicted_records records/t5_ft_ft_experiment_dev.pkl \
  --development_sql data/dev.sql \
  --development_records records/ground_truth_dev.pkl

Project Structure

• train_t5.py — training and generation loop
• load_data.py, t5_utils.py — preprocessing and utilities
• evaluate.py — EM and execution-based evaluation
• utils.py — SQL execution environment

Takeaway

Fine-tuning seq2seq models works well for Text-to-SQL, but schema design, decoding stability, and evaluation setup are equally important. Execution-based evaluation offers a clearer view of real-world performance than string matching alone.