BIRD-SQL Incorrect Generator

Generate plausible but incorrect SQL queries for the BIRD benchmark dataset

This tool automatically generates incorrect SQL queries for natural language questions in the BIRD benchmark. The generated queries are syntactically valid but produce incorrect results, simulating realistic mistakes.

BIRD Datasets Generate Queries View Results Analysis
Query Summary

6

Natural Language Queries

18

Incorrect SQL Queries

6

Databases

Database Distribution:
Database Queries
student_1 1
restaurant_1 1
library_1 1
california_schools 1
superhero 1
student_club 1
Getting Started
How to use this tool:
  1. Step 1: Browse available datasets to explore the database schemas and queries: View Datasets
  2. Step 2: Generate incorrect SQL queries using the web interface: Generate Incorrect Queries
  3. Step 3: View and analyze the generated incorrect queries:
    View Results Analysis
About the Query Validator

Our system uses a sophisticated validation process to ensure generated incorrect queries are:

  • Syntactically valid (can be executed without errors)
  • Semantically incorrect (produce different results than the correct query)
  • Realistic (represent common human mistakes)
Command Line Option (Advanced):

For bulk processing, you can also use the command line: 

python -m src.main --num_queries 100 --min_errors 1 --max_errors 5 --analyze