Dataset Card for Spider

Table of Contents

• Dataset Description
  • Dataset Summary
  • Supported Tasks and Leaderboards
  • Languages
• Dataset Structure
  • Data Instances
  • Data Fields
  • Data Splits
• Dataset Creation
  • Curation Rationale
  • Source Data
  • Annotations
  • Personal and Sensitive Information
• Considerations for Using the Data
  • Social Impact of Dataset
  • Discussion of Biases
  • Other Known Limitations
• Additional Information
  • Dataset Curators
  • Licensing Information
  • Citation Information
  • Contributions

Dataset Description

• Homepage: https://yale-lily.github.io/spider
• Repository: https://github.com/taoyds/spider
• Paper: https://www.aclweb.org/anthology/D18-1425/
• Paper: https://arxiv.org/abs/1809.08887
• Point of Contact: Yale LILY

Dataset Summary

Spider is a large-scale complex and cross-domain semantic parsing and text-to-SQL dataset annotated by 11 Yale students. The goal of the Spider challenge is to develop natural language interfaces to cross-domain databases.

Supported Tasks and Leaderboards

The leaderboard can be seen at https://yale-lily.github.io/spider

Languages

The text in the dataset is in English.

Dataset Structure

Data Instances

What do the instances that comprise the dataset represent?

Each instance is natural language question and the equivalent SQL query

How many instances are there in total?

What data does each instance consist of?

[More Information Needed]

Data Fields

• db_id: Database name
• question: Natural language to interpret into SQL
• query: Target SQL query
• query_toks: List of tokens for the query
• query_toks_no_value: List of tokens for the query
• question_toks: List of tokens for the question

Data Splits

train: 7000 questions and SQL query pairs dev: 1034 question and SQL query pairs

[More Information Needed]

Dataset Creation

Curation Rationale

[More Information Needed]

Source Data

Initial Data Collection and Normalization

Who are the source language producers?

[More Information Needed]

Annotations

The dataset was annotated by 11 college students at Yale University

Annotation process

Who are the annotators?

Personal and Sensitive Information

[More Information Needed]

Considerations for Using the Data

Social Impact of Dataset

Discussion of Biases

[More Information Needed]

Other Known Limitations

Additional Information

The listed authors in the homepage are maintaining/supporting the dataset.

Dataset Curators

[More Information Needed]

Licensing Information

The spider dataset is licensed under the CC BY-SA 4.0

[More Information Needed]

Citation Information

@inproceedings{yu-etal-2018-spider,
    title = "{S}pider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-{SQL} Task",
    author = "Yu, Tao  and
      Zhang, Rui  and
      Yang, Kai  and
      Yasunaga, Michihiro  and
      Wang, Dongxu  and
      Li, Zifan  and
      Ma, James  and
      Li, Irene  and
      Yao, Qingning  and
      Roman, Shanelle  and
      Zhang, Zilin  and
      Radev, Dragomir",
    editor = "Riloff, Ellen  and
      Chiang, David  and
      Hockenmaier, Julia  and
      Tsujii, Jun{'}ichi",
    booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing",
    month = oct # "-" # nov,
    year = "2018",
    address = "Brussels, Belgium",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/D18-1425",
    doi = "10.18653/v1/D18-1425",
    pages = "3911--3921",
    archivePrefix={arXiv},
    eprint={1809.08887},
    primaryClass={cs.CL},
}

Contributions

Thanks to @olinguyen for adding this dataset.