Dataset Card for "ScienceQA"

Dataset Description

• Homepage: https://scienceqa.github.io/
• Repository: https://scienceqa.github.io/#dataset
• Paper: https://arxiv.org/abs/2209.09513
• Leaderboard:
• Point of Contact: https://lupantech.github.io/

Dataset Summary

ScienceQA is collected from elementary and high school science curricula, and contains 21,208 multimodal multiple-choice science questions. Out of the questions in ScienceQA, 10,332 (48.7%) have an image context, 10,220 (48.2%) have a text context, and 6,532 (30.8%) have both. Most questions are annotated with grounded lectures (83.9%) and detailed explanations (90.5%). The lecture and explanation provide general external knowledge and specific reasons, respectively, for arriving at the correct answer. To the best of our knowledge, ScienceQA is the first large-scale multimodal dataset that annotates lectures and explanations for the answers.

ScienceQA, in contrast to previous datasets, has richer domain diversity from three subjects: natural science, language science, and social science. Questions in each subject are categorized first by the topic (Biology, Physics, Chemistry, etc.), then by the category (Plants, Cells, Animals, etc.), and finally by the skill (Classify fruits and vegetables as plant parts, Identify countries of Africa, etc.). ScienceQA features 26 topics, 127 categories, and 379 skills that cover a wide range of domains.

Supported Tasks and Leaderboards

The dataset is prepared to used it for visual question-answering.

Languages

The dataset is in english

Dataset Structure

Data Fields

• image: This field has the image, which is the context given to the model.
• question: This field incorporates the question that has to answer the model from the image context.
• choices: Multiple choice selection.
• answer: The answer from the multiple choice.
• solution: The chain of thought process of the solution selection.
• CTH: A flag that indicates whether it doesnt have chain of thought in that row.

Data Splits

The dataset is split in 80% train and 20% test.

Considerations for Using the Data

The dataset is well balanced in order to get really got result when used in multimodal models.

Additional Information

Dataset Curators

The curators of this dataset where the students from the Masters degree in Computation and Inteligent Systems from University of Deusto.

Citation Information

@inproceedings{lu2022learn,
    title={Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering},
    author={Lu, Pan and Mishra, Swaroop and Xia, Tony and Qiu, Liang and Chang, Kai-Wei and Zhu, Song-Chun and Tafjord, Oyvind and Clark, Peter and Ashwin Kalyan},
    booktitle={The 36th Conference on Neural Information Processing Systems (NeurIPS)},
    year={2022}
}