Dataset Card for "TheoremQA"

Introduction

We propose the first question-answering dataset driven by STEM theorems. We annotated 800 QA pairs covering 350+ theorems spanning across Math, EE&CS, Physics and Finance. The dataset is collected by human experts with very high quality. We provide the dataset as a new benchmark to test the limit of large language models to apply theorems to solve challenging university-level questions. We provide a pipeline in the following to prompt LLMs and evaluate their outputs with WolframAlpha.

How to use TheoremQA

from datasets import load_dataset

dataset = load_dataset("TIGER-Lab/TheoremQA")

for d in dataset['test']:
  print(d)

Arxiv Paper:

https://arxiv.org/abs/2305.12524

Code

https://github.com/wenhuchen/TheoremQA/tree/main