datasets, pandas, croissantEach row in the dataset includes:
question: The query posed in the dataset.answer: The correct response.category: The domain of the question (e.g., math, science).src: The source of the question.id: A unique identifier for each entry.chain_of_thoughts: Step-by-step reasoning steps leading to the answer.labels: labels of the step-by-step reasoning steps, where 1 refers to a correct step, -1 refers to incorrect ones.parsed_answer: Parsed answer from the chain_of_thoughts.parsed_answer_correctness: Boolean indicating if the parsed answer is correct.The dataset is available via the Hugging Face datasets library:
from datasets import load_dataset
dataset = load_dataset("UW-Madison-Lee-Lab/MMLU-Pro-CoT-Train-Labeled")
5 commits
datasets, pandas, croissantEach row in the dataset includes:
question: The query posed in the dataset.answer: The correct response.category: The domain of the question (e.g., math, science).src: The source of the question.id: A unique identifier for each entry.chain_of_thoughts: Step-by-step reasoning steps leading to the answer.labels: labels of the step-by-step reasoning steps, where 1 refers to a correct step, -1 refers to incorrect ones.parsed_answer: Parsed answer from the chain_of_thoughts.parsed_answer_correctness: Boolean indicating if the parsed answer is correct.The dataset is available via the Hugging Face datasets library:
from datasets import load_dataset
dataset = load_dataset("UW-Madison-Lee-Lab/MMLU-Pro-CoT-Train-Labeled")
5 commits