README
Dataset Card for RLPR-Evaluation
News:
- [2025.06.23] 📃 Our paper detailing the RLPR framework and its comprehensive evaluation using this suite is accessible at here!
Dataset Summary
We include the following seven benchmarks for evaluation of RLPR:
Mathematical Reasoning Benchmarks:
- MATH-500 (Cobbe et al., 2021)
- Minerva (Lewkowycz et al., 2022)
- AIME24
General Domain Reasoning Benchmarks:
- MMLU-Pro (Wang et al., 2024): A multitask language understanding benchmark with reasoning-intensive questions. We randomly sample 1000 prompts for a balance of efficiency and variance.
- GPQA (Rein et al., 2023): Graduate-level questions across disciplines. We use the highest-quality GPQA-diamond subset.
- TheoremQA (Chen et al., 2023): Assesses the ability to apply theorems to solve complex science problems (Math, Physics, etc.). We use 800 high-quality questions, removing 53 multimodal instructions.
- WebInstruct (Validation Split) (Ma et al., 2025): A held-out validation split from WebInstruct, designed as an accessible benchmark for medium-sized models. We uniformly sample 1k prompts and apply 10-gram deduplication, resulting in 638 distinct questions.
This multi-faceted suite allows for a thorough evaluation of reasoning capabilities across diverse domains and difficulty levels.
Usage
from datasets import load_dataset
data = load_dataset("openbmb/RLPR-Evaluation")
Data Fields
The dataset contains the following fields for each sample:
| Key | Description | |
|---|---|---|
| 0 | data_source | Identifier for the specific benchmark or split. |
| 1 | prompt | The input question or problem statement, potentially with context or instructions. |
| 2 | ability | The domain or category of the task. |
| 3 | reward_model | Dictionary containing the ground_truth answer, essential for scoring. |
| 4 | extra_info | Benchmark-specific metadata, such as answer_type, category, difficulty, id, or split. |
| 5 | uid | The uid for item in the dataset |
Citation
If you use the RLPR framework or refer to our evaluation methodology using this suite, please cite our paper. Additionally, please cite the original papers for any component benchmarks you use:
@misc{yu2025rlprextrapolatingrlvrgeneral,
title={RLPR: Extrapolating RLVR to General Domains without Verifiers},
author={Tianyu Yu and Bo Ji and Shouli Wang and Shu Yao and Zefan Wang and Ganqu Cui and Lifan Yuan and Ning Ding and Yuan Yao and Zhiyuan Liu and Maosong Sun and Tat-Seng Chua},
year={2025},
eprint={2506.18254},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://huggingface.co/papers/2506.18254},
}
Similar repos
README
Dataset Card for RLPR-Evaluation
News:
- [2025.06.23] 📃 Our paper detailing the RLPR framework and its comprehensive evaluation using this suite is accessible at here!
Dataset Summary
We include the following seven benchmarks for evaluation of RLPR:
Mathematical Reasoning Benchmarks:
- MATH-500 (Cobbe et al., 2021)
- Minerva (Lewkowycz et al., 2022)
- AIME24
General Domain Reasoning Benchmarks:
- MMLU-Pro (Wang et al., 2024): A multitask language understanding benchmark with reasoning-intensive questions. We randomly sample 1000 prompts for a balance of efficiency and variance.
- GPQA (Rein et al., 2023): Graduate-level questions across disciplines. We use the highest-quality GPQA-diamond subset.
- TheoremQA (Chen et al., 2023): Assesses the ability to apply theorems to solve complex science problems (Math, Physics, etc.). We use 800 high-quality questions, removing 53 multimodal instructions.
- WebInstruct (Validation Split) (Ma et al., 2025): A held-out validation split from WebInstruct, designed as an accessible benchmark for medium-sized models. We uniformly sample 1k prompts and apply 10-gram deduplication, resulting in 638 distinct questions.
This multi-faceted suite allows for a thorough evaluation of reasoning capabilities across diverse domains and difficulty levels.
Usage
from datasets import load_dataset
data = load_dataset("openbmb/RLPR-Evaluation")
Data Fields
The dataset contains the following fields for each sample:
| Key | Description | |
|---|---|---|
| 0 | data_source | Identifier for the specific benchmark or split. |
| 1 | prompt | The input question or problem statement, potentially with context or instructions. |
| 2 | ability | The domain or category of the task. |
| 3 | reward_model | Dictionary containing the ground_truth answer, essential for scoring. |
| 4 | extra_info | Benchmark-specific metadata, such as answer_type, category, difficulty, id, or split. |
| 5 | uid | The uid for item in the dataset |
Citation
If you use the RLPR framework or refer to our evaluation methodology using this suite, please cite our paper. Additionally, please cite the original papers for any component benchmarks you use:
@misc{yu2025rlprextrapolatingrlvrgeneral,
title={RLPR: Extrapolating RLVR to General Domains without Verifiers},
author={Tianyu Yu and Bo Ji and Shouli Wang and Shu Yao and Zefan Wang and Ganqu Cui and Lifan Yuan and Ning Ding and Yuan Yao and Zhiyuan Liu and Maosong Sun and Tat-Seng Chua},
year={2025},
eprint={2506.18254},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://huggingface.co/papers/2506.18254},
}