GSM-Ranges Dataset

📄 Paper: Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
🔗 GitHub Repository: GSM-Ranges GitHub

What is GSM-Ranges?

GSM-Ranges is a dataset generator built upon the GSM8K benchmark. It systematically modifies numerical values in math word problems to assess the robustness of large language models (LLMs) across a broad spectrum of numerical scales. By introducing numerical perturbations, GSM-Ranges evaluates how well LLMs generalize mathematical reasoning to out-of-distribution numerical values.

Each perturbation level includes 50 sets of 100 questions, resulting in 5,000 problems per level. The dataset also includes the original 100 base questions from GSM8K for comparison.

Example Problem

How the Perturbation Levels Work

GSM-Ranges applies six levels of numerical perturbation to each question:

1. Level 1 - Numbers are replaced with others of the same digit length.
2. Level 2 - Numbers range between 100 and 1,000.
3. Level 3 - Numbers range between 1,000 and 10,000.
4. Level 4 - Numbers range between 10,000 and 100,000.
5. Level 5 - Numbers range between 100,000 and 1,000,000.
6. Level 6 - Numbers range between 1,000,000 and 10,000,000.

These perturbations maintain the logical structure of the original problems while introducing numerical diversity. For more details on the methodology, refer to the paper:
👉 Mathematical Reasoning in Large Language Models.

Generate Your Own GSM-Ranges Dataset

This dataset is a sample. To generate your own randomized GSM-Ranges dataset, visit the GitHub repository:
👉 GSM-Ranges GitHub

Citation

If you use GSM-Ranges in your research, please cite:

@article{shresthakim2025gsmranges,
  title={Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges},
  author={Shrestha, Safal and Kim, Minwu and Ross, Keith},
  journal={arXiv preprint arXiv:2502.08680},
  year={2025}
}