Dataset Card for BeyondX

• Dataset Description
• Paper Information
• Dataset Examples
• Leaderboard
• Dataset Usage
  • Data Downloading
  • Data Format
  • Data Visualization
  • Data Construction
  • Formulate-and-Solve
• License
• Citation

Dataset Description

BeyondX is a novel algebra reasoning benchmark within multi-unknown, which addresses a limitation that existing math datasets are dominated by problems with at most two unknowns. In total, BeyondX includes 464 examples generated from 2 different source datasets.

Paper Information

• Project Page: https://johnsonkao0213.github.io/Formulate_and_Solve/
• Paper: https://arxiv.org/abs/2407.05134
• Code: https://github.com/johnsonkao0213/Formulate_and_Solve
• Visualization: https://johnsonkao0213.github.io/Formulate_and_Solve/#visualization
• Leaderboard: https://johnsonkao0213.github.io/Formulate_and_Solve/#leaderboard

Dataset Examples

Examples of our dataset under each unknown and original source problem:

1. Source Problem (Two Unknown)

2. Three Unknown

3. Four Unknown

4. Five Unknown

Leaderboard

🏆 The leaderboard for the BeyondX (464 examples) including three, four, and five unknowns is available here.

Dataset Usage

Data Downloading

All the data examples were divided into three subsets: BeyondX_3, BeyondX_4, and BeyondX_5.

• BeyondX_3: 194 examples for three unknowns.
• BeyondX_4: 158 examples for four unknowns.
• BeyondX_5: 112 examples for five unknowns.

You can download this dataset by the following command (make sure that you have installed Huggingface Datasets):

from datasets import load_dataset

dataset = load_dataset("Johnson0213/BeyondX")

Here are some examples of how to access the downloaded dataset:

# print the first example on the BeyondX
print(dataset["train"][0])
print(dataset["train"][0]['id']) # print the problem id 
print(dataset["train"][0]['problem']) # print the problem text 
print(dataset["train"][0]['system_of_equations']) # print the system of equations
print(dataset["train"][0]['ans']) # print the answer
print(dataset["train"][0]['source']) # print the data source

Data Format

The dataset is provided in json format and contains the following attributes:

{
    "problem": [string] The problem text,
    "system_of_equations": [list] The system of equations for the problem,
    "ans": [list] The correct answer for the problem,
    "id": [int] The problem ID, e.g., 1,
    "source": [string] The source dataset from which the problem was taken
}

Data Visualization

🎰 You can interactively explore the dataset here.

Data Construction

The BeyondX dataset is derived from two collected datasets: ALG514 and DRAW-1K. To efficiently generate a large corpus of multi-unknown problems, we developed a novel pipeline that automatically expands existing problems to N unknowns, please refer to our GitHub repository and Paper for details here.

Formulate-and-Solve

🔔 To evaluate our proposed SoTA prompting method Formulate-and-Solve on BeyondX, please refer to our GitHub repository and Paper for details here.

License

The new contributions to our dataset are distributed under the CC BY-SA 4.0 license.

The copyright of the questions belongs to the original authors, and the source of every original question can be found in the source field. Alongside this license, the following conditions apply:

• Purpose: The dataset was primarily designed for use as a test set.
• Commercial Use: The dataset can be used commercially as a test set, but using it as a training set is prohibited. By accessing or using this dataset, you acknowledge and agree to abide by these terms in conjunction with the CC BY-SA 4.0 license.

Citation

@misc{kao2024solvingxbeyondlarge,
      title={Solving for X and Beyond: Can Large Language Models Solve Complex Math Problems with More-Than-Two Unknowns?}, 
      author={Kuei-Chun Kao and Ruochen Wang and Cho-Jui Hsieh},
      year={2024},
      eprint={2407.05134},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2407.05134}, 
}