RPG Dataset

Research Plan Generation dataset with three subsets covering ML, Arxiv, and PubMed research papers. Each subset contains research tasks with evaluation rubrics and reference solutions.

Dataset Statistics

Loading the Dataset

from datasets import load_dataset

# Load a specific subset
ml_data = load_dataset("facebook/research-plan-gen", "ml")
arxiv_data = load_dataset("facebook/research-plan-gen", "arxiv")
pubmed_data = load_dataset("facebook/research-plan-gen", "pubmed")

# Access splits
train_data = ml_data['train']
test_data = ml_data['test']

# Get a sample
sample = train_data[0]
print(sample['Goal'])

Dataset Schema

Each sample contains:

• Goal (string): The research task or objective to be accomplished
• Rubric (list of strings): List of evaluation criteria for assessing the generated plan
• Reference solution (string): A reference solution, which is a Llama4-maverick generated summary of how the authors addressed the research task
• article_id (string): Unique identifier for the source article
• q_id (string): Question/task identifier (is a sha256 first 16 chars hash of the goal)
• Subdomain (string): Research subdomain (populated for Arxiv subset, empty string for ML and Pubmed)
• Category (string): Research category (populated for Arxiv subset and ML test, empty string for ML train and Pubmed)
• Identifier (string): Additional identifier field to find the original paper. Openreview forum id for ml papers, arxiv identifier for arxiv papers, pmid for pubmed papers.

Example

{
  'Goal': 'You are tasked with fine-tuning a Large Multimodal Model...',
  'Rubric': [
    'The proposed method should be parameter-efficient...',
    'The method should allow for intuitive control...',
    ...
  ],
  'Reference solution': 'To fine-tune a Large Multimodal Model...',
  'article_id': 'zxg6601zoc',
  'q_id': 'a396a61f2da8ce60',
  'Subdomain': '',
  'Category': '',
  'Identifier': 'zxg6601zoc'
}

Citation

If you use this dataset, please cite:

@article{goel2025training,
  title={Training AI Co-Scientists Using Rubric Rewards},
  author={Goel, Shashwat and Hazra, Rishi and Jayalath, Dulhan and Willi, Timon and Jain, Parag and Shen, William F and Leontiadis, Ilias and Barbieri, Francesco and Bachrach, Yoram and Geiping, Jonas and Whitehouse, Chenxi},
  journal={arXiv preprint arXiv:2512.23707},
  year={2025}
}

License

The Data is released CC-by-NC and is intended for benchmarking purposes only. The goal, grading rubrics and solutions are outputs of Llama 4, and subject to the Llama 4 license ( https://github.com/meta-llama/llama-models/tree/main/models/llama4). If you use this portion of the data to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Llama” at the beginning of any such AI model name. Third party content pulled from other locations are subject to its own licenses and you may have other legal obligations or restrictions that govern your use of that content.