verifiable-corpus

This is the corpus from "Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning".

Code: https://github.com/jonhue/ttc

Introduction

We study how large language models (LLMs) can continually improve at reasoning on their target tasks at test-time. We propose an agent that assembles a task-specific curriculum, called test-time curriculum (TTC-RL), and applies reinforcement learning to continue training the model for its target task. Our experiments demonstrate that reinforcement learning on a test-time curriculum consistently improves the model on its target tasks, across a variety of evaluations and models.

Overview

Fields:

• kind: Indicates problem type. One of math, code, verifier
• dataset: Indicates original dataset. One of dapo_math, math, gsm8k, livecodebench, taco, primeintellect, webinstruct
• description: Contains the problem / question (without boilerplate)
• problem: Contains the problem / question (inluding boilerplate for training)
• answer: Contains the answer for all datasets except code
• tests: Contains a dictionary with test cases for validating code, serialized to json
  • inputs - input for each test case
  • outputs - output for each test case
  • testtype - either "functional" or "stdin"
  • fn_name - may be "" if not functional or if the first function of the output should be taken
  • context - should be included in namespace during execution of solution attempts
  • time_limit - time limit in seconds

Other fields are maintained from the original datasets.

Sample Usage

To get started with the dataset and reproduce results from the paper, follow these steps:

Installation & Setup

Clone the repository and add to PYTHONPATH:

git clone --recurse-submodules https://github.com/jonhue/ttc
export PYTHONPATH=.../ttc:$PYTHONPATH

Install additional libraries and the modified version of verl:

pip install -r requirements.txt
pip install -e TTRL/verl/.;
pip install -e activeft/.;

This repository builds on the Test-Time Reinforcement Learning (TTRL) and the Volcano Engine Reinforcement Learning (verl) libraries. Please refer to the documentation of these libraries for basic functionality and setup.

Corpus Creation

To generate the corpus, run:

python data/train/create_dataset.py

Dataset Preprocessing

Use the generate_verl_data.sh script to create datasets for training:

DATA_PATH=... # Specify your data path
bash generate_verl_data.sh Qwen/Qwen3-8B lasgroup/verifiable-corpus math-ai/aime25 $DATA_PATH false 500000 true false false true

Training

To start TTC-RL training on the generated dataset:

bash training/verl_training.sh Qwen/Qwen3-8B lasgroup_verifiable-corpus_math-ai_aime25_500000

Creation

1. Clone https://github.com/jonhue/ttc and add to PYTHONPATH
2. Remove dataset_dict.json and train directory
3. Run python data/train/create_dataset.py
4. Run git lfs install
5. Commit and push

Citation

If you find this work helpful, please cite us.

@article{hubotter2025learning,
	title        = {Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning},
	author       = {H{\"u}botter, Jonas and Diaz-Bone, Leander and Hakimi, Ido and Krause, Andreas and Hardt, Moritz},
	year         = 2025,
	journal      = {arXiv preprint arXiv:2510.04786}
}

@inproceedings{hubotter2024efficiently,
	title        = {Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs},
	author       = {H{\"u}botter, Jonas and Bongni, Sascha and Hakimi, Ido and Krause, Andreas},
	year         = 2025,
	booktitle    = {ICLR}
}