CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning

This is the official dataset for the CLaRa paper which contains training and evaluation data for the CLaRa model, organized into three main categories: pretraining, instruction tuning, and end-to-end tuning.

Dataset Structure

1. Pretraining Data

• pretraining: Large-scale pretraining data for the compressor learning
  • Format: JSONL with fields: data_type, question, answers, docs

2. Instruction Tuning Data

• instruction_tuning: Training data for answering questions based on compressed documents representation
  • Format: JSONL with fields: question, docs, gold_answer, answer
• inst_eval_*_with_pos: Evaluation sets in oracle setting (guaranteed to contain positive documents)
• inst_eval_*_no_pos: Evaluation sets in normal setting
  • Datasets: 2wiki, hotpotqa, musique, nq

3. End-to-End Tuning Data

• e2e_training_*_with_pos: Training sets in oracle setting (guaranteed to contain positive documents)
• e2e_training_*_no_pos: Training sets in normal setting
• e2e_eval_*_with_pos: Evaluation sets in oracle setting (guaranteed to contain positive documents)
• e2e_eval_*_no_pos: Evaluation sets in normal setting
  • Datasets: ours_2wiki, ours_hotpotqa, ours_musique, ours_nq
  • Format: JSONL with fields: question, answer, docs, pos_index

Usage

from datasets import load_dataset

# Load pretraining data
pretrain = load_dataset("apple/CLaRa_multi_stage", "pretraining")

# Load instruction tuning data
inst_tune = load_dataset("apple/CLaRa_multi_stage", "instruction_tuning")

# Load end-to-end tuning data (normal setting)
e2e_train = load_dataset("apple/CLaRa_multi_stage", "e2e_training_ours_2wiki_no_pos")

# Load end-to-end tuning data (oracle setting with positive documents)
e2e_train_oracle = load_dataset("apple/CLaRa_multi_stage", "e2e_training_ours_2wiki_with_pos")

Data Fields

Pretraining Format

• data_type: Type of data (e.g., "qa")
• question: List of questions
• answers: List of corresponding answers
• docs: List of document contexts

Instruction Tuning Format

• question: The question string
• docs: List of retrieved documents
• gold_answer: Ground truth answer
• answer: Model-generated answer

End-to-End Tuning Format

• question: The question string
• answer: The answer string
• docs: List of retrieved documents
• pos_index: Index of the positive document (the document that helps the model answer the question) in the docs list

More information:

• Paper: CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning (https://arxiv.org/abs/2511.18659)
• GitHub: https://github.com/apple/ml-clara

Citation

If you use this dataset, please cite:

@misc{he2025clarabridgingretrievalgeneration,
      title={CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning}, 
      author={Jie He and Richard He Bai and Sinead Williamson and Jeff Z. Pan and Navdeep Jaitly and Yizhe Zhang},
      year={2025},
      eprint={2511.18659},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2511.18659}, 
}

License

MIT License