zai-org/DeepDive

Dataset

DeepDive Dataset

34

2 commits

1 linked in READMEs

updated Mar 17, 2026

See the code

README

DeepDive Dataset

GitHub arXiv Dataset Model

Overview

This is the training dataset for DeepDive, an automated approach for training deep search agents with complex, multi-step reasoning capabilities. The dataset is constructed through automated knowledge graph random walks, entity obfuscation, and difficulty filtering to create challenging questions that require sophisticated search and retrieval skills.

Data Synthesis Pipeline

Dataset Statistics

ComponentSplitSizeDescription
Total Datasetqa_sft, qa_rl3,250Complete collection of QA pairs
SFT Portionqa_sft1,016Question-answer pairs for Supervised Fine-Tuning
↳ SFT Trajectoriestrajectories_sft858Search trajectories from SFT QA pairs via reject sampling
RL Portionqa_rl2,234Question-answer pairs for Reinforcement Learning

Data Structure

qa_sft and qa_rl Split

Fields:

  • id: Unique identifier for the QA pair
  • question: Multi-hop reasoning question requiring search
  • answer: Ground truth answer
  • conversation: [] (empty)

trajectories_sft Split

Fields:

  • id: Unique identifier for the trajectory
  • question: The original question
  • answer: Ground truth answer
  • conversation: List of conversation turns showing the search process (role, content)

Usage

from datasets import load_dataset

dataset = load_dataset("zai-org/DeepDive")

# Access splits
sft_data = dataset["qa_sft"]
rl_data = dataset["qa_rl"]
trajectories = dataset["trajectories_sft"]

License

This dataset is released under the Apache License 2.0.

Citation

If you find DeepDive useful for your research, please cite our paper:

@misc{lu2025deepdiveadvancingdeepsearch,
      title={DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL},
      author={Rui Lu and Zhenyu Hou and Zihan Wang and Hanchen Zhang and Xiao Liu and Yujiang Li and Shi Feng and Jie Tang and Yuxiao Dong},
      year={2025},
      eprint={2509.10446},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2509.10446},
}

zai-org/DeepDive

Dataset

DeepDive Dataset

34

2 commits

1 linked in READMEs

updated Mar 17, 2026

See the code

README

DeepDive Dataset

GitHub arXiv Dataset Model

Overview

This is the training dataset for DeepDive, an automated approach for training deep search agents with complex, multi-step reasoning capabilities. The dataset is constructed through automated knowledge graph random walks, entity obfuscation, and difficulty filtering to create challenging questions that require sophisticated search and retrieval skills.

Data Synthesis Pipeline

Dataset Statistics

ComponentSplitSizeDescription
Total Datasetqa_sft, qa_rl3,250Complete collection of QA pairs
SFT Portionqa_sft1,016Question-answer pairs for Supervised Fine-Tuning
↳ SFT Trajectoriestrajectories_sft858Search trajectories from SFT QA pairs via reject sampling
RL Portionqa_rl2,234Question-answer pairs for Reinforcement Learning

Data Structure

qa_sft and qa_rl Split

Fields:

  • id: Unique identifier for the QA pair
  • question: Multi-hop reasoning question requiring search
  • answer: Ground truth answer
  • conversation: [] (empty)

trajectories_sft Split

Fields:

  • id: Unique identifier for the trajectory
  • question: The original question
  • answer: Ground truth answer
  • conversation: List of conversation turns showing the search process (role, content)

Usage

from datasets import load_dataset

dataset = load_dataset("zai-org/DeepDive")

# Access splits
sft_data = dataset["qa_sft"]
rl_data = dataset["qa_rl"]
trajectories = dataset["trajectories_sft"]

License

This dataset is released under the Apache License 2.0.

Citation

If you find DeepDive useful for your research, please cite our paper:

@misc{lu2025deepdiveadvancingdeepsearch,
      title={DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL},
      author={Rui Lu and Zhenyu Hou and Zihan Wang and Hanchen Zhang and Xiao Liu and Yujiang Li and Shi Feng and Jie Tang and Yuxiao Dong},
      year={2025},
      eprint={2509.10446},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2509.10446},
}