AnnoyRAG Dataset

The AnnoyRAG dataset, introduced in Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning, employs entity anonymization to isolate LLMs' parametric knowledge. This design enables more precise evaluation of how effectively LLMs integrate retrieved information in RAG systems.

Dataset Details

Dataset Description

The basic statistical information of the dataset is as follows:

• Languages: [English&Chinese]

Dataset Sources

All raw data in this study are sourced from the original texts of four classic novels: Water Margin, Dream of the Red Chamber, Moby-Dick, and Middlemarch. The copyrights of all these works have entered the public domain.

Dataset Structure

1. annoyrag_chs_qa.parquet & annoyrag_eng_qa.parquet

1. question: questions designed as input queries to assess RAG capabilities.
2. answer: final answers for reference.
3. relations: potentially relevant knowledge graph relations (reference only).
4. entities: potentially relevant knowledge graph entities (reference only).
5. query_type: The question types include multiple-choice questions and de-anonymization tasks.

2. annoyrag_chs_text_chunks.parquet & annoyrag_eng_text_chunks.parquet

1. idx: positional indices for chunk ordering.
2. title: section titles after anonymization processing.
3. chunk: The anonymized corpus has been chunked with a maximum size of 1,000 characters per segment. To prevent semantic truncation, we implemented a 100-character overlap between consecutive chunks.

License

• Dataset: cc-by-nc-4.0.
• Generated using DeepSeek-V3-0324.

Citation

@misc{dong2025youtugraphragverticallyunifiedagents,
      title={Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning}, 
      author={Junnan Dong and Siyu An and Yifei Yu and Qian-Wen Zhang and Linhao Luo and Xiao Huang and Yunsheng Wu and Di Yin and Xing Sun},
      year={2025},
      eprint={2508.19855},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2508.19855}, 
}