🔥 News

• Coming soon:  🎉🎉 Support the generation of comprehensive surveys in all fields on the online website.

• 2025.06:  🎉🎉 We released the code of SurveyForge.

• 2025.05:  🎉🎉 Congratulations: SurveyForge was accepted by ACL-2025 main conference.

Introduction

Survey papers are vital in scientific research, especially with the rapid increase in research publications. Recently, researchers have started using LLMs to automate survey creation for improved efficiency. However, LLM-generated surveys often fall short compared to human-written ones, particularly in outline quality and citation accuracy. To address this, we introduce SurveyForge, which first creates an outline by analyzing the structure of human-written outlines and consulting domain-related articles. Then, using high-quality papers retrieved by our scholar navigation agent, SurveyForge can automatically generate and refine the content of the survey.

Moreover, to achieve a comprehensive evaluation, we construct SurveyBench, which includes 100 human-written survey papers for win-rate comparison and assesses AI-generated survey papers across three dimensions: reference, outline, and content quality.

🤔How to try out SurveyForge?

• First you need to download the database from [SurveyForge Database].

• Then you need to download the embedding model from [gte-large-en-v1.5]

• Finally, configure your API KEY and data path in code/run_demo.py and run the command as follows:

cd code && python run_demo.py

🕵️‍♂️How to evaluate the quality of the survey paper?

We offer SurveyBench, a benchmark for academic research and evaluating the quality of AI-generated surveys.

SurveyBench Download

Currently , SurveyBench consists of approximately 100 human-written survey papers across 10 distinct topics, carefully curated by doctoral-level researchers to ensure thematic consistency and academic rigor. The supported topics and the core references corresponding to each topic are as follows:

More support topics coming soon!

🧑‍💻You can evaluate the survey by:

cd SurveyBench && python test.py --is_human_eval

Note set is_human_eval True for human survey evaluation, False for generated surveys.

If you want to evaluate your method on SurveyBench, please follow the format:

generated_surveys
|-- 3D Gaussian Splatting
    |-- exp_1
        |-- ref.json
    |-- exp_2
        |-- ref.json
...
|-- Graph Neural Networks
...

📝Examples

Acknowledgements

We sincerely thank the AutoSurvey for laying the foundation in automated survey generation and analysis. SurveyForge is developed on top of the AutoSurvey framework, and we remain committed to continuous innovation and delivering ever more powerful, flexible solutions for automated survey research.

Citations

@article{yan2025surveyforge,
  title={Surveyforge: On the outline heuristics, memory-driven generation, and multi-dimensional evaluation for automated survey writing},
  author={Yan, Xiangchao and Feng, Shiyang and Yuan, Jiakang and Xia, Renqiu and Wang, Bin and Zhang, Bo and Bai, Lei},
  journal={arXiv preprint arXiv:2503.04629},
  year={2025}
}