AI-powered Research and Review Agents [ICLR 2025 / ACL 2025]

Update:

[03/05/2026] We have opensourced the DeepReviewer-v2 in Github and the DeepReviewer 2.0 online platform is now live. It is free for all scholars. Try it out at deepscientist.cc.

[04/26/2025] We hosted AI Co-scientist Discussion in ICLR 2025, over 300 people gathered together!

[04/06/2025] We have collected 400 papers related to AI Scientists in our Awesome-AI-Scientist GitHub repository. If you're interested in this field, don't miss out!

[03/22/2025] We've just rolled out an exciting new feature for https://ai-researcher.net! 🎉 Now you can directly read arXiv papers with unprecedented ease! 📚✨

Transform any arXiv link from: https://arxiv.org/abs/2503.08569 -> https://ai-researcher.net/abs/2503.08569

🔍 Overview

CycleResearcher is a comprehensive open-source ecosystem for AI-powered academic research and review. Our system features three integrated components:

• CycleResearcher: Generates high-quality research papers
• CycleReviewer: Provides detailed academic reviews
• DeepReviewer: Delivers multi-perspective review simulations with self-verification

By creating a complete feedback loop between research generation and evaluation, we aim to:

• 🤖 Automate academic research processes
• 📝 Provide rigorous, multi-perspective research reviews
• 🔄 Establish research-review feedback loops
• 🚀 Accelerate scientific discovery

🚀 Getting Started

Installation

pip install ai_researcher

Using CycleResearcher

# Import necessary libraries
from ai_researcher import CycleResearcher
from ai_researcher.utils import print_paper_summary

# Initialize CycleResearcher with the default 12B model
researcher = CycleResearcher(model_size="12B")

# Load references from BibTeX file
with open('cycleresearcher_references.bib', 'r') as f:
    references_content = f.read()

# Generate a paper with specific references
generated_papers = researcher.generate_paper(
    topic = "AI Researcher",
    references = references_content,
    n = 1  # Generate a single paper
)

# Print summary of generated paper
print_paper_summary(generated_papers[0])

Using CycleReviewer

# Import necessary libraries
from ai_researcher import CycleReviewer

# Initialize CycleReviewer with the default 8B model
reviewer = CycleReviewer(model_size="8B")

# Review a paper (assuming paper_text contains the paper content)
review_results = reviewer.evaluate(paper_text)

# Print review results
print(f"Average score: {review_results[0]['avg_rating']}")
print(f"Decision: {review_results[0]['paper_decision']}")

Using DeepReviewer

# Import necessary libraries
from ai_researcher import DeepReviewer

# Initialize DeepReviewer with 14B model
deep_reviewer = DeepReviewer(model_size="14B")

# Review a paper with multiple simulated reviewers in Standard Mode
review_results = deep_reviewer.evaluate(
    paper_text,
    mode="Standard Mode",  # Options: "Fast Mode", "Standard Mode", "Best Mode"
    reviewer_num=4         # Simulate 4 different reviewers
)

# Print review results
for i, review in enumerate(review_results[0]['reviews']):
    print(f"Reviewer {i+1} Rating: {review.get('rating', 'N/A')}")
    print(f"Reviewer {i+1} Summary: {review.get('summary', 'N/A')[:100]}...")

Launching DeepReviewer Best Mode

Using OpenScholar

OpenScholar is a retrieval-augmented generation-based academic research question-answering system. For detailed usage instructions, please refer to the OpenScholar directory.

Quick Start Guide for OpenScholar

1. Apply for Semantic Scholar API Key: Visit Semantic Scholar API

2. Start Model Services:

   # For Linux/Mac users
   cd OpenScholar
   chmod +x start_models.sh
   ./start_models.sh
   

3. Start API Service:

   python openscholar_api.py \
       --s2_api_key YOUR_SEMANTIC_SCHOLAR_API_KEY \
       --reranker_path OpenSciLM/OpenScholar_Reranker
   

4. Using the API:

   import requests
   
   # Send questions to OpenScholar API
   response = requests.post("http://localhost:38015/batch_ask", json={
       "questions": ["How do retrieval-augmented LMs perform in knowledge-intensive tasks?"]
   })
   
   result = response.json()
   print("OpenScholar Answer:", result["results"][0]["output"])
   

Best Mode

DeepReviewer's Best Mode provides the most comprehensive review experience, including background knowledge search, multi-reviewer simulation, and self-verification:

# Use Best Mode for in-depth review
review_results = deep_reviewer.evaluate(
    paper_text,
    mode="Best Mode",      # Most comprehensive review mode
    reviewer_num=6,        # Simulate 6 different reviewers
    enable_search=True,    # Enable background knowledge search
    self_verification=True # Enable self-verification
)

📊 Model Evaluation

🧠 Models & Datasets

Models Overview

CycleResearcher Models

Model Name	Pre-training Language Model	HF Link
CycleResearcher-ML-12B	Mistral-Nemo-Instruct-2407	🤗 link
CycleResearcher-ML-72B	Qwen2.5-72B-Instruct	🤗 link
CycleResearcher-ML-123B	Mistral-Large-2	🤗 link

CycleReviewer Models

Model Name	Pre-training Language Model	HF Link
CycleReviewer-ML-Llama3.1-8B	Llama3.1-8B-Instruct	🤗 link
CycleReviewer-ML-Llama3.1-70B	Llama3.1-70B-Instruct	🤗 link
CycleReviewer-ML-Pro-123B	Mistral-Large-2	🤗 link

DeepReviewer Models

Model Name	Parameters	HF Link
DeepReviewer-7B	7B	🤗 link
DeepReviewer-14B	14B	🤗 link

Datasets

💡 Features

DeepReviewer Review Modes

DeepReviewer offers three distinct review modes to accommodate different use cases:

AI Detection

Detect if content was generated by AI models:

from ai_researcher import AIDetector

# Initialize AI detector
detector = AIDetector(device='cpu')

# Analyze the generated paper
detection_result = detector.analyze_paper(paper)

print("Detection Results:")
print(f"Probability of AI generation: {detection_result['probability'] * 100:.2f}%")
print(f"Confidence Level: {detection_result['confidence_level']}")

📚 Tutorials and Demos

We have prepared comprehensive tutorials to help users understand and utilize our models:

• Tutorial 1: Getting Started with CycleResearcher 🚀
• Tutorial 2: Understanding CycleReviewer 📝
• Tutorial 3: Mastering DeepReviewer 🔍
• Tutorial 4: Creating an End-to-End Research Workflow 🔄

📄 License

This code and the models' weights are provided under the CycleResearcher-License. See the LICENSE.md file for details.

📚 Citation

If CycleResearcher is helpful to your work, please cite our paper:

@inproceedings{
weng2025cycleresearcher,
title={CycleResearcher: Improving Automated Research via Automated Review},
author={Yixuan Weng and Minjun Zhu and Guangsheng Bao and Hongbo Zhang and Jindong Wang and Yue Zhang and Linyi Yang},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=bjcsVLoHYs}
}

if DeepReviewer is helpful to your work, please cite our paper:

@misc{zhu2025deepreviewimprovingllmbasedpaper,
      title={DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process}, 
      author={Minjun Zhu and Yixuan Weng and Linyi Yang and Yue Zhang},
      year={2025},
      eprint={2503.08569},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2503.08569}, 
}

📮 Contact

• Submit an Issue
• Email: zhuminjun@westlake.edu.cn