Open (and Free) Deep Research

An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. If you like this project, please consider starring it :) and checking out my LinkedIn

Originally based on the project by @dzhng

The goal of this repo is to provide the completely free and local implementation of a deep research agent - e.g. an agent that can refine its research direction overtime and deep dive into a topic. It'll always be kept at <500 LoC so it is easy to understand and build on top of.

How It Works

flowchart TB
    subgraph Input
        Q[User Query]
        B[Breadth Parameter]
        D[Depth Parameter]
    end

    DR[Deep Research] -->
    SQ[SERP Queries] -->
    PR[Process Results]

    subgraph Results[Results]
        direction TB
        NL((Learnings))
        ND((Directions))
    end

    PR --> NL
    PR --> ND

    DP{depth > 0?}

    RD["Next Direction:
    - Prior Goals
    - New Questions
    - Learnings"]

    MR[Markdown Report]

    %% Main Flow
    Q & B & D --> DR

    %% Results to Decision
    NL & ND --> DP

    %% Circular Flow
    DP -->|Yes| RD
    RD -->|New Context| DR

    %% Final Output
    DP -->|No| MR

    %% Styling
    classDef input fill:#7bed9f,stroke:#2ed573,color:black
    classDef process fill:#70a1ff,stroke:#1e90ff,color:black
    classDef recursive fill:#ffa502,stroke:#ff7f50,color:black
    classDef output fill:#ff4757,stroke:#ff6b81,color:black
    classDef results fill:#a8e6cf,stroke:#3b7a57,color:black

    class Q,B,D input
    class DR,SQ,PR process
    class DP,RD recursive
    class MR output
    class NL,ND results

Features

• Iterative Research: Performs deep research by iteratively generating search queries, processing results, and diving deeper based on findings
• Intelligent Query Generation: Uses LLMs to generate targeted search queries based on research goals and previous findings
• Depth & Breadth Control: Configurable parameters to control how wide (breadth) and deep (depth) the research goes
• Smart Follow-up: Generates follow-up questions to better understand research needs
• Comprehensive Reports: Produces detailed markdown reports with findings and sources
• Concurrent Processing: Handles multiple searches and result processing in parallel for efficiency

Requirements

• Node.js environment
• Enough Hardware to run:
  • Firecrawl
  • DeepSeek R1 1.5B

Setup

1. Clone the repository
2. Install dependencies:

npm install

The LLM and Firecrawl is installed when the project runs for the first time automatically :)

Usage

Run the research assistant:

npm start

You'll be prompted to:

1. Enter your research query
2. Specify research breadth (recommended: 3-10, default: 6)
3. Specify research depth (recommended: 1-5, default: 3)
4. Answer follow-up questions to refine the research direction

The system will then:

1. Generate and execute search queries
2. Process and analyze search results
3. Recursively explore deeper based on findings
4. Generate a comprehensive markdown report

The final report will be saved as output.md in your working directory.

How It Works

1. Initial Setup

   • Takes user query and research parameters (breadth & depth)
   • Generates follow-up questions to understand research needs better

2. Deep Research Process

   • Generates multiple SERP queries based on research goals
   • Processes search results to extract key learnings
   • Generates follow-up research directions

3. Recursive Exploration

   • If depth > 0, takes new research directions and continues exploration
   • Each iteration builds on previous learnings
   • Maintains context of research goals and findings

4. Report Generation

   • Compiles all findings into a comprehensive markdown report
   • Includes all sources and references
   • Organizes information in a clear, readable format