🔬 Intelligent Research Paper Explainer v2.0

An autonomous research agent designed for deep-level academic analysis. This system moves beyond text-parsing by integrating Spatial Reasoning and Vision-Language Models (VLM) to interpret a paper's full context, including complex diagrams, tables, and mathematical derivations.

🌟 Advanced Project Capabilities

1. Hierarchical Chunking (HiChunk)

The system treats documents as structured graphs rather than flat strings.

• Structural Context Awareness: Identifies academic sectors (Methodology, Discussion, etc.) via regex-based section mapping.
• Parent-Child Retrieval: Chunks are recursively split and tagged with parent metadata (Section Head & Page), ensuring the AI never loses context during retrieval.

2. Dual-Embedding Pipeline (Vision + Text)

A world-class RAG architecture that indexes multi-modal data:

• Text Path: Uses qwen3-embedding:0.6b via Ollama for lightweight, high-speed semantic indexing.
• Vision Path: Utilizes Qwen3-VL-Embedding-2B (HuggingFace) to embed figures and charts directly into the vector space.
• Visual RAG: Allows you to ask questions like "Explain the trend in Figure 3" by retrieving the embedded visual features.

3. Intelligent Hardware Orchestration

A custom IntelligentModelManager acts as the project's brain to manage your hardware resources:

• VRAM Safeguards: Automatically detects your GPU limits and applies NF4 Quantization (4-bit) for 7B+ models while keeping 2B models in FP16 for quality.
• Sequential Batching: Implements "Phase 2" parsing where pages are processed in batches with mandatory VRAM clearing between runs to prevent OOM (Out Of Memory) errors.

4. High-Fidelity VLM Parsing (Phase 1 & 2)

• Phase 1 (Render): Uses pdftoppm for 4K high-resolution extraction at 300 DPI.
• Phase 2 (Parse): Employs MiniCPM-o-4.5 to transcribe full structured Markdown, capturing high-fidelity tables and LaTeX equations that standard OCR misses.

🛠️ Technical Implementation

Core Components

• Vector Engine: Dual-mode support for FAISS (In-Memory) and ChromaDB (Persistent).
• Inference Engines: Support for local HuggingFace (Transformers) and Ollama backends.
• Document Logic: Multi-layered extraction using PyMuPDF, pdfplumber, and OpenCV for diagram detection.

📂 Project Structure

• backend/main_intelligent.py: The FastAPI coordination layer.
• backend/model_manager.py: Hardware-aware model switcher with VRAM cleanup.
• backend/document_processors.py: High-fidelity VLM parsing and image extraction logic.
• backend/rag_qa.py: Hierarchical retrieval and conversation memory management.
• backend/vector_database.py: Dual-embedding storage (Text + Vision).

⚡ Setup & Launch

1. Prepare Environment:

   pip install -r requirements.txt
   sudo apt-get install poppler-utils  # Required for pdftoppm
   

2. Start API:

   python -m backend.main_intelligent