GenAI Roadmap - 2025

OCR with Multimodal | Vision Language Models

Complex PDF Parsing

Comprehensive example code for extracting content from complex PDFs with mixed elements, including text and image data extraction. Includes two Streamlit apps:

1. PDF Parser & RAG Evaluator (pdf_parser_app.py) - Parse PDFs with 13 different parsers + ask questions using RAG
2. VLM OCR App (vlm_ocr_app.py) - Extract text from images using Vision Language Models (Claude, Gemini, GPT-4o, Mistral-OCR, Ollama, OmniAI)

Also, check -> PDF Parsing Guide

YouTube Video: Walkthrough on setup and running the app

Implementation Options

1. Paid - API Based Methods

2. Open Weight - Local Methods

Setup Instructions

Prerequisites

• Python 3.10 or higher
• pip (Python package installer)

Installation

1. Clone the repository:

   git clone https://github.com/genieincodebottle/parsemypdf.git
   cd parsemypdf
   

2. Create a virtual environment:

   pip install uv  # if uv not installed
   uv venv
   .venv\Scripts\activate  # On Linux/Mac -> source .venv/bin/activate
   

3. Install dependencies:

   uv pip install -r requirements.txt
   

4. Configure environment variables:

   Rename .env.example to .env and add the API keys you need.

   You don't need ALL keys. Only add keys for the parsers/LLMs you want to use. Start with a free one.

   # --- Free-tier (no credit card) ---
   GROQ_API_KEY=your_key_here       # Free - https://console.groq.com/keys
   GOOGLE_API_KEY=your_key_here     # Free - https://aistudio.google.com/apikey
   
   # --- Paid ---
   ANTHROPIC_API_KEY=your_key_here  # https://console.anthropic.com/settings/keys
   OPENAI_API_KEY=your_key_here     # https://platform.openai.com/api-keys
   MISTRAL_API_KEY=your_key_here    # https://console.mistral.ai/api-keys
   UNSTRUCTURED_API_KEY=your_key_here # https://unstructured.io/api-key-free
   LLAMA_CLOUD_API_KEY=your_key_here  # https://cloud.llamaindex.ai/api-key
   OMNI_API_KEY=your_key_here       # https://app.getomni.ai/settings/account
   
   # Azure Document Intelligence (optional)
   AZURE_DI_ENDPOINT=your_endpoint  # https://portal.azure.com
   AZURE_DI_KEY=your_key_here
   

5. Install Ollama & Models (optional, for local processing):

   • Download Ollama:
     • Windows: https://ollama.com/download/windows (Requires Windows 10 or later)
     • Linux: curl https://ollama.ai/install.sh | sh
   • Pull required models:

     ollama pull llama3.1
     ollama pull x/llama3.2-vision:11b
     ollama pull gemma3:4b
     ollama pull qwen2.5vl:7b
     ollama pull minicpm-v:8b
     

6. Run the PDF Parser & RAG Evaluator app:

   streamlit run pdf_parser_app.py
   

7. Run the VLM OCR app:

   streamlit run vlm_ocr_app.py
   

8. Run individual parsers:

   • Place PDF files in the input/ directory
   • Run any parser script from the parser/ folder

Example PDFs (in input/ folder)

Important Notes

• System resources needed for local multimodal model operations
• API keys required for API/cloud-based implementations
• Factor in PDF complexity (tables, merged cells, scanned documents, handwritten text, multi-column layouts, rotated text, embedded images) when selecting a parser
• All frameworks, libraries, and multimodal models provided in one place for testing
• Ghostscript is required for Camelot (pip install ghostscript + system install)
• torch is a heavy dependency (~2GB+). It is required for HuggingFace embeddings and local models. If you only need API-based parsers, you can skip it