📄 PDF Question Answering Pipeline

Welcome to the PDF Question Answering Pipeline! 🚀 This project provides a comprehensive solution to convert PDF documents into text, perform Optical Character Recognition (OCR), and evaluate the extracted text using various advanced models for question answering tasks. Whether you're dealing with large-scale document processing or need robust text analysis, this pipeline has got you covered! 🛠️

📜 Features

• 📚 PDF to Image Conversion: Efficiently convert multi-page PDF files into high-quality images.
• 📝 Image to Text (OCR): Extract text from images using state-of-the-art OCR techniques.
• 🔗 Text Combination: Merge extracted text files into a single, cohesive document for easier analysis.
• 🧠 Model Evaluation for QA: Assess the quality of the extracted text using multiple evaluation models specifically designed for question answering tasks.
• ⚡ Parallel Processing: Utilize multi-threading and multi-processing to speed up conversions and evaluations.
• 📊 Detailed Metrics: Generate comprehensive metrics to evaluate model performance in question answering.

📊 Results Summary

Note: The scores and times are empirical and based on experiments performed on my M1 machine, please run them on your machine to get an accurate estimate on your hardware.

📦 Installation

1. Clone the Repository

   git clone https://github.com/yourusername/pdf-processing-pipeline.git
   cd pdf-processing-pipeline
   

2. Install Dependencies

   Ensure you have Python 3.x installed. Then, install the required Python libraries:

   pip install -r requirements.txt
   

3. Additional Setup

   • Tesseract OCR: Make sure Tesseract is installed on your system. You can download it from here.
   • Ollama API: Ensure the Ollama API is running locally if you're using models that rely on it.

🛠️ Usage

Follow these steps to process your PDF documents and evaluate the extracted text for question answering:

1. Convert PDF to Images

   python pdf2img.py
   

   This will convert your PDF files into images and save them in the output_images directory.

2. Perform OCR on Images

   python img2ocr.py
   

   This step extracts text from the images and saves the output in the ocr_output directory.

3. Combine Extracted Texts

   python combine_texts.py
   

   Merges all text files into a single combined_output.txt for streamlined analysis.

4. Evaluate with Models for QA

   Choose the evaluator based on your requirements:

   • Contextual LLM Evaluator

     python long_text_only_new.py
     

   • LayoutLMv3 Evaluator

     python layoutlm.py
     

   • Document Evaluator (UDOP and Donut)

     python udop.py
     python donut.py
     

   • Short Text Only Model Evaluator

     python short_text_only_model.py
     

📝 Requirements

• Programming Language: Python 3.x
• Libraries:
  • pdf2image
  • pytesseract
  • transformers
  • evaluate
  • natsort
  • joblib
  • tqdm
  • Pillow
  • requests
  • numpy
  • torch
• Other Tools:
  • Tesseract OCR
  • Ollama API (if using related evaluators)

📂 Project Structure

pdf-processing-pipeline/
├── pdf2img.py
├── img2ocr.py
├── combine_texts.py
├── long_text_only_new.py
├── layoutlm.py
├── udop.py
├── donut.py
├── short_text_only_model.py
├── requirements.txt
├── README.md
└── LICENSE

📝 License

Copyright © 2023 Shrey Mishra. All rights reserved.

This project is licensed under the following terms:

• You are allowed to use this code solely for academic purposes, specifically for assignment evaluations.
• No part of this code may be used in any commercial environment or for profit.
• No modifications, adaptations, or redistributions of this code are permitted.
• You must not use this code for any other purpose, including but not limited to personal projects, commercial applications, or any form of distribution.

By using this code, you agree to these terms.

📞 Contact Information

Shrey Mishra
PhD Candidate at ENS PARIS
Email: [email protected]

---

✨ Happy Processing! If you have any questions or need further assistance, feel free to open an issue or reach out. We're here to help! 😊