Sanskrit OCR & Translation System

AI-Powered Sanskrit OCR for Printed & Handwritten Text with Hindi and English Translation

---

Overview

The Sanskrit OCR & Translation System is an AI-powered application designed to extract Sanskrit text from:

• Printed Sanskrit documents
• Handwritten Sanskrit notes
• Sanskrit PDFs

and automatically translate the extracted text into:

• English
• Hindi

The project combines:

• Computer Vision
• Optical Character Recognition (OCR)
• Transformer-based NLP
• Retrieval-Augmented Generation (RAG)

to create a complete Sanskrit document understanding pipeline.

---

Key Features

• Printed Sanskrit OCR using Tesseract
• Handwritten Sanskrit OCR using TrOCR
• Sanskrit PDF OCR Support
• Sanskrit → English Translation
• Sanskrit → Hindi Translation
• RAG-based Sanskrit Knowledge Retrieval
• OCR History Storage
• Streamlit Web Interface
• CPU-Compatible Inference
• Synthetic Sanskrit Dataset Generation
• TrOCR Fine-Tuning Pipeline
• ChromaDB Vector Database Integration

---

System Architecture

                    ┌──────────────────┐
                    │   User Upload    │
                    │ Image / PDF File │
                    └────────┬─────────┘
                             │
                             ▼
                 ┌──────────────────────┐
                 │  Image Preprocessing │
                 │ OpenCV + PIL         │
                 └────────┬─────────────┘
                          │
                          ▼
              ┌───────────────────────────┐
              │        OCR Engine         │
              ├───────────────────────────┤
              │ Printed → Tesseract OCR  │
              │ Handwritten → TrOCR      │
              └────────┬──────────────────┘
                       │
                       ▼
             ┌──────────────────────┐
             │  Text Postprocessing │
             └────────┬─────────────┘
                      │
                      ▼
           ┌──────────────────────────┐
           │     Translation Layer    │
           │       NLLB-200 Model     │
           └────────┬─────────────────┘
                    │
                    ▼
       ┌──────────────────────────────┐
       │ RAG + ChromaDB Retrieval     │
       └────────┬─────────────────────┘
                │
                ▼
      ┌───────────────────────────────┐
      │ Streamlit Web Interface       │
      └───────────────────────────────┘

---

Tech Stack

---

OCR Pipeline

Printed OCR Pipeline

The printed OCR workflow includes:

1. Image Upload
2. Image Preprocessing
3. Noise Removal
4. Sharpening
5. Thresholding
6. Sanskrit OCR using Tesseract
7. Text Cleanup
8. Translation

Printed OCR Accuracy

• Approximate accuracy:
  • 80–90% on clean printed Sanskrit text

---

Handwritten OCR Pipeline

The handwritten OCR workflow includes:

1. Handwritten Image Upload
2. Adaptive Image Preprocessing
3. TrOCR Transformer Inference
4. Sequence Generation
5. OCR Postprocessing
6. Translation

---

TrOCR Model

Current CPU-Friendly Model

MODEL_NAME = "microsoft/trocr-small-handwritten"

Why this model?

• Faster CPU inference
• Lightweight deployment
• Lower RAM usage
• Better compatibility with Render/Streamlit deployment

---

GPU Recommendation

If GPU is available, better handwritten OCR performance can be achieved using:

MODEL_NAME = "microsoft/trocr-base-handwritten"

or

MODEL_NAME = "microsoft/trocr-large-handwritten"

Advantages of GPU Models

• Better handwriting recognition
• Higher OCR accuracy
• Better contextual decoding
• Improved sequence generation

---

Translation Pipeline

The project uses the following multilingual translation model:

facebook/nllb-200-distilled-600M

Supported Translation Directions

• Sanskrit → English
• Sanskrit → Hindi

---

RAG (Retrieval-Augmented Generation)

The system includes a lightweight RAG pipeline for Sanskrit knowledge retrieval.

Components

• Sanskrit Corpus Storage
• Sentence Embeddings
• ChromaDB Vector Database
• Semantic Retrieval

Purpose

• Contextual Sanskrit assistance
• Similar text retrieval
• Knowledge-based augmentation

---

Synthetic Dataset Generation

A synthetic Sanskrit dataset generation pipeline was created using:

• PIL
• Devanagari fonts
• Noise injection
• Blur augmentation
• Rotation augmentation

Generated Dataset

• 1000+ synthetic Sanskrit image samples

---

TrOCR Fine-Tuning

Fine-tuning experiments were performed using:

• HuggingFace Transformers
• PyTorch
• Synthetic Sanskrit handwritten dataset

Training Details

• CPU-based fine-tuning
• CER monitoring
• Sequence-to-sequence optimization

---

Project Folder Structure

sanskrit_ocr/
│
├── app.py
├── requirements.txt
├── render.yaml
├── apt.txt
│
├── ocr/
│   ├── tesseract_engine.py
│   ├── trocr_engine.py
│   └── preprocess.py
│
├── translation/
│   ├── translate.py
│   └── language_utils.py
│
├── rag/
│   ├── rag_pipeline.py
│   ├── chroma_db/
│   └── corpus_data/
│
├── database/
│   ├── history.db
│   └── history_db.py
│
├── fine_tuning/
│   ├── generate_synthetic.py
│   └── train_trocr.py
│
├── data/
│   ├── fonts/
│   └── synthetic/
│
└── .streamlit/
    └── config.toml

---

Installation Guide

Clone Repository

git clone https://github.com/avinashsingh539/Sanskrit-ocr.git
cd Sanskrit-ocr

---

Create Virtual Environment

Windows

python -m venv .venv

---

Activate Virtual Environment

Windows PowerShell

.\.venv\Scripts\Activate.ps1

Linux / MacOS

source .venv/bin/activate

---

Install Dependencies

pip install -r requirements.txt

---

Run Application

streamlit run app.py

---

PDF Support

Windows

Install Poppler and add:

Library/bin

to system PATH.

Linux

sudo apt install poppler-utils

---

Deployment Support

The project is deployment-ready for:

• Render
• HuggingFace Spaces
• Streamlit Cloud

---

Future Improvements

• Better Sanskrit handwriting recognition
• GPU fine-tuning support
• Sanskrit grammar correction
• Sanskrit semantic search
• Sanskrit speech synthesis
• Better OCR line segmentation
• Cloud-based inference optimization

---

Authors

Avinash Singh

SY Computer Engineering Mini Project

---

Acknowledgements

Special thanks to:

• HuggingFace Transformers
• Microsoft TrOCR
• Facebook NLLB
• OpenCV
• Streamlit
• Tesseract OCR
• ChromaDB
• PyTorch