Patent Classification AI

An AI-powered web application for classifying patent documents using fine-tuned BERT models with explainability features.

Overview

This project provides an intelligent system for automatically classifying patent documents into CPC (Cooperative Patent Classification) categories using state-of-the-art NLP models. The application features:

• Multi-level Classification: Hierarchical patent classification (levels 0-3)
• Model Interpretability: Word attribution visualization showing which terms influenced predictions
• Multiple Model Support: Compatible with BERT, SciBERT, BigBird, and T5 architectures
• Web Interface: User-friendly Flask-based interface for document upload and classification
• Authentication System: Secure user management with SQLAlchemy
• Flexible Prediction Methods: Multiple threshold strategies for optimal classification

Key Features

Machine Learning

• Fine-tuned SciBERT models optimized for scientific/technical text
• Support for multiple prediction strategies (avg, max, threshold-based)
• Batch prediction capabilities for efficient processing
• GPU acceleration with automatic CPU fallback
• Model explainability using transformers-interpret

Web Application

• Modern, responsive Bootstrap-based UI
• Real-time classification with progress indicators
• Visual word attribution heatmaps
• User authentication and session management
• RESTful API endpoints for integration

Project Structure

patent-classification-ai/
├── app/                      # Flask web application
│   ├── __init__.py          # Application factory
│   ├── routes.py            # URL routes and views
│   ├── models.py            # Database models
│   ├── forms.py             # WTForms definitions
│   ├── templates/           # Jinja2 HTML templates
│   └── static/              # CSS, JavaScript, images
├── ml/                       # Machine learning components
│   ├── predictor.py         # Main prediction engine
│   ├── data_processing.py   # Data loading and preprocessing
│   ├── custom_classes.py    # Custom PyTorch datasets/trainers
│   └── labels.py            # CPC label definitions
├── training/                 # Model fine-tuning scripts
│   ├── finetuning.py        # Base BERT fine-tuning
│   ├── finetuning_scibert.py # SciBERT fine-tuning
│   └── finetuning_t5.py     # T5 model fine-tuning
├── config.py                 # Configuration management
├── run.py                    # Application entry point
├── requirements.txt          # Python dependencies
├── .env.example             # Environment variables template
└── README.md                # This file

Installation

Prerequisites

• Python 3.9 or higher
• CUDA-capable GPU (optional, but recommended for training)
• 8GB+ RAM

Setup

1. Clone the repository

   git clone https://github.com/yourusername/patent-classification-ai.git
   cd patent-classification-ai
   

2. Create a virtual environment

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   

3. Install dependencies

   pip install -r requirements.txt
   

4. Configure environment variables

   cp .env.example .env
   # Edit .env with your configuration
   

5. Download or train models

   • Place your fine-tuned models in the project root directory
   • Default expected path: model lvl 0 - balanced/
   • Or train your own using scripts in training/

6. Initialize the database

   python run.py  # Database tables are auto-created on first run
   

Usage

Running the Web Application

python run.py

The application will start on http://localhost:5000

API Usage

Quick Classification (returns top predicted class):

curl -X POST http://localhost:5000/classify \
  -H "Content-Type: application/json" \
  -d '{
    "input_data": "Your patent text here...",
    "type": "quick"
  }'

Full Classification (returns predictions + word attributions):

curl -X POST http://localhost:5000/classify \
  -H "Content-Type: application/json" \
  -d '{
    "input_data": "Your patent text here...",
    "type": "full"
  }'

Training Custom Models

# Fine-tune SciBERT on your dataset
python -m training.finetuning_scibert

# Fine-tune T5 model
python -m training.finetuning_t5

Technology Stack

Backend

• Flask - Web framework
• Flask-Login - Authentication
• Flask-SQLAlchemy - ORM for database management
• Flask-WTF - Form handling and validation

Machine Learning

• PyTorch - Deep learning framework
• Transformers (Hugging Face) - Pre-trained model library
• transformers-interpret - Model explainability
• scikit-learn - Metrics and utilities

Data Processing

• pandas - Data manipulation
• BeautifulSoup - HTML parsing for patent documents
• tqdm - Progress bars

Frontend

• Bootstrap 4 - UI framework
• Vanilla JavaScript - Client-side interactions

Model Architecture

The system uses SciBERT (BERT pre-trained on scientific publications) as the base model, fine-tuned for multi-label patent classification:

• Input: Patent text (claims, descriptions, or both)
• Tokenization: SciBERT vocabulary (uncased, 512 max tokens)
• Architecture: BERT encoder + classification head
• Loss Function: Binary Cross-Entropy with Logits (multi-label)
• Output: CPC classification codes with confidence scores

Configuration

Key settings in config.py:

SECRET_KEY            # Flask secret key (set in .env!)
DATABASE_URL          # SQLAlchemy database URI
MODEL_DIR             # Directory containing trained models
LOG_LEVEL             # Logging verbosity

Deployment

Production Considerations

1. Set environment to production

   export FLASK_ENV=production
   

2. Use a production WSGI server

   gunicorn -w 4 -b 0.0.0.0:5000 run:app
   

3. Set a strong SECRET_KEY

   export SECRET_KEY="your-cryptographically-strong-key"
   

4. Configure a production database

   • Use PostgreSQL or MySQL instead of SQLite
   • Set DATABASE_URL environment variable

Performance

• Classification Speed: ~0.5s per document (GPU), ~2s (CPU)
• Model Size: ~440MB (SciBERT-base)
• Memory Usage: ~2GB (inference), ~8GB (training)

License

This project is licensed under the MIT License.

Acknowledgments

• SciBERT: Allen AI for the pre-trained scientific BERT model
• Hugging Face: For the excellent Transformers library
• EPO: European Patent Office for patent data and CPC classification system