VLM Predictive Maintenance System

A comprehensive Vision Language Model (VLM) based system for predictive maintenance that detects wear & tear, rust, cracks, fire, smoke, sparks, and other equipment issues without requiring training data.

🎯 Why VLMs for Predictive Maintenance?

Traditional CV Approach ❌

• Requires thousands of labeled images
• Separate models for each defect type
• Expensive data collection
• Retraining for new defect types

VLM Approach ✅

• Pre-trained on billions of images - already knows what defects look like
• Zero-shot detection - no training data needed
• Natural language queries - easy to customize
• Context-aware - distinguishes normal vs abnormal conditions
• Multi-defect detection - finds all issues in one pass

🚀 Features

• Multiple VLM Support: GPT-4V, Claude 3, Gemini, LLaVA (local)
• Real-time Monitoring: Continuous camera feed analysis
• Comprehensive Detection: Wear, tear, rust, cracks, fire, smoke, sparks, leaks, etc.
• Severity Assessment: Automatic risk classification
• Alert System: Webhooks, email notifications for critical issues
• Report Generation: JSON and human-readable reports
• Camera Integration: Webcam, IP cameras, RTSP streams

📋 Installation

1. Clone/Setup

cd C:\VLM

2. Create Virtual Environment

python -m venv venv
.\venv\Scripts\Activate.ps1

3. Install Dependencies

pip install -r requirements.txt

4. Configure API Keys

Copy .env.example to .env and add your API keys:

copy .env.example .env

Edit .env and add your keys:

OPENAI_API_KEY=your_key_here
ANTHROPIC_API_KEY=your_key_here
GOOGLE_API_KEY=your_key_here

🎮 Quick Start

Single Image Analysis

from maintenance_analyzer import MaintenanceAnalyzer

# Initialize with your preferred VLM
analyzer = MaintenanceAnalyzer("gpt4v")  # or "claude", "gemini", "llava"

# Analyze an image
result = analyzer.analyze_equipment("equipment.jpg")

# Print summary
print(analyzer.generate_summary(result))

Live Camera Monitoring

from monitoring_system import MonitoringSystem

# Start continuous monitoring
monitor = MonitoringSystem(
    vlm_type="gpt4v",
    camera_source=0,  # Webcam
    check_interval=300  # Check every 5 minutes
)

monitor.start_monitoring()

Batch Analysis

# Analyze multiple images
images = ["img1.jpg", "img2.jpg", "img3.jpg"]
results = analyzer.batch_analyze(images)

📚 Examples

All examples are in the examples/ folder:

1. example_single_image.py - Analyze a single equipment image
2. example_batch_analysis.py - Analyze multiple images at once
3. example_live_camera.py - Capture and analyze from camera
4. example_continuous_monitoring.py - Run 24/7 monitoring
5. example_vlm_comparison.py - Compare different VLMs
6. example_safety_check.py - Quick critical safety check

Run any example:

python examples\example_single_image.py

🔧 Supported VLMs

Cloud-based (API)

Local (No API)

🎯 Detected Defects

The system can detect:

• ✅ Wear and Tear: Surface degradation, material loss
• ✅ Rust & Corrosion: Oxidation, material decay
• ✅ Cracks: Fractures, splits, breaks
• ✅ Sparks: Electrical arcing, abnormal sparks
• ✅ Fire: Flames, burning, heat damage
• ✅ Smoke: Visible smoke, haze
• ✅ Leaks: Fluid leaks, oil stains
• ✅ Misalignment: Improper positioning
• ✅ Loose Parts: Bolts, screws appearing loose
• ✅ Overheating: Heat discoloration

📊 Output Format

{
  "defects_found": [
    {
      "type": "rust_corrosion",
      "severity": "medium",
      "confidence": 0.85,
      "location": "bottom left corner of housing",
      "description": "Surface rust visible on metal casing",
      "recommended_action": "Clean and apply protective coating"
    }
  ],
  "overall_condition": "fair",
  "immediate_action_required": false,
  "summary": "Minor surface corrosion detected, no critical issues"
}

🔔 Alert System

Configure alerts in .env:

ALERT_EMAIL=maintenance@company.com
ALERT_WEBHOOK_URL=https://your-webhook.com

Alerts are triggered for:

• 🔴 Critical severity defects
• 🔥 Fire or smoke detection
• ⚡ Immediate action required

📁 Project Structure

VLM/
├── config.py                 # Configuration
├── vlm_base.py              # Base VLM interface
├── vlm_gpt4.py              # GPT-4 Vision implementation
├── vlm_claude.py            # Claude 3 implementation
├── vlm_gemini.py            # Gemini implementation
├── vlm_llava.py             # LLaVA local implementation
├── maintenance_analyzer.py   # Main analyzer
├── camera_capture.py        # Camera integration
├── monitoring_system.py     # Continuous monitoring
├── alert_system.py          # Alert handling
├── examples/                # Usage examples
├── output/                  # Captured images
└── reports/                 # Analysis reports

🔑 API Key Setup

OpenAI (GPT-4V)

1. Go to https://platform.openai.com/api-keys
2. Create new API key
3. Add to .env: OPENAI_API_KEY=sk-...

Anthropic (Claude)

1. Go to https://console.anthropic.com/
2. Get API key
3. Add to .env: ANTHROPIC_API_KEY=sk-ant-...

Google (Gemini)

1. Go to https://makersuite.google.com/app/apikey
2. Create API key
3. Add to .env: GOOGLE_API_KEY=AIza...

💡 Tips

1. Start with Gemini - Free tier, good for testing
2. Use GPT-4V for production - Best accuracy
3. Try LLaVA for privacy - Runs locally, no data sent to cloud
4. Adjust check intervals - Balance between coverage and cost
5. Test with sample images - Verify before deploying

🛠️ Customization

Add Custom Prompts

Edit vlm_base.py to add custom inspection prompts:

CUSTOM_PROMPT = """
Look specifically for:
- Your specific defect type
- Your specific conditions
...
"""

Adjust Sensitivity

Edit config.py:

CONFIDENCE_THRESHOLD = 0.7  # Higher = fewer false positives

📞 Support

For issues or questions:

1. Check the examples folder
2. Review the configuration in config.py
3. Test with a simple image first

📄 License

MIT License - Free to use and modify

---

Ready to start? Run: python examples\example_single_image.py