AI Subtitles

Local-first AI video transcription with speaker diarization, semantic search, and RAG-powered chat.

Features

• Local AI Transcription - Faster Whisper runs on your machine, no API costs
• Speaker Diarization - Automatically identifies and labels different speakers
• Multi-format Support - MP4, MP3, WAV, WebM, MKV, and more
• Multi-language - Auto-detection and translation using MarianMT
• Semantic Search - Find content by meaning with vector embeddings
• Visual Search - CLIP-powered search by describing what you see
• Audio Analysis - Detect laughter, applause, music, and emotions
• RAG Chat - Ask questions about your video with context-aware answers
• Background Jobs - Queue large files for async processing
• Real-time Updates - Live progress via Supabase
• Share Links - Generate public links to share results
• Subtitle Export - WebVTT and SRT with translation support
• Multiple LLMs - Ollama (local), Groq, OpenAI, Anthropic, Grok

Tech Stack

Quick Start

Prerequisites

• Node.js 18+, Python 3.9+, FFmpeg
• HuggingFace token (for speaker diarization)

Backend

cd backend
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env  # Edit with your settings
uvicorn main:app --reload --port 8000

Frontend

cd frontend
npm install
npm run dev

Open http://localhost:5173

Architecture

flowchart TB
    subgraph Frontend["Frontend (React)"]
        UI[UI] --> API[API Client]
        API --> RT[Supabase Realtime]
    end

    subgraph Cloud["Cloud Services"]
        GCS[(GCS)]
        SB[(Supabase + pgvector)]
    end

    subgraph Backend["Backend (FastAPI)"]
        TR[Transcription] --> WH[Whisper]
        SR[Speaker] --> PY[Pyannote]
        CR[Chat] --> SB
        CR --> LLM[LLM Providers]
    end

    API --> TR & SR & CR
    RT <--> SB
    TR --> SB
    TR --> GCS

Troubleshooting

Documentation

• Configuration Guide - All environment variables
• API Reference - Complete endpoint documentation
• Architecture - Detailed system diagrams
• Speaker Diarization Setup
• Production Deployment

Project Structure

ai-subs/
├── frontend/          # React + TypeScript
│   ├── src/
│   │   ├── components/
│   │   ├── hooks/
│   │   ├── services/
│   │   └── types/
│   └── package.json
├── backend/           # FastAPI + ML
│   ├── routers/       # API endpoints
│   ├── services/      # Business logic
│   ├── models/        # Pydantic schemas
│   └── main.py
└── docs/              # Documentation

Contributing

Contributions welcome! Please open issues or submit pull requests.

Acknowledgments

Faster Whisper | Pyannote | Ollama | CLIP | PANNs