✨ Features

• 🎥 Multi-Platform Support: Works with YouTube, TikTok, Bilibili, Apple Podcasts, SoundCloud, and 30+ more
• ⚡ Subtitle-First Architecture: For platforms with native subtitles (e.g. YouTube), transcripts are extracted instantly — no audio download needed. Whisper is only a fallback, making the whole pipeline dramatically faster
• 🎬 Original Video Download: Keep the source video alongside the transcript. It downloads in parallel with transcription, previews inline in the results card, and saves with one click. On the Whisper path the audio is extracted from that same file, so the video is only fetched once. Toggle Keep original video off when you want text only
• 📁 Local File Upload: Drag-and-drop or pick a file — .txt (treated as transcript text), .mp3, .mp4, .m4a, .wav, .webm, .mkv, .ogg, .flac. Media is normalized with FFmpeg for Whisper, then runs the same optimize → translate → summarize pipeline as URLs
• 🗣️ Intelligent Transcription: High-accuracy speech-to-text using Faster-Whisper when subtitles aren't available
• 🤖 AI Text Optimization: Automatic typo correction, sentence completion, and intelligent paragraphing
• 🌍 Multi-Language Summaries: Generate intelligent summaries in 11 languages
• ⚙️ Conditional Translation: Auto-translates the transcript when the summary language differs from the source language
• 🔧 Bring Your Own Model: Configure any OpenAI-compatible API endpoint (OpenAI, OpenRouter, local LLM, etc.) directly in the UI — enter your API Base URL and API Key, then click Fetch to auto-discover available models
• 📡 Live Progress: Server-sent events stream real-time status, with a badge showing whether the job took the subtitle or Whisper path
• 📱 Mobile-Friendly: Responsive layout, dark UI

📑 Contents

• Quick Start
• Usage Guide
• Use It From Claude / Codex / Scripts
• API Reference
• Technical Architecture
• Configuration Options
• FAQ
• Supported Languages
• Performance Tips
• Contributing

🚀 Quick Start

Prerequisites

• Python 3.8+
• FFmpeg (required for yt-dlp audio extraction, merging downloaded video, and normalizing uploaded media)
• An API key from any OpenAI-compatible provider (OpenAI, OpenRouter, etc.) — configurable directly in the UI, no server-side env var needed

Installation

git clone https://github.com/wendy7756/AI-Video-Transcriber.git
cd AI-Video-Transcriber

chmod +x install.sh
./install.sh

git clone https://github.com/wendy7756/AI-Video-Transcriber.git
cd AI-Video-Transcriber

# Using Docker Compose (easiest)
cp .env.example .env
# Edit .env if you want server-side defaults (optional)
docker-compose up -d

# Or using Docker directly
docker build -t ai-video-transcriber .
docker run -p 8000:8000 --env-file .env ai-video-transcriber

# macOS (PEP 668) strongly recommends a virtualenv
python3 -m venv venv
source venv/bin/activate
python -m pip install --upgrade pip
pip install -r requirements.txt

brew install ffmpeg                              # macOS
sudo apt update && sudo apt install ffmpeg       # Ubuntu/Debian
sudo yum install ffmpeg                          # CentOS/RHEL

# Only if you prefer server-side defaults — otherwise configure in the UI
export OPENAI_API_KEY="your_api_key_here"
export OPENAI_BASE_URL="https://openrouter.ai/api/v1"   # any OpenAI-compatible endpoint

Start the Service

python3 start.py

Then open http://localhost:8000.

Production mode (recommended for long videos) — disables hot-reload so the SSE connection stays stable across 30–60+ minute tasks:

python3 start.py --prod

source venv/bin/activate
export OPENAI_API_KEY=your_api_key_here                  # optional: server-side default
# export OPENAI_BASE_URL=https://openrouter.ai/api/v1    # optional: server-side default
python3 start.py --prod

📖 Usage Guide

1. Choose your input — URL or file

• Video / podcast URL: paste a link from YouTube, Bilibili, or any supported platform
• Local file: drag a file onto the dashed upload area, or click to browse. The same Transcribe button starts the job. Uploads use the same API route as URLs (POST /api/process-video with a multipart file), which helps when a reverse proxy only allows that path

2. Pick your options

• Summary Language — the output language for the summary
• Keep original video — on by default. Downloads the source video (≤720p) so you can preview and save it with the results. Turn it off for text-only jobs to save bandwidth and disk

3. (Optional) Configure your AI model — click AI Settings to expand

• Enter your API Base URL (e.g. https://openrouter.ai/api/v1) and API Key
• Click Fetch to auto-load available models, then select one — or leave blank for the server default
• Credentials are stored in your browser's localStorage, never sent anywhere but your chosen provider

4. Start processing — click Transcribe. For URL jobs a badge shows the active mode:

For local uploads, media is normalized with FFmpeg then transcribed with Whisper. Plain .txt files skip download and Whisper entirely, going straight into the text pipeline.

5. Review the results

• Transcript and AI Summary tabs are always present; a Translation tab appears automatically when the transcript language differs from your summary language
• Each tab has its own purple download icon — click it to save that file without switching tabs
• Download original video sits at the right of the tab row, next to an inline player showing the source file and its size

🤖 Use It From Claude / Codex / Scripts

Besides the web UI there is a headless entry point, so agents and scripts can run the same pipeline with no server and no browser.

CLI

venv/bin/python transcribe.py "https://www.youtube.com/watch?v=VIDEO_ID" --json
venv/bin/python transcribe.py talk.mp4 -l zh --no-video
venv/bin/python transcribe.py notes.txt --no-llm          # no API key needed

--json puts a machine-readable result on stdout and keeps progress on stderr. Exit codes: 0 success, 2 bad input, 1 download/transcode failure.

AI provider settings for agents

For CLI/agent use, configure the OpenAI-compatible provider with environment variables:

export OPENAI_API_KEY="your_api_key_here"
export OPENAI_BASE_URL="https://openrouter.ai/api/v1"
export OPENAI_TRANSLATION_MODEL="gpt-4o"  # optional

For one-off CLI runs you can also pass --api-key, --base-url, and --model, but environment variables are safer because API keys do not end up in shell history.

Codex App plugin

The repo also ships a skills-only Codex plugin:

.codex-plugin/plugin.json
skills/video-transcribe/SKILL.md

This repository root is the plugin root. Import or install this folder as a local plugin in Codex App, then start a new task and select AI Video Transcriber from Plugins. Shipping the plugin files does not install the plugin automatically; after changing the plugin, refresh or reinstall it and start a new task so Codex reloads the skill. The skill wraps the same CLI pipeline above, so the machine running Codex still needs this repo's venv and ffmpeg.

Codex App does not provide a plugin-specific settings panel for this skills-only plugin. To switch from OpenRouter to another OpenAI-compatible endpoint, update the environment variables available to the Codex task, or ask Codex to run the CLI with --base-url / --model for that specific run.

This plugin intentionally does not bundle the local stdio MCP server. If you want Codex to call the transcribe_video MCP tool directly, register the MCP server separately as shown below.

Claude Code skill

The repo ships .claude/skills/video-transcribe/SKILL.md, so Claude Code picks it up automatically when you work in this directory — just ask it to transcribe a link. To use it from anywhere, copy the folder to ~/.claude/skills/.

MCP server (Claude Code, Claude Desktop, Codex)

The project includes an optional stdio MCP server. It is not registered automatically; add it once per client.

pip install "mcp>=2.0"

# From the repository root:
claude mcp add video-transcriber \
  -e OPENAI_API_KEY=your_api_key_here \
  -e OPENAI_BASE_URL=https://openrouter.ai/api/v1 \
  -- "$(pwd)/venv/bin/python" "$(pwd)/mcp_server.py"

codex mcp add \
  --env OPENAI_API_KEY=your_api_key_here \
  --env OPENAI_BASE_URL=https://openrouter.ai/api/v1 \
  video-transcriber -- "$(pwd)/venv/bin/python" "$(pwd)/mcp_server.py"

If you prefer editing Codex config directly, add this to ~/.codex/config.toml:

[mcp_servers.video-transcriber]
command = "/abs/path/venv/bin/python"
args = ["/abs/path/mcp_server.py"]

[mcp_servers.video-transcriber.env]
OPENAI_API_KEY = "your_api_key_here"
OPENAI_BASE_URL = "https://openrouter.ai/api/v1"

For Claude Desktop, add the same command and args to your Claude Desktop MCP configuration:

{
  "mcpServers": {
    "video-transcriber": {
      "command": "/abs/path/venv/bin/python",
      "args": ["/abs/path/mcp_server.py"],
      "env": {
        "OPENAI_API_KEY": "your_api_key_here",
        "OPENAI_BASE_URL": "https://openrouter.ai/api/v1"
      }
    }
  }
}

To change providers later, update the MCP client's saved environment variables or remove and re-add the MCP server with the new OPENAI_BASE_URL / model settings, then restart or reload the client if it keeps MCP servers running.

Exposes one tool, transcribe_video, returning the transcript, summary, optional translation, file paths, and a no_speech flag. Verify the wiring with venv/bin/python mcp_server.py --selftest, then check client registration with claude mcp list or codex mcp list.

no_speech matters: when the source has no speech, the pipeline skips the LLM entirely and returns empty text. Agents should report that rather than guessing at the content — feeding an empty transcript to an LLM produces confident fabrications.

🔌 API Reference

All endpoints are served from the same origin as the UI.

# Transcribe a URL, skipping the video download
curl -X POST http://localhost:8000/api/process-video \
  -F "url=https://www.youtube.com/watch?v=VIDEO_ID" \
  -F "summary_language=en" \
  -F "download_video=0"

🛠️ Technical Architecture

Backend — FastAPI · yt-dlp (download & subtitle extraction) · FFmpeg (audio extraction, video merge, upload normalization to mono 16 kHz) · Faster-Whisper (transcription) · OpenAI-compatible API (optimization, translation, summary)

Frontend — vanilla HTML5/CSS3/ES6+ · Marked.js (Markdown rendering) · Font Awesome (icons) · SSE for live progress

Processing Pipeline

URL ──┬─→ probe subtitles ──found──→ parse VTT/SRT ─────────────┐
      │                                                         │
      │   (in parallel, if "Keep original video" is on)          ├─→ optimize
      └─→ download video (≤720p) ──→ extract audio ──→ Whisper ──┘   → translate*
                                                                     → summarize
File ─→ normalize with FFmpeg ──→ Whisper ───────────────────────┘   → results
       (.txt skips straight to the text pipeline)                    (* when languages differ)

Project Structure

AI-Video-Transcriber/
├── backend/
│   ├── main.py             # FastAPI app, routes, task orchestration
│   ├── pipeline.py         # Pure helpers shared by web/CLI/MCP (incl. no-speech guard)
│   ├── video_processor.py  # yt-dlp: subtitles, audio, original video
│   ├── transcriber.py      # Faster-Whisper transcription
│   ├── summarizer.py       # Transcript optimization + summary
│   ├── translator.py       # Conditional translation
│   └── llm_sanitize.py     # Post-process LLM output (strip boilerplate)
├── static/
│   ├── index.html          # UI markup + styles
│   └── app.js              # Frontend logic, SSE, i18n
├── temp/                   # Generated transcripts, summaries, media
├── Dockerfile
├── docker-compose.yml
├── .env.example
├── requirements.txt
├── install.sh
├── start.py                # Startup script (--prod disables hot reload)
├── transcribe.py           # Headless CLI (agents, scripts, cron)
├── mcp_server.py           # MCP server exposing the transcribe_video tool
├── .codex-plugin/
│   └── plugin.json         # Codex App plugin manifest
├── skills/
│   └── video-transcribe/   # Codex plugin skill wrapping the CLI
└── .claude/skills/
    └── video-transcribe/   # Claude Code skill wrapping the CLI

⚙️ Configuration Options

Environment Variables

All are optional — the app runs with defaults and accepts AI credentials from the UI.

Whisper Model Size Options

All sizes are multilingual; tiny–medium also ship English-only variants.

🔧 FAQ

# Delete generated files older than 7 days
find temp -type f -mtime +7 ! -name 'tasks.json' -delete

git clone https://github.com/wendy7756/AI-Video-Transcriber.git
cd AI-Video-Transcriber
cp .env.example .env      # edit for server-side defaults (optional)

docker-compose up -d      # recommended

# Or manually
docker build -t ai-video-transcriber .
docker run -p 8000:8000 --env-file .env ai-video-transcriber

docker ps                                                    # running containers
docker logs ai-video-transcriber-ai-video-transcriber-1      # logs
docker-compose down                                          # stop
docker-compose build --no-cache                              # rebuild

# Reduce memory usage
WHISPER_MODEL_SIZE=tiny

# Limit container memory
docker run -m 1g -p 8000:8000 --env-file .env ai-video-transcriber

# Monitor
docker stats ai-video-transcriber-ai-video-transcriber-1

curl -I https://www.youtube.com/     # platform access
curl -I https://openrouter.ai        # AI provider
docker pull hello-world              # Docker Hub

🎯 Supported Languages

Transcription — 100+ languages via Whisper, with automatic language detection.

Summaries & Translation — English, Chinese (Simplified), Japanese, Korean, Spanish, French, German, Italian, Portuguese, Russian, Arabic.

Interface — English and Chinese, switchable from the top-right.

📈 Performance Tips

Hardware

• Minimum: 4 GB RAM, dual-core CPU
• Recommended: 8 GB RAM, quad-core CPU
• Ideal: 16 GB RAM, multi-core CPU, SSD

Processing time estimates

Times assume Keep original video is off. With it on, subtitle-mode jobs are bounded by the video download rather than the transcript — the download overlaps the AI steps, so the added wall-clock is usually well under the download time itself.

🤝 Contributing

Issues and Pull Requests are welcome!

1. Fork the project
2. Create a feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Acknowledgments

• yt-dlp — powerful video downloading tool
• Faster-Whisper — efficient Whisper implementation
• FastAPI — modern Python web framework
• OpenAI — intelligent text processing API

📞 Contact

For questions or suggestions, please open an Issue or contact Wendy.

🚀 Try the Full Product — sipsip.ai

This tool is the open-source part of sipsip.ai.

The full product goes further:

• 📧 Daily email briefs — follow your favorite creators and get an AI-curated digest in your inbox every morning
• ⚡ Transcribe & summarize any video or podcast on demand
• 🌐 Multi-language support across all features

Free to start — no credit card required.

➡️ sipsip.ai

More from the same developer

• Recording a video? Try the best free teleprompter — use the browser version or download the app versionfor a better experience.
• Growing a product on Pinterest? GetPinFast is a Pinterest growth tool for more traffic and conversions.
• Writing through your thoughts? 3 Pages Daily helps you remember the past, make sense of the present, and shape what comes next.
• Looking for side hustle or small business tools? Clara Vale offers free templates, calculators, and planners for budgeting, finance, bookkeeping, and taxes.

⭐ If you find this project helpful, please consider giving it a star!