Doppelganger

TTS voice cloning Discord bot.

Two TTS engines are supported:

• Chatterbox TTS - zero-shot voice cloning from a short reference audio clip, no fine-tuning needed
• Orpheus TTS - LoRA fine-tuned voices via vLLM for higher quality on specific characters

Quick Start

# Install dependencies (requires CUDA GPU)
uv sync

# Start Postgres
make docker-db

# Run database migrations
make migrate

# Copy and configure environment
cp .env.example .env

# Build dashboard
make frontend

# Start dev server at http://localhost:8000
make dev

Dependencies

• Python 3.12+ and uv
• Docker and Docker Compose
• NVIDIA GPU with CUDA (8-16 GB VRAM for inference)
• FFmpeg (for Discord voice audio)
• Node.js + pnpm (for frontend build)

Discord Bot

The bot runs inside the FastAPI process. Starting the API starts the bot. See docs/bot-setup.md for Discord Developer Portal setup.

Dashboard

The Svelte dashboard is served at http://localhost:8000:

• Dashboard - request metrics, recent activity
• Queue - view, cancel, and bump TTS requests
• Cache - manage audio cache entries, playback, download
• Characters - register, delete, and tune character voices
• Users - view users, blacklist/unblacklist
• Config - read-only view of current settings
• Metrics - per-character and per-user request breakdowns
• System - GPU stats, engine status, cache hit rate, uptime

Voice Cloning

Chatterbox (Zero-Shot)

Register a character with a 5-30 second WAV reference clip. Around 10 seconds of clean mono speech at 22050 Hz works best.

# Upload via API
curl -X POST "http://localhost:8000/api/characters?name=my-character" \
  -F "audio=@/path/to/reference.wav"

# Or place manually and restart
# voices/my-character/reference.wav

Orpheus (Fine-Tuned)

Train a LoRA adapter from multiple audio clips for higher quality. See docs/lora-tuning.md for the full guide.

export CUDA_VISIBLE_DEVICES=0
export CHARACTER=my_character

# Prepare audio clips (3-13s each, normalized)
make prepare-audio ARGS="raw_audio/$CHARACTER/ prepared/$CHARACTER/"

# Transcribe with Whisper
make transcribe ARGS="prepared/$CHARACTER/ --model large"

# Train LoRA adapter
make train-lora ARGS="$CHARACTER prepared/$CHARACTER/ --device cuda --epochs 1"

The voice registry auto-detects adapter files (adapter_config.json) and routes to the Orpheus engine.

API

OpenAPI docs at http://localhost:8000/docs.

Health and Status

TTS

Characters

Queue

Requests

Users

Cache

Other

Configuration

Application

Database

Chatterbox TTS

Orpheus TTS

Discord

Development

make test             # All tests (unit + integration)
make test-unit        # Unit tests only (no Docker)
make test-integration # Integration tests (needs Docker)
make check            # Format + lint + type-check
make fmt              # Format with ruff
make lint             # Lint with ruff

Docker

make docker-db        # PostgreSQL only
make docker-up        # API + PostgreSQL
make docker-down      # Stop all containers
make vllm             # Start vLLM for Orpheus
make psql             # Connect to database

References

• repos
  • https://github.com/canopyai/Orpheus-TTS
  • https://github.com/resemble-ai/chatterbox
  • https://github.com/openai/whisper
  • https://github.com/hubertsiuzdak/snac
  • https://github.com/ytdl-org/youtube-dl
  • https://github.com/pyannote/pyannote-audio
• tested/unused repos
  • https://github.com/RVC-Boss/GPT-SoVITS
  • https://huggingface.co/coqui/XTTS-v2
  • https://huggingface.co/SWivid/F5-TTS
  • https://github.com/idiap/coqui-ai-TTS (maintained version)