Live Speech Transcription powered by FastRTC ⚡️ and Local Whisper 🤗

This project uses FastRTC to handle live interaction between audio input and text output, and open-source Automatic Speech Recognition (ASR) models via Transformers.

Quick Start 🚀

System Requirements

• Python ≥3.10
• ffmpeg
• CUDA-compatible GPU (optional, for faster inference)

# Clone and enter directory
git clone https://github.com/sofi444/realtime-transcription-fastrtc
cd realtime-transcription-fastrtc

# Set up environment
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Run with default settings
python main.py

Visit the URL shown in the terminal (default: https://localhost:7860) to start transcribing!

Detailed Installation

1. Environment Setup

Choose your preferred package manager:

uv venv --python 3.11 && source .venv/bin/activate
uv pip install -r requirements.txt

python -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

2. Install ffmpeg

brew install ffmpeg

sudo apt update
sudo apt install ffmpeg

Configuration

The application can be configured using environment variables. Create a .env file in the project root with the following variables:

Model Selection

You can use any Whisper model version or other ASR model from Hugging Face. The default whisper-large-v3-turbo is recommended as it's lightweight, performant and multi-lingual.

We use batch size 1 to start transcribing as soon as a chunk is available.

Docker 🐳

I provide a Docker setup for both CPU and GPU: Dockerfile.cpu and Dockerfile.cuda, helpful if you want to deploy the app in a container.

The Dockerfiles use uv as environment and package manager.

Dockerfile.cuda includes Flash Attention installation for faster inference (https://github.com/Dao-AILab/flash-attention).

# Using docker-compose
docker-compose --profile cpu up --build

# Or build manually
docker build -f Dockerfile.cpu -t realtime-transcription-fastrtc-cpu .

# Using docker-compose
docker-compose --profile cuda up --build

# Or build manually
docker build -f Dockerfile.cuda -t realtime-transcription-fastrtc-cuda .

Deploying on HF Spaces 🤗

1. Create a new Space on HuggingFace
2. Select Docker SDK
3. Choose your hardware (CPU/GPU)
4. Set variables and secrets in the Space's settings (see here)
5. Clone your Space's repository
6. Copy the app contents to the new repository
7. Rename either Dockerfile.cpu or Dockerfile.cuda to Dockerfile based on your hardware choice (see previous section about Docker 🐳)
8. Push to your Space!

For deployed environments:

• Set APP_MODE=deployed
• Set TURN_PROVIDER to your chosen provider (cloudflare, hf-cloudflare, or twilio)
• Configure the corresponding TURN server credentials in your Space's secrets

Additional Resources

• FastRTC documentation - Learn about audio stream and Voice Activity Detection (VAD) parameters
• Whisper configuration options
• Available ASR models on Hugging Face