STT Pipeline

Advanced Speech-to-Text Microservices Architecture

The STT Pipeline is a robust, modular system designed for high-accuracy audio transcription, enhancement, and speaker diarization. It primarily processes Vietnamese audio but is extensible to other languages.

Requirements

Before you begin, ensure your system meets the following requirements:

• Operating System: Linux (Ubuntu 20.04+ recommended) or WSL2.
• Containerization: Docker Engine & Docker Compose.
• Frontend Runtime: Bun (v1.0+).
• Hardware (Recommended): NVIDIA GPU with installed NVIDIA Container Toolkit for hardware acceleration (ASR, Diarization, Enhancement).

Documentation

For detailed information about the system, please refer to the docs/ directory:

• Project Overview
• Architecture
• Frontend Documentation
• API Contracts
• Instruction Guide

Installation

Follow these steps to set up and run the STT Pipeline:

1. Run Setup Script

Execute the quick setup script to initialize directories, create environment files, and check for dependencies.

./scripts/setup.sh

2. Configure Environment

The setup script creates a .env file. You must edit this file to configure essential services.

nano .env

Critical Configuration:

• HF_TOKEN: Required for Speaker Diarization. Get your token from Hugging Face and accept the user agreements for pyannote/speaker-diarization-3.1 and pyannote/segmentation-3.0.
• ENABLE_DIARIZATION: Optional (default: false). Set to true to enable speaker diarization and alignment. Note: requiring Hugging Face token and additional services.
• LLM_BASE_URL: Set this if you plan to use LLM for post-processing refinement (see Troubleshooting below).

3. Download Models

Download the AI models required for the pipeline.

./scripts/model-downloaders/download_models.sh

• Note: This script downloads ASR (Whisper), VAD (Silero), and Enhancement models.

4. Build and Start Services

Build the Docker images and start the backend microservices.

docker compose build
docker compose up -d

Wait for the services to initialize. You can check the status with docker compose ps or view logs with docker compose logs -f.

5. Start Frontend Application

Navigate to the frontend application directory and start the server using Bun.

cd frontend/app
bun install
bun start

The application will be accessible at: http://localhost:8107

Troubleshooting

LLM Models Not Accessible

The pipeline uses an OpenAI-compatible endpoint for text refinement. If the default configuration is not working or unreachable:

1. Change Provider: You can switch to any service that supports the OpenAI API format (e.g., OpenAI, Ollama, vLLM, DeepSeek).
2. Update .env: Modify the LLM_BASE_URL, LLM_API_KEY, and LLM_MODEL variables to match your provider.

Examples of Supported Providers:

• OpenAI:

  LLM_BASE_URL=https://api.openai.com/v1/
  LLM_API_KEY=sk-your-openai-key
  LLM_MODEL=gpt-4o
  

• DeepSeek:

  LLM_BASE_URL=https://api.deepseek.com/v1/
  LLM_API_KEY=sk-your-deepseek-key
  LLM_MODEL=deepseek-chat
  

ASR Model Failure or High VRAM Usage

If the ASR service (Faster Whisper) fails to start, crashes, or gives Out Of Memory (OOM) errors:

1. Use Smaller Models: The default model large-v3 requires significant VRAM (~10GB). You can switch to a smaller model size.

   • Edit .env:

     # Change to a smaller model
     ASR_MODEL_SIZE=medium
     # Or use int8 quantization
     ASR_COMPUTE_TYPE=int8
     

2. Adjust Performance Settings:

   • ASR_NUM_WORKERS: Reduce the number of workers (e.g., set to 1).
   • ASR_DEVICE: Switch to cpu if no GPU is available (slower).

Remote Access (Ngrok)

To expose the services to the internet for testing with the Input Client:

1. Configure NGROK_AUTHTOKEN in .env.
2. Start with ngrok profile: docker compose --profile ngrok up -d.
3. Retrieve the public URL: ./scripts/ops/get_ngrok_url.sh.

See Ngrok Deployment Guide for details.