Premium, low-latency multilingual voice agent pipeline with high-fidelity support for English, Hindi, and Hinglish. Optimized for real-time conversational AI.
| Component | Model / Tool | Optimization |
|---|---|---|
| STT | faster-whisper (Turbo) | CUDA float16, Background Queue |
| LLM (EN/HI) | Gemma-3-1B-IT | GGUF Q4_K_M (llama-cpp) |
| LLM (Mix) | Qwen2.5-3B-Instruct | GGUF Q4_K_M (llama-cpp) |
| TTS (Primary) | OmniVoice (bf16) | 8-step Diffusion, RNG Locked |
| TTS (Fallback) | Kokoro TTS | Lightweight English Fallback |
| Backend | FastAPI | Asynchronous Streaming |
python -m venv venv
# Windows
venv\Scripts\activate
# Linux
source venv/bin/activate
pip install -r requirements.txt
The server will automatically download necessary model weights on the first run.
python SERVER_UPGRADED_PREMIUM.py
Note: Ensure CUDA drivers are properly configured for llama-cpp-python and torch.
Run the interactive client in a separate terminal.
python client.py
Features: Interactive menu, VAD-based recording, and real-time streaming playback.
graph LR
A[Client Microphone] -->|VAD + Opus| B(FastAPI Server)
B --> C[Whisper Turbo STT]
C -->|Text + Lang| D{Language Router}
D -->|EN/HI| E[Gemma-3-1B]
D -->|Hinglish| F[Qwen2.5-3B]
E & F -->|Streaming Text| G[OmniVoice TTS]
G -->|Interleaved Chunks| H(Client Playback)
| Turn Stage | Latency | Optimization |
|---|---|---|
| Upload | 0.9 - 1.2s | Opus compression |
| STT | 0.4 - 1.0s | faster-whisper turbo |
| LLM | 90 - 180ms | GGUF llama-cpp |
| TTS | 0.3 - 0.7s | 8-step diffusion |
| Total perceived | ~1.5 - 2.0s | Pipelining |
SERVER_UPGRADED_PREMIUM.py: High-performance FastAPI server with trilingual routing.client.py: Advanced CLI client with VAD and streaming playback.PROJECT_GUIDE.md: Detailed technical documentation and architectural deep-dive.requirements.txt: Standardized dependency list.n_ctx in the LLM loading section of the server code.LD_LIBRARY_PATH (Linux) or PATH (Windows) includes your CUDA/cuDNN bin directories.sounddevice and pyaudio are properly installed. pip install sounddevice soundfile pyaudio.MIT License.
Premium, low-latency multilingual voice agent pipeline with high-fidelity support for English, Hindi, and Hinglish. Optimized for real-time conversational AI.
| Component | Model / Tool | Optimization |
|---|---|---|
| STT | faster-whisper (Turbo) | CUDA float16, Background Queue |
| LLM (EN/HI) | Gemma-3-1B-IT | GGUF Q4_K_M (llama-cpp) |
| LLM (Mix) | Qwen2.5-3B-Instruct | GGUF Q4_K_M (llama-cpp) |
| TTS (Primary) | OmniVoice (bf16) | 8-step Diffusion, RNG Locked |
| TTS (Fallback) | Kokoro TTS | Lightweight English Fallback |
| Backend | FastAPI | Asynchronous Streaming |
python -m venv venv
# Windows
venv\Scripts\activate
# Linux
source venv/bin/activate
pip install -r requirements.txt
The server will automatically download necessary model weights on the first run.
python SERVER_UPGRADED_PREMIUM.py
Note: Ensure CUDA drivers are properly configured for llama-cpp-python and torch.
Run the interactive client in a separate terminal.
python client.py
Features: Interactive menu, VAD-based recording, and real-time streaming playback.
graph LR
A[Client Microphone] -->|VAD + Opus| B(FastAPI Server)
B --> C[Whisper Turbo STT]
C -->|Text + Lang| D{Language Router}
D -->|EN/HI| E[Gemma-3-1B]
D -->|Hinglish| F[Qwen2.5-3B]
E & F -->|Streaming Text| G[OmniVoice TTS]
G -->|Interleaved Chunks| H(Client Playback)
| Turn Stage | Latency | Optimization |
|---|---|---|
| Upload | 0.9 - 1.2s | Opus compression |
| STT | 0.4 - 1.0s | faster-whisper turbo |
| LLM | 90 - 180ms | GGUF llama-cpp |
| TTS | 0.3 - 0.7s | 8-step diffusion |
| Total perceived | ~1.5 - 2.0s | Pipelining |
SERVER_UPGRADED_PREMIUM.py: High-performance FastAPI server with trilingual routing.client.py: Advanced CLI client with VAD and streaming playback.PROJECT_GUIDE.md: Detailed technical documentation and architectural deep-dive.requirements.txt: Standardized dependency list.n_ctx in the LLM loading section of the server code.LD_LIBRARY_PATH (Linux) or PATH (Windows) includes your CUDA/cuDNN bin directories.sounddevice and pyaudio are properly installed. pip install sounddevice soundfile pyaudio.MIT License.