🎙️ Concierge Platform

An AI-powered digital assistant backend for So Drama! Entertainment — receives visitor speech transcriptions over WebSocket, generates conversational responses via a local LLM, and returns synchronised text tokens + cloned-voice audio in real time.

Tech Stack

---

Prerequisites

• Docker and Docker Compose (v2)
• NVIDIA GPU with drivers installed
• nvidia-container-toolkit
• An LLM backend: LM Studio, VLLM, or SGLang (see services/inference/)

---

Quick Start

# 1. Clone the repo
git clone <repo-url> && cd concierge-platform

# 2. Configure environment
cp .env.example .env
# Edit .env — set HF_TOKEN and LLM settings

# 3. Build and start
docker compose up -d --build

# 4. Verify
curl http://localhost:8001/health
# → {"status": "ok"}

The server will be available at http://localhost:8001.

---

Project Structure

concierge-platform/
├── services/
│   ├── concierge/          # FastAPI WebSocket gateway (CPU)
│   ├── tts/                # gRPC TTS microservice (GPU)
│   └── inference/          # VLLM/SGLang model serving configs
├── tools/
│   └── benchmark/          # Benchmarking suite
├── proto/                  # Shared protobuf definitions
├── context/                # Knowledge base (floor plans, company info)
├── reference_audio/        # TTS voice cloning references
├── docs/                   # API docs, architecture, system prompt
│   ├── API.md
│   ├── PERSONALITY.md
│   └── architecture.md
├── docker-compose.yml      # Full-stack orchestration
└── .github/workflows/      # CI pipeline

---

Documentation

---

Development

Linting & Formatting

# Install ruff
pip install ruff

# Lint
ruff check .

# Format
ruff format .

Running Tests

cd services/concierge
uv sync
uv run pytest tests/ -v

Pre-commit Hooks

pip install pre-commit
pre-commit install

---

Docker Commands

# Build & start all services
docker compose up -d --build

# View logs
docker compose logs -f

# Start a specific inference engine
cd services/inference
docker compose up -d vllm-qwen-9b

# Stop everything
docker compose down

---

Reference Audio (Voice Cloning)

Place a .wav file in reference_audio/:

• Format: WAV (PCM), 5–15 seconds of clean speech
• Active voice: Set via REFERENCE_AUDIO_PATH in .env

---

Troubleshooting