IELTS Listening & Speaking Agent — Full Plan

Architecture at a glance

Microphone
   │
   ▼
[STT] nvidia/parakeet_realtime_eou_120m-v1   (CPU)
   │  transcript text
   ▼
[LLM] gemma-2-2b-it via LM Studio            (GPU)
   │  examiner reply
   ▼
[TTS] nvidia/tts_en_fastpitch + HiFi-GAN     (CPU)
   │  audio waveform
   ▼
Speakers

---

VRAM Budget (4 GB total, ~2804 MiB already used by model)

STT and TTS deliberately run on CPU to leave all VRAM for the LLM. The 120M parakeet model is fast enough on a modern CPU (~200 ms RTF). FastPitch synthesis of a 15-word sentence takes ~0.5 s on CPU.

---

LM Studio Settings (apply in the model's load dialog)

---

LM Studio API — Correct endpoint

The agent uses the OpenAI-compatible endpoint (not /api/v1/chat):

POST http://localhost:1234/v1/chat/completions

{
  "model": "gemma-2-2b-it",
  "messages": [
    {"role": "system", "content": "You are an IELTS examiner…"},
    {"role": "user",   "content": "Student answer here"}
  ],
  "max_tokens": 256,
  "temperature": 0.7
}

The /api/v1/chat endpoint you quoted is LM Studio's native v1 stateful API (added in v0.4.0) which uses input instead of messages[]. Both work, but the OpenAI-compat endpoint is simpler and more stable.

---

STT Model — parakeet_realtime_eou_120m-v1

• Size: 120M parameters — very lightweight
• Input: 16 kHz mono WAV
• Feature: Built-in End-of-Utterance (EOU) detection — knows when you've finished speaking
• Latency: ~100-200 ms on CPU for a 10-second utterance
• NeMo load: ASRModel.from_pretrained("nvidia/parakeet_realtime_eou_120m-v1")

---

TTS Models — FastPitch + HiFi-GAN

• FastPitch: Text → mel spectrogram (fast, ~0.3 s for short sentences)
• HiFi-GAN: Mel spectrogram → waveform (lightweight vocoder)
• Output: 22 050 Hz mono audio
• NeMo load:
  • FastPitchModel.from_pretrained("nvidia/tts_en_fastpitch")
  • HifiGanModel.from_pretrained("nvidia/tts_hifigan")

---

How to run

# 1. Set up environment (first time only)
bash setup.sh

# 2. Activate venv
source .venv/bin/activate

# 3. Start LM Studio → load gemma-2-2b-it → Start Server

# 4. Run the agent
python agent.py

---

Conversation flow

1. Agent speaks an opening IELTS Part 1 question
2. Press ENTER to start recording
3. Speak your answer — agent auto-detects when you stop
4. Agent transcribes → LLM generates feedback + next question → TTS speaks it
5. Repeat; type q to end session

---

Tuning tips