Voxtral Mini 4B Realtime

This is a 4-bit quantized MLX conversion of mistralai/Voxtral-Mini-4B-Realtime-2602, Mistral AI's streaming speech-to-text model.

Runs via mlx-audio.

Key Details

See also: fp16 variant

Usage

pip install mlx-audio[stt]

from mlx_audio.stt.utils import load

model = load("mlx-community/Voxtral-Mini-4B-Realtime-2602-4bit")

# Transcribe audio
result = model.generate("audio.wav")
print(result.text)

# Streaming transcription
for chunk in model.generate("audio.wav", stream=True):
    print(chunk, end="", flush=True)

# Adjust transcription delay (lower = faster but less accurate)
result = model.generate("audio.wav", transcription_delay_ms=480)

Recommended Settings

Benchmarks (from upstream)

FLEURS (13 languages, WER%)

Long-form English (WER%)

Architecture

• Causal audio encoder (~0.6B) with sliding window attention — enables true streaming
• Language model decoder (~3.4B) based on Ministral-3B with adaptive RMS norm conditioned on transcription delay
• 4x downsampling from encoder to decoder (frame rate = 12.5 Hz)
• Both components use sliding window attention for unbounded audio length

More Info

• Original model card
• mlx-audio GitHub
• Mistral announcement