Gipformer - Efficient Vietnamese Speech Recognition

Highlights

• State-of-the-art accuracy — Demonstrates top-tier performance across major Vietnamese ASR benchmarks, delivering highly precise and reliable transcription quality.
• Robust handling of telephonic domains — Excels in processing challenging, noisy real-world call center recordings across all major Vietnamese regional accents.
• Domain-leading accuracy — Best WER on the technology, finance, education, public administration and medical benchmarks.
• Outstanding parameter efficiency — Ranks among the smallest ASR models currently available.
• Seamless edge deployment — Its naturally low resource requirements enable ultra-fast inference on mobile and embedded systems, making it perfectly suited for offline, on-device applications.
• Built-in data privacy — By supporting full local execution, the model ensures sensitive audio data is processed securely on-device, eliminating the need for third-party cloud services.
• gipformer-68M-rnnt and gipformer1.5-68M-rnnt are both based on the Zipformer Transducer architecture.

Benchmark Results (WER%)

Lower is better; bold = best in each column.

Normalization: Both predictions and labels are normalized before computing WER — lowercased, punctuation removed, and numbers converted to spoken form.

Dataset Descriptions

Private test sets (call center domain):

• tele-medium — Call center recordings with medium difficulty
• tele-hard-north — Low-quality call center audio, hard-to-hear speakers — Northern Vietnamese accent
• tele-hard-middle — Low-quality call center audio, hard-to-hear speakers — Central Vietnamese accent
• tele-hard-south — Low-quality call center audio, hard-to-hear speakers — Southern Vietnamese accent

Public test sets:

• vi-asr-tech-test — Technology: consumer electronics reviews, software tutorials, programming and IT walkthroughs
• vi-asr-edu-test — Education: study-abroad consulting, exam guidance, university and training-course introductions
• vi-asr-finance-test — Finance: stock market commentary, trading platforms, banking and crypto
• vi-asr-pubadmin-test — Public administration: administrative procedures, licensing guidance, civil records
• vivos — Vietnamese read speech corpus
• Common-Voice — Mozilla Common Voice, Vietnamese subset
• vlsp-t1 — VLSP 2020 ASR Shared Task 1
• VietMed — Vietnamese medical domain
• MultiMED — Multi-domain medical conversations
• LSVSC — Large-Scale Vietnamese Speech Corpus
• Fleurs — Google's Few-shot Learning Evaluation of Universal Representations of Speech, Vietnamese subset
• ViMD — Vietnamese Multi-Domain

Call Center Domain: Where It Matters Most

Call center ASR is one of the most challenging real-world domains — noisy phone lines, overlapping speech, diverse regional accents, and spontaneous conversation. The gipformer family holds the best result on all call center test sets.

Quick Start

This project uses uv for dependency management. Install uv first (instructions), then:

# Core (ONNX inference) — works on every platform (CPU/GPU/mobile)
uv sync

# + PyTorch / icefall stack (research & fine-tuning) — Linux + CUDA only
uv sync --extra pytorch

uv sync creates an isolated .venv and installs the pinned dependencies from pyproject.toml / uv.lock. Activate it once, then run python ... directly — all examples below assume an activated venv:

source .venv/bin/activate          # Linux / macOS

ONNX Inference (Recommended)

The simplest way to run the model. Supports CPU, GPU, mobile, and embedded devices.

# Full infer with fp32
python infer_onnx.py --audio data/audio1.wav

# INT8 quantized (smaller & faster)
python infer_onnx.py --audio data/audio1.wav --quantize int8

# Multiple files
python infer_onnx.py --audio data/audio1.wav data/audio2.wav data/audio3.wav

# Pick a model version (default: 1.5)
python infer_onnx.py --audio data/audio1.wav --version 1

# Load from a local model directory instead of downloading
python infer_onnx.py --audio data/audio1.wav --model-dir [path-to-model]

PyTorch Inference (Advanced)

For research and fine-tuning. Requires a Linux machine with CUDA. Install the extra first with uv sync --extra pytorch.

# Basic usage
python infer_pytorch.py --audio data/audio1.wav

# Use GPU
python infer_pytorch.py --audio data/audio1.wav --device cuda

# Multiple files
python infer_pytorch.py --audio data/audio1.wav data/audio2.wav data/audio3.wav

# Pick a model version (default: 1.5)
python infer_pytorch.py --audio data/audio1.wav --version 1

# Load from a local model directory instead of downloading
python infer_pytorch.py --audio data/audio1.wav --model-dir [path-to-model]

Citation

@misc{gipformer,
  title={Gipformer - Efficient Vietnamese Speech Recognition},
  author={G-Group AI Lab},
  year={2026},
  url={https://huggingface.co/g-group-ai-lab/gipformer1.5-68M-rnnt}
}

License

This project is licensed under the MIT License.

Acknowledgments

• k2
• icefall
• sherpa-onnx