Gipformer - Efficient Vietnamese Speech Recognition

Highlights

• State-of-the-art accuracy — Demonstrates top-tier performance across major Vietnamese ASR benchmarks, delivering highly precise and reliable transcription quality.
• Robust handling of telephonic domains — Excels in processing challenging, noisy real-world call center recordings across all major Vietnamese regional accents.
• Outstanding parameter efficiency — Ranks among the smallest ASR models currently available.
• Seamless edge deployment — Its naturally low resource requirements enable ultra-fast inference on mobile and embedded systems, making it perfectly suited for offline, on-device applications.
• Built-in data privacy — By supporting full local execution, the model ensures sensitive audio data is processed securely on-device, eliminating the need for third-party cloud services.
• gipformer-65M-rnnt is based on Zipformer Transducer architecture.

Benchmark Results (WER%)

Lower is better. Bold = best result.

Normalization: Both predictions and labels are normalized before computing WER — lowercased, diacritics removed, and numbers converted to spoken form.

Dataset Descriptions

Private test sets (call center domain):

• tele-medium — Call center recordings with medium difficulty
• tele-difficult-north — Low-quality call center audio, hard-to-hear speakers — Northern Vietnamese accent
• tele-difficult-middle — Low-quality call center audio, hard-to-hear speakers — Central Vietnamese accent
• tele-difficult-south — Low-quality call center audio, hard-to-hear speakers — Southern Vietnamese accent

Public test sets:

• MultiMED — Multi-domain medical conversations
• VietMed — Vietnamese medical domain
• vlsp-2020-task-1 — VLSP 2020 ASR Shared Task 1
• vlsp-2020-task-2 — VLSP 2020 ASR Shared Task 2
• LSVSC — Large-Scale Vietnamese Speech Corpus
• Fleurs — Google's Few-shot Learning Evaluation of Universal Representations of Speech (Vietnamese subset)
• ViMD — Vietnamese Multi-Domain
• vivos — Vietnamese read speech corpus

Call Center Domain: Where It Matters Most

Call center ASR is one of the most challenging real-world domains — noisy phone lines, overlapping speech, diverse regional accents, and spontaneous conversation. gipformer-65M-rnnt delivers dominant performance across all call center test sets.

Quick Start

Installation

Hardware: All experiments were conducted on an NVIDIA RTX 4090.

# Python 3.8+ required
pip install -r requirements.txt

ONNX Inference (Recommended)

The simplest way to run the model. Supports CPU, GPU, mobile, and embedded devices.

# Full infer with fp32
python infer_onnx.py --audio data/audio1.wav

# INT8 quantized (smaller & faster)
python infer_onnx.py --audio data/audio1.wav --quantize int8

# Multiple files
python infer_onnx.py --audio data/audio1.wav data/audio2.wav data/audio3.wav

PyTorch Inference (Advanced)

For research and fine-tuning. Requires CUDA toolkit and additional dependencies.

# Basic usage
python infer_pytorch.py --audio data/audio1.wav

# Use GPU
python infer_pytorch.py --audio data/audio1.wav --device cuda

# Multiple files
python infer_pytorch.py --audio data/audio1.wav data/audio2.wav data/audio3.wav

Note: On first run, the script automatically downloads the model (~280MB) and clones icefall (~50MB) for model architecture code.

Citation

@misc{gipformer,
  title={Gipformer - Efficient Vietnamese Speech Recognition},
  author={G-Group AI Lab},
  year={2026},
  url={https://huggingface.co/g-group-ai-lab/gipformer-65M-rnnt}
}

License

This project is licensed under the MIT License.

Acknowledgments

• k2
• icefall
• sherpa-onnx