VibeASR.cpp

VibeASR.cpp is the official inference runtime for VibeVoice-ASR-BitNet — enabling real-time multilingual speech recognition on CPU through heterogeneous quantization (I8_S for VAE + I2_S for LM).

To enable efficient edge CPU deployment, we replace the original Qwen2.5-7B language model with Qwen2.5-1.5B, achieving only modest accuracy degradation (1–4% absolute WER increase) while reducing the total model size from 4.62 GB to 1.58 GB. Combined with custom SIMD kernels and operator fusion in the ggml framework, VibeVoice-ASR-BitNet achieves 1.6–2.3× faster inference than Whisper.cpp at comparable model sizes, with real-time capability (RTF < 1) on low-resource CPUs.

📄 Tech Report  |  🤗 Models  |  ✨ Demo  |  🏠 GeneralAI

Key Results

Model Size

Inference Performance

RTF (Real-Time Factor) on audio input, excluding one-time model loading. Bold = RTF < 1 (real-time). All measurements use audio clips in the 10s–30s range; the Whisper.cpp comparison (large-v3-turbo) runs the exact same clip through both engines with greedy decoding.

Accuracy (WER%)

Note: The accuracy benchmarks above are evaluated on standard-accent speech corpora. Performance on accented or dialectal speech not represented in the training data may degrade more significantly, as is common with ASR models trained on specific data distributions.

Quick Start

Requirements

• Python ≥ 3.9, CMake ≥ 3.14, GCC/Clang with C++11 support
• ~2 GB disk space (code + quantized models)

Windows users: MSVC is not supported — the build requires GCC or Clang (MinGW-w64 recommended). See Notes for Windows below.

One-Command Setup

git clone --recursive https://github.com/microsoft/VibeASR.cpp.git
cd VibeASR.cpp
pip install -r requirements.txt
python setup_env.py

Manual Build

git clone --recursive https://github.com/microsoft/VibeASR.cpp.git
cd VibeASR.cpp

# Build
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

# Download pre-quantized models
pip install huggingface_hub
huggingface-cli download microsoft/VibeVoice-ASR-BitNet --local-dir models/vibeasr

Usage

CLI Inference

./build/bin/asr_infer \
    --vae-model models/vibeasr/vibeasr-vae-encoder-i8_s.gguf \
    --lm-model models/vibeasr/vibeasr-lm-i2_s-embed-q6_k.gguf \
    --audio input.wav -t 4

Web Demo (Gradio)

pip install gradio soundfile numpy

python demo/gradio_asr_demo.py --port 7860 \
    --vae-model models/vibeasr/vibeasr-vae-encoder-i8_s.gguf \
    --lm-model models/vibeasr/vibeasr-lm-i2_s-embed-q6_k.gguf

Notes for Windows

Windows builds require GCC or Clang — MSVC is rejected by src/CMakeLists.txt. MinGW-w64 (e.g. WinLibs) is recommended. Use the MinGW Makefiles generator, and keep the MinGW bin dir on your PATH at runtime so the executables find their DLLs:

cmake -B build -G "MinGW Makefiles" -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_MAKE_PROGRAM=mingw32-make
cmake --build build --target asr_infer -j

• Build with the command above rather than setup_env.py (its clang probe assumes a POSIX shell).
• Gradio: run python demo/gradio_asr_demo.py --port 7860 (model paths come from the script's MODEL_CONFIGS, not --vae-model/--lm-model; pass --bin build/bin/asr_infer.exe if needed).

Model Conversion

For most users, downloading pre-quantized models from HuggingFace is recommended. To convert from SafeTensors yourself:

Step 1: SafeTensors → F32 GGUF

# LM (BitNet) — handles weight preprocessing and config flattening automatically
python utils/convert_lm_to_gguf.py <safetensors-dir>

# VAE Tokenizer
python utils/convert_vae_to_gguf.py <safetensors-dir>

Step 2: F32 GGUF → Quantized GGUF

# VAE Tokenizer: F32 → I8_S
./build/bin/llama-quantize \
    <safetensors-dir>/vibeasr-vae-encoder-f32.gguf \
    <safetensors-dir>/vibeasr-vae-encoder-i8_s.gguf \
    I8_S 1 1

# LM: F32 → I2_S (with Q6_K embeddings)
./build/bin/llama-quantize --token-embedding-type Q6_K \
    <safetensors-dir>/vibeasr-lm-f32.gguf \
    <safetensors-dir>/vibeasr-lm-i2_s-embed-q6_k.gguf \
    I2_S 1 1

Citation

@article{xu2025vibeasrbitnet,
    title={VibeVoice-ASR-BitNet Technical Report},
    author={Xu, Songchen and Song, Ting and Huang, Shaohan and Peng, Zhiliang and Xia, Yan and Tu, Yujie and Huang, Xin and Yu, Jianwei and Dong, Li and Wei, Furu},
    journal={arXiv preprint arXiv:2607.21075},
    year={2025}
}

License

This project is licensed under the MIT License.