Breeze ASR 26 - Taiwanese Hokkien Speech Recognition

MediaTek Research Breeze 3：讓 AI 聽懂台語、說出台味、守護台灣 https://www.mediatek.com/zh-tw/tek-talk-blogs/mediatek-research-breeze-3

Integration with HuggingFace's Breeze ASR 26 model for automatic speech recognition in Taiwanese Hokkien (台語/Taigi).

Overview

Breeze ASR 26 is a state-of-the-art automatic speech recognition (ASR) model developed by MediaTek Research for Taiwanese Hokkien. It:

• Fine-tuned from OpenAI's Whisper-large-v2 architecture
• Trained on ~10,000 hours of synthetic Taiwanese Hokkien speech
• Outputs transcriptions in Mandarin Chinese characters
• Achieves competitive Character Error Rate (CER) ~30.13%
• Available on HuggingFace Model Hub

Installation

Clone the repository

cd breeze-asr-26-project

Quick Start

# 1. 安裝依賴
uv sync

# 2. 下載模型到本地（約 3-6 GB，只需執行一次）
uv run download-model

# 3. 啟動即時轉錄
uv run live-transcribe

Method 1: Simple Pipeline

from transformers import pipeline

# Load model
pipe = pipeline("automatic-speech-recognition", model="MediaTek-Research/Breeze-ASR-26")

# Transcribe
result = pipe("audio.wav")
print(result['text'])

Method 2: Using the Wrapper Class

from main import BreezASR26

# Initialize
asr = BreezASR26()

# Transcribe file
result = asr.transcribe_file("your_audio.wav")
print(f"Text: {result['text']}")

Method 3: Real-time Microphone (NEW! 🎤)

# Interactive mode with menu
uv run live-transcribe

# Or use in code:
from main import BreezASR26

asr = BreezASR26()
result = asr.start_microphone_stream(duration=5.0)
print(result['text'])

See MICROPHONE_GUIDE.md for detailed microphone documentation.

Method 4: Advanced - Direct Model Access

import torch
import librosa
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq

# Load model
processor = AutoProcessor.from_pretrained("MediaTek-Research/Breeze-ASR-26")
model = AutoModelForSpeechSeq2Seq.from_pretrained("MediaTek-Research/Breeze-ASR-26")
model = model.to("cuda" if torch.cuda.is_available() else "cpu")

# Load audio
audio, sr = librosa.load("audio.wav", sr=16000)

# Process & transcribe
inputs = processor(audio, sampling_rate=sr, return_tensors="pt")
with torch.no_grad():
    output = model.generate(**inputs)

# Decode
transcription = processor.batch_decode(output, skip_special_tokens=True)
print(transcription[0])

File Structure

breeze-asr-26-project/
├── requirements.txt          # Python dependencies
├── README.md                 # This file
├── main.py                   # Main BreezASR26 wrapper class
├── example_inference.py      # Example usage scripts
└── sample_audio.wav          # (Optional) Sample audio file

Features

The BreezASR26 class provides:

• transcribe_file(audio_path) - Transcribe from audio file
• transcribe_array(audio_array, sampling_rate) - Transcribe from numpy array
• transcribe_url(audio_url) - Transcribe from URL
• load_audio(audio_path, sr) - Load audio using librosa
• Automatic GPU detection and usage

Requirements

• Python 3.8+
• PyTorch 2.0.0+
• Transformers 4.36.0+
• librosa 0.10.0+
• ~6GB GPU VRAM (or CPU, but slower)

Usage Examples

Example 1: Basic Transcription

python main.py

Example 2: Run with Custom Audio

from main import BreezASR26

asr = BreezASR26()
result = asr.transcribe_file("path/to/your/audio.wav")
print(f"Transcription: {result['text']}")

Example 3: Batch Processing

from pathlib import Path
from main import BreezASR26

asr = BreezASR26()
audio_files = Path("audio_folder").glob("*.wav")

for audio_file in audio_files:
    result = asr.transcribe_file(str(audio_file))
    print(f"{audio_file.name}: {result['text']}")

Model Details

Supported Audio Formats

• WAV
• MP3
• FLAC
• OGG
• And other formats supported by librosa

Performance

• GPU (CUDA): ~real-time or faster
• GPU (Apple Silicon): ~real-time or faster
• CPU: 2-5x slower than real-time (depends on CPU)

Troubleshooting

OutOfMemoryError

• Use a GPU with more VRAM
• Process shorter audio segments
• Use CPU with longer processing time

Model Download Issues

• Check internet connection
• Ensure HuggingFace model is accessible
• Set HF_HOME environment variable if needed

Audio Not Processing

• Verify audio format is supported
• Check audio sampling rate (should be ~16kHz)
• Ensure audio contains Taiwanese Hokkien speech

References

• Breeze ASR 26 on HuggingFace
• MediaTek Research
• Transformers Documentation

License

Model: Licensed by MediaTek Research Code: MIT License

Support

For issues with the model, visit: https://huggingface.co/MediaTek-Research/Breeze-ASR-26/discussions