whisper-small-arabic

Fine-tuned openai/whisper-small for Arabic speech recognition.

Results

Evaluated on 300 held-out clips from the Mozilla Common Voice Arabic test split. Baseline and fine-tuned numbers use identical decoding configuration (greedy, language=arabic, task=transcribe).

Training data

Mozilla Common Voice — Arabic, the crowdsourced Arabic speech corpus distributed by the Mozilla Foundation. Train/dev/test were combined, shuffled with seed=42, and the first 25,000 clips were used for fine-tuning. Audio is at 16 kHz mono.

Training procedure

Single Tesla T4 (16 GB) on Kaggle, mixed-precision fp16. Long audio is handled by Whisper's standard 30-second windowing.

Hyperparameters

Two-stage learning rate schedule

The first 2,000 steps used learning_rate = 1e-5 and converged to WER ≈ 21.3. Validation loss had plateaued, so training was resumed from the 2,000-step checkpoint at the lower learning_rate = 5e-6 for another 2,000 steps. The smaller LR squeezed out an additional point of WER and produced the published checkpoint at step 3,000.

Validation curve

Learning rate is dropped from 1e-5 to 5e-6 after step 2,000. Step 3,000 is the published checkpoint (lowest validation WER).

The published checkpoint is step 3,000, the lowest WER on the held-out validation split. Training loss continued to fall after that point but validation loss flattened and WER ticked back up — classic mild overfitting, so the earlier checkpoint was preserved as the final model.

Quick use

import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

REPO = "Omar10lfc/whisper-small-arabic"
processor = WhisperProcessor.from_pretrained(REPO)
model     = WhisperForConditionalGeneration.from_pretrained(REPO)
model.generation_config.language = "arabic"
model.generation_config.task     = "transcribe"

speech, _ = librosa.load("audio.wav", sr=16000, mono=True)
features  = processor(speech, sampling_rate=16000, return_tensors="pt").input_features
ids       = model.generate(features, max_new_tokens=440)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

For long audio (lectures, podcasts), use the HF pipeline with chunking:

from transformers import pipeline
asr = pipeline(
    "automatic-speech-recognition",
    model=REPO,
    chunk_length_s=30,
    stride_length_s=5,
    device=0,                    # or -1 for CPU
)
print(asr("lecture.wav", generate_kwargs={"language": "arabic", "task": "transcribe"})["text"])

Intended use & limitations

• Intended use: Modern Standard Arabic speech-to-text — lectures, news, podcasts, dictation. Used as the ASR front-end of an Arabic audio understanding pipeline (Whisper → AraBART summarization → CAMeL-BERT/FAISS semantic search).
• Out of scope: code-switching with English beyond a few words, speaker diarization, regional dialects under-represented in Common Voice (Maghrebi, Khaleeji), very long form (>30 s without chunking).
• Bias: Common Voice is volunteer-recorded; speaker demographics skew toward MSA-fluent contributors and may underrepresent dialects, accents, and acoustic conditions outside that distribution.

Citation

@misc{whisper-small-arabic,
  title  = {whisper-small-arabic: Fine-tuned Whisper for Arabic on Mozilla Common Voice},
  author = {{Omar10lfc}},
  year   = {2026},
  howpublished = {Hugging Face},
  note = {Fine-tune of openai/whisper-small on Mozilla Common Voice Arabic, WER 20.61.}
}

Built on top of:

@article{radford2022whisper,
  title  = {Robust Speech Recognition via Large-Scale Weak Supervision},
  author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and others},
  journal= {arXiv preprint arXiv:2212.04356},
  year   = {2022}
}

FP16 (Unquantized) vs. CTranslate2 INT8 Quantization Comparison

Does 8-bit quantization degrade accuracy? Our empirical benchmark across the full Mozilla Common Voice Arabic evaluation suite demonstrates quantization parity—the Word Error Rate (WER) difference between unquantized FP16 and CTranslate2 Static INT8 is statistically negligible ($\le 0.5$ to $0.8$ percentage points), while reducing disk size and VRAM by ~50% and accelerating inference by up to $3.6\times$:

Production Recommendations

• Deploy CTranslate2 INT8 (*-ct2-int8): Recommended for production APIs, serverless containers (Modal, RunPod, Hugging Face Spaces), and edge environments to cut cold-start load times and memory by half while maximizing throughput.
• Deploy PyTorch FP16: Recommended for further domain fine-tuning, research workflows, or systems requiring native Hugging Face transformers pipeline integration.