telaffuz-yz — XLS-R-300M Turkish Phoneme CTC

Fine-tuned from facebook/wav2vec2-xls-r-300m for Turkish phoneme-level CTC recognition. Developed as the acoustic backbone of a Turkish pronunciation assessment system (MSc thesis, Istanbul University–Cerrahpaşa, 2026).

Model Description

Phoneme Inventory

The 54-token vocabulary covers the full Turkish phoneme inventory used in the thesis pipeline:

• 8 short vowels: a e i ɯ o œ u y
• 8 long vowels: aː eː iː ɯː oː œː uː yː
• 1 vowel allophone: æ
• 20 consonant phonemes: p b t d k ɡ t͡ʃ d͡ʒ f v s z ʃ ʒ h m n l ɾ j
• 7 base consonant allophones: c ɟ ɲ ŋ ɫ β β̞
• 5 pedagogical allophones: pʰ tʰ kʰ cʰ ɾ̞̊
• 1 suprasegmental: ˈ (primary stress)
• Special tokens: <pad> <unk> <blank> <spc>

This inventory is aligned with the rule-based G2P pipeline that produces reference transcriptions.

Training Data

Fine-tuned on ~310 hours of Turkish read speech from two sources:

Only segments with verified publisher transcripts are used (Whisper-transcribed segments excluded). Duration filter: 0.5 s ≤ clip ≤ 15 s.

Split composition (speaker-aware 90/5/5):

Speaker assignment uses SHA1 hashing on speaker IDs so no speaker appears in more than one split.

Training Procedure

scripts/train_wav2vec.py --model xls-r-300m --epochs 8 --batch-size 4 --grad-accum 8 \
    --lr 1e-4 --warmup-steps 500 --early-stopping-patience 5 \
    --early-stopping-threshold 0.001 --seed 42

Checkpoint selection: Best dev-set PER across all evaluations (every 400 steps). HuggingFace Trainer with load_best_model_at_end=True.

Hardware: NVIDIA GeForce RTX 3090 (24 GB VRAM), AMD Ryzen 9 5900X, 64 GB RAM, CUDA 12.4, PyTorch 2.6.0+cu124, Transformers 4.57.6.

Training time: ~20 GPU-hours (59,888 steps, single GPU).

Evaluation Results

Evaluated on the held-out test set (15,277 utterances, 661,008 reference phones) with atomic IPA token decoding (multi-character tokens like t͡ʃ, aː treated as single units):

The companion MMS-1B model achieves 4.13% PER on the same test set.

Usage

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import torch, soundfile as sf

model_id = "onesvat/telaffuz-yz-wav2vec-xlsr300m"
processor = Wav2Vec2Processor.from_pretrained(model_id)
model = Wav2Vec2ForCTC.from_pretrained(model_id)
model.eval()

audio, sr = sf.read("speech.wav", dtype="float32")
# resample to 16 kHz if needed

inputs = processor(audio, sampling_rate=16000, return_tensors="pt", padding=True)
with torch.no_grad():
    logits = model(**inputs).logits

pred_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(pred_ids)
print(transcription)  # IPA phoneme sequence

Intended Use

This model is intended for:

• Turkish pronunciation assessment and coaching systems
• Turkish phoneme recognition research
• Downstream tasks requiring Turkish phone-level posteriors

It is not intended for general Turkish ASR (word-level transcription). The vocabulary is phonemic, not graphemic.

Limitations

• Trained on read speech only; performance on spontaneous/conversational speech may be lower.
• Long vowels (aː, eː, …) are treated as distinct phoneme classes; recognition accuracy varies by phoneme (see per-phoneme error analysis in the thesis).
• Duration filter (0.5–15 s) means very short or very long utterances were excluded from training.

Citation

If you use this model, please cite:

@mastersthesis{nesvat2026telaffuz,
  author  = {Onur Nesvat},
  title   = {Türkçe Telaffuz Değerlendirme Sistemi},
  school  = {İstanbul Üniversitesi-Cerrahpaşa},
  year    = {2026},
}

License

Apache-2.0 (inherited from facebook/wav2vec2-xls-r-300m).