example_audios/Tianjin_1002906_0_0.wav

Using the Voxlect, we obtain the following probability

Dialect: Jiang-Huai       Probability: 0.002
Dialect: Jiao-Liao        Probability: 0.051
Dialect: Ji-Lu            Probability: 0.169 (Target dialect)
Dialect: Lan-Yin          Probability: 0.001
Dialect: Mandarin         Probability: 0.765
Dialect: Southwestern     Probability: 0.000 
Dialect: Zhongyuan        Probability: 0.013
Dialect: Yue              Probability: 0.000

If you are a Mandarin speaker, you will notice that the first 2 words are spoken likely to be Tianjin dialect, but after that, the generated speech sounds more like standard Mandarin, and our model captures this correctly.

Quick Example 2 - MMS-LID-256 Spanish Dialect

# Load libraries
import torch
import torch.nn.functional as F
from src.model.dialect.mms_dialect import MMSWrapper

# Label List
spanish_dialect_list = [
    "Andino-Pacífico", 
    "Caribe and Central", 
    "Chileno",
    "Mexican", 
    "Penisular", 
    "Rioplatense", 
]
    
# Find device
device = torch.device("cuda") if torch.cuda.is_available() else "cpu"

# Load model from Huggingface
mms_model = MMSWrapper.from_pretrained("tiantiaf/voxlect-spanish-dialect-mms-lid-256").to(device)
mms_model.eval()

# Load data, here just zeros as the example
# Our training data filters output audio shorter than 3 seconds (unreliable predictions) and longer than 15 seconds (computation limitation)
# So you need to prepare your audio to a maximum of 15 seconds, 16kHz and mono channel
max_audio_length = 15 * 16000
data = torch.zeros([1, 16000]).float().to(device)[:, :max_audio_length]
mms_logits, mms_embeddings = mms_model(data, return_feature=True)
    
# Probability and output
mms_prob = F.softmax(mms_logits, dim=1)
print(spanish_dialect_list[torch.argmax(mms_prob).detach().cpu().item()])

Given that the Voxlect Benchmark paper is still under peer-review, we provide limited set of models and model weights before the review is concluded. But below are the models we currently put out.

MMS-LID-256 Models

Whisper-Large V3 Models

Whisper-small Models

Responsible Use: Users should respect the privacy and consent of the data subjects, and adhere to the relevant laws and regulations in their jurisdictions when using Voxlect.

❌ Out-of-Scope Use

• Clinical or diagnostic applications
• Surveillance
• Privacy-invasive applications
• No commercial use

If you like our work or use the models in your work, kindly cite the following. We appreciate your recognition!

@article{feng2025voxlect,
  title={Voxlect: A Speech Foundation Model Benchmark for Modeling Dialects and Regional Languages Around the Globe},
  author={Feng, Tiantian and Huang, Kevin and Xu, Anfeng and Shi, Xuan and Lertpetchpun, Thanathai and Lee, Jihwan and Lee, Yoonjeong and Byrd, Dani and Narayanan, Shrikanth},
  journal={arXiv preprint arXiv:2508.01691},
  year={2025}
}

@article{feng2025vox,
  title={Vox-Profile: A Speech Foundation Model Benchmark for Characterizing Diverse Speaker and Speech Traits},
  author={Feng, Tiantian and Lee, Jihwan and Xu, Anfeng and Lee, Yoonjeong and Lertpetchpun, Thanathai and Shi, Xuan and Wang, Helin and Thebaud, Thomas and Moro-Velazquez, Laureano and Byrd, Dani and others},
  journal={arXiv preprint arXiv:2505.14648},
  year={2025}
}