Raon-OpenTTS

Open Models and Data for Robust Text-to-Speech

Technical Report | Raon-OpenTTS-1B

Highlights

• Fully open: both model weights and training data are publicly available.
• Large-scale training: 510.1K hours of quality-filtered speech (Raon-OpenTTS-Core), drawn from a 615K-hour open pool (Raon-OpenTTS-Pool) comprising 11 English datasets.
• Competitive with closed-data SOTA: matches or outperforms MaskGCT, VoxCPM, CosyVoice 3, and Qwen3-TTS on standard benchmarks while being the first system that is simultaneously open-weight and open-data at this scale.
• Two model sizes: 0.3B and 1B parameters, both based on the F5-TTS DiT architecture.

Model Zoo

Both models use character-level tokenization (vocab size 5,512) with text_dim=512, and are trained on 80-channel log mel-spectrograms at 16 kHz (hop=256). A pretrained HiFi-GAN vocoder (16 kHz, LibriTTS) is used for waveform synthesis.

Benchmark Results

Bold marks the best result and the Raon-OpenTTS rows. All numbers are from the paper (arXiv:2605.20830).

Seed-TTS-Eval (English)

WER measured via Whisper-large-v3; speaker similarity (SIM) via WavLM-large.

CV3-Eval

WER on CV3-EN and CV3-Hard-EN; SIM via ERes2Net, DNSMOS for perceptual quality (CV3-Hard-EN).

Raon-OpenTTS-Eval

Covers 4 acoustic regimes (Clean, Noisy, Wild, Expressive) across 12 datasets with 6K prompt-text pairs. Overall is computed over all evaluation samples.

Installation

git clone https://github.com/krafton-ai/Raon-OpenTTS.git
cd Raon-OpenTTS
pip install -e .

# With evaluation dependencies (WER, SIM, DNSMOS)
pip install -e ".[eval]"

Vocoder

We use a HiFi-GAN vocoder fine-tuned on LibriTTS at 16 kHz (originally from speechbrain/tts-hifigan-libritts-16kHz). Our standalone loader requires no speechbrain dependency.

mkdir -p pretrained_models
huggingface-cli download speechbrain/tts-hifigan-libritts-16kHz generator.ckpt --local-dir pretrained_models

Quick Start: Inference

python -m f5_tts.infer.infer_cli \
    --config src/f5_tts/configs/03b.yaml \
    --ckpt_dir checkpoints/Raon-OpenTTS-0.3B \
    --ckpt_name model_last.pt \
    --lst_path data/librispeech_pc_test_clean_cross_sentence.lst \
    --audio_root data/librispeech/test-clean \
    --output_dir output/inference

VAD-based Duration Estimation

The inference pipeline uses VAD-trimmed reference length for generation-length estimation, while the original (non-trimmed) audio is used as the conditioning signal. A dynamic silence threshold adapts to the speaker's volume, and a minimum speech rate (12 chars/sec) is enforced to prevent excessively long generations.

from f5_tts.infer.utils_infer import infer_process

audio, sr, _ = infer_process(ref_audio, ref_text, gen_text, model, vocoder)

Training

Both models are trained from the Raon-OpenTTS-Pool HuggingFace dataset using the core split (quality-filtered).

Launch training

# 0.3B model (1 nodes x 8 GPUs)
accelerate launch --multi_gpu --mixed_precision bf16 \
    --num_processes 8 --num_machines 1 \
    -m f5_tts.train.train --config-name=03b

# 1B model (1 nodes x 8 GPUs)
accelerate launch --multi_gpu --mixed_precision bf16 \
    --num_processes 8 --num_machines 1 \
    -m f5_tts.train.train --config-name=1b

Adapting to different hardware

If you use a different number of GPUs or batch size, recompute total_updates_per_epoch with a dry run:

# Run one step and check log output for "TOTAL UPDATES <N>"
accelerate launch --multi_gpu --mixed_precision bf16 \
    --num_processes <num_gpus> \
    -m f5_tts.train.train --config-name=03b
# Then set: total_updates_per_epoch = TOTAL_UPDATES / (epochs x num_gpus)

Evaluation

We evaluate on 3 benchmarks measuring intelligibility (WER) and speaker similarity (SIM):

# Run evaluation across all benchmarks
bash src/f5_tts/eval/run_infer_eval.sh

Data

Raon-OpenTTS-Pool (615K hours, 11 English speech datasets) is publicly available on HuggingFace

Raon-OpenTTS-Core (510.1K hours, 194.5M segments) is the quality-filtered subset used for training. It is obtained by applying a combined filter based on DNSMOS, WER, and VAD rank scores, removing the bottom 15% of Raon-OpenTTS-Pool. The core split in the HuggingFace dataset corresponds to Raon-OpenTTS-Core.

Acknowledgement

This project is built upon F5-TTS by SWivid. We thank the authors for their excellent open-source work.

License

This project is licensed under Apache 2.0.

Citation

@article{kim2026raonopentts,
    title={Raon-OpenTTS: Open Models and Data for Robust Text-to-Speech},
    author={Kim, Semin and Chung, Seungjun and Moon, Taehong and Lee, Sangheon and Ahn, Minyoung and Lee, Keon and Kim, Nam Soo and Cho, Jaewoong and Schmidt, Ludwig and Lee, Kangwook and Park, Dongmin},
    journal={arXiv preprint arXiv:2605.20830},
    year={2026}
}