Irodori-TTS

Flow Matching-based Japanese Text-to-Speech model using a Rectified Flow Diffusion Transformer (RF-DiT) over DACVAE continuous latents. Based on Echo-TTS.

Performance

Benchmarked on NVIDIA H100 PCIe, 40 steps, ~18s audio, --no-ref mode:

Installation

git clone https://github.com/kadirnar/fast-irodori.git
cd fast-irodori
uv sync

Quick Start

Python API

from irodori_tts.inference.infer import infer

# Basic inference
infer(
    text="今日はいい天気ですね。",
    hf_checkpoint="Aratako/Irodori-TTS-500M-v2",
    no_ref=True,
    output_wav="output.wav",
)

# Optimized inference (x12.5 faster)
infer(
    text="今日はいい天気ですね。",
    hf_checkpoint="Aratako/Irodori-TTS-500M-v2",
    no_ref=True,
    model_precision="bf16",
    codec_precision="bf16",
    block_cache=True,
    optimize_codec=True,
    output_wav="output.wav",
)

# VoiceDesign (caption-conditioned)
infer(
    text="今日はいい天気ですね。",
    hf_checkpoint="Aratako/Irodori-TTS-500M-v2-VoiceDesign",
    caption="落ち着いた女性の声で、やわらかく自然に読み上げてください。",
    no_ref=True,
    output_wav="output.wav",
)

# With reference audio (voice cloning)
infer(
    text="今日はいい天気ですね。",
    hf_checkpoint="Aratako/Irodori-TTS-500M-v2",
    ref_wav="path/to/reference.wav",
    output_wav="output.wav",
)

SGLang-Compatible Server

from irodori_tts.serving.sglang_adapter import launch_server

launch_server(
    model_path="Aratako/Irodori-TTS-500M-v2",
    port=8000,
    optimize=True,
)

# Generate audio via OpenAI-compatible API
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "今日はいい天気ですね。", "seconds": 20}' \
  --output output.wav

# Get JSON timings
curl -X POST http://localhost:8000/v1/audio/speech/json \
  -H "Content-Type: application/json" \
  -d '{"input": "今日はいい天気ですね。", "seconds": 20}'

SGLang Python API (Offline)

from irodori_tts.serving.sglang_adapter import IrodoriTTSGenerator

gen = IrodoriTTSGenerator.from_pretrained(
    model_path="Aratako/Irodori-TTS-500M-v2",
    optimize=True,
)

result = gen.generate(text="今日はいい天気ですね。", seconds=20.0)
result.save("output.wav")

print(result.timings)
# {'ttft_ms': 0.4, 'ttfa_ms': 86.0, 'end_to_end_ms': 86.1,
#  'generation_time_ms': 85.0, 'sampling_ms': 62.0, 'decode_ms': 23.0,
#  'audio_duration_s': 18.2}

gen.shutdown()

Benchmark

from benchmark import run_benchmark

# Baseline
run_benchmark()

# All optimizations
run_benchmark(
    optimize_codec=True,
    model_precision="bf16",
    codec_precision="bf16",
    compile_blocks=True,
    block_cache=True,
    block_cache_fn=1,
    block_cache_velocity_skip=2,
)

Web UI

uv run irodori-app                # Base model (port 7860)
uv run irodori-app-voicedesign    # VoiceDesign (port 7861)

Optimizations

Training

1. Prepare Data

uv run irodori-prepare-manifest \
  --dataset myorg/my_dataset \
  --audio-column audio \
  --text-column text \
  --output-manifest data/train_manifest.jsonl \
  --latent-dir data/latents \
  --device cuda

2. Train

# Single GPU
uv run irodori-train \
  --config configs/train_500m_v2.yaml \
  --manifest data/train_manifest.jsonl

# Multi-GPU DDP
uv run torchrun --nproc_per_node 4 -m irodori_tts.training.train \
  --config configs/train_500m_v2.yaml \
  --manifest data/train_manifest.jsonl

# LoRA fine-tuning
uv run irodori-train \
  --config configs/train_500m_v2_lora.yaml \
  --manifest data/train_manifest.jsonl \
  --init-checkpoint path/to/model.safetensors

3. Convert Checkpoint

uv run irodori-convert-checkpoint outputs/checkpoint_final.pt

Project Structure

irodori_tts/
  config.py                    # Model / Train / Sampling configs
  model/                       # DiT architecture, RF sampling, LoRA
  text/                        # Tokenizer, text normalization
  audio/                       # DACVAE codec (fast-dacvae optimized)
  training/                    # Train loop, dataset, optimizer
  inference/                   # CLI inference, runtime engine
  serving/                     # SGLang-compatible server + offline API
  app/                         # Gradio web UIs
  tools/                       # Data prep, checkpoint conversion
configs/                       # YAML training presets

License

• Code: MIT License
• Model Weights: See base model card and VoiceDesign model card

Acknowledgments

• Irodori-TTS — Original codebase
• Echo-TTS — Architecture and training design
• fast-dacvae — Optimized DACVAE codec
• cache-dit — Block caching algorithm
• SGLang — Serving framework

Citation

@misc{irodori-tts,
  author = {Chihiro Arata},
  title = {Irodori-TTS: A Flow Matching-based Text-to-Speech Model},
  year = {2026},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/Aratako/Irodori-TTS}}
}