Fish Audio Drama 3 — single-speaker voice cloning

A Gradio app that runs fishaudio/drama-3 (35B MoE, 3B active per token) in-process on a ZeroGPU large slice (RTX Pro 6000, 48 GB) via one @spaces.GPU function. Record or upload a voice sample, type what that sample says, and generate a new line in that voice.

Hardware: this Space must run on ZeroGPU hardware (Settings → Hardware → ZeroGPU). On CPU hardware the page opens, but Generate asks you to flip the hardware.

Inference uses the official PyTorch path of the model, vendored under fish_speech/ (adapted from hugging-apps/fish-audio-drama-3 via akhaliq/drama-3-tts): the 32.2B FP8 MoE experts stay in FP8 — dequantised they would need ~67 GB and would not fit the slice. The model loads at module scope, as the ZeroGPU docs recommend: built on the meta device, materialised on CPU with FP8 experts, then moved to cuda, so startup costs no GPU time. spaces is imported before CUDA is initialized.

How to use

The page has five controls:

1. Reference audio — upload a clip or record with the microphone. The clip must be 0.5–30 seconds of speech. A longer clip is rejected; it is not silently trimmed, because the transcript has to match the whole recording.
2. Reference transcript — type exactly what is spoken in the reference audio. This text only conditions the voice. It is not the line that gets synthesized.
3. Text to generate — type the new words you want that voice to say. Plain text is enough; it is mapped to the reference speaker internally. You do not need to write S0: or a speaker token.
4. Generate — one click runs one synthesis. Requests are queued, and this model accepts one generation at a time.
5. Generated audio — play or download the result. The page returns a single audio file.

Example:

• Reference transcript: Hello, nice to meet you. My name is Chenyang.
• Text to generate: Have a nice day and enjoy Southern California sunshine.

The output should say the new line in the reference voice, not repeat the introduction.

This page is a single-speaker preview. Multi-speaker scenes, streaming, and the HTTP API are documented with SGLang Omni:

• Fish Audio Drama 3 cookbook: install, voice cloning, and the HTTP API
• SGLang Omni: the GitHub repo — please give it a star
• Technical report: architecture of the 35B MoE model
• Fish Audio Playground: try Fish Audio voices in the browser
• Fish Speech: source repository for the model

Sampling is fixed: temperature=1.0, top_p=0.9, top_k=30, and max_new_tokens=500 per chunk. The token budget is also capped by the context window, so a chunk never asks for more frames than the prompt leaves free. Long text is split into chunks inside that single generation call.

Each click books its own GPU slot with a dynamic @spaces.GPU(duration=...) estimate (≤120 s requested; ZeroGPU books 1.5× on this hardware). GPU time counts against the visitor's daily ZeroGPU quota — sign in for a larger allowance.

Acknowledgments

It took us over half a year to bring SGLang Omni back to life. Thanks to all the people who helped us along the way.

Thanks so much for AK help to set up the space.

SGLang Omni team: Chenyang Zhao, Jingwen Gu, Shenggui Li, Richard Wang, Yichi Zhang, Xuesong Ye, Jiaxin Deng, Ratish P, Jiaxuan Luo, Xinyu Lu, Wenyao Gao, Ao Sun, Jintao Quaige Li, Yuhao Chen, Yuhan Zhou, Xuxiang Tian, Ruoyi Zhang, Junnan Li, Yixiao Zeng, Chenchen Hong, Huapeng Zhou, Yang Gao, Chaoye Fei, Yiyang Zh, Kang, Yitian Gong, Yuan Luo, Hao Jin, Yifei Gao, Gaokai Zhang, Lifan Shen, Hui Shen, Kejing Xia, Jinxin Li, Dongming Shen, Jun Liu, Yue Yin, Zhihao Guo, Jisen Li, Ruoyu He, Xuanyi Li, Chenye Zhu, Liang Zhen, Xiaodong Ye, Yizhi Zhen, Kaizuo Li, Chonghan Liu, Wei Guan, Zhuofeng Li

FishAudio Team: Shijia Liao, Shidong Li, Xuehao Yang, Jiahua Liu

Notes

• Weights are preloaded at build time via preload_from_hub; first boot still takes a while while the ~40 GB checkpoint is packed.
• Model license: Fish Audio Community License — free non-commercial/research demo. Built with Fish Audio.