ComfyUI OmniVoice

ComfyUI custom nodes for OmniVoice.

Zero-shot voice cloning, voice design, multilingual TTS, pronunciation helpers, and standard ComfyUI model-folder support in one custom node package.

Highlights

• zero-shot voice cloning
• voice design
• multilingual TTS
• pronunciation helpers
• standard ComfyUI model folder support
• download progress logging for missing model files

Compatibility Warning

[!WARNING] This node is currently pinned to transformers==5.3.0. OmniVoice needs a Transformers build that includes HiggsAudioV2TokenizerModel. If your ComfyUI environment is still pinned to older transformers releases, OmniVoice will fail before model loading even starts.

Quick check inside the same Python environment used by ComfyUI:

python -m pip show transformers

Or list installed packages and filter for it:

python -m pip list | findstr transformers

If you get an import error mentioning HiggsAudioV2TokenizerModel, your current transformers build is too old for OmniVoice.

Be aware that some other custom nodes still depend on older Hugging Face stacks. If you mix them in one ComfyUI environment, you can run into dependency conflicts. In that case, the cleanest solution is usually a separate ComfyUI environment for OmniVoice.

At a Glance

Included Nodes

Model Layout

Download model weights manually into:

ComfyUI/models/OmniVoice/model.safetensors
ComfyUI/models/OmniVoice/audio_tokenizer/model.safetensors

ASR models used by OmniVoice Transcribe Reference are stored here:

ComfyUI/models/OmniVoice/asr/<repo-id>/

The node also stores these support files under ComfyUI/models/OmniVoice/:

• config.json
• tokenizer.json
• tokenizer_config.json
• chat_template.jinja
• audio_tokenizer/config.json
• audio_tokenizer/preprocessor_config.json

Installation

1. Copy this folder into:

ComfyUI/custom_nodes/ComfyUi_OmniVoice

2. Install dependencies into the same Python environment used by ComfyUI:

python -m pip install -r requirements.txt

3. Make sure your ComfyUI environment already has compatible torch and torchaudio.
4. Restart ComfyUI.

Recommended dependency set from this repo:

transformers==5.3.0
accelerate
huggingface_hub
pydub
numpy
soundfile

Quick Start

Voice Cloning with Built-in ASR

Workflow: workflow_omnivoice_voice_clone.json

Flow:

1. LoadAudio
2. OmniVoice Transcribe Reference
3. OmniVoice Load Model
4. OmniVoice Create Voice Clone Prompt
5. OmniVoice Generate
6. PreviewAudio or SaveAudioMP3

Voice Design without Reference Audio

Workflow: workflow_omnivoice_voice_design.json

Flow:

1. OmniVoice Load Model
2. OmniVoice Build Voice Design
3. OmniVoice Generate
4. PreviewAudio or SaveAudioMP3

What OmniVoice Create Voice Clone Prompt Does

This node is not an ASR node. It prepares the reusable voice-clone conditioning object that OmniVoice needs for cloning.

In practice it:

• takes the reference waveform
• takes the reference transcript
• tokenizes the reference audio
• builds the voice_clone_prompt passed into OmniVoice Generate

So:

• OmniVoice Transcribe Reference answers: "what was said?"
• OmniVoice Create Voice Clone Prompt answers: "how should this voice be conditioned for cloning?"

Supported Generation Controls

OmniVoice Generate exposes:

• language
• seed
• num_step
• guidance_scale
• speed
• duration
• t_shift
• denoise
• preprocess_prompt
• postprocess_output
• layer_penalty_factor
• position_temperature
• class_temperature
• audio_chunk_duration
• audio_chunk_threshold

The language widget includes the full OmniVoice language list plus Auto.

Voice Design

OmniVoice voice design supports the upstream attribute set:

• gender
• age
• pitch
• whisper
• English accents
• Chinese dialects

It does not support arbitrary emotion labels such as angry, happy, or sad in the official instruct vocabulary.

Pronunciation Control

English

Use bracketed CMU / ARPABET:

You could probably still make [IH1 T] look good.

Chinese

Use pinyin with tone numbers directly in the text, for example:

... ZHE2 ... SHE2 ... MA1 ...

Download Logging

When the node downloads missing files from Hugging Face, it logs:

• which file is being downloaded
• file size
• progress percent
• downloaded / total size
• overall ASR progress for multi-file downloads

Credits

• OmniVoice: https://github.com/k2-fsa/OmniVoice
• Hugging Face model: https://huggingface.co/k2-fsa/OmniVoice