ComfyUI-Kaola-ACE-Step

ComfyUI custom nodes for ACE-Step 1.5 - Commercial-grade music generation.

Features

• 🎵 Text to Music - Generate music from text
• 🎭 Cover Generation - Style transfer
• 🎨 Audio Repaint - Local audio editing
• 💡 Simple Mode - Natural language to music
• 📝 Format Sample - Enhance user input
• 🔍 Understand Audio - Analyze audio codes
• 🎤 Audio Transcription - Transcribe lyrics from audio
• 📋 Audio Captioning - Generate detailed music descriptions
• 🎸 Extract/Lego/Complete - Track extraction and manipulation (Base model)
• 🧹 Clear VRAM - Free GPU memory after operations

Quick Start

1. Install ACE-Step

git clone https://github.com/ACE-Step/ACE-Step-1.5.git
cd ACE-Step-1.5
pip install -e .

2. Install ComfyUI Nodes

Standard Installation:

cd ComfyUI/custom_nodes
git clone https://github.com/kana112233/ComfyUI-kaola-ace-step.git
cd ComfyUI-kaola-ace-step
pip install -r requirements.txt

For ComfyUI Portable:

# Navigate to your ComfyUI portable folder
cd ComfyUI_windows_portable/ComfyUI/custom_nodes
git clone https://github.com/kana112233/ComfyUI-kaola-ace-step.git

# Use the embedded Python to install requirements
cd ComfyUI-kaola-ace-step
../../python_embeded/python.exe -m pip install -r requirements.txt

# You must also install ACE-Step in the embedded Python:
cd ../../../
git clone https://github.com/ACE-Step/ACE-Step-1.5.git
python_embeded/python.exe -m pip install -e ACE-Step-1.5

3. Download Models

# Using huggingface-cli
huggingface-cli download ACE-Step/Ace-Step1.5 --local-dir ComfyUI/models/Ace-Step1.5

# Or using hfd (faster)
pip install hfd
hfd ACE-Step/Ace-Step1.5 --local-dir ComfyUI/models/Ace-Step1.5

4. Restart ComfyUI

Node Settings

Model Directory Structure

The checkpoint_dir dropdown scans all folders directly under ComfyUI/models/. You can choose any folder from the root models directory as your ACE-Step base folder.

Example layout:

ComfyUI/models/
├── Ace-Step1.5/            # Select "Ace-Step1.5" in the node
│   ├── acestep-v15-turbo/
│   ├── vae/
│   └── ...
└── My-Custom-Folder/       # Select "My-Custom-Folder" in the node
    ├── acestep-v15-turbo/
    ├── vae/
    └── ...

Workflow Examples

See examples/ directory for ready-to-use workflows.

Nodes

📖 Detailed usage guide: See USAGE.md for comprehensive documentation.

ACE_STEP_TRANSCRIBER

Audio transcription node powered by ACE-Step Transcriber (Qwen2.5-Omni-7B).

Features

• 🌍 50+ Languages - Chinese, English, Japanese, Korean, etc.
• 🎵 Lyrics Transcription - Specialized for singing voice
• 🏷️ Structure Tags - Auto identifies [Verse], [Chorus], [Bridge], etc.
• ⏱️ Timestamps - Via prompt instruction
• ✂️ Chunked Processing - Handle long audio automatically

Installation

Download the model to your models folder:

# Using huggingface-cli
huggingface-cli download ACE-Step/acestep-transcriber --local-dir ComfyUI/models/acestep-transcriber

# Or place inside Ace-Step1.5 folder
huggingface-cli download ACE-Step/acestep-transcriber --local-dir ComfyUI/models/Ace-Step1.5/acestep-transcriber

Parameters

ACE_STEP_Captioner

Audio captioning node powered by ACE-Step Captioner (Qwen2.5-Omni-7B).

Features

• 🎼 Musical Style Analysis - Identifies genres, sub-genres, and stylistic influences
• 🎸 Instrument Recognition - Detects and describes 1000+ instrument types
• 🎭 Structure & Progression - Analyzes musical arrangement (intro, verse, chorus, bridge, etc.)
• 🔊 Timbre Description - Captures tonal qualities, textures, and sonic characteristics

Performance

Accuracy surpasses Gemini Pro 2.5 in music description tasks.

Installation

Download the model to your models folder:

# Using huggingface-cli
huggingface-cli download ACE-Step/acestep-captioner --local-dir ComfyUI/models/Ace-Step1.5/acestep-captioner

# Or using hfd (faster)
pip install hfd
hfd ACE-Step/acestep-captioner --local-dir ComfyUI/models/Ace-Step1.5/acestep-captioner

Parameters

Outputs

Requirements

• Python: 3.10+
• GPU: 6GB+ VRAM recommended
• Disk: ~8GB for models

6. Using LoRA Models (e.g., Chinese New Year)

Use the ACE-Step LoRA Loader node to apply LoRA adaptations.

1. Place your LoRA model folder (containing adapter_config.json) in ComfyUI/models/loras/ or ComfyUI/models/Ace-Step1.5/loras/.
2. Add the ACE-Step LoRA Loader node.
3. Select your LoRA from the dropdown.
4. Connect the lora_info output to any ACE-Step generation node (TextToMusic, etc.).
5. Adjust strength as needed (default 1.0).

---

Troubleshooting

"Model path not found"

Ensure models are in ComfyUI/models/Ace-Step1.5/ with the correct subdirectory structure (see above).

Audio Transcription Technical Roadmap

graph TD
    Start[🚀 Start: Select Audio Transcription Solution] --> Domain{Core Use Case?}

    %% Branch 1: Music & Lyrics
    Domain -- Music / Lyrics / Structure --> Music[Music Domain]
    Music --> Hardware1{VRAM > 30GB?}
    Hardware1 -- Yes --> ACE[<b>ACE-Step Transcriber</b><br>Base: Qwen2.5-Omni-7B]
    ACE --> ACEDep[<u>Features</u>:<br>✅ 50+ Languages Support<br>✅ Lyrics + Structure Tags<br>✅ Timestamps via Prompt]
    Hardware1 -- No --> Heart[<b>HeartTranscriptor-oss</b><br>0.8B Params]

    %% Branch 2: Speech & Meetings
    Domain -- Speech / Meetings / Translation --> Speech[Speech Domain]
    Speech --> Duration{Duration & Coherence?}

    %% VibeVoice Path
    Duration -- Long Meeting --> Vibe[<b>VibeVoice-ASR</b><br>64K Context]

    %% Whisper Path
    Duration -- General / Short Audio --> Whisp[<b>Whisper-large-v3</b><br>Mature Ecosystem]
    Whisp --> Speed{Speed Priority?}
    Speed -- Fast --> WhispFast[torch.compile]
    Speed -- Long Audio --> WhispLong[Pipeline Chunking]

    %% Styles
    classDef model fill:#e1f5fe,stroke:#01579b,stroke-width:2px;
    classDef dep fill:#fff3e0,stroke:#ff6f00,stroke-width:1px,stroke-dasharray: 5 5;
    class ACE,Heart,Vibe,Whisp,WhispFast,WhispLong model;
    class ACEDep dep;

📋 Technical Dependency Matrix

License

MIT License - see LICENSE for details.

Links

• ACE-Step GitHub
• ComfyUI
• Changelog