ComfyUI-Kimodo

A ComfyUI plugin that wraps Kimodo — NVIDIA's kinematic motion diffusion model for generating high-quality 3D human and humanoid robot motions from text prompts with optional kinematic constraints.

中文说明

Features

• Text-to-Motion Generation — Describe a motion in natural language, get 3D joint positions and rotations
• Multiple Skeleton Types — SOMA human body, SMPLX, and Unitree G1 humanoid robot
• Kinematic Constraints — Optional JSON constraints for pose keyframes, end-effector positions, 2D paths
• Multi-Prompt Segments — Chain multiple motion descriptions with smooth transitions
• Multiple Samples — Generate batch of motion variations from the same prompt
• NPZ Export — Save motion data (joint positions, rotations, foot contacts, trajectories)
• BVH Export — Export to BVH format for animation software (SOMA skeletons)
• FBX Export (Mixamo) — Retarget motion onto Mixamo-rigged FBX characters and export animated FBX
• 2D Preview — Skeleton stick-figure visualization as ComfyUI IMAGE output
• HuggingFace Auto-Download — Models download automatically on first use (~17GB VRAM)

Nodes

Modular Workflow (Recommended)

Preview & Export

Installation

Clone this repository into your ComfyUI custom_nodes directory:

cd ComfyUI/custom_nodes
git clone https://github.com/jtydhr88/ComfyUI-Kimodo.git

Install dependencies:

cd ComfyUI-Kimodo
pip install -r requirements.txt

The kimodo package itself will be auto-installed on first launch if needed.

Restart ComfyUI. The Kimodo nodes will appear under the Kimodo category.

Models

Models download automatically from HuggingFace on first use:

Manual Model Download

Kimodo's text encoder uses Meta Llama 3 8B, which is a gated model on HuggingFace. You need to:

1. Visit https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct and request access
2. Create a token at https://huggingface.co/settings/tokens
3. Log in and download all required models:

# Log in to HuggingFace
huggingface-cli login

# Text encoder: Llama 3 base model (gated, requires access approval)
huggingface-cli download meta-llama/Meta-Llama-3-8B-Instruct

# Text encoder: LLM2Vec adapters
huggingface-cli download McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp
huggingface-cli download McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised

# Kimodo model (pick the one you want to use)
huggingface-cli download nvidia/Kimodo-SOMA-RP-v1

Motion Correction (Optional)

The motion_correction C++ module provides foot-skate cleanup post-processing. You have two options:

Option A: Use prebuilt binary (Windows + Python 3.11 only)

# Copy the prebuilt files into your Python environment
cp -r prebuilt/win_amd64_cp311 <your-python-env>/Lib/site-packages/motion_correction

Or add the prebuilt/win_amd64_cp311 directory to your Python path.

Option B: Build from source (any platform)

Requires CMake 3.15+ and a C++17 compiler (MSVC / GCC / Clang).

cd kimodo/MotionCorrection
pip install -e .

Verify: python -c "import motion_correction; print('OK')"

Without this module, set post_processing = False in the Generate node. The motion will still work but may have foot-sliding artifacts.

FBX Export (Optional)

To use the Kimodo Export FBX (Mixamo) node, install the FBX SDK Python bindings:

pip install fbxsdkpy --extra-index-url https://gitlab.inria.fr/api/v4/projects/18692/packages/pypi/simple

You also need a Mixamo-rigged FBX character file. Download one from Mixamo (select "Without Skin" or "T-Pose" for best results).

Usage

Modular Workflow (Recommended)

Load Model → Text Encode → Sampler → Post Process → Export/Preview
                                ↑
                         (constraints_json)

1. Add Kimodo Load Model — select a model variant
2. Add Kimodo Text Encode — enter text prompt (reusable across different seeds)
3. Add Kimodo Sampler — set duration, seed, diffusion steps
4. Add Kimodo Post Process — optional foot-skate cleanup
5. Add Kimodo Preview / Kimodo Export BVH / Kimodo Export FBX — visualize or save

Parameters

Multi-Prompt

Separate motion segments with periods in the prompt:

A person walks forward. They stop and wave hello. They turn around and sit down.

Each segment gets the specified duration.

Output Format

The NPZ output contains:

• posed_joints — Joint positions [T, J, 3]
• global_rot_mats — Joint rotation matrices [T, J, 3, 3]
• root_positions — Root trajectory [T, 3]
• foot_contacts — Foot contact labels [T, 4]
• global_root_heading — Root heading angle [T]

Credits

This plugin wraps Kimodog.

License

Apache-2.0