Part of the LongCat-Video-Avatar 1.5 — MLX collection.

LongCat-Video-Avatar-1.5-bf16-dmd-merged (MLX)

Apple MLX bf16 weights for LongCat-Video-Avatar-1.5 — Meituan's audio-driven video diffusion model — with the DMD step-distillation LoRA pre-merged into the DiT weights. Recommended variant for inference: produces the same outputs as the base + LoRA combination but in 8 sampling steps with no LoRA loading at runtime.

TL;DR

Quick start

# 1. Pull weights (~43 GB)
hf download mlx-community/LongCat-Video-Avatar-1.5-bf16-dmd-merged \
    --local-dir ./weights

# 2. Set up inference (Python 3.12)
git clone https://github.com/xocialize/longcat-avatar-mlx
cd longcat-avatar-mlx
python3.12 -m venv .venv
.venv/bin/pip install -e ".[parity]"
.venv/bin/pip install librosa Pillow imageio imageio-ffmpeg

# 3. Run end-to-end
.venv/bin/python scripts/run_inference.py \
    --weights ./weights/.. \
    --variant merged \
    --num-frames 93 \
    --out output.mp4

Programmatic usage:

import json
import pathlib
import mlx.core as mx

from longcat_video_avatar.pipeline_mlx import LongCatAvatarPipeline, PipelineConfig
from longcat_video_avatar.models.autoencoder_kl_wan import AutoencoderKLWan
from longcat_video_avatar.models.avatar.longcat_video_dit_avatar import (
    LongCatVideoAvatarTransformer3DModel,
)
from longcat_video_avatar.models.umt5 import UMT5EncoderModel
from longcat_video_avatar.models.whisper import WhisperEncoder

W = pathlib.Path("./weights")

vae = AutoencoderKLWan.from_config(json.loads((W/"vae/config.json").read_text()))
vae.load_weights(str(W/"vae/diffusion_pytorch_model.safetensors"), strict=False)

# (Load umT5/Whisper/DiT similarly via from_config + load_weights; see
# scripts/run_inference.py:build_pipeline() for the sharded-load helper.)

pipeline = LongCatAvatarPipeline(vae=vae, text_encoder=umt5,
                                 audio_encoder=whisper, dit=dit,
                                 config=PipelineConfig())
video = pipeline(image=ref_image_chw, audio_mel=mel_features,
                 text_embeds=text_embeds, text_mask=text_mask,
                 uncond_embeds=neg_embeds, uncond_mask=neg_mask)

Variants

Performance

Tested on Apple M5 Max (128 GB unified memory):

Peak memory at 480 × 832: ~50 GB unified. The VAE attention layers are routed to a CPU stream by default (see CLAUDE.md L10) to recover strict fp32 precision; net perf hit is negligible.

Layout

LongCat-Video-Avatar-1.5-bf16-dmd-merged/
├── README.md                           # this file
├── pipeline_config.json
├── vae/
│   ├── config.json
│   └── diffusion_pytorch_model.safetensors        # ~254 MB (bf16)
├── text_encoder/
│   ├── config.json
│   ├── model.safetensors.index.json
│   └── model-{00001,00002,00003}-of-00003.safetensors   # ~11 GB total (bf16)
├── audio_encoder/                                  # Whisper-large-v3 ENCODER only
│   ├── config.json
│   └── model.safetensors                           # ~1.3 GB (bf16)
├── dit/                                            # DMD LoRA pre-merged
│   ├── config.json
│   ├── diffusion_pytorch_model.safetensors.index.json
│   └── diffusion_pytorch_model-{00001..00007}-of-00007.safetensors   # ~33 GB total
├── scheduler/
│   └── scheduler_config.json                       # FlowMatchEuler, shift=7.0
└── tokenizer/                                       # umT5 tokenizer files
    ├── tokenizer.json
    ├── tokenizer_config.json
    └── special_tokens_map.json

Source weights

Provenance, in case you want to verify or re-derive these weights:

Conversion recipe: recipes/convert_longcat_avatar.py in the companion GitHub repo. Run with --variant merged --out <dir> to reproduce these weights from Meituan's PT sources.

Numerical conventions preserved from upstream

• fp32 internal compute for RMSNorm / LayerNorm / AdaLN modulation (Meituan's _FP32 suffix convention) — adaLN_modulation and gamma weights are stored fp32 to defend this.
• Negative velocity flip before scheduler step (noise_pred = -noise_pred) — Meituan's DiT outputs -v; we flip to +v for FlowMatchEulerDiscreteScheduler.step.
• 3-pass disentangled CFG combiner: uncond + s_t·(cond − uncond_text) + s_a·(uncond_text − uncond). Defaults: s_t = s_a = 4.0 (matches PT DMD distillation).
• DMD distilled sigma schedule — 8 sigmas spanning [1.0, 0.124] computed by guidance.get_dmd_distilled_sigmas. The pipeline overwrites the trailing sentinel sigma (mlx-arsenal appends 1.0; we replace with 0.0 to actually denoise to clean at the final step — see CLAUDE.md L17).

License

MIT. Matches upstream Meituan LongCat-Video license. Adapted code from Blaizzy/mlx-video (MIT) for some MLX op primitives; full attribution in LICENSE.

Citation

@misc{longcat-avatar-mlx,
  title  = {longcat-avatar-mlx: Apple MLX port of LongCat-Video-Avatar-1.5},
  author = {xocialize},
  year   = {2026},
  url    = {https://github.com/xocialize/longcat-avatar-mlx},
}

@techreport{meituan2026longcat,
  title       = {LongCat-Video-Avatar 1.5 Technical Report},
  author      = {Meituan LongCat Team},
  institution = {Meituan},
  year        = {2026},
  url         = {https://github.com/meituan-longcat/LongCat-Video},
}