LTX 2.3 Studio

A single-process Gradio app that wraps LTX-2.3 — Lightricks' open 22B video generation model — under one focused UI. Six modes (text · image · audio · lipsync · keyframe · style) sharing the same ComfyUI All-In-One workflow. Runs locally on Apple Silicon (MPS) or NVIDIA (CUDA), deploys to Hugging Face Spaces (ZeroGPU).

→ Live demo: https://huggingface.co/spaces/techfreakworm/LTX2.3-Studio

---

What's inside

Six modes wired through the same ComfyUI All-In-One workflow. Each mode exposes only the inputs it actually consumes — the form stays short and focused.

Every mode carries Fast / Balanced / Quality presets (steps × 1, × 1.5, × 3). A per-mode ZeroGPU duration estimator adapts the call timeout to the requested workload.

---

Quick start (local)

Requires Python 3.11, ~80 GB free disk for the weight set, and ~24 GB VRAM (CUDA) or ~32 GB unified memory (Apple Silicon).

git clone --recurse-submodules https://github.com/techfreakworm/ltx2.3-AIO-generator
cd ltx2.3-AIO-generator
bash setup.sh           # creates .venv, installs ComfyUI + pinned custom nodes + app deps
source .venv/bin/activate
python app.py           # http://127.0.0.1:7860

The first run resolves model weights into your HF cache (~/.cache/huggingface/hub/) and symlinks them into comfyui/models/<comfy_type>/. Subsequent starts skip the download. Expect ~70 GB of weights pulled on a cold first run.

Apple Silicon notes. PYTORCH_ENABLE_MPS_FALLBACK=1 is set automatically so the few MPS-unsupported ops fall back to CPU. ComfyUI's VRAM autodetect picks the right tier; override with LTX23_AIO_VRAM=lowvram|normalvram|highvram if you need to force one.

LAN access (phone / tablet on the same WiFi): python app.py binds 0.0.0.0:7860. Visit http://<your-LAN-IP>:7860 from another device. On macOS, allow inbound for python in System Settings → Network → Firewall if the connection refuses.

Quick start (HF Spaces)

This repo is a Gradio Space. The Pro tier provides ZeroGPU (A10G) access and the per-call duration budget needed for the Balanced and Quality presets.

git remote add space https://huggingface.co/spaces/<your-handle>/LTX2.3-Studio
git push space master:main       # local branch is master; HF Space deploys from main

⚠ The refspec master:main matters. The local default branch is master (GitHub convention); the HF Space deploys from main. A bare git push space master creates an orphan remote branch that does NOT trigger a deploy.

The Space's preload_from_hub directive (see the YAML at the top of this file) bakes ~111 GB of weights into the build image. app.py:_bootstrap() then:

1. Clones ComfyUI + pinned custom nodes into ~/comfyui on cold start (ZeroGPU container freezes preserve them across calls)
2. Mirrors the read-only preload cache into ~/hf-cache-rw/ — works around the build-user-vs-runtime-user permissions trap (preloaded files are root-owned; we run as uid 1000 and can't write to them, so any lazy download to the cache would fail with Permission denied)
3. Stages seed input files into comfyui/input/ so workflow loaders don't error before any user upload arrives

Subsequent requests hit warm cache — no network traffic on inference 2+.

ZeroGPU duration estimator. Each generate call carries a dynamic @spaces.GPU(duration=N) calculated from mode, preset, and frame count. Clamped at [60, 900] s. On timeout ("GPU task aborted"), the handler auto-retries once at 2× duration.

---

Architecture

                                     ┌──────────────────────────────────┐
                          browser ──▶│   app.py — Gradio Blocks         │
                                     │   header · drawer · 6 mode tabs  │
                                     └──────────────────┬───────────────┘
                                                        │
                                                        ▼
                                     ┌──────────────────────────────────┐
                                     │   backend.py                     │
                                     │   ComfyUILibraryBackend          │
                                     │   @spaces.GPU(duration=callable) │
                                     │   calls PromptExecutor directly  │
                                     └──────────────────┬───────────────┘
                                                        │
       ┌──────────────┬──────────────┬──────────────────┴──────┬──────────────────┐
       ▼              ▼              ▼                         ▼                  ▼
   modes.py       models.py      workflow.py                ui.py              tools/
   per-mode       walk + ensure  load + patch               per-mode form      extract_modes.py
   parameterize   from HF cache  API-format JSON            builders           (regen workflows/)
                                                        │
                                                        ▼
                                     ┌──────────────────────────────────┐
                                     │   comfyui/                       │
                                     │   submodule (local)              │
                                     │   runtime clone at ~/comfyui     │
                                     │   on HF Spaces                   │
                                     │                                  │
                                     │   ├── custom_nodes/ (pinned SHAs)│
                                     │   └── models/ → HF cache symlinks│
                                     └──────────────────────────────────┘

One backend, one process. The @spaces.GPU decorator is the only divergence between local and Spaces runtime. ComfyUI manages VRAM via its tiered presets — no empty_cache() sprinkling needed elsewhere.

Workflow as data. Each of the six modes is a user-exported API-format JSON in workflows/. The mode handler patches a deep-copied template (modes.parameterize_fn) and hands it to ComfyUI's PromptExecutor. Updating the master workflow is a three-step ritual: edit in the ComfyUI editor → export → python tools/extract_modes.py --master ... --out workflows.

---

Project layout

.
├── app.py              # Gradio Blocks entry, _bootstrap, _on_generate, mode tabs
├── backend.py          # ComfyUILibraryBackend, @spaces.GPU, duration estimator
├── modes.py            # MODE_REGISTRY + per-mode parameterize_fn + node-id constants
├── models.py           # MODEL_REGISTRY, walk_workflow_for_models, ensure_models
├── ui.py               # render_status, _render_idle, mode-form layout primitives
├── workflow.py         # load_template, set_input helpers
├── workflows/          # API-format mode JSONs (do not hand-edit)
│   ├── t2v.json
│   ├── i2v.json
│   ├── a2v.json
│   ├── lipsync.json
│   ├── keyframe.json
│   └── style.json
├── assets/seed_inputs/ # placeholder image / audio / video for cold-start staging
├── tools/
│   └── extract_modes.py  # regenerate workflows/ from a master ComfyUI export
├── docs/
│   ├── future_improvements.md
│   └── superpowers/{specs,plans}/  # spec + implementation plans per feature
├── tests/              # L1 + L3 in CI; L2 with --comfy-real; L4 GPU smoke
├── README.md           # this file (HF Space YAML + project intro)
├── CLAUDE.md           # project facts + gotchas (what & why)
├── AGENTS.md           # tool-agnostic agent rulebook
├── SKILLS.md           # process / debugging / deployment (how)
├── requirements.txt    # pinned deps
├── pyproject.toml      # ruff + pytest config (py311)
├── setup.sh            # venv + ComfyUI + custom nodes bootstrap
└── comfyui/            # git submodule (local) / runtime clone target (Spaces)

---

Tech stack

• Gradio 5.50 — UI shell, native components, gr.Progress(track_tqdm=True)
• ComfyUI — library-mode PromptExecutor (pinned commit; submodule locally, runtime-cloned on Spaces)
• LTX-2.3 22B by Lightricks — primary diffusion transformer (BF16 weights via Kijai/LTX2.3_comfy)
• Gemma 3 12B by Google — multimodal text encoder (requires the full 5-shard model — text-only checkpoints crash on meta-tensor allocation in SDPA)
• Custom nodes (pinned SHAs in app.CUSTOM_NODES_PINNED):
  • Lightricks/ComfyUI-LTXVideo — LTX sampler / decoder nodes
  • kijai/ComfyUI-KJNodes — utility nodes
  • rgthree/rgthree-comfy — Power-Lora-Loader
  • Kosinkadink/ComfyUI-VideoHelperSuite — video I/O
  • pythongosssss/ComfyUI-Custom-Scripts — string / dict helpers
  • city96/ComfyUI-GGUF — GGUF transformer loader
  • Fannovel16/comfyui_controlnet_aux — DWPose for Lipsync/Style preprocessors
  • evanspearman/ComfyMath — math nodes for the workflow's keyframe path
  • Smirnov75/ComfyUI-mxToolkit — utility nodes
  • DoctorDiffusion/ComfyUI-MediaMixer — FinalFrameSelector
• HF Spaces ZeroGPU (A10G) — @spaces.GPU(duration=…) for queue-priority signalling and per-call timeout

---

Design

Theme: Topaz Cinema Slate — slate substrate #1A1F26, warm amber accent #E0A458 used sparingly, IBM Plex Sans throughout. Defined as _TOPAZ_THEME + _CUSTOM_CSS in app.py.

Layout: hamburger drawer. Pinned 220 px sidebar at ≥1024 px (mode buttons + model status + settings); below 1024 px it slides in as a fixed overlay via the .aio-shell.drawer-open class. The header carries a live mode tag (T2V/A2V/I2V/LIPSYNC/KEY/STYLE) updated by JS without a server round-trip.

Spec, plan, and design rationale live under docs/superpowers/specs/ and docs/superpowers/plans/.

---

Notes on running

• First inference is slow. Cold-start workflow validation + model load on the active node graph takes ~30 – 90 s. Subsequent calls within the same session reuse loaded models.
• VRAM tier is auto-detected; override with LTX23_AIO_VRAM=lowvram|normalvram|highvram.
• ZeroGPU duration cap. The per-call estimator clamps to [60, 900] s. If a generation aborts with "GPU task aborted", the handler retries once at 2× duration. The duration field is the queue-priority signal, not a billing cap.
• Output directory. Local: comfyui/output/LTX2.3/. Spaces: ~/comfyui/output/LTX2.3/. Both are whitelisted via allowed_paths= on launch (Gradio 5 file-access policy).
• Local LAN testing. Bound to 0.0.0.0:7860. macOS firewall: allow inbound for python if a connection from your phone refuses.

---

License

MIT for the AIO app code (see LICENSE).

• ComfyUI is GPL-3.0.
• LTX-2.3 and Lightricks-published LoRAs / auxiliaries retain Lightricks' open-source licensing — see the individual model cards on Hugging Face.
• Gemma 3 weights are subject to Google's Gemma Terms of Use.
• Each pinned custom node retains its own license; see the linked repositories.

Credits

• LTX-2.3 by Lightricks
• ComfyUI by comfyanonymous
• Gemma 3 by Google DeepMind
• All-In-One ComfyUI workflow that this app wraps — by Danielle Falco (FutuTek)
• Workflow nodes by Lightricks, kijai, rgthree, Kosinkadink, pythongosssss, city96, Fannovel16, evanspearman, Smirnov75, DoctorDiffusion

Built by @techfreakworm — drop a ♥ on the Space if it's useful, and follow there for what's next.