Lattice Compositor

---

Vision

Lattice bridges the gap between professional motion graphics and AI video generation.

The AI video generation community has powerful models (Wan 2.1, AnimateDiff, MotionCtrl, VACE) but creating the conditioning data these models need — depth maps, motion trajectories, attention masks — requires jumping between multiple tools. Lattice solves this by bringing industry-standard animation tools directly into ComfyUI.

Core Philosophy

1. Deterministic by Design — Every frame is reproducible. Scrub to frame 50, scrub back to frame 10, scrub to frame 50 again — identical output, guaranteed. This is critical for AI video workflows.

2. AI-Native Export — One-click export to formats AI models actually consume: matte sequences for IP Adapter, camera trajectories for MotionCtrl, point paths for Wan Move.

3. Professional Tools, Zero Cost — Keyframe animation, bezier curves, particle systems, expressions, 3D cameras — the full professional toolkit, open source.

---

What is Lattice?

Lattice is a professional-grade motion graphics compositor built for the ComfyUI ecosystem. It brings industry-standard animation tools to AI video workflows, enabling you to create stunning motion graphics, animated text, particle systems, and 3D camera animations — then export directly to AI video models like Wan 2.1, AnimateDiff, and MotionCtrl.

Why Lattice?

Perfect for: AI video creators, motion designers, VFX artists, and content creators using Stable Diffusion video workflows.

---

Screenshots

---

Project Status

See PROJECT_STATUS.md for detailed status and PRD.md for product requirements.

---

Features

Layer System (17 Types)

Animation System

• 35 Easing Functions — Linear, ease-in/out, spring, elastic, bounce, back, and custom bezier
• Expression Language — wiggle(), repeatAfter(), bounce(), inertia(), elastic()
• Curve Editor — Fine-tune animation curves with bezier handles
• Property Linking — Connect any property to any other property
• 16 Semantic Keyframe Shapes — Visual encoding of easing type

Particle System

• 24 Built-in Presets — Fire, smoke, snow, rain, confetti, magic, and more
• Deterministic Simulation — Scrub-safe with checkpoint system
• 7 Emitter Shapes — Point, line, circle, box, sphere, ring, spline
• Physics Forces — Gravity, wind, turbulence, attractors, vortices
• Sub-emitters — Particles that spawn particles
• Collision Detection — Bounce, slide, stick behaviors

3D Camera System

• 22 Camera Presets — Orbit, dolly, crane, reveal, spiral, and more
• Depth of Field — Realistic focus with bokeh
• Camera Shake — Handheld, earthquake, subtle variations
• Trajectory Export — MotionCtrl, CameraCtrl, Uni3C formats

Audio Reactivity

• Beat Detection — Sync animations to music
• Frequency Bands — React to bass, mid, high frequencies
• Stem Separation — React to drums, vocals, instruments
• Audio-to-Property Mapping — Drive any parameter with audio
• Audio Path Animation — Animate objects along paths driven by audio

Effect Pipeline (22+ Effects)

AI Integration

• Natural Language Agent — Describe animations: "fade in the title over 1 second"
• GPT-4o & Claude Support — Choose your preferred model
• 30+ Tool Actions — Create layers, add keyframes, apply effects
• Conversation Memory — Iterative refinement: "make it faster"

Export Formats

---

Installation

ComfyUI Extension (Recommended)

# Navigate to your ComfyUI custom_nodes folder
cd ComfyUI/custom_nodes

# Clone the repository
git clone https://github.com/justinfleek/ComfyUI-Lattice-Compositor.git

# Install Python dependencies
pip install -r ComfyUI-Lattice-Compositor/requirements.txt

Restart ComfyUI. The Lattice icon will appear in your sidebar. No build step required — pre-built JavaScript files are included.

Windows with Portable/Embedded Python

If you're using ComfyUI's portable distribution with embedded Python:

# Navigate to custom_nodes
cd "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes"

# Clone
git clone https://github.com/justinfleek/ComfyUI-Lattice-Compositor.git

# Install with embedded Python
& "C:\ComfyUI_windows_portable\python_embeded\python.exe" -m pip install -r ComfyUI-Lattice-Compositor/requirements.txt

Requirements

The requirements.txt includes minimal dependencies since most are already in ComfyUI:

numpy
Pillow
scipy  # For depth-to-normal conversion

Development Setup

For contributors or those wanting to modify the source:

# Navigate to the UI source
cd lattice-compositor/ui

# Install Node.js dependencies
npm install

# Start development server (hot reload)
npm run dev
# Opens at http://localhost:5173

# Run test suite
npm test

# Build for production (outputs to web/js/)
npm run build

---

Quick Start

Basic Workflow

1. Open Compositor — Click the Lattice icon in the ComfyUI sidebar
2. Add a Layer — Right-click canvas → Add Layer → Text/Shape/Particles
3. Animate — Click the keyframe icon (◆) next to any property
4. Preview — Press Space to play, drag the playhead to scrub
5. Export — Click Export button → Select format → Export

Using the AI Agent

1. Open the AI tab in the right panel
2. Type: "Create a title that fades in and slides up"
3. The agent creates layers, keyframes, and effects automatically
4. Refine: "add a glow effect", "make it bounce"

Requires: Set OPENAI_API_KEY or ANTHROPIC_API_KEY environment variable before starting ComfyUI.

Frame Count Formula (4n+1 Pattern)

AI video models require frame counts following frames = (seconds × 16) + 1:

---

Architecture

┌─────────────────────────────────────────────────────────────────┐
│  PRESENTATION LAYER (Vue 3.5 + TypeScript)                      │
│  ├── 146 Vue Components                                         │
│  ├── PrimeVue UI Library                                        │
│  ├── 6 Theme Gradients (Violet, Ocean, Sunset, Forest, Ember)   │
│  └── Design Tokens System                                       │
├─────────────────────────────────────────────────────────────────┤
│  STATE LAYER (Pinia 2.2)                                        │
│  ├── compositorStore - Main project state                       │
│  ├── playbackStore - Timeline playback                          │
│  ├── historyStore - Undo/redo (50 snapshots)                    │
│  ├── audioStore - Audio analysis & reactivity                   │
│  └── assetStore - Asset management                              │
├─────────────────────────────────────────────────────────────────┤
│  ENGINE LAYER (Three.js r170)                                   │
│  ├── LatticeEngine - Main rendering facade                         │
│  ├── MotionEngine - Deterministic frame evaluation              │
│  ├── LayerManager - 17 layer implementations                    │
│  ├── ParticleSystem - Checkpoint-based simulation               │
│  └── EffectProcessor - Canvas-based effect pipeline             │
├─────────────────────────────────────────────────────────────────┤
│  SERVICE LAYER (80+ Services)                                   │
│  ├── Animation: interpolation, easing, expressions              │
│  ├── Audio: FFT analysis, beat detection, mapping               │
│  ├── Particles: CPU simulation, GPU rendering                   │
│  ├── 3D: camera trajectories, depth parallax                    │
│  ├── Effects: blur, color, distort, generate                    │
│  └── Export: mattes, trajectories, video                        │
├─────────────────────────────────────────────────────────────────┤
│  BACKEND (Python + ComfyUI)                                     │
│  ├── AI Inference: DepthAnything, SAM, NormalCrafter            │
│  ├── Export: Frame sequences, trajectories                      │
│  └── API Routes: /lattice/* endpoints                              │
└─────────────────────────────────────────────────────────────────┘

The Determinism Rule

For AI video generation, every frame must be reproducible. Lattice guarantees:

evaluate(frame, project) always returns identical results for identical inputs.

• Seeded RNG — Mulberry32 algorithm for deterministic randomness
• Checkpoint System — Particles restore state on scrub
• Pure Evaluation — No Math.random(), no Date.now(), no accumulation

---

Keyboard Shortcuts

---

Documentation

See docs/ folder for technical documents and _archived/ for historical documentation.

---

Known Limitations

Requires External Setup

See FEATURE_AUDIT.md for complete feature-to-UI mapping.

---

Terminology

Lattice uses clear, descriptive terminology:

---

Acknowledgments

Lattice incorporates techniques and inspiration from these outstanding projects:

Audio Reactive

• ATI_AudioReactive — Beat detection, amplitude curves
• Yvann-Nodes — Stem separation, IPAdapter scheduling
• RyanOnTheInside — Flex Features paradigm

Depth & Parallax

• ComfyUI-Depthflow-Nodes — 2.5D parallax
• DepthAnything — Depth estimation

Export Formats

• Wan-Move — Point trajectories
• Time-to-Move — Cut-and-drag motion
• MotionCtrl — Camera trajectories
• AnimateDiff CameraCtrl — Camera control

AI Models

• SAM2 — Segmentation
• Sapiens — Human pose estimation

---

Contributing

Contributions are welcome! Please read CLAUDE.md for architecture details and coding conventions.

# Clone
git clone https://github.com/justinfleek/ComfyUI-Lattice-Compositor.git
cd ComfyUI-Lattice-Compositor/ui

# Install dependencies
npm install

# Development server
npm run dev

# Run tests
npm test

# Build
npm run build

Contribution Ideas

• Additional particle presets
• New effect implementations
• Export format support (new AI video models)
• UI/UX improvements
• Documentation and tutorials

---

License

MIT License — see LICENSE for details.

---