AI Sprite Sheet Creator

Local web app that turns a text description or reference image into pixel-art 2D game character sprite sheets — generated on your own GPU with Stable Diffusion XL.

How it works (accuracy pipeline)

1. Base character — 4 candidates are generated with SDXL + pixel-art LoRA from your description (plus an optional reference image via IP-Adapter). You pick the one that is exactly right; its color palette is extracted and locked.
2. 8-direction turnaround — FLUX.1 Kontext (an identity-preserving editing model) transforms the approved base into S, SE, E, NE, N, NW, W, SW views, driven by per-direction pose skeletons through the RefControl pose-transfer LoRA. Retry any view that drifted, then lock.
3. Movements, one at a time — pick a direction and a movement (40 movements: idle, run, attacks, wall slide, swim…), hit Generate. Every frame is a Kontext pose-transfer: input is [that direction's locked view | deterministic 3D-rigged skeleton], output is the same character in that exact pose. Frames are then background-removed, k-centroid downscaled, snapped to the locked palette, height-normalized to the skeleton's predicted size, and foot-anchored. Bad frame? Regenerate just that frame.
4. Export — one packed sheet.png (one row per direction+movement) with sheet.json metadata (cell size, rows, frame counts, fps, loop).

Generation speed on an RTX 5060 Ti: base candidates ~40 s each; turnaround views and movement frames ~3.5 min each (FLUX 12B at 28 steps — fewer steps were tested and degrade pose accuracy, so quality wins). An 8-frame run cycle takes ~30 min. The two engines share the GPU and are swapped automatically.

Requirements

• Windows, NVIDIA GPU with ≥12 GB VRAM (built on an RTX 5060 Ti 16 GB)
• Python 3.11+
• ~15 GB disk for model weights (downloaded automatically on first run)

Setup

python -m venv .venv
.venv\Scripts\pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128
.venv\Scripts\pip install -r requirements.txt

Run

Double-click start.bat, or:

.venv\Scripts\python -m uvicorn backend.main:app --port 8000

Open http://localhost:8000. The first generation loads/downloads the models (one time, ~13 GB); after that a movement of 6 frames takes a couple of minutes.

Set ASSETS_CREATOR_FULL_GPU=1 to keep all models resident on the GPU (faster, needs the full 16 GB free).

Output layout

output/<character_id>/
  character.json        # full state (resumable from the UI)
  base_raw.png          # approved 1024px base (identity anchor)
  base_sprite.png       # pixel-art base sprite
  views/                # 8-direction turnaround (raw + sprite)
  anims/<DIR_movement>/ # frames f0..fN.png, raw frames, preview.gif
  sheet.png, sheet.json # packed export

Models used

Adding movements

Movements are pure data: add an entry in backend/poses/animations.py (build()) using the parametric Pose rig — it automatically works for all 8 directions.

Smoke test (no web UI)

.venv\Scripts\python scripts\smoke_test.py

Generates a base knight + one East run frame into scripts/smoke_out/.