Manifest + Render GPU Demos

A collection of AI demo apps that run web UIs on Barney and dispatch GPU inference jobs to Render GPU. The GPU inference demos use lazy auto-deploy for their model pipelines — interact with the UI and the system handles GPU provisioning automatically.

Demos

How It Works

All demos (except the dashboard) follow the same two-service architecture:

┌─────────────────────┐         ┌──────────────────────┐
│   Control Plane     │         │   Render Compute API  │
│   (Barney)          │────────▶│   api.compute.x.io    │
│                     │  HMAC   │                       │
│  FastAPI + Web UI   │  auth   │  GPU job lifecycle    │
└────────┬────────────┘         └──────────────────────┘
         │
         │  HTTP proxy (per-model routing)
         ▼
┌─────────────────────┐   ┌─────────────────────┐
│  Inference Server   │   │  Inference Server   │
│  (Model A)          │   │  (Model B)          │
│  GPU + Docker       │   │  GPU + Docker       │
│  (weights baked in) │   │  (weights baked in) │
└─────────────────────┘   └─────────────────────┘

• Control plane — Lightweight Python web app deployed on Barney. Serves the browser UI, manages per-model GPU job lifecycle through the Render Compute API, and proxies generation requests to the appropriate inference server.
• Inference server — GPU-powered Docker container deployed on Render GPU. Loads a specific model and exposes /generate and /health endpoints. Model weights are baked into each Docker image.

Key Design Patterns

• Lazy auto-deploy — Inference servers start on first request, not upfront.
• Idle auto-stop — Jobs auto-cancel after configurable idle timeout (default 30 days).
• One image per model — Each model gets its own Docker image with code, dependencies, and weights baked in.
• VRAM-based scheduling — Jobs specify minimum VRAM; Render assigns the best available GPU.
• In-memory state — No database. On restart, running jobs are re-adopted by title matching.

User Flow

1. Open the web UI and see available models with status badges
2. Select a model, provide input (prompt, audio, etc.), click Generate
3. If the model isn't running, the system auto-deploys it
4. UI polls until ready, then generation happens automatically
5. After idle timeout, the model's Render job is auto-cancelled
6. Next request for that model triggers a fresh deploy

Prerequisites

• Python 3.12+
• A Render GPU account with API credentials (public key + secret key)
• Docker with BuildKit support (for building/pushing inference images)
• An NVIDIA GPU with CUDA (only for running inference servers locally)

Quick Start

Each demo has its own README with detailed install and build instructions. Here's the general pattern:

1. Set up environment variables

# Required for all control planes
export RENDER_API_KEY="your-render-public-key"
export RENDER_SECRET_KEY="your-render-secret-key"

# Model configuration (JSON map — one image per model)
export RENDER_INFERENCE_MODELS='{
  "ModelName": {
    "image": "your-registry/inference-image:tag",
    "min_vram_gb": 8
  }
}'

2. Run a control plane

cd render-image-gen  # or any control plane folder
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 8000

3. Build and push an inference image

cd render-image-gen-inference  # or any inference folder

# Download model weights (one-time) — see each inference folder's README
# for the exact command (some use AUDIOCRAFT_CACHE_DIR or --output-dir)
HF_HOME=./model-cache python3 download_model.py <model_id>

# Build and push (build.sh pushes automatically)
./build.sh <model_id> <image_tag>

4. (Optional) Run inference locally

Requires an NVIDIA GPU with CUDA:

cd render-image-gen-inference
pip install -r requirements.txt
MODEL_ID=stabilityai/sdxl-turbo uvicorn main:app --host 0.0.0.0 --port 8000

See each folder's README for demo-specific details.

Project Structure

manifest-render-demo/
├── render-image-gen/                 # Text-to-image control plane
├── render-image-gen-inference/       # Text-to-image inference (diffusers)
├── render-music-gen/                 # Text-to-music control plane
├── render-music-gen-inference/       # Text-to-music inference (ACE-Step, DiffRhythm2)
├── render-hum-music-gen/             # Hum-to-music control plane
├── render-hum-music-gen-inference/   # Hum-to-music inference (MusicGen Melody)
├── render-voice-clone/               # Voice clone control plane
├── render-voice-clone-inference/     # Voice clone inference (XTTS-v2 + MuseTalk)
├── render-dashboard/                 # Render Network metrics dashboard
├── DEVELOPMENT.md                    # Architecture deep-dive and developer guide
└── SECURITY.md                       # Security model and trust boundaries

Environment Variables

All GPU demo control planes share the same core environment variables (the dashboard has its own configuration — see its README):

*One of RENDER_INFERENCE_MODELS or RENDER_INFERENCE_IMAGE is required. RENDER_INFERENCE_TIMEOUT default varies by demo: 120s for image-gen, 300s for music demos, 600s for voice clone. See each folder's README for the exact default.

All inference servers share these environment variables (MODEL_ID does not apply to voice-clone-inference, which uses a fixed pipeline):

Documentation

Each service folder also has its own README with install, build, and run instructions.

Notes

• Job state is in-memory. On startup, control planes re-adopt active Render jobs by title matching using a fingerprint derived from INFERENCE_SECRET.
• Deploy timeout. Jobs stuck in PENDING/ASSIGNED longer than RENDER_DEPLOY_TIMEOUT_MINUTES (default 150) are auto-cancelled.
• Port mapping. Render assigns random external ports. Control planes resolve inference URLs by matching the description field in node_urls (container port), not the port field (external port).
• TLS cert pinning. When INFERENCE_SECRET is explicitly set, both sides derive a deterministic Ed25519 TLS certificate from the secret. See SECURITY.md for details.