LingBot-World NF4 Quantized

Pre-quantized NF4 weights for LingBot-World video generation model. This is a complete, self-contained package - no additional downloads required.

Features

• 4-bit NF4 quantization via bitsandbytes - fits in 32GB VRAM
• Pre-quantized weights - no runtime quantization overhead
• Complete package - includes T5 encoder, VAE, and diffusion models

Quick Start

# Clone the repo
git clone https://huggingface.co/cahlen/lingbot-world-base-cam-nf4
cd lingbot-world-base-cam-nf4

# Install dependencies
pip install -r requirements.txt

# Generate a video
python generate_prequant.py \
    --image your_image.jpg \
    --prompt "A cinematic video of the scene" \
    --frame_num 81 \
    --output output.mp4

Model Contents

Total size: ~30GB (vs ~85GB for full precision models)

Usage

Basic Generation

python generate_prequant.py \
    --image input.jpg \
    --prompt "Your prompt here" \
    --frame_num 81 \
    --size "480*832" \
    --output output.mp4

Parameters

With Camera Control

python generate_prequant.py \
    --image input.jpg \
    --prompt "Your prompt" \
    --action_path /path/to/camera_poses/ \
    --frame_num 81

Camera pose directory should contain:

• poses.npy: Shape [num_frames, 4, 4] - camera transformation matrices
• intrinsics.npy: Shape [num_frames, 4] - [fx, fy, cx, cy]

Requirements

• Python 3.10+
• CUDA 11.8+ (tested with CUDA 12.x)
• ~32GB VRAM (RTX 4090, RTX 5090, A100, etc.)

Quantization Details

The diffusion models are quantized using bitsandbytes NF4 with double quantization:

{
  "format": "bnb_nf4",
  "double_quant": true,
  "compute_dtype": "bfloat16",
  "blocksize": 64
}

This achieves ~3.9x compression while maintaining generation quality.

License

This model is based on LingBot-World and follows its license terms.

Citation

@misc{lingbot-world-nf4,
  title={LingBot-World NF4 Quantized},
  year={2025},
  url={https://huggingface.co/cahlen/lingbot-world-base-cam-nf4}
}