4DAnyone: Create Anyone in 4D from a Casual Monocular Video

Project Page  |  Paper

4DAnyone turns a casual monocular video into multi-view videos, enabling downstream 4DGS reconstruction.

[!note] 4DAnyone is a multi-view video model that:

• generates dozens of synchronized, view-consistent videos from a single monocular video.
• requires 22 GB of peak CUDA memory, enabling inference on consumer GPUs.
• averages 27 seconds per 121-frame video on a single RTX 4090.

News

• 2026-09-05: Reduced peak GPU memory below 24 GB, enabling inference on consumer GPUs (RTX 4090).
• 2026-09-02: Released 4DAnyone-Turbo, achieving a 5.58× denoising speedup over 4DAnyone-Base.
• 2026-08-28: Achieved a 1.42× end-to-end speedup for the complete 24-view generation pipeline.
• 2026-08-28: Reduced peak GPU memory below 32 GB while slightly improving speed.

Installation

git clone https://github.com/ant-research/4DAnyone.git
cd 4DAnyone
git submodule update --init third_party/GVHMR

conda create -n 4danyone python=3.11 -y
conda activate 4danyone
pip install -r requirements.txt

For faster inference, optionally install FlashAttention-3 or SageAttention. The installed backend is enabled automatically.

Missing models and examples are downloaded automatically on first use. You can also download them manually:

python scripts/download_smplx.py
python scripts/download_model.py
python scripts/download_example.py

Inference

This repository provides two models: 4DAnyone-Base with the standard denoising schedule and the distilled 4DAnyone-Turbo for faster four-step denoising. 4DAnyone-Turbo is enabled by default for faster inference while maintaining generation quality comparable to 4DAnyone-Base. See Inference performance for GPU memory, inference speed, and generation quality benchmarks.

4DAnyone supports flexible target-view counts, pitch layers, and yaw coverage. Here are several common camera configurations:

6-View Full Orbit

A compact 360° layout for basic coverage. Start here for an initial test.

python inference.py \
    --video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
    --output_dir "data/fdanyone/pexels/2785536-uhd_2160_3840_25fps" \
    --views_per_layer 6

24-View Full Orbit

A dense 360° layout with broad angular coverage, suitable for 4DGS reconstruction.

python inference.py \
    --video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
    --output_dir "data/fdanyone/pexels/2785536-uhd_2160_3840_25fps" \
    --views_per_layer 24

48-View Full Orbit, Three Pitch Layers

This layout distributes views across three pitch rings for broader coverage, enabling free-viewpoint 4DGS rendering.

python inference.py \
    --video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
    --output_dir "data/fdanyone/pexels/2785536-uhd_2160_3840_25fps" \
    --views_per_layer 16 --layer_pitches '[-10,15,35]'

24-View Frontal Arc, Two Pitch Layers

A two-layer layout for dense coverage across the frontal 180° arc.

python inference.py \
    --video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
    --output_dir "data/fdanyone/pexels/2785536-uhd_2160_3840_25fps" \
    --views_per_layer 12 --layer_pitches '[0,30]' --start_yaw -90 --yaw_span 180

Key Arguments

Run python inference.py --help for the full list.

• video_path: path to the source video.
• output_dir: output directory for the current clip. Defaults to data/fdanyone/<clip>.
• views_per_layer: number of evenly spaced views per pitch layer. It must be divisible by 4 or 6.
• layer_pitches: pitch angles in degrees, one per layer. Positive values place cameras above the subject. Total views are views_per_layer × len(layer_pitches).
• start_yaw: horizontal angle of the first view, in degrees. Yaw 0 is the front view.
• yaw_span: horizontal range covered by each camera layer, in degrees.
• gpu_ids: GPU IDs used for parallel pose/VAE view stages and target denoising. Defaults to all visible GPUs.
• enable_turbo: whether to use 4DAnyone-Turbo. Enabled by default.

Output

The output directory contains:

<clip>/                           # input filename without its extension
├── metadata.json                 # run settings, timings, resources
├── cameras.json                  # intrinsics and poses for N target views
├── gvhmr/                        # reusable motion recovery
│   ├── motion.json               # source timeline and motion metadata
│   └── motion.safetensors        # motion tensors
├── skeletons/00.mp4 ... <N-1>.mp4  # pose conditioning for each target view
└── videos/
    ├── sparse/{00,04,09,12,14,19}.mp4  # RCP videos
    └── dense/00.mp4 ... <N-1>.mp4  # target videos

Completed outputs are never overwritten. After a failed or interrupted run, rerun with the same --output_dir to reuse completed motion recovery and restart generation.

Custom Data

Use an input video with:

• a single person in a full-body or upper-body shot.
• no large camera movements, clear footage.
• 1080p or higher, 9:16 portrait aspect ratio, at least 121 frames.

3DGS Reconstruction

See the nerfstudio guide for details.

Roadmap

Peak Memory Optimization

• Reduce peak GPU memory below 32 GB.
• Further reduce peak GPU memory below 24 GB.

Inference Acceleration

• Optimize inference speed through multi-GPU parallelism.
• Accelerate inference via few-step model distillation.

Reconstruction

• Support 3DGS reconstruction with nerfstudio.
• Support 4DGS reconstruction with an open-source method.

Citation

If you find 4DAnyone useful or interesting, please cite our work and consider giving the repository a star ⭐:

@article{jin2026fdanyone,
  title={4DAnyone: Create Anyone in 4D from a Casual Monocular Video},
  author={Jin, Yudong and Xie, Tao and Zhang, Qihang and Shen, Zehong and Xu, Zhen and Shen, Yujun and Bao, Hujun and Zhou, Xiaowei and Xu, Yinghao},
  journal={arXiv preprint arXiv:2608.20335},
  year={2026},
  url={https://arxiv.org/abs/2608.20335}
}