WonderZoom: Multi-Scale 3D World Generation

WonderZoom: Multi-Scale 3D World Generation

Jin Cao*, Koven Yu*, Jiajun Wu

(* denotes equal contribution)

Overview

WonderZoom generates multi-scale 3D worlds from a single image. Starting from an input photograph, it builds a 3D Gaussian Splatting scene that supports continuous zoom-in navigation, revealing new details and objects at each scale.

The scene is made of scale-adaptive Gaussian surfels: each primitive knows the scale it was generated at, and the renderer fades it in and out with the viewing scale. A progressive detail synthesizer grows the scene interactively:

• zoom-in adds finer-scale content with Chain-of-Zoom super-resolution and scale-consistent depth registration;
• camera moves and auxiliary views extend the scene with Gen3C videos;
• object insertion (optional) places new objects at a new scale with Step1X-Edit.

What is released

One server, run.py, serves one page, splat-main/index_gen.html, at http://localhost:7747/ in both modes.

run.py --view replaces run_render_only.py and splat-main/index_stream.html of the first release.

The generation pipeline runs the three video and image models (Gen3C, Chain-of-Zoom and Step1X-Edit) as persistent worker processes in their own environments. On a single 46-48 GB GPU, the default auto GPU policy lets the models take turns on the GPU and parks the idle ones in host RAM. Scripts install the environments, download pinned checkpoints and check the installation.

License

Non-commercial use only. Both viewing (run.py --view) and generation depend on components that do not allow commercial use. Read THIRD_PARTY_LICENSES.md before you use or redistribute the code, the weights or the generated scenes.

WonderZoom's own code: the license has not been chosen yet (TODO(authors)). Until the authors add a LICENSE file, no license is granted for it.

Non-commercial components:

Other terms to be aware of:

• Gen3C guardrail. The Gen3C-Cosmos-7B and Cosmos tokenizer weights are under the NVIDIA Open Model License, whose rights terminate if the model's safety guardrails are disabled. WonderZoom runs Gen3C with the guardrail disabled by default, as in the paper runs (services.gen3c.disable_guardrail: true in config/services.yaml). Set it to false to run the guardrail; see THIRD_PARTY_LICENSES.md.
• Stable Video Diffusion img2vid-xt (GeometryCrafter's base model) is under the Stability AI Community License. Stable Diffusion 2 inpainting (object insertion) is under CreativeML Open RAIL++-M, which has use-based restrictions.

Quick start: view a released scene

1. Install the main environment. It builds PyTorch3D and the 3DGS CUDA extensions, which takes about 30-60 min on 16 CPUs.

git clone https://github.com/jin-cao-tma/WonderZoom.git && cd WonderZoom
bash scripts/install_env_main.sh          # creates the conda env wz-main
conda activate wz-main

2. Download the pre-generated scenes (~7.8 GB) into gaussian/:

bash scripts/download_checkpoints.sh --scenes
# or, with the Hugging Face CLI:
huggingface-cli download TmaKiss/WonderZoom --repo-type dataset --include "gaussian/*.pth" --local-dir ./

3. Start the server in view mode:

python run.py --view --example_config config/more_examples/street.yaml

--view loads the scene file of the config (pth_path), renders it at the config's orig_H x orig_W (else gen_H x gen_W) and uses the config's orbit code for Space. It loads no generation model and starts no worker. Start-up takes about 35 s for the largest released scene.

You can also pass a .pth file directly (--pth_path implies --view):

python run.py --view --pth_path ./gaussian/gau_bird3_complete1080.pth \
    --example_config config/more_examples/street.yaml

Add --dry_run to print the resolved scene path, render size and stream settings and exit before loading the scene.

Streaming quality: view mode streams JPEG frames of quality 80, at most 1080 px on the longest edge (--stream_quality 80 --stream_max_size 1080). If the viewer lags over SSH, lower them, e.g. --stream_max_size 512 --stream_quality 40.

4. Open the page. The server binds to 127.0.0.1:7747. If it is remote, forward the port first:

ssh -N -L 7747:localhost:7747 <your-server>

Then open http://localhost:7747/. Pass --host 0.0.0.0 to serve other machines directly. The page also works as a local file: splat-main/index_gen.html?server=localhost:7747.

In view mode the page is titled "WonderZoom Viewer", hides the generation controls and shows idle with Viewing <file> in the status line. Generation keys are refused; start run.py without --view to generate. Stop the server with Ctrl+C.

Quick start: generation

# 1. Clone
git clone https://github.com/jin-cao-tma/WonderZoom.git && cd WonderZoom

# 2. Build the environments wz-main, wz-gen3c and wz-coz (add --objects for object insertion: wz-step1x + GroundedSAM)
bash scripts/install_all.sh [--objects]

# 3. Accept the SD3-Medium license (Stability AI Non-Commercial Research Community License) on
#    https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers, then log in
conda activate wz-main && huggingface-cli login      # or: export HF_TOKEN=hf_...

# 4. Download the checkpoints: the groups for the three default environments (~111 GB) ...
bash scripts/download_checkpoints.sh --core --gen3c --coz
#    ... or everything, including object insertion (~168 GB)
# bash scripts/download_checkpoints.sh --all

# 5. On a GPU node, check every registered environment (add --objects after install_all.sh --objects)
python scripts/check_install.py

# 6. Start the server
bash scripts/run_server.sh --example_config config/more_examples/street.yaml

The server binds to 127.0.0.1:7747. Forward the port from your laptop and open the generation UI:

ssh -L 7747:localhost:7747 <your-server>     # then open http://localhost:7747/

The status line at the top of the page reports progress. The main models load first. The model workers then start in the background while the initial scene is built, and the status becomes idle once the server accepts requests. A request that needs a worker that is still loading waits for it. Every session is written to runs/<example_name>/<YYYYmmdd-HHMMSS>/.

To generate from your own image, run bash scripts/run_server.sh --image /path/to/photo.jpg --name my_scene. Relative paths are resolved against your current directory first, then against the repository root.

Details: docs/INSTALL.md for environments and troubleshooting, docs/GENERATION_GUIDE.md for workflows and configuration.

Key map (generation UI)

Object insertion: type an object (e.g. a ladybug) into the box at the top of the page and press Enter. The object is inserted at the end of the next zoom-in.

Viewing a saved scene

X saves the scene in the same format as the released scenes, so run.py --view can load it. Pass the session's config.yaml as the scene config: it records the generation resolution and the scene's orbit code. Run it in the wz-main environment, on another port if the generation server is still running:

python run.py --view --pth_path runs/street/<YYYYmmdd-HHMMSS>/scenes/street_000.pth \
    --example_config runs/street/<YYYYmmdd-HHMMSS>/config.yaml --port 7748

Then forward port 7748 and open http://localhost:7748/. See Viewing saved scenes and, for .splat export, tools/export_splat.py.

Documentation

• docs/INSTALL.md: the four environments, build settings (MAX_JOBS, TORCH_CUDA_ARCH_LIST), checkpoints, check_install.py and troubleshooting.
• docs/GENERATION_GUIDE.md: workflows (camera move, zoom-in, object insertion, HQ views, crack fixing, undo/save, viewing and exporting scenes), your own images, the config reference, config/services.yaml and the GPU policies.
• docs/HARDWARE.md: GPU memory, host RAM, disk and timings.
• THIRD_PARTY_LICENSES.md: licenses of the bundled code and the downloaded models, including non-commercial terms.

Hardware requirements

See docs/HARDWARE.md for per-model numbers and multi-GPU profiles.

Paper-to-code map

Project structure

WonderZoom/
├── run.py                   # the server (Flask + Socket.IO): generation, or --view for saved / released scenes; serves splat-main/index_gen.html
├── config/
│   ├── base-config.yaml     # every scene key with its default
│   ├── custom_template.yaml # used by run.py --image
│   ├── more_examples/       # per-scene configs
│   └── services.yaml        # model workers, GPUs, checkpoint paths
├── services/                # worker clients, GPU arbiter; services/workers/ runs inside each worker env
├── scripts/                 # install_*.sh, download_checkpoints.sh, check_install.py, run_server.sh, ...
├── third_party/             # pins.env, checksums, WonderZoom files added to the upstream clones
├── tools/export_splat.py    # export a saved scene to .splat
├── tests/                   # worker smoke tests, headless end-to-end driver
├── docs/                    # INSTALL, GENERATION_GUIDE, HARDWARE
├── THIRD_PARTY_LICENSES.md, LICENSES/   # third-party license summary and license texts
├── envs/, requirements/     # conda toolchain and pinned pip requirements (no top-level requirements.txt; use the install scripts)
├── models/, scene/, gaussian_renderer/, util/, utils/, marigold_lcm/   # WonderZoom core
├── GeometryCrafter/, MoGe/, RepViT/   # vendored (trimmed) dependencies
├── submodules/              # 3DGS rasterizer and simple-knn CUDA extensions
├── splat-main/              # web page (index_gen.html, main_stream.js)
├── external/                # pinned upstream clones (created by the scripts; gitignored)
├── checkpoints/             # Gen3C, Step1X-Edit and other weights (created by the scripts; gitignored)
├── gaussian/                # released scenes (download_checkpoints.sh --scenes)
└── runs/                    # generation sessions (gitignored)

Citation

@misc{wonderzoom,
    title={WonderZoom: Multi-Scale 3D World Generation},
    author={Jin Cao and Hong-Xing Yu and Jiajun Wu},
    year={2025},
    eprint={2512.09164},
    archivePrefix={arXiv},
    primaryClass={cs.CV},
    url={https://arxiv.org/abs/2512.09164}
}

TODO

• Release rendering and interactive visualization code
• Release Chain-of-Zoom integration for multi-scale zoom-in generation
• Release Gen3C integration for high-quality novel view synthesis
• Release Step1X-Edit integration for object editing

Related Project

• [CVPR2025 Highlight] WonderWorld: Interactive 3D Scene Generation from a Single Image

Acknowledgement

We appreciate the authors of the following projects for sharing their code: 3D Gaussian Splatting, LucidDreamer (CC BY-NC-SA 4.0; several files in utils/, scene/ and arguments_in.py are derived from it, see THIRD_PARTY_LICENSES.md), WonderWorld, MoGe, GeometryCrafter, Marigold, OneFormer, RepViT-SAM, PyTorch3D, Chain-of-Zoom, Gen3C, Step1X-Edit, Stable Diffusion, Grounded-Segment-Anything, VGGT, Kornia, and INR-Harmonization.