MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layouts

🏗️ Pipeline Overview

📜 TODO List

• Release ${\color{#8D88E2}MajutsuDataset}$
• Release ${\color{#6EB4F9}MajutsuCity}$
  • Layout Generation model
  • Material texture fintuned model
  • Framework code
• Release ${\color{#FF7FAE}MajutsuAgent}$ code

🪄 Usage

Installation

Requirements: Linux, Python 3.10, CUDA 12.4, and PyTorch 2.6.0.

git clone https://github.com/LongHZ140516/MajutsuCity.git
cd MajutsuCity

conda env create -f environment.yml
conda activate majutsucity

Place the official Hunyuan3D-Omni and Hunyuan3D-2.1 source checkouts at external/Hunyuan3D-Omni and external/Hunyuan3D-2.1. Compile the two Hunyuan3D-2.1 paint extensions as described in requirements/README.md.

Models and local paths

Copy the example configuration and edit it for your machine:

cp configs/paths.example.yaml configs/paths.local.yaml

All relative paths in configs/paths.local.yaml are resolved from the repository root. GPU allocation is configured with runtime.qwen_gpu_groups and runtime.hunyuan_gpus.

The pipeline uses local checkpoints for layout and height generation, image generation and editing, background removal, 3D reconstruction and painting, and surface/skybox generation. Expand the panel below for their exact paths and official download pages.

models/
├── MajutsuCity/
│   ├── layout/
│   │   ├── layout_generation.ckpt
│   │   └── height_generation.ckpt
│   └── texture/
│       ├── Texture_Lora.safetensors
│       └── SkyBox_Lora.safetensors
├── LongCLIP-L/longclip-L.pt
├── Qwen-Image-Edit-2509/
├── Qwen-Image/
├── RMBG-2.0/
├── Hunyuan3D-Omni/
├── Hunyuan3D-2.1/
├── dinov2-large/
└── dinov2-giant/

End-to-end generation with Scene Design API

The recommended entry point is scripts/generate_city.py. An OpenAI-compatible Chat Completions endpoint can turn one scene description into coordinated prompts for the layout, buildings, props, surface materials, and skybox, then run the complete city-generation pipeline.

export MAJUTSUCITY_LLM_BASE_URL="https://your-provider.example/v1"
export MAJUTSUCITY_LLM_API_KEY="your-api-key"
export MAJUTSUCITY_PLANNER_MODEL="your-model"

python scripts/generate_city.py \
  --config configs/paths.local.yaml \
  --case-name planned_city \
  --style Cyberpunk \
  --scene-prompt "A compact coastal city with a connected road network"

The generated Scene Plan is saved to output/<case>/scene_plan.json and can be reused without another planning request by passing --scene-plan <path>. Visual asset review is optional: set MAJUTSUCITY_REVIEW_MODEL and add --enable-vlm-review only when review and retry are required.

python scripts/generate_city.py \
  --config configs/paths.local.yaml \
  --case-name stockholm_cyberpunk \
  --style Cyberpunk \
  --layout examples/pair_cases/layout/SE-Stockholm_64.png \
  --depth examples/pair_cases/depth/SE-Stockholm_64.png

python scripts/generate_city.py \
  --config configs/paths.local.yaml \
  --case-name generated_cyberpunk_city \
  --style Cyberpunk \
  --layout-prompt "A compact coastal city with a connected road network"

For each building, the end-to-end pipeline samples the layout-derived gray mesh to points_ascii.ply, uses the edited image and point cloud to generate shape with Hunyuan3D-Omni, and applies PBR texture with Hunyuan3D-2.1 Paint. It also generates ground, road, vegetation, water, and skybox textures before Blender assembly. Existing intermediate results are reused; add --overwrite to regenerate them. Use --shape-backend 2.1 only for the image-only compatibility path.

Outputs

output/<case>/
├── inputs/                         # canonical layout and depth
├── scene_plan.json                 # only with Scene Design
├── assets_<style>/                 # buildings and environment assets
├── scenes/<style>/
│   ├── city_scene.blend
│   ├── preview.png
│   └── assembly_report.json
└── pipeline_run.json

For lower-level options, run python scripts/generate_city.py --help or python scripts/run_pipeline.py --help. Additional references:

• Environment construction
• Standalone layout inference
• Third-party notices

🙏 Acknowledgements

• Some visual design inspirations (e.g., icons and layout ideas) are adapted from awesome-framework-gallery, Arknights and Arknights: Endfield.
• The Texture/Skybox LoRA is trained on the basis of flymyai-lora-trainer.

📝 Citation

@article{huang2025majutsucity,
  title={MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layouts},
  author={Huang, Zilong and He, Jun and Huang, Xiaobin and Xiong, Ziyi and Luo, Yang and Ye, Junyan and Li, Weijia and Chen, Yiping and Han, Ting},
  journal={arXiv preprint arXiv:2511.20415},
  year={2025}
}