Pixal3D generates high-fidelity 3D assets from a single image. Instead of loosely injecting image features via attention, it explicitly lifts pixel features into 3D through back-projection, establishing direct pixel-to-3D correspondences.

🔬 This Fork

This is a fork of the original Pixal3D repo that adds texture-only inference with automatic pose alignment: texturing an externally-supplied mesh (e.g. a Hunyuan3D output) from a single photo, even when the photo is not shot from the mesh's canonical front.

Added files

Modified files

Everything else is unchanged from upstream.

📌 Branches

🚀 Installation

First set up the base TRELLIS.2 environment, then:

pip install -r requirements.txt

# Replace xx with your CUDA arch and desired build worker count
NATTEN_CUDA_ARCH="xx" NATTEN_N_WORKERS=xx pip install natten==0.21.0 --no-build-isolation

pip install https://github.com/LDYang694/Storages/releases/download/20260430/utils3d-0.0.2-py3-none-any.whl

If flash_attn is unavailable, prefix any command with ATTN_BACKEND=sdpa.

🖼️ Inference

Image → 3D (generate a GLB from a single image):

python inference.py --image assets/images/0_img.png --output ./output.glb
python inference.py --image assets/images/0_img.png --output ./output.glb --low_vram   # ~10-12GB VRAM
python inference.py --image assets/images/0_img.png --output ./output.glb --resolution 1024

Default resolution is 1536 (standard) or 1024 (low-VRAM); override with --resolution.

Texture an external mesh (this fork) — pose-aligned texturing of a supplied mesh from a photo:

python texture_inference.py --mesh mesh.glb --image photo.png --output ./textured.glb --low_vram
python texture_inference.py --mesh mesh.glb --image photo.png --output ./textured.glb --no_pose      # skip pose alignment
python texture_inference.py --mesh mesh.glb --image photo.png --output ./textured.glb --debug_proj   # dump projection overlays

Key options: --resolution (voxel grid, default 1024), --texture_size (atlas, default 2048), --fov (radians, override MoGe estimate), --bbox_padding (default 1.1).

Web demo (Gradio):

python app.py
python app.py --low_vram   # or: LOW_VRAM=1 python app.py

🔧 Training

Pixal3D is trained as a three-stage cascade, each stage progressively increasing resolution. All stages use pixel-aligned projection conditioning and view-aligned latents (2 views by default). Within a stage, train the lowest resolution first, then fine-tune upward by setting finetune_ckpt in the config to the previous checkpoint.

First prepare data following data_toolkit/README.md, then:

python train.py --config <CONFIG_JSON> --output_dir <OUTPUT_DIR> --data_dir '<DATA_DIR_JSON>'

--data_dir is a JSON string describing the dataset layout. Stages, configs, and required keys:

# --- Stage 1: Sparse Structure (32 → 64) ---
python train.py --config configs/gen/ss_flow_img_dit_1_3B_32_bf16_proj_finetune.json \
  --output_dir results/ss_32 \
  --data_dir '{"ObjaverseXL_sketchfab": {"base": "datasets/ObjaverseXL_sketchfab", "ss_latent": "datasets/ObjaverseXL_sketchfab/ss_latents/ss_enc_conv3d_16l8_fp16_64_view", "render_cond": "datasets/ObjaverseXL_sketchfab/renders_cond"}}'

python train.py --config configs/gen/ss_flow_img_dit_1_3B_32_bf16_proj_finetune_ft64.json \
  --output_dir results/ss_ft64 \
  --data_dir '{"ObjaverseXL_sketchfab": {"base": "datasets/ObjaverseXL_sketchfab", "ss_latent": "datasets/ObjaverseXL_sketchfab/ss_latents/ss_enc_conv3d_16l8_fp16_64_view", "render_cond": "datasets/ObjaverseXL_sketchfab/renders_cond"}}'

# --- Stage 2: Shape (256 → 512 → 1024) ---
python train.py --config configs/gen/slat_flow_img2shape_dit_1_3B_256_bf16_proj_finetune.json \
  --output_dir results/shape_256 \
  --data_dir '{"ObjaverseXL_sketchfab": {"base": "datasets/ObjaverseXL_sketchfab", "shape_latent": "datasets/ObjaverseXL_sketchfab/shape_latents/shape_enc_next_dc_f16c32_fp16_256_view", "render_cond": "datasets/ObjaverseXL_sketchfab/renders_cond"}}'
# ...then _ft512 (shape_..._512_view) and _ft1024 (shape_..._1024_view) configs.

# --- Stage 3: Texture (256 → 512 → 1024) ---
python train.py --config configs/gen/slat_flow_imgshape2tex_dit_1_3B_256_bf16_proj_finetune.json \
  --output_dir results/tex_256 \
  --data_dir '{"ObjaverseXL_sketchfab": {"base": "datasets/ObjaverseXL_sketchfab", "shape_latent": "datasets/ObjaverseXL_sketchfab/shape_latents/shape_enc_next_dc_f16c32_fp16_256_view", "pbr_latent": "datasets/ObjaverseXL_sketchfab/pbr_latents/tex_enc_next_dc_f16c32_fp16_256_view", "render_cond": "datasets/ObjaverseXL_sketchfab/renders_cond"}}'
# ...then the _512 and _ft1024 texture configs with matching 512/1024 latent dirs.

🤗 Acknowledgements

Built upon Trellis.2, Direct3D-S2, and Trellis. Pose alignment uses MoGe-2. We sincerely thank the authors for their outstanding work.

📄 Citation

@article{li2026pixal3d,
    title={Pixal3D: Pixel-Aligned 3D Generation from Images},
    author={Li, Dong-Yang and Zhao, Wang and Chen, Yuxin and Hu, Wenbo and Guo, Meng-Hao and Zhang, Fang-Lue and Shan, Ying and Hu, Shi-Min},
    journal={arXiv preprint arXiv:2605.10922},
    year={2026}
}

📜 License

Released under the MIT License. Third-party components remain under their original terms; see NOTICE.