---

Rust workspace for clean-room 3D inference, contract inspection, and weight canonicalization.
CUDA-first runtime for Pi3, Pi3X, and TripoSR.

Lux3D / candle-3d

Table of Contents

• What is this?
• Key Features
• Demo
• Repository Layout
• Quick Start
• Performance
• System Requirements
• License

What is this?

Lux3D / candle-3d is a Rust 2024 workspace that provides:

• a Candle-first inference runtime
• contract inspection for supported model families
• canonical weight normalization and validation
• geometry export tooling for point clouds and meshes

Supported model families:

• pi3
• pi3x
• triposr

Key Features

• lux3d-core implements contracts, runtime loading, export logic, and weight validation.
• lux3d-cli exposes inspect, weights normalize, and run.
• lux3d-server-core exposes an OpenAI-style async HTTP API for Pi3, Pi3X, and TripoSR generation.
• Python baseline tooling lives in tools/python_baseline/README.md.
• Model-family-specific licensing can be inspected through the CLI before redistribution.
• Experimental: Vulkan and WGPU inference via --device vulkan / --device wgpu (build with --features vulkan / --features wgpu). Not production-ready — may be unstable, produce wrong results, or not work at all depending on hardware and drivers. CUDA is the only verified backend.

Demo

https://github.com/user-attachments/assets/0a9f766f-68d3-482a-9c94-c55d0c355808

Repository Layout

Quick Start

Workspace Checks

cargo metadata --no-deps
cargo run -p lux3d-cli -- --help

Inspect Contracts

cargo run -p lux3d-cli -- inspect --repo-root <runtime-root> pi3
cargo run -p lux3d-cli -- inspect --repo-root <runtime-root> pi3x
cargo run -p lux3d-cli -- inspect --repo-root <runtime-root> triposr

Normalize Canonical Weights

cargo run -p lux3d-cli -- weights normalize --repo-root <runtime-root> --raw-model-dir <raw-pi3-dir> --output-dir <canonical-pi3-dir> pi3
cargo run -p lux3d-cli -- weights normalize --repo-root <runtime-root> --raw-model-dir <raw-pi3x-dir> --output-dir <canonical-pi3x-dir> pi3x
cargo run -p lux3d-cli -- weights normalize --repo-root <runtime-root> --raw-model-dir <raw-triposr-dir> --output-dir <canonical-triposr-dir> triposr

Run Inference

# Pi3 -> PLY with an explicit canonical package directory (layout matches 3d/canonical-weights/pi3)
cargo run -p lux3d-cli -- run pi3 --model-path <canonical-pi3-dir> --source <input-sequence> --output <output-file.ply>

# Pi3X core -> PLY with an explicit canonical package directory (layout matches 3d/canonical-weights/pi3x)
cargo run -p lux3d-cli -- run pi3x --model-path <canonical-pi3x-dir> --source <input-sequence> --conditions <conditions-file> --output <output-file.ply>

# Pi3X VO -> PLY with an explicit canonical package directory (layout matches 3d/canonical-weights/pi3x)
cargo run -p lux3d-cli -- run pi3x --model-path <canonical-pi3x-dir> --source <input-video> --vo --chunk-size 8 --overlap 4 --conf-threshold 0.05 --inject-condition pose,depth,ray --output <output-file.ply>

# TripoSR -> OBJ with an explicit canonical package directory (layout matches 3d/canonical-weights/triposr)
cargo run -p lux3d-cli -- run triposr --model-path <canonical-triposr-dir> --source <input-image> --mc-resolution 256 --mc-threshold 25.0 --output <output-file.obj>

# Auto-download canonical packages from Hugging Face into the user cache
cargo run -p lux3d-cli -- run pi3 --source <input-sequence> --output <output-file.ply>

Experimental backends (Vulkan / WGPU): select a non-CUDA device with --device:

cargo run -p lux3d-cli --features vulkan -- run triposr --device vulkan --source <input-image> --output <output-file.obj>
cargo run -p lux3d-cli --features wgpu -- run pi3 --device wgpu --source <input-sequence> --output <output-file.ply>

These paths are experimental and not verified in CI. Inference may be unstable, produce incorrect output, or fail entirely depending on your GPU, driver, and OS. Use CUDA for production workloads.

Run the HTTP Server

cargo run -p lux3d-server -- --host 127.0.0.1 --port 8080
curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/v1/models

Generation endpoints accept multipart/form-data and return 202 Accepted with an async job object:

Common fields:

• model — required for point clouds (pi3 or pi3x), optional for meshes (triposr default)
• source — required upload (.zip frame archive, .mp4 video, or image)
• conditions — optional Pi3X conditions file
• options — optional JSON string with family-specific parameters

Workflow:

# 1. Create a mesh generation job
curl -X POST http://127.0.0.1:8080/v1/meshes/generations `
  -F "model=triposr" `
  -F "source=@input.png" `
  -F "options={\"mc_resolution\":256,\"mc_threshold\":25.0}"

# 2. Poll job status
curl http://127.0.0.1:8080/v1/generations/gen_...

# 3. Download completed asset
curl http://127.0.0.1:8080/v1/generations/gen_.../content --output mesh.obj

Embed the router in another Axum app:

use lux3d_server_core::{
    lux3d_for_server_builder::Lux3dForServerBuilder,
    lux3d_server_router_builder::Lux3dServerRouterBuilder,
};

let lux3d = Lux3dForServerBuilder::new().build()?;
let routes = Lux3dServerRouterBuilder::new()
    .with_lux3d(lux3d)
    .build()
    .await?;
let app = axum::Router::new().nest("/api/lux3d", routes);

Performance

Measured inference time ([stage] infer: preprocessing + neural pass, single iteration per process, weights excluded) on an NVIDIA GeForce RTX 3060 12 GiB, Windows (WDDM), candle wgpu/vulkan fork rev efb506c9, candle-3d (2026-09-06):

Warm-loop criterion (10 samples, steady state — the candle-bench protocol, one back-to-back window): vulkan 3.14 s / 4.90 s per iter (1.16×/1.20× of CUDA 2.72 s / 4.09 s), wgpu 8.21 s / 24.0 s (3.0×/5.9×). The biggest single lever was a fused LayerNorm+RoPE kernel (rope_layernorm.comp/.wgsl in the fork): one dispatch per q/k replaces the slow-path LayerNorm, both surrounding copies, and the 8-op rope chain at every rope site, cutting wgpu pi3 from 54.9 s to 8.21 s warm.

Correctness: outputs of every backend × model combination were verified against the CUDA reference meshes (bounding box / center / mean delta within 1% of max extent, vertex/face counts within 0.5% — all PASS); the fused kernel is additionally unit-tested exact to 0.0 diff against the composed form on both backends.

Memory behavior under repeated inference (10-iteration loop, same process):

• CUDA — flat; the decoder attention scores buffer is bounded to 768 MiB by query-axis chunking (LUX3D_MAX_SDPA_SCORES_BYTES), which removes a ~4 GiB transient that previously fragmented the CUDA memory pool and stalled a 12 GiB card ~100×.
• Vulkan — flat ~7.5 GiB; retention is bounded by the inflight byte budget (256 MiB × grace band 8) and a 2 GiB reusable GPU buffer pool (CANDLE_VK_INFLIGHT_GRACE, CANDLE_VK_POOL_MAX_BYTES). Batch caps are tuned from flush-reason profiling: transfer bytes 512 MiB (CANDLE_VK_MAX_BATCH_TRANSFER_BYTES), descriptor sets 8× dispatches — closing the batch per big activation copy cost ~4-9 ms of WDDM fence-signal latency per submission.
• WGPU — 10-iteration pi3x benchmark completes with a 10.4 GiB VRAM peak and zero errors (previously OOM'd by iteration ~3); free pool / recycle backlog / in-flight retention are each byte-capped (CANDLE_WGPU_POOL_MAX_BYTES, CANDLE_WGPU_INFLIGHT_MAX_BYTES).

Known gaps: the remaining Vulkan/CUDA warm gap (1.16–1.20×) is the structural WDDM submit tax (~4-9 ms per submission on a strictly dependent op chain) — a platform property code cannot reach; Linux without WDDM would close it. WGPU remains 3.0–5.9× behind warm (down from 13.9–19.5×), dominated by WGSL GEMM quality plus the same inter-pass WDDM gaps. A criterion harness for whole-model benches lives in crates/lux3d-core/benches/ (run one (backend, model) pair per process: LUX3D_BENCH_DEVICE=… LUX3D_BENCH_MODEL=… cargo bench -p lux3d-core --bench bench_main --features vulkan,wgpu).

System Requirements

• Rust 1.85 or newer
• Cargo with support for edition = "2024"
• Python 3.x for baseline tooling and normalization
• CUDA-capable NVIDIA hardware for verified runtime inference
• Vulkan and WGPU backends (via FerrisMind/candle wgpu/vulkan fork): verified for correctness and memory stability on RTX 3060 (see Performance); slower than CUDA, WGPU substantially so.
• Metal backend on macOS is not tested in this repository, but may be supported theoretically.
• Canonical model package directories supplied with --model-path, or downloadable from Hugging Face into the user cache

License

The code and documentation in this repository are licensed under Apache 2.0. See LICENSE.

Upstream model assets and canonicalized model artifacts keep their original licenses and usage restrictions. Review model-family-specific terms before redistribution.