Overview

EPO (Edge-based Pose Optimization) is an optimization framework specifically designed to enhance Structure-from-Motion reconstructions generated by 3D foundation models, without the need for explicit feature tracks. We use Gradient Descent to learn a per-scene pose-MLP and a set of weights improving camera parameters and geometry. We demonstrate that EPO achieves geometric precision comparable to, or even exceeding, traditional Bundle Adjustment, while reducing runtime by up to 80% and operating efficiently on consumer-grade hardware.

Releases

v1.2 — EPO is now even faster 🚀

The optimization loop was profiled and rebuilt around two findings: most of the per-step cost was not in the Triton kernels but in redundant data movement and in kernel-launch dispatch (~600 tiny launches per step). v1.2 removes both — edge points, DT fields, and pad masks are now read by image index directly inside the fused kernels, the geometry prologue (pose MLP → intrinsics → unprojection, forward and backward) is captured once as a CUDA graph and replayed, and the default batch size goes 128 → 1024.

EPO is now 2.4× faster than v1.1 (133 → 318 it/s mean over the 38-scene benchmark set, RTX 4090) and 8.4× faster than the torch backend wall-clock. On bicycle (MipNeRF360), the EPO optimization itself now takes ~5 s.

Accuracy is preserved by construction: every change is bit-exact against the previous loop (same kernels, same FP order), verified by identical AUC@5/@3/@1 on all 38 benchmark scenes. The larger batch size can shift individual scenes slightly (the optimizer sees different mini-batch compositions), but dataset means stay within ±0.5 AUC@5. Kernel-level optimizations that measured faster but perturbed AUC (e.g. reduction-order changes) were deliberately rejected.

v1.1 — Dense depth via Any2Full

EPO refines depth only at the edge pixels it samples, so the depths.pth it exports is sparse (~8% of pixels) and its point cloud is correspondingly thin. v1.1 adds a depth-completion stage built on Any2Full: it takes EPO's refined (sparse) depths as guidance and completes them into full-resolution dense maps, giving you a dense point cloud on top of the refined poses — poses, intrinsics, and the sparse model are left untouched.

python demo_epo.py \
    --images_path bicycle/images \
    --output_path out/bicycle \
    --densify                         # New added to denisfy EPO output

This writes a dense_<model>_epo COLMAP model next to sparse_<model>_epo. See Densifying EPO's depths for the standalone API, the weights, and the point-count/DBSCAN caveats. Completion is a single feed-forward pass per image, so it adds only ~25 s for a 150-image scene — on bicycle (MipNeRF360, RTX 4090) the whole --densify run takes ~90 s end-to-end: ~65 s for VGGT + EPO and ~25 s for the densification on top.

Installation

Prerequisites

• Python 3.10
• CUDA-compatible GPU
• One of: Conda or python3-venv (any pip-only flow)

Option A — Conda Environment File

git clone --recursive https://github.com/mattiadurso/epo.git
cd epo

conda env create -f environment.yml
conda activate epo

Option B — Conda + Manual Pip Install

conda create -n epo python=3.10 -y
conda activate epo

pip install joblib \
            kornia \
            matplotlib \
            numpy \
            opencv-python \
            pandas \
            pycolmap \
            rerun-sdk \
            torch \
            torchvision \
            triton \
            tqdm

# Only needed to run the demo notebooks (demo.ipynb); not required for the library or demo_epo.py
pip install git+https://github.com/mattiadurso/mylib.git

ℹ️ triton is Linux-only and requires a CUDA build of torch. On systems without CUDA, install torch from the official selector first, then run the rest of the pip install line without triton — EPO will fall back to the PyTorch reference path (backend="torch").

Submodules

wrapper/ provides swappable drivers for several 3D foundation models, each a thin driver over a pristine git submodule under third_party/ plus the shared pycolmap-4 conversion helper wrapper/np_to_colmap.py and the common base class wrapper/base_wrapper.py. Select one via --model on demo_epo.py (default vggt); the full list lives in the WRAPPERS registry in wrapper/__init__.py:

Each wrapper is also runnable on its own for a quick smoke test — handy to check a single model end-to-end without going through demo_epo.py or a dataset config. It writes a COLMAP model + depths.pth to --output_path using the model's registered weights (override with --model_path); for batch runs across benchmark datasets use wrapper/run_for_dataset.py instead:

python wrapper/vggt_wrapper.py --images_path scene/images/1 --output_path out/sparse

Only vggt's submodule is needed to run EPO's own demo/reconstructions; EPO refines any reconstruction in the expected layout without any of them. third_party/lightglue is a further submodule needed only for VGGT's optional Bundle-Adjustment path (use_ba=True); the default feed-forward path — including demo_epo.py — never imports it. If you cloned without --recursive:

wrapper/any2full_wrapper.py (over third_party/Any2Full) is a different kind of driver and is therefore not in the WRAPPERS registry: it takes an existing COLMAP reconstruction whose depths are sparse — EPO's own export, whose depths.pth only carries depth at the sampled edge pixels — and completes them into dense maps, passing the poses through untouched. See Densifying EPO's depths.

git submodule update --init --recursive

To actually run a model (e.g. via demo_epo.py), also install its dependencies — see the top of each wrapper module for the exact extras and any model-specific gotchas (e.g. vggt_omega's checkpoint is gated on Hugging Face: request access, then pass a local path via --model_path). For VGGT itself, install only these extras — do not run pip install -r third_party/vggt/requirements.txt, as its pinned torch/numpy versions would downgrade and break the EPO environment:

pip install huggingface_hub einops safetensors

3DFM Results

We report models results averaged by dataset at mattiadurso.com/epo.

Usage

Step 1 — Prepare Input Images

📦 A few demo scenes can be downloaded here. These are already pre-processed with VGGT, thus steps 1 and 2 can be skipped.

Organize your images using the following structure. Images can be grouped by camera if multiple cameras are used:

bicycle/
└── images/
    ├── 1/                   # Images for Camera Group 1
    │   ├── _DSC8679.jpg
    │   ├── _DSC8680.jpg
    │   └── ...
    └── 2/                   # Images for Camera Group 2 (if more)
        ├── _DSC9001.jpg
        └── ...

Step 2 — Run a 3D Foundation Model

Run a 3DFM (e.g., VGGT) on your images and export the reconstruction in COLMAP format and the dense depth maps with the following layout:

bicycle/
└── sparse/
    ├── cameras.bin          # Camera intrinsic parameters
    ├── images.bin           # Camera extrinsics and image registration
    ├── points3D.bin         # Sparse 3D point cloud
    └── depths.pth           # Dense depth maps: torch.save'd dict
                             # {image_stem: {"depth": (H, W), "confidence": (H, W) (optional)}}

Step 3 — Run EPO

from epo import EPO

epo = EPO(
    reconstruction_path="bicycle/sparse",
    images_path="bicycle/images",
    depths_path="bicycle/sparse/depths.pth",
    backend="triton",                           # "torch" for the reference path
)
epo(early_stop="pose", gt_path="<path_to_gt>")  # gt_path optional — enables evaluation at runtime
epo.to_colmap("out/sparse", save_points=True)

End-to-end demo (3DFM → EPO)

demo_epo.py runs steps 2–3 for you: the selected --model (vggt by default; see the Submodules table for the full list) on a folder of images, then EPO directly on its in-memory output (EPO.from_ff), writing sparse_<model> and sparse_<model>_epo under --output_path:

python demo_epo.py \
    --images_path bicycle/images \
    --output_path out/demo \
    --model vggt \               # or vggt_omega, dvlt, da3, mapanything, pi3x
    --gt_path <path_to_gt>      # Optional — enables quantitative evaluation at runtime

Pass --model_output <dir> to reuse a previous run's reconstruction + depths.pth instead of re-running the model; both modes produce the same refinement. Pass --model_path <path/or/repo-id> to load the model from a local checkpoint instead of its default Hugging Face download.

Notes

• depths.pth keys must match the relative image paths without extension (e.g., cam_<j>/image_<i>).
• Ground truth data must be provided in COLMAP format, with matching image names (e.g., cam_<j>/image_<i>.jpg).

Programmatic use — feed-forward output

If you already have a 3DFM's output as tensors in memory, you can skip the COLMAP / depths.pth export and feed EPO directly:

from epo import EPO

# ff_data: dict keyed by "cam_id/image_name"
# Each value is a dict with the per-image tensors (already at images_size).
# Pose is world-to-camera (T_cw), matching COLMAP / PoseModule.
ff_data = {
    "cam_<j>/image_<i>.jpg": {
        "image":     image_tensor,      # (3, H, W) float in [0, 1]
        "depth":     depth_tensor,      # (H, W)
        "pose":      pose_tensor,       # (3, 4) or (4, 4), world-to-camera
        "intrinsic": intrinsic_tensor,  # (3, 3) pinhole
        # "confidence": conf_tensor,    # (H, W), optional
    },
    ...
}

epo = EPO.from_ff(
    ff_data,
    backend="triton",
)
epo()
epo.to_colmap("out/sparse", save_points=True)

All other EPO(...) kwargs are forwarded. Images and depths must already be at images_size (no internal resize). With single_camera_per_folder=True (default) all images under the same "cam_id/" folder share one jointly-optimized camera; otherwise each image gets its own. Every wrapper in wrapper/ (see the Submodules table) returns an EPO-ready ff_data built this way from its forward() method (see demo_epo.py).

Citation

If you find this work useful, please consider citing:

@inproceedings{durso2026epo,
  title     = {Boosting 3D Foundation Models with Edge-Based Pose Optimization},
  author    = {Mattia D'Urso and Christian Sormann and Mattia Rossi and Friedrich Fraundorfer},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026},
}

---