ViPE: Video Pose Engine for Geometric 3D Perception

TL;DR: ViPE is a useful open-source spatial AI tool for annotating camera poses and dense depth maps from raw videos!

ViPE estimates camera intrinsics, camera motion, and dense near-metric depth maps from unconstrained raw videos, including pinhole, wide-angle, and 360-degree panorama footage.

News

• 2026/09: Added long-sequence SLAM (pose_only_long) for arbitrarily long videos, with GPU/CPU memory bounded by a sliding keyframe window instead of growing with video length. Uses the MoGe v2 (moge2-l) keyframe-depth prior by default.
• 2026/06: 🚀🚀🚀 Released ViPE 1.2.0: 2.7x speed-up with no loss of accuracy, enabled by CUDA fused kernels, model and pipeline caching, prefetching, and other optimizations.
• 2026/05: Merged Panorama estimation pipeline & bump release version to 1.0.0.
• 2026/01: Integration with Depth-Anything 3 for depth estimation (use dav3 pipeline).
• 2025/10: Add support to run on wide-angle videos.
• 2025/09: Add support to run Lyra pipeline.
• 2025/08: Initial release of ViPE.

Installation

# From PyPI
pip install nvidia-vipe

# From source (conda for CUDA/native deps, uv for the Python env)
conda env create -f envs/cu128.yml
conda activate cu128
uv sync
uv run vipe infer YOUR_VIDEO.mp4

See docs/installation.md for details (dev/docs dependency groups, etc).

Long-Sequence Videos

For videos too long for the default pipeline's fixed-size keyframe buffer, use the pose_only_long pipeline: it retires old keyframes to a compact trajectory ledger as it streams, so GPU and CPU memory stay bounded (roughly constant, not growing with video length) regardless of how many frames the video has. It uses the MoGe v2 (moge2-l) keyframe-depth prior by default.

# Single video
uv run vipe infer YOUR_LONG_VIDEO.mp4 -p pose_only_long -o vipe_results/

# A folder of videos: every .mp4 in the folder is processed in turn, through a single
# loaded model instance, writing each video's trajectory to vipe_results/pose/<name>.npz
uv run vipe infer YOUR_VIDEO_FOLDER/ -p pose_only_long -o vipe_results/

License

This project will download and install additional third-party models and softwares. Note that these models or softwares are not distributed by NVIDIA. Review the license terms of these models and projects before use. This source code, except for the Unik3D part (which is under the BY-NC-SA 4.0 license) , is released under the Apache 2 License.