Overview

OrthoTrack is a training-free system for continuous, absolute, metrically scaled 6-DoF UAV trajectory estimation using public orthophotos and surface models as map priors—without GPS and without post-hoc alignment.

---

🚀 How it works

1. Keyframe Localization — RoMa matches the UAV image against a DOP orthophoto crop. Correspondences are lifted to metric 3D via the DSM and solved with PnP (RANSAC).
2. Inter-Frame Tracking — Lucas-Kanade optical flow propagates the 2D–3D correspondences to subsequent frames, yielding an absolute metric pose at every frame.
3. Re-localization — When tracked points degrade, a new keyframe triggers another DOP match to anchor the trajectory.

---

🛠️ Installation

Requirements:

• Python ≥ 3.10
• CUDA GPU (required for the dense feature matcher)
  • ≥ 8 GB VRAM — use --fine_matcher turbo (or fast)
  • ≥ 16 GB VRAM — use default --fine_matcher precise
• conda or venv for environment management

git clone https://github.com/cvg/orthotrack.git
cd orthotrack

conda create -n orthotrack python=3.10 -y
conda activate orthotrack
# Install a CUDA build of PyTorch for your platform first if needed:
#   https://pytorch.org/get-started/locally/
bash setup.sh

setup.sh installs deps from requirements.txt (including Open3D, NumPy, and OpenCV versions needed for mesh rendering), clones RoMaV2 into thirdparty/, and installs it editable. RoMaV2 weights download on the first tracker run (torch.hub).

For headless mesh render (dataset/create_movingdrone.py --render), install EGL/OpenGL system libs if needed:

sudo apt-get install -y libegl1 libgles2 libgl1

⚠️ Set PYTHONPATH=. when running Python scripts from the project root.

---

📊 Datasets

See dataset/README.md for MovingDrone, UAVD4L, and UAVScenes. To author new MovingDrone scenes (mesh + GES trajectory, or frames + priors), use dataset/create_movingdrone.py — see dataset/movingdrone.md.

Minimal MovingDrone download (local path is always scenes/; the server hosts files under sequences/):

PYTHONPATH=. python -c "
from dataset.MovingDrone import MovingDrone
MovingDrone(dataset_dir='data/MovingDrone', sequences=['airport10'],
            predownload=True, load_dop=True, load_dsm=True)
"

---

🏃 Running OrthoTrack

CLI defaults: --fine_matcher precise, --coarse_matcher base, --skip_frames 1, --max_image_dim 1920.
Examples below use turbo where noted for GPUs with ~8 GB VRAM.

1. On a Dataset Scene (recommended first run)

PYTHONPATH=. python scripts/run_tracking.py \
    --sequence_dir data/MovingDrone/scenes/airport10 \
    --fine_matcher turbo \
    --output outputs/tracking/airport10 \
    --end_frame 30 \
    --save_keyframe_vis

Drop --end_frame for the full video. On ≥16 GB VRAM, omit --fine_matcher turbo to use the default precise matcher.

2. On Custom User Data

Provide an MP4 (or frame folder), a GeoTIFF DOP, and a GeoTIFF DSM:

bash scripts/run_custom_data.sh path/to/video.mp4 path/to/ortho.tif path/to/dem.tif outputs/my_custom_flight

Or directly:

PYTHONPATH=. python scripts/run_tracking.py \
    --footage path/to/video.mp4 \
    --dop path/to/ortho.tif \
    --dsm path/to/dem.tif \
    --output outputs/my_custom_flight \
    --fine_matcher turbo \
    --save_keyframe_vis

Place intrinsics.json next to the video (or pass --intrinsics) when camera calibration is known; otherwise add --force_calibration.

3. Benchmarking on the Test Set

Loops over test scenes in splits.json:

PYTHONPATH=. python scripts/run_orthotrack_benchmark.py \
    --data_root data/MovingDrone \
    --output_dir outputs/benchmark \
    --fine_matcher turbo

Extra run_tracking.py flags (e.g. --save_keyframe_vis, --flow_method raft) are forwarded.

4. Evaluating Foundation Models & SLAM Baselines (optional)

Requires separate third-party installs — not covered by setup.sh. See EVALUATION.md.

---

✅ Best practices for tracking quality

These settings matter more than matcher speed for accurate trajectories:

Interactive demo: Hugging Face Space — bundled examples use 0.2 m/px geodata and show live resolution warnings for custom uploads.

---

🔍 Common CLI options

Run PYTHONPATH=. python scripts/run_tracking.py --help for the full argument list.

---

📁 Outputs

After processing a sequence, all data is saved to your specified --output directory:

• results.csv — Per-frame 6-DoF absolute metric pose estimates.
• results.json — Comprehensive log containing processing times, device info, and configurations.
• tracking_results.png — An overhead trajectory plot of the flight path.
• keyframes/ — If --save_keyframe_vis is enabled, visualization images for each keyframe trigger.

---

🎬 LoD Mesh Overlay Video

OrthoTrack includes a standalone script to generate a video with 3D building wireframes dynamically projected onto the drone footage using the estimated 6-DoF poses.

Example overlays are shown on the project page. Generate one locally after tracking:

On a Dataset Sequence

PYTHONPATH=. python scripts/visualize_results.py \
    --sequence_dir data/MovingDrone/scenes/airport10 \
    --results outputs/tracking/airport10/results.json \
    --output outputs/lod_overlay.mp4

On Custom User Data

If you are using custom data, you can provide the video and a custom LoD mesh directly (.npz, .obj, or .ply):

PYTHONPATH=. python scripts/visualize_results.py \
    --video path/to/video.mp4 \
    --lod path/to/custom_mesh.ply \
    --results outputs/my_custom_flight/results.json \
    --output outputs/lod_overlay_custom.mp4

📜 Citation

If you use OrthoTrack or MovingDrone in your research, please cite:

@inproceedings{dhaouadi2026orthotrack,
  title     = {OrthoTrack: Continuous 6-DoF UAV Trajectory Estimation Anchored in Public Orthophotos},
  author    = {Dhaouadi, Oussema and Bauer, Zuria and Meier, Johannes and Wysocki, Olaf and Pollefeys, Marc and Cremers, Daniel},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}

⚖️ License

This code is released under the CC BY-NC 4.0 license (non-commercial use only). Third-party components retain their original licenses.