SIH 2026 — UAV 3D Reconstruction & Benchmark Studio

Single-Pass Drone Video to Accurate 3D Model Generation System (SIH26158)
Comprehensive repository featuring the Production End-to-End 3D Reconstruction Pipeline (COLMAP, FastMap, Global SfM, VGGT, Georeferencing, Point Cloud Processing, Gaussian Splatting) alongside the Empirical Research Benchmark Studio (ORB-SLAM3, DROID-SLAM, DPVO, DUSt3R, MASt3R).

---

🌟 System Architecture

This repository contains two interconnected systems:

1. Production 3D Reconstruction Pipeline (reconstruction/)

A production-grade, state-machine orchestrated 3D reconstruction pipeline built across 10 developmental milestones:

• Input Ingestion & Quality Filtering: Automatic FPS sampling, Laplacian blur detection, and extreme brightness filtering.
• Dynamic Masking: Ultralytics YOLO11 segmentation to mask transient dynamic objects (vehicles, pedestrians).
• Adaptive Multi-Route Pose Engine: FastMap -> COLMAP Global SfM -> VGGT fallback hierarchy with strict photogrammetric validation gates.
• GPS / IMU Georeferencing: Sim(3) 7-DoF Umeyama alignment against UTM/local coordinates with ellipsoidal altitude referencing.
• Point Cloud Optimization: Statistical outlier removal (k-NN), radius outlier removal, and voxel grid downsampling.
• Gaussian Splatting / Splatfacto: Nerfstudio dataset formatting, camera convention transforms (+X right, +Y up, +Z back), and Gaussian Splat training wrapper.
• Full Orchestrator (reconstruction/pipeline.py): PipelineStateMachine with step resumption (--resume), failure isolation, resource monitoring, and comprehensive processing_report.json generation.

2. Research Benchmark Studio (pipelines/, models/, evaluation/, ui/)

An empirical evaluation suite comparing Visual SLAM and Feedforward 3D Foundation Models:

• Models Evaluated: ORB-SLAM3, DROID-SLAM, DPVO, DUSt3R, and MASt3R.
• Hybrid Pipelines (A–E): Pairing visual odometry with dense geometric regression.
• Hardware Telemetry & Metrics: Real-time measurement of ATE RMSE, RPE, Chamfer distance, CPU %, VRAM, and FPS.
• Interactive UI (ui/app.py): Streamlit & Plotly 3D visual studio with side-by-side comparison, 3D point cloud explorer, and trajectory viewer.

---

📂 Repository Layout

sih-drone-3d/
├── reconstruction/                 # Step 1-10 Production Reconstruction Pipeline
│   ├── pipeline.py                 # Central PipelineStateMachine Orchestrator (Step 10)
│   ├── masking/                    # YOLO11 dynamic object segmentation & masking
│   ├── pose_engine/                # Adaptive pose engine (FastMap, Global SfM, VGGT)
│   ├── colmap/                     # Feature extraction, matching, and reconstruction
│   ├── validation/                 # Photogrammetric quality gating & camera pose validation
│   ├── georeferencing/             # GPS/IMU timestamp sync, Sim(3) alignment, CRS transforms
│   ├── pointcloud/                 # Point cloud filtering (Statistical, Radius, Voxel) & PLY export
│   └── splatting/                  # Nerfstudio Splatfacto dataset generation & splat export
├── tests/                          # Automated test suite (139 tests across Steps 1–10)
├── dataset/                        # Drone datasets, sample generator, video, frames, metadata
│   ├── video/                      # Drone video files (drone_video.mp4)
│   ├── frames/                     # Extracted PNG frames with metadata.json
│   ├── metadata/                   # GPS & IMU telemetry logs, camera intrinsics
│   ├── ground_truth/               # Ground truth TUM poses and 3D point clouds
│   ├── frame_extractor.py          # Configurable frame extraction CLI (1, 2, 5, 10 FPS)
│   └── sample_generator.py         # Realistic 1080p drone dataset synthesizer
├── models/                         # Research Model Adapters
│   ├── base_adapter.py             # BaseModelAdapter & BenchmarkResult dataclass
│   ├── orbslam3/                   # ORB-SLAM3 adapter, config.yaml, Linux Dockerfile
│   ├── droidslam/                  # DROID-SLAM dense SLAM adapter & config.yaml
│   ├── dpvo/                       # DPVO deep patch visual odometry adapter
│   ├── dust3r/                     # DUSt3R ViT pointmap regression adapter
│   └── mast3r/                     # MASt3R metric 3D feature matching adapter
├── pipelines/                      # Hybrid Benchmark Pipelines (A–E)
│   ├── pipeline_a_orbslam_depth.py # Video -> ORB-SLAM3 -> Dense Depth -> Point Cloud
│   ├── pipeline_b_droid_dense.py   # Video -> DROID-SLAM Dense SLAM -> 3D Model
│   ├── pipeline_c_dpvo_dense.py    # Video -> DPVO Odometry -> Dense Reconstruction
│   ├── pipeline_d_dust3r.py        # Video -> DUSt3R Multi-View Alignment
│   ├── pipeline_e_mast3r.py        # Video -> MASt3R Metric 3D Matching
│   └── hybrid_pipeline_runner.py   # Full benchmark runner & summary generator
├── evaluation/                     # Quantitative Evaluation Metrics
│   ├── metrics_runtime.py          # Hardware profiling (CPU, RAM, latency, FPS)
│   ├── metrics_trajectory.py       # ATE RMSE & RPE evaluation with Sim(3) alignment
│   ├── metrics_reconstruction.py   # Chamfer distance, point density, bbox volume
│   └── qualitative_eval.py         # 6-axis qualitative evaluation rubric
├── visualization/                  # 3D Visualizers
│   ├── trajectory_3d.py            # Plotly 3D camera trajectory & frustum visualizer
│   ├── depth_colormap.py           # Scientific colorized depth maps (Turbo/Viridis)
│   ├── pointcloud_viewer.py        # Interactive 3D point cloud explorer
│   └── side_by_side.py             # Synchronized multi-model comparison matrix
├── ui/                             # Interactive Streamlit Web Studio
│   ├── app.py                      # Interactive Benchmark Studio
│   └── styles.css                  # Modern glassmorphic theme styling
├── runs/                           # Benchmark execution outputs & summary CSV
├── BENCHMARK_RESULTS.md            # Benchmark analysis & architectural recommendations
├── MODEL_STATUS.md                 # Hardware diagnosis & model status audit
├── SIH26158_Master_Technical_Context.docx
└── SIH_Drone_3D_Reconstruction_Technical_Proposal.docx

---

🚀 Quick Start

1. Run the Production Pipeline (Step 10)

Execute the complete end-to-end reconstruction pipeline on drone video or image directory:

python -m reconstruction.pipeline --input dataset/video/drone_video.mp4 --output output/ --method auto

Or with GPS georeferencing and resumption:

python -m reconstruction.pipeline --input dataset/frames/ --gps dataset/metadata/gps.csv --output output/ --resume

2. Run the Benchmark Studio (Streamlit UI)

Launch the interactive web benchmark studio:

streamlit run ui/app.py

3. Run Automated Tests

Execute the complete verification test suite (139 tests):

pytest -v