UniCV

UniCV is a unified, extensible framework for computer vision models that operate across heterogeneous input and output representations. It wraps state-of-the-art models — depth estimators, Gaussian splat predictors, mesh generators, and more — behind a single, composable VisionModule interface.

---

Philosophy

Modern computer vision has fragmented into dozens of incompatible APIs: each model ships with its own preprocessing, its own output format, and its own integration burden.

The architecture and design philosophy of UniCV is inspired by modular deep learning ecosystems such as pytorch and HuggingFace's transformers, as well as recent efforts toward foundation models and generalist perception systems in computer vision. Rather than prescribing fixed pipelines (e.g. RGB → Depth or RGB → Mesh), UniCV abstracts vision algorithms as composable transformations between representation spaces.

The core abstraction of UniCV is VisionModule, which defines a standardized interface for mapping any combination of visual input modalities to any combination of output modalities. These modalities include, but are not limited to:

• RGB images
• Depth maps
• Point clouds
• Meshes
• Gaussian splats and other implicit or semi-implicit scene representations

Concrete vision algorithms—such as monocular depth estimation, RGB-to-point-cloud reconstruction, or RGB-D refinement—are implemented as subclasses of this abstract interface. Existing models available online (e.g. DepthPro, MiDaS, CDM, or point-cloud reconstruction networks) can be redefined within this framework without altering their internal logic, allowing them to be seamlessly integrated into a shared system.

UniCV is hence designed to accommodate the full spectrum of modern 3D perception: classical CNNs, ViT-based backbones, implicit neural representations, Gaussian splatting, and diffusion-based generation.

This abstraction enables UniCV to decouple input modality, latent processing, and output representation, encouraging reuse, composition, and extension of vision algorithms. Models may share encoders, latent spaces, or decoders, and can be combined or chained to support progressive or multi-stage reconstruction pipelines.

The conceptual motivation for UniCV is closely aligned with the emergence of foundation models for perception, where a single system is expected to reason across tasks, representations, and data sources. By enforcing a common interface at the representation level, UniCV facilitates cross-representation supervision, multi-task learning, and interoperability between otherwise incompatible vision methods.

Motivation

UniCV aims to support vision systems that are:

1. Modality-agnostic, capable of ingesting arbitrary combinations of visual inputs (e.g. RGB, depth, point clouds) without architectural redesign.
2. Representation-agnostic, able to emit multiple scene representations from a shared latent abstraction.
3. Algorithm-agnostic, allowing existing and future computer vision models to be wrapped, extended, or replaced under a common interface.
4. Composable, enabling complex pipelines to be constructed by chaining or jointly training multiple VisionModule-based modules.
5. Extensible, supporting both classical CV algorithms and modern deep learning approaches, including implicit scene representations and neural rendering techniques.
6. Foundation-ready, serving as an architectural substrate for training large, generalist vision models capable of cross-task and cross-representation transfer.

In addition to standard convolutional and Transformer-based architectures, UniCV is designed to accommodate emerging paradigms such as implicit neural representations, Gaussian splatting, and hybrid geometric–neural pipelines, enabling a unified experimental platform for next-generation 3D perception systems.

---

Quick Start

pip install unicv                  # core (torch only)
pip install unicv[pretrained]      # + huggingface_hub, timm, safetensors

from unicv.models.depth_anything_3 import DepthAnything3Model

model  = DepthAnything3Model.from_pretrained(variant="vit_l")
result = model(rgb=image_tensor)   # {Modality.DEPTH: (B, 1, H, W)}

Every model follows the same interface — only input_spec and output_modalities differ. See the Getting Started guide for all models.

---

Implemented Models

See the full model catalogue for planned models.

---

Documentation

Read the full documentation →

---

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines on adding new models, writing tests, and submitting pull requests.

Code of Conduct

This project follows the Contributor Covenant. By participating, you agree to uphold a welcoming and respectful environment for everyone.

License

MIT — see LICENSE.