Welcome to pyvisim!

pyvisim is a computer vision library for computing image similarities using traditional and deep learning methods.

📚 Documentation: https://mechacritter.github.io/Python-Visual-Similarity/

Table of Contents

• Status
• Overview
  • Accelerated Computation
  • Examples
• Installation
• Contributing
• Get in Touch
• License

Status

🚀 The first stable release v1.0.0 is out!

Overview

The goal of pyvisim is to become the largest collection of image similarity metrics, varying from traditional methods like PSNR, SSIM, Fisher Vectors, and VLAD to deep learning methods like CLIP and Siamese Networks. Furthermore, advanced image similarity search and reranking algorithms are provided to allow users to refine the search results as desired. For more details, please refer to the documentation provided.

Currently, one would need to install numerous libraries to use all features mentioned above (for example, scikit-image + opencv-python for Fisher Vectors, SSIM, open-clip for CLIP Embedder, and faiss for Approximate Nearest Neighbors Search). pyvisim attempts to close this gap by implementing as many metrics as possible using only numpy, scipy (for conventional metrics), and optionally torch (for deep learning metrics) as core dependencies, plus making them more user-friendly with a simple Object-Oriented code design.

[!TIP] hnsw is provided as a built-in ANNs algorithm, backed by hnswlib. However, a FAISS index can also be passed to the image store in case you would like to use other search algorithms with this library. Just note that pyvisim does not install faiss.

Accelerated Computation

Cython kernels and C++ libraries are used for some metrics to accelerate computation significantly compared to all reference libraries on the CPU. See, for example, benchmark results of the SSIM implementation.

Examples

Structural Similarity (see documentation here):

from pyvisim.dense.structural import SSIM

ssim = SSIM()
similarity_score = ssim.similarity_score(image1, image2)
print(f"Similarity Score: {similarity_score}")

One-Shot similarity computation using the CLIPEmbedder(see documentation here):

from pyvisim.neural_networks import ClipEmbedder

# Declare the Clip Embedder
embedder = ClipEmbedder()

# Compute the similarity score. By default, cosine similarity is used.
similarity_score = embedder.similarity_score(image1, image2)
print(f"Similarity Score: {similarity_score}")

Image retrieval (see documentation here):

from pyvisim.neural_networks import ClipEmbedder
from pyvisim.retrieval.image_store import InMemoryImageEmbeddingStore

embedder = ClipEmbedder()

image_store = InMemoryImageEmbeddingStore(
    image_paths=train_image_paths,
    embedder=embedder,
    search_index="hnsw",
    index_params={"graph_degree": 16, "build_candidates": 200},
)
image_store.build_store()  # embeds the gallery and builds the index

candidates = image_store.retrieve_top_k_similar(image, k=5)[0]  # one Candidate per match
for candidate in candidates:
    print(candidate.path, candidate.score)

The alpha query expansion and the k-reciprocal re-rankingcan additionally be used to refine the retrieval results, improving mean Average Precision (see the documentation):

from pyvisim.retrieval.reranking import KReciprocalReranker

pool = image_store.retrieve_top_k_similar(image, k=100, query_expansion=True)[0]
best = KReciprocalReranker(image_store).rerank(pool, top_k=5)

For more examples, please refer to the tutorials.

Installation

To install the slim version (without deep learning features):

pip install pyvisim

Additional features include (note: these pull in heavy dependencies like torch):

# For deep learning features and the OxfordFlowerDataset
pip install "pyvisim[nn]"

All experiments in this project were made on the Oxford Flower Dataset [7], for which I have created a custom dataset class. For more details on the dataset, please refer to the documentation.

Contributing

See the contributing guidelines.

Get in Touch

Feel free to:

• Open an issue on GitHub.
• Write me an email at mechacritter@outlook.com.
• Connect on LinkedIn to follow my work and share your thoughts.

License

This project is licensed under the terms of the MIT license.