ImageSearch

A semantic image search engine over the Amazon Products 2023 dataset, powered by local CLIP embeddings and background-removal inference — all running on local GPU via ONNX Runtime + DirectML (Also CUDA). No external APIs or cloud inference where used in this application.

Application is still in development and many things with be improved in future

---

How It Works

1. Setup pipeline — on first run, downloads models, tokenizer files, and the dataset from Hugging Face automatically.
2. Data seeding — the dataset is parsed, converted from .arrow to JSON files, and inserted into MS SQL Server. For each product, the pipeline:
   • Strips the background from the product image using RMBG-2.0 (background removal model), so embeddings focus on the product itself rather than noise and reduces vector size.
   • Generates a CLIP image embedding from the cleaned image for image based search.
   • Generates a CLIP text embedding from the product title for meaning based search. e.g: Apple Airpods, Samsung EarBuds -> Earphone, Headphone, ect...
   • Stores metadata (title, price, category, etc.) in MS SQL Server.
   • Stores both vectors in Qdrant.
3. Search — at query time, your input (text or image) is embedded using the same CLIP models and compared against both vector collections. Results are scored and ranked using a simple weighted fusion:

The top 10 products by combined score are returned.

---

Models

All models run locally via ONNX Runtime with DirectML GPU acceleration.

---

Tech Stack

• .NET 8 — ASP.NET Core Web API, layered architecture, EF Core
• ONNX Runtime + DirectML — local GPU inference (Works with CUDA too)
• MS SQL Server — product metadata storage
• Qdrant — vector similarity search (separate collections for image and text vectors)
• Hugging Face — model and dataset source

---

Dataset

AMAZON-Products-2023 — a large-scale Amazon product catalog with titles, images, categories, and pricing metadata.