Introduction

Visual prompting offers a powerful alternative to traditional training. Instead of curating thousands of labeled images, you simply show the model one or a few examples of what you are looking for. The model effectively "learns" instantly, detecting and segmenting similar objects in new images or live video streams without retraining.

Key Features

• Library & Application: A unified framework providing a modular Python library for research and development, and a Full-Stack Application for deploying those algorithms on live video streams.
• Simple & Modular API: A composable design where developers can mix and match components (backbones, matchers) to create custom pipelines.
• Algorithms & Models: A wide collection of ready-to-use zero-shot and few-shot algorithms (e.g., SAM 2, Matcher, GroundedSAM) and foundation models.
• Hardware Acceleration: Built-in support for model optimization and export to OpenVINO™ for fast inference on Intel hardware (CPU, GPU, NPU).
• Multiple Backends: Seamless switching between PyTorch for flexibility/research and OpenVINO for optimized deployment.

Getting Started

Geti Instant Learn consists of two core components:

• Python Library: The foundation for research and zero/few shots algorithm development.
• Full Stack Application: Leverages the library to enable real-time inference on live streams, video files, and images.

Prerequisites

• Python 3.13
• uv (Python package manager)
• Just (Command runner)
• Node.js (v24.2.0) (Required only for the UI Application)
• Docker (Optional, for containerized deployment)

Geti Instant Learn Library

Install the library:

cd library
uv sync --extra xpu    # Intel XPU (recommended)
uv sync --extra cpu    # CPU only
uv sync --extra cuda    # CUDA support

Or with pip:

pip install ./library[xpu]  # or [cpu], [cuda]

SAM3: Zero-Shot Text Prompting

SAM3 performs zero-shot segmentation using text prompts (category names) or bounding boxes — no reference mask needed. You provide a list of categories you want to segment in any image.

from instantlearn.models import SAM3
from instantlearn.data import Category, Sample

# Initialize SAM3 with automatic device selection
model = SAM3()

# SAM3 is zero-shot — no fit() required. Just provide categories per sample.
predictions = model.predict([
    Sample(image_path="library/examples/assets/coco/000000286874.jpg", categories=[Category(0, "elephant")]),
    Sample(image_path="library/examples/assets/coco/000000173279.jpg", categories=[Category(0, "elephant")]),
])

Tip: Calling model.fit(sample) is optional for SAM3. If called, the fitted categories are reused for all subsequent predict() calls so you don't need to specify categories on every target sample. If not called, categories are taken from each target sample directly.

For more examples of SAM3 capabilities, see the SAM3 aerial & maritime notebook.

Since SAM3 requires a text prompt for every sample (unless fit() is used), this is where Matcher comes in — you fit once with a reference mask (one-shot) and predict on any number of new images without providing prompts again.

Matcher: One-Shot Visual Prompting

from instantlearn.models import Matcher
from instantlearn.data import Sample

# Initialize Matcher with automatic device selection
model = Matcher()

# Create reference sample (auto-loads image and mask from paths)
ref_sample = Sample(
    image_path="library/examples/assets/coco/000000286874.jpg",
    mask_paths="library/examples/assets/coco/000000286874_mask.png",
)

# Fit once on reference
model.fit(ref_sample)

# Predict on multiple target images — no prompts needed
predictions = model.predict([
    "library/examples/assets/coco/000000390341.jpg",
    "library/examples/assets/coco/000000173279.jpg",
    "library/examples/assets/coco/000000267704.jpg",
])

# Access results for each image
for pred in predictions:
    masks = pred.masks  # Predicted segmentation masks

Models select a compatible device automatically. See Manual Device Selection to select a specific physical device.

For interactive mask generation with SAM, CLI usage, and benchmarking, see the Library README.

Geti Instant Learn Application

Full-stack web interface for real-time inference.

Configure Hugging Face access as described in the Application Documentation

Deploy models on live video streams, cameras, and video files.

just device=xpu application/dev   # (device can be "xpu", "cuda", or "cpu")

Access at: http://localhost:3000

View Application Documentation →

Supported Models and Algorithms

Geti Instant Learn supports a variety of foundation models and visual prompting algorithms, optimized for different performance needs.

Visual Prompting Algorithms

Foundation Models (Backbones)

Documentation

Community

• To report a bug or submit a feature request, please open a GitHub issue.
• Ask questions via GitHub Discussions.

License

Geti Instant Learn is licensed under the Apache License 2.0.

FFmpeg is an open source project licensed under LGPL and GPL. See https://www.ffmpeg.org/legal.html. You are solely responsible for determining if your use of FFmpeg requires any additional licenses. Intel is not responsible for obtaining any such licenses, nor liable for any licensing fees due, in connection with your use of FFmpeg.