Prism — Multimodal AI Search Dashboard

Prism is a local-first, zero-API-key multimodal content indexing and search dashboard. It runs entirely in the browser using WebGPU — no server, no cloud, no data leaves your machine. Drop images, audio, or video files and Prism will caption, transcribe, embed, and index them for instant semantic search.

Quick Start

# Clone the repository
git clone <repo-url>
cd prism

# Install dependencies
npm install

# Start the dev server
npm run dev

Open http://localhost:5173 in a Chromium-based browser with WebGPU enabled.

Browser Requirements

Note: Firefox and Safari do not fully support WebGPU yet. Prism will show a warning if WebGPU is unavailable.

How It Works

Three AI Models

Prism uses three AI models, each running in its own dedicated Web Worker via Transformers.js v4:

Models load lazily on first use — not on app startup.

Ingest Workflow

1. Drop files onto the ingest view (images, audio, or video)
2. Images → thumbnail generation → Florence-2 (caption + OCR) → Jina CLIP embedding → saved to IndexedDB
3. Audio/Video → audio extraction (mono 16kHz) → Whisper transcription → Jina CLIP embedding → saved to IndexedDB

Search Workflow

1. Text search: Type a natural language query → Jina CLIP embeds the query → cosine similarity against all indexed items
2. Image search: Drop or select an image → Jina CLIP embeds the image → cosine similarity search
3. Results ranked by similarity score, displayed as a grid

JSON Export

Click the Export button in the top bar to download all indexed data as a JSON file. Embeddings are serialized as regular arrays. The file is timestamped (e.g., prism-export-2026-03-24.json).

Tech Stack

Project Structure

src/
├── workers/         # Web Workers (Florence-2, Whisper, Jina CLIP)
├── lib/             # Utilities (db, vectorStore, export, workerBridge, fileUtils)
├── store/           # Zustand stores (model, index, UI)
├── components/      # React components
│   ├── layout/      # AppShell, Sidebar, TopBar
│   ├── ingest/      # DropZone, FileQueue, FileQueueItem
│   ├── search/      # SearchBar, ResultCard, ResultsGrid
│   ├── models/      # ModelStatusBar, ModelBadge
│   └── shared/      # Toast, EmptyState, ProgressBar, ErrorBoundary
├── hooks/           # Pipeline hooks (useIngestPipeline, useAudioPipeline, useSearch)
├── App.jsx
├── main.jsx
└── index.css

Known Limitations

• WebGPU only — no CPU/WASM fallback in v1
• Brute-force search — scales to ~10k items; approximate nearest-neighbor planned for v2
• No persistence outside browser — IndexedDB is per-origin; clearing browser data deletes the index
• Large files — processing very large images (>10MB) or audio (>5min) may be slow depending on GPU
• Single tab — model memory is per-tab; multiple tabs will each load their own models
• No server — no sync, sharing, or multi-device support

Architecture

For detailed architecture documentation, see docs/design/DESIGN.md.

License

This project is provided as-is for personal and educational use.