Home Security Intelligence

Turn "dumb" security cameras into an intelligent threat detection system — 100% local, no cloud APIs required.

📸 Static Screenshots

Dashboard	Timeline	Entities

Alerts	Analytics	AI Performance

Profiling	Tracing	Operations

What You Get: AI-Powered Risk Reasoning

The brain of this system is a vision-language model that assesses every event — VlmAnalyzer asks the ai-vlm llama.cpp serve for a verdict on each batch. Model identity is config (VLM_MODEL_PATH), not code. It runs entirely on your hardware:

Also included:

• Real-time dashboard — camera grid, activity feed, risk gauge, telemetry
• Detections → events — time-window batching turns many frames into one explained "event"
• Identity lookups — faces, license plates and person re-ID matched against your own registrations, in-process, alongside every VLM verdict
• Local-first — runs on your hardware; footage stays on your network
• First run — the API returns 503 for everything except setup and health until you register the first admin in the dashboard; that's the setup guard working, not a broken install

Video Analytics Features

The system provides comprehensive video analytics capabilities:

Detection and Analysis

Zone Intelligence

Person and Vehicle Identification

Analytics and Reporting

All four live on the Analytics API:

AI Model Zoo

models.yml is the single source of truth for every model the system downloads and loads — name, download size, VRAM footprint, and which service runs it. The tables below are drawn from it, except where a model runs inside the gateway's Triton process instead of the backend model zoo (that process is budgeted in VRAM Requirements; the Triton-served rows' vram_mb — e.g. osnet 100, threat 300 — cover only the backend-side load).

Two AI Services

The VLM runs in the default compose set (up -d starts it; until UR-18 it sat behind a compose profile that had to be named explicitly) and the backend soft-depends on it (docker-compose.prod.yml:639-651 lists postgres, redis, ai-gateway and go2rtc, never ai-vlm; the dial is the AI_VLM_URL env var at :554 — the backend has no depends_on entry for it, so the backend degrades instead of failing to boot). ai-gateway's Triton repository holds exactly {yolo26, reid, threat}; GATEWAY_MODEL_SET accepts only vlm and hard-raises on anything else. ai-vlm is the only LLM service; there is no ai-llm.

Object Detector

Three YOLO26 variants ship (n/s/m; m is the default). Measured end-to-end latency, RTX A5500 at 640x640, batch 1 — details and throughput curves in YOLO26 Benchmarks:

Lookup Models

Three in-process lookup legs run alongside every VLM verdict and answer identity questions (whose face, which plate, which person) against your own registrations. The rest of the catalogue backs those legs and the Triton models the gateway serves. Backend-zoo models become resident at boot when BACKEND_MODEL_PRELOAD=true loads their enabled: true + preload: true rows; nothing loads them on demand at runtime, and a leg whose model is absent answers unavailable instead. Nothing evicts: the zoo has no eviction pass, so never_evict/priority rows are parsed but have no consumer (see backend/main.py's preload note). Complete models.yml inventory:

Model Priority System

Every model carries a priority in models.yml — the intended eviction order for a future VRAM-budget pass. No eviction pass exists today, so the field is parsed into ModelConfig and read by nobody (the contract is documented in docs/reference/models.md):

• CRITICAL / HIGH / MEDIUM / LOW: most rows ship at medium (the two CPU face rows are low); no row is marked critical
• never_evict: true is the intended pin (no live row sets it)
• preload: true is honored: it loads the row at startup alongside enabled: true — today osnet + the two face rows

Downloading Models

# Download the models the setup_lib rule selects (5 of the 10 live
# models.yml rows, 763MB of models.yml size_mb — the script header carries the
# exact re-derivation, and a guard test pins the fetch list to the rule).
# Reads only the AI_MODELS_PATH shell variable (not .env — export it to match
# setup.py's choice, below) and the target must already exist and be writable:
# setup.py sudo-creates it; otherwise sudo mkdir -p + chown first.
./ai/download_models.sh

# Custom models directory (default /export/ai_models; the containers mount this)
AI_MODELS_PATH=/path/to/models ./ai/download_models.sh

VRAM Budget

Sizing is dominated by the VLM identity in VLM_MODEL_PATH / VLM_MMPROJ_PATH: VLM_GPU_LAYERS=auto (default) offloads as many layers as the card allows, so there is no fixed tier table. Budget the GGUF pair's layers, the gateway's Triton process, and ~1.0GB of lookup-model headroom — the full table lives in VRAM Requirements.

The backend ModelManager (backend/services/model_zoo.py) exposes an @asynccontextmanager load() that releases VRAM on exit, but in production the only load trigger is the boot preload sweep (BACKEND_MODEL_PRELOAD=true in backend/main.py's lifespan) — no runtime path calls load(); a leg whose model is absent answers unavailable. Preloaded models stay resident until shutdown (unload_all() runs in the same lifespan), and the zoo has no eviction pass; the HTTP load/unload endpoints return 501 because Triton models are resident by design (--model-control-mode=none).

Model Status API

curl http://localhost:8000/api/system/models
curl http://localhost:8000/api/system/models/<name>/status

[!NOTE] The status API reads ai-gateway health — Triton readiness unioned across the mounted routers (/yolo26, /enrich-lt) plus in-process state from the backend ModelManager for models without a Triton mapping. Load/unload (POST /api/system/models/<name>/load and /unload) return 501 by design: Triton runs with --model-control-mode=none, gateway models are resident, and the gateway exposes no preload/unload surface — see backend/api/routes/model_management.py for the full semantics. vram-summary sums models.yml's estimated vram_mb per router; Triton does not expose per-model VRAM over HTTP.

Quick Start

Prerequisites: Linux host + NVIDIA GPU + Podman with GPU passthrough (the docker CLI works as a shim only via the optional podman-docker package)

# 1. Run setup — generates .env from .env.example with your ports, paths and secrets,
#    offers to download the models, and ends by deploying the stack
python setup.py

# 2. Download AI models (763 MB — the 5 rows the shared setup_lib rule selects
#    from models.yml; minutes on a fast connection. The VLM GGUF pair is separate:
#    you place it at VLM_MODEL_PATH / VLM_MMPROJ_PATH yourself, ledger D5).
#    Skip if you let setup.py do it. The script reads only the AI_MODELS_PATH shell
#    variable — not .env — so match what setup.py wrote, and the target directory
#    must already exist and be writable (setup.py sudo-creates it).
AI_MODELS_PATH=$(grep -m1 ^AI_MODELS_PATH .env | cut -d= -f2-) ./ai/download_models.sh

# 3. Start everything (you only need this if setup.py's auto-deploy was skipped —
#    reboot prompt, --defaults, or Ctrl-C; see the note below for PODMAN_SOCKET)
podman compose -f docker-compose.prod.yml up -d

[!NOTE] This project targets Podman: the compose file uses Podman-specific keys (userns_mode: keep-id, :U mount flags, CDI devices:), so plain docker compose does not run it as-is. backend and alloy also hard-require PODMAN_SOCKET in .env; python setup.py deploy appends it automatically. If you skipped the deploy phase, enable the socket first:

systemctl --user enable --now podman.socket
echo "PODMAN_SOCKET=/run/user/$(id -u)/podman/podman.sock" >> .env

Verify:

curl http://localhost:8000/api/system/health   # Backend health
open http://localhost:8080                     # Dashboard (HTTP)
# Or: open https://localhost:8444              # Dashboard (HTTPS; enable with SSL_ENABLED=true)

Then open the dashboard: first run requires you to register the first admin account (the API returns 503 for everything except setup and health until you do).

[!TIP] Run just core services: podman compose -f docker-compose.prod.yml up -d postgres redis backend frontend ai-gateway ai-vlm (ai-gateway serves detection on /yolo26; ai-vlm is in the default set, so a plain up -d starts it too — no flag needed.)

Operations & Monitoring

The monitoring ships in the same compose file — no separate stack to start. The URLs below all bind 127.0.0.1 by default, the dashboard included, so open them on the host itself (or forward an SSH tunnel). Set EXPOSE_LAN=true and re-run setup.py to publish the frontend's ports on 0.0.0.0 for LAN clients, which arms the auth gate; fence that with your firewall.

Port numbers come from .env (GRAFANA_PORT, PROMETHEUS_PORT, …). Full guide: Monitoring & Observability.

First five minutes at 2am

# What's up / what's down (this project uses Podman)
podman ps -a --format "table {{.Names}}\t{{.Status}}"
podman compose -f docker-compose.prod.yml ps

# Logs — a whole service, or one container
podman compose -f docker-compose.prod.yml logs --tail=50 backend
# backend has no container_name: the real name is <project>-backend-1 (compose v2)
# or <project>_backend_1 (podman-compose v1) — discover it:
podman logs --tail=50 "$(podman ps --format '{{.Names}}' | grep -E -- '[-_]backend[-_]?[0-9]?$')"

# Restart one service without touching the rest
podman compose -f docker-compose.prod.yml restart backend

Host-run dev-mode logs land in logs/backend.log (./scripts/dev.sh logs tails it).

Health endpoints, cheapest first — health/live answers without touching dependencies:

A /health 200 from ai-vlm proves the server answers, not that it loaded the mmproj projector. Check podman logs ai-vlm 2>&1 | grep -i mmproj when verdicts arrive as verification_failed with a NULL risk score.

Service control and self-healing details: Service Control.

# 1. One-time setup
python setup.py
uv sync --extra dev            # Python deps (repo-root .venv — dev.sh sources this one)
cd frontend && npm ci          # Frontend deps (CI uses npm; bun.lock + bunfig.toml exist — if you use bun, `bun run test`, never `bun test`; see frontend/bunfig.toml)

# 2. PostgreSQL — dev.sh does NOT start it, and the backend dies on boot without it
#    (init_db in the FastAPI lifespan). .env's DATABASE_URL/REDIS_URL point at the
#    container hostnames `postgres`/`redis`, so run a dev container and override:
podman run -d --name dev-postgres \
  -e POSTGRES_USER=security -e POSTGRES_DB=security \
  -e POSTGRES_PASSWORD="$(grep -m1 ^POSTGRES_PASSWORD .env | cut -d= -f2-)" \
  -p 127.0.0.1:5432:5432 docker.io/library/postgres:16-alpine
export DATABASE_URL="postgresql+asyncpg://security:$(grep -m1 ^POSTGRES_PASSWORD .env | cut -d= -f2-)@localhost:5432/security"
export REDIS_URL=redis://localhost:6379/0

# 3. Start Redis + backend (uvicorn :8000) + frontend (Vite dev server :8444, HTTPS)
./scripts/dev.sh start
./scripts/dev.sh status        # Also: stop | restart | logs

# Start the AI servers on the host (separate terminals)
./ai/start_detector.sh   # YOLO26 — reads PORT/YOLO26_PORT, defaults to 8090
# The reasoning engine has no host-run script: the shipped llama.cpp serve is the
# ai-vlm container — start it with
#   podman compose -f docker-compose.prod.yml up -d ai-vlm

# Then point the HOST-RUN backend (dev.sh) at them. The host detector serves its
# endpoints at the root (/health, /detect), so no router suffix — and these exports
# never reach a containerized backend (compose hardcodes the AI gateway URLs —
# AI_VLM_URL is the interpolated exception; no env_file).
export YOLO26_URL=http://localhost:8090
# The /enrich-lt lane (threat + re-ID Triton models, published on 127.0.0.1:8090 by
# the ai-gateway container) is read as a readiness target by
# backend/api/routes/model_management.py. Leave USE_AI_GATEWAY=false (the Settings
# default): with it true the detector client ignores YOLO26_URL and sends detection
# back to the gateway.
export ENRICHMENT_LIGHT_URL=http://localhost:8090/enrich-lt

Architecture

Detailed Documentation:

/export/foscam/{camera_name}/

Contributing

Start with the Developer Hub, then:

./scripts/validate.sh  # Full validation (lint + typecheck + tests)

Issue tracking: Linear

License

Licensed under Apache License 2.0. See LICENSE.

Acknowledgments

Ultralytics YOLO · Qwen3-VL · llama.cpp · NVIDIA Triton · FastAPI · Tremor