DeepStream + Triton playground

A playground for investigating and simplifying work with NVIDIA DeepStream 9.1 and Triton Inference Server — building tools and utilities that make DeepStream much more user-friendly.

The centerpiece is apps/ds_pipeline, a self-contained Python toolkit (element factories, source/output bins, metadata iterators, probe factories, config loading) that shrinks a DeepStream app to its model-specific logic. Around it: a set of in-house pipelines (YOLO11x/YOLO26x, RF-DETR detection + segmentation, Grounding DINO, SAHI tiled inference, multi-RTSP fan-out), the NVIDIA Python reference samples, and reproducible accuracy/latency research.

Run it

./docker/launch.sh -b      # build the dev image (nvcr.io/nvidia/deepstream:9.1-triton-multiarch base)
./docker/launch.sh -d      # drop into a dev shell with everything mounted
./docker/launch.sh -r apps/deepstream-test1/deepstream_test_1.py   # run one app directly

Host requirements: NVIDIA driver ≥ 595.58.03, docker + nvidia-container-toolkit. An X11 display is used for on-screen sinks (falls back per app).

Inside the container the repo is mounted at /workspace, plus convenience mounts the apps' configs rely on:

Repo map

Highlights

• ds_pipeline — build a full detection pipeline in ~40 lines; probes via make_pgie_tensor_probe(parse_fn, config) so an app only writes its tensor-parsing function. Errors raise, run_pipeline returns an exit code, .dot graphs dump on failure.
• SAHI tiled inference as a Triton ensemble (apps/deepstream-yolo26x-sahi-triton) — slicing → batched YOLO26x → merge, entirely server-side; studied quantitatively in research/.
• RF-DETR detection + instance segmentation with raw-tensor parsing and GPU mask post-processing (apps/deepstream-rfdetr, apps/deepstream-rfdetr-seg).
• Grounding DINO open-vocabulary detection through an nvinferserver Triton ensemble with text prompts (apps/deepstream-grounding-dino).
• Multi-RTSP fan-out — one inference pass tee'd into N mounts at different resolutions/framerates (apps/deepstream-multi-rtsp-out).

DeepStream 9.1 notes

The repo targets DS 9.1 (CUDA 13.2, TensorRT 10.16, Triton 26.03). TensorRT engines built under 9.0 do not load on 9.1 — rebuild them (see docs/deepstream-9.1-upgrade.md). The Python apps use the pyds bindings, which NVIDIA deprecated in 9.1 in favor of pyservicemaker; pyds still builds and works (the Dockerfile builds it from source).