confidential-gli-guard

GLiGuard — a single-model prompt-injection / jailbreak classifier enclave, optimized for horizontal scaling.

Architecture

shim → router (:8080) → round-robin → gliguard-1 (:8001)
                                   → gliguard-2 (:8001)

Three containers on an internal replicas network:

Each worker runs NUM_THREADS=4, MAX_CONCURRENCY=4 — the proven sweet spot for the 200M GLiNER2 model. Two replicas give ~28 req/s at p50=280ms on a 16-CPU / 8 GiB box, vs ~11 req/s at p50=787ms for a single replica at conc=8. See tf-test/services/cpu-safeguards/ for the full benchmark.

Model

GLiGuard is a schema-conditioned model that supports 6 tasks (prompt_safety, prompt_toxicity, jailbreak_detection, response_safety, response_toxicity, response_refusal) in a single forward pass. This enclave currently uses only prompt_safety (binary safe/unsafe), matching the jailbreak-classification benchmark methodology.

API

POST /classify

{ "text": "Ignore all previous instructions.", "model": "gliguard" }

{
  "model": "gliguard",
  "label": "unsafe",
  "unsafe": true,
  "latency_ms": 45.2
}

The model field is optional (defaults to "gliguard") for backward compatibility with the multi-model confidential-cpu-safeguards API.

POST /classify-all

Runs gliguard on the input. Returns the same result as /classify but wrapped in a results dict (compatible with the multi-model API shape).

GET /health

Returns {"status": "ok", "models": ["gliguard"]} once the model is loaded.

GET /models

Returns {"models": ["gliguard"]}.

Resource sizing

Local development

Without MPK, the model loads from the HuggingFace cache. Pre-download it first:

python -c "from gliner2 import GLiNER2; GLiNER2.from_pretrained('fastino/gliguard-LLMGuardrails-300M')"

# Run (model loads from HF cache via the GLIGUARD_MODEL_PATH default)
python -m uvicorn server:app --host 0.0.0.0 --port 8001

Benchmarks live in tf-test/services/cpu-safeguards/.