Privacy Gateway — startup repair and project review

Latest patch (0.8.1): corrects GLiNER labeling a language word as PERSON in explicit Russian language requests, including Привет ты понимаешь по русский?. See the review appendix for its deliberately limited scope.

Start with START_HERE.md. The review and validation limits are in PROJECT_REVIEW.md.

This revision fixes setup error handling, adds local diagnostics, pins the tested Gradio version, checks model availability, and repairs conversation-history privacy checks. Your exact Windows traceback was not supplied, so this is a verified repair of the issues found in the archive, not a claim that your particular failure was reproduced.

A local-first FastAPI + Gradio privacy gateway for LLM applications. Sensitive source conversation state remains local, while a separate placeholder-sanitized conversation is maintained for any external LLM call.

This version includes RAM-only multi-turn sessions, Presidio + contextual rules + local GLiNER, EntityMerger, semantic placeholders, final deterministic leak scanning with safe local auto-remediation/re-scan, and the stakeholder pipeline inspector. The independent local privacy reviewer remains the next major security phase.

Current flow

                         LOCAL COMPUTER

User / Gradio
    |
    v
RAM-only PrivacySession
    |-- local_history      = original user + restored/local assistant text
    |-- mapping            = <PERSON_1> -> real value
    |-- sanitized_history  = placeholder-safe cloud mirror
    |
    v
Presidio + Context Rules + GLiNER -> EntityMerger -> placeholders
    -> FinalLeakScanner -> safe auto-remediation/re-scan -> EgressGate
    |                                             |
    | LOCAL_ONLY                                  | ALLOW_EXTERNAL
    v                                             v
Ollama gets local_history                  LiteLLM gets sanitized_history only
    |                                             |
    +---------------- response -------------------+
                          |
                    restore locally

Privacy behavior

Default storage mode: RAM only

By default:

• original chat history is stored only in Python process memory;
• placeholder mappings are stored only in process memory;
• sanitized history is stored only in process memory;
• no chat transcript is intentionally persisted to disk;
• logs contain metadata only, never prompt text or mapping values;
• sessions become eligible for expiry after 60 minutes by default; pruning occurs on subsequent store operations;
• restarting the gateway removes all session state;
• Clear conversation deletes the corresponding backend session immediately from the application's in-memory store.

Python/Windows cannot provide a mathematical guarantee of immediate physical RAM zeroization. Clearing removes application references and all configured persistent copies.

Stable placeholders across turns

Exact sensitive values already known in the session reuse the same placeholder:

Turn 1:
Sarah Johnson -> <PERSON_1>

Turn 2:
Sarah Johnson -> <PERSON_1>

Alias/pronoun resolution such as Sarah -> previously seen Sarah Johnson is not implemented yet; that belongs with the future contextual NER/reviewer layer.

Install

Requirements:

• Python 3.11 (recommended for this setup script), or 3.12 for manual setup
• Ollama for local responses
• a LiteLLM-supported provider only when you explicitly enable cloud egress

On Windows run:

setup_windows.bat

Or manually:

python -m venv .venv
# Activate .venv first (Windows: .venv\Scripts\activate)
python -m pip install -r requirements.txt
python -m spacy download en_core_web_lg
python scripts/prefetch_gliner.py

Copy .env.example to .env, then configure your local Ollama model. The application loads this project-local .env automatically at startup.

First check installed models:

ollama list

Then set one exact installed model name:

OLLAMA_MODEL=qwen2.5:3b
FALLBACK_MODE=local_llm
EXTERNAL_ENABLED=false
SESSION_TTL_MINUTES=60

If OLLAMA_MODEL is blank, Ollama is stopped, or the model is not installed, the UI fails closed with LOCAL_LLM_UNAVAILABLE; it does not send the original prompt externally.

Run

run_windows.bat

or:

python -m app.diagnostics
python -m uvicorn app.main:app --host 127.0.0.1 --port 8000 --no-access-log

Open:

http://127.0.0.1:8000/ui/

API docs:

http://127.0.0.1:8000/docs

API multi-turn example

First request without a session ID:

POST /v1/chat
{
  "message": "Email John Smith at john@example.com"
}

The response includes a local session_id. Reuse it:

POST /v1/chat
{
  "message": "Now draft a shorter version for John Smith",
  "session_id": "<returned-session-id>"
}

Clear it:

DELETE /v1/sessions/<session-id>

External mode

Cloud egress remains OFF by default.

When eventually enabled:

EXTERNAL_ENABLED=true
EXTERNAL_MODEL=openai/<model-name>

The external client receives the sanitized multi-turn history, never local_history or the placeholder mapping.

Tests

pytest -q

Important tests cover:

• no original PII in mocked LiteLLM payloads;
• stable placeholder reuse across conversation turns;
• local Ollama receiving real local multi-turn history;
• external LLM receiving sanitized multi-turn history only;
• clearing a session removes mappings and both histories.

Next security phase

After testing this UI/session architecture, add:

1. independent Ollama PrivacyReviewer;
2. strict structured reviewer JSON;
3. reviewer-driven iterative re-anonymization;
4. expanded secrets and adversarial tests;
5. Russian/Kazakh multilingual detection.

Do not describe this system as guaranteeing 100% anonymity. Its security goal is defense in depth with fail-closed external egress.

Stakeholder pipeline demo (v0.4)

Open http://127.0.0.1:8000/ui/ after running run_windows.bat. The UI now acts as a live privacy-pipeline inspector:

1. original user prompt (local/sensitive);
2. Presidio detections;
3. placeholder-anonymized prompt;
4. local-only placeholder mapping;
5. final deterministic leak scan / egress decision;
6. exact sanitized conversation payload eligible for a cloud LLM;
7. raw LLM response before restoration;
8. locally restored final answer.

For stakeholder demos, local Ollama simulates the external/cloud LLM. It receives only the sanitized message history plus the placeholder-preservation system instruction. This demonstrates the intended cloud boundary without requiring an API key and without calling a cloud LLM (local HTTP is still used). The normal /v1/chat endpoint is unchanged and continues to enforce the production-style gateway behavior.

The demo trace contains original sensitive values and the local mapping, so it is intentionally UI-only: it is not returned by /v1/chat, is not exposed as a named Gradio API action, and must not be written to application logs. Run the app bound to 127.0.0.1, as provided by run_windows.bat.

The current inspector reflects the implemented layers: Presidio + contextual rules + GLiNER → EntityMerger → placeholders → final leak scanner → safe local auto-remediation/re-scan → egress boundary → LLM → response validation → restoration. The independent Ollama privacy reviewer remains the next major phase.

Stakeholder demo fixes (v0.5)

The pipeline inspector now includes two safeguards exposed by the latest demo:

• PERSON span refinement: if Presidio/spaCy absorbs an imperative command into a person span (for example Email John Smith), the command verb is kept in the prompt and only John Smith is placeholderized. The sanitized result becomes Email <PERSON_1> ..., not <PERSON_1> ....
• LLM response placeholder validation: the demo checks that the LLM reused only placeholders already present in the local mapping. If the model invents a token such as <INTERNAL_PROJECT_1>, the demo retries once with a stricter allowed-placeholder instruction. Unknown placeholders are never written into future cloud-safe conversation history.

The stakeholder UI therefore shows an additional 6B — Response placeholder validation step between the raw LLM response and local restoration.

Contextual business-sensitive detection

The stakeholder demo now runs multiple independent local detectors before placeholderization:

• Microsoft Presidio for classic PII;
• deterministic contextual recognizers for high-confidence organization/project/location/floor/room/time patterns;
• GLiNER (gliner-community/gliner_small-v2.5) for local contextual NER;
• EntityMerger to resolve duplicate and overlapping findings.

Example:

John Smith about the KazMunayGas project ... Izumrudnyi Business Center ... 29th floor room 2905 at 6pm today

can become:

<PERSON_1> about the <ORGANIZATION_1> project ... <LOCATION_1> ... <LOCATION_DETAIL_1> at <DATE_TIME_1>

setup_windows.bat downloads the GLiNER model once. run_windows.bat then sets Hugging Face/Transformers offline mode so the stakeholder demo does not unexpectedly fetch model files at runtime.

Final scanner auto-remediation (v0.7)

The final deterministic scanner is still authoritative, but ordinary residual PII with a concrete span can now be repaired locally before the turn is blocked. Example: if the first detector pass masks 6pm today and this week but the final Presidio re-scan additionally identifies some other day as DATE_TIME, the gateway creates another semantic placeholder locally, re-runs the final scanner, and proceeds only if the re-scan passes.

High-risk secret findings and invariant failures such as KNOWN_VALUE, API keys, private keys, bearer tokens, database URLs, password assignments, or malformed/ambiguous findings are not auto-remediated and remain fail-closed blockers. The stakeholder UI shows the initial final-scan finding, any local remediation pass, the final cloud-safe prompt, and the re-scan result.