Nivora — PII Redaction Review Prototype

SprintFour Hackathon · Problem Statement 1: Trust and Explainability Built by Amritha S Nidhi (CB.SC.U4CSE23404)

---

The Problem

Marcus has a sensitive document he wants to process through an AI tool. He has heard stories of files marked "redacted" where the sensitive content was still recoverable underneath. He will not adopt a tool he cannot interrogate. Every redaction needs a reason. Every non-redaction needs a reason too. Silence reads as a miss.

The standard approach to this problem produces a document with black bars and no further explanation. The user is expected to trust the bars. Marcus does not.

---

The Insight

Most redaction tools explain what they hid. Nivora treats kept-visible content with equal seriousness.

When a location like "Houston" remains in the document, a reviewer clicking it sees: LOCATION type, heuristic-judged at 55% confidence, kept visible because city-level location alone is a low-entropy identifier — and a direct acknowledgment that this was a judgment call, not a certainty. When "HIPAA §164.506" is left in, the explanation is different: rule-matched at 97%, confidently cleared because it is a legal reference, not personal data.

Silence is not trust. Explaining the kept-visible decisions is as important as explaining the redacted ones.

---

What the Reviewer Sees

The document renders with three distinct visual states:

• Solid black blocks — permanently removed content. A lock icon. No recovery path.
• Labeled tokens — anonymized content recoverable via a token map. [PERSON_1], [EMAIL_1]
• Dotted coloured underlines — content that was scanned and deliberately left visible.

Clicking any of the three opens a panel showing: PII type, detection method (rule-matched / heuristic / Presidio NER / GLiNER), confidence score on a visual bar, treatment mode, and a one-sentence plain-English reason. Borderline confidence (below 0.50) renders in amber rather than blue — so a 38% marital status match looks visually different from a 99% SSN match.

A confidence threshold slider re-evaluates all spans client-side in real time. Dragging it changes what is redacted and what is not, live, and the explanation panel surfaces the override: "Nivora originally chose redacted. At your current threshold, this span is kept visible."

An AI Preview toggle shows exactly what a downstream AI model will receive — redacted spans become blank, anonymized spans become their token, kept-visible spans remain unchanged.

A Risk Dashboard aggregates all detections into a document-level risk level (LOW → CRITICAL) with combination-risk notes when name, location, and date appear together.

Trust is not binary. The slider makes it tunable.

---

Architecture

The system is organised into four swim lanes:

No database, no authentication, no live LLM call. All detection runs in Python on the uploaded document. The only persistent state is an in-memory session dictionary and a calibration.json file for reviewer feedback.

---

Detection Pipeline

Detection runs in four sequential stages followed by a merge step. The multi-stage design separates high-confidence deterministic detection from progressively softer heuristic and model-based detection, keeping confidence scores honest. A 0.99 on an email regex and a 0.60 on a name heuristic are genuinely different claims — they should look different to the reviewer.

Stage 1 — Rule Layer (detection/rule_layer.py)

Deterministic regex detectors. Confidence 0.92–1.00. All matches tagged rule_matched.

Direct identifiers:

Financial identifiers:

Other:

Ordering matters: MRN, NPI, and HIPAA patterns run before the generic POLICY_NUMBER pattern. Without this, MRN-008473 would be labelled as POLICY_NUMBER.

---

Stage 2 — Judgment Layer (detection/judgment_layer.py)

Heuristic detectors for unstructured PII. Confidence 0.35–0.90. All matches tagged heuristic or heuristic_judged. The softer confidence range is a design choice — it accurately represents that these are educated guesses, not certifications.

Name detection:

_SECTION_WORDS (200+ terms) prevents document section headers — HOLDINGS BREAKDOWN, CURRICULUM VITAE, ACCOUNT SUMMARY — from matching as person names.

Name scoring detail. Base score 0.60. +0.20 if a title cue precedes the phrase within 40 characters. −0.25 for each word matching _NON_NAME. Score < 0.40 → dropped. Score 0.40–0.61 → kept_visible. Score ≥ 0.62 → redacted with anonymize mode.

Context detectors:

---

Stage 3 — Presidio NER (detection/presidio_layer.py)

Microsoft Presidio backed by spaCy en_core_web_lg. Adds NLP named-entity recognition and Presidio's built-in regex recognizers. No-ops gracefully if Presidio or spaCy is not installed.

---

Stage 4 — GLiNER Zero-Shot NER (detection/gliner_layer.py)

GLiNER enables zero-shot NER against arbitrary label lists, selected per document profile. A fine-tuned checkpoint (trained_pii_model/checkpoint-1000) is available. Custom entity labels can be added at runtime via POST /api/custom-rule. No-ops gracefully if GLiNER is not installed.

---

Merge + Score (detection/merge.py)

Spans from all four stages are merged into a single deduplicated list:

1. Sort by start position → stage number (lower = higher priority) → span length → confidence
2. Overlap resolution — higher-priority or higher-confidence span wins; losers are discarded
3. Severity assignment — each span gets CRITICAL / HIGH / MEDIUM / LOW / INFO / CLEAR from the unified SEVERITY_MAP
4. Sequential IDs — spans are re-sorted by position and assigned span_1, span_2, …
5. Risk score — compute_risk_score() sums severity scores and adds combination risk when name + location, name + date, or name + medical info appear together

---

PS3 — Correction Review

A second-pass system surfaces issues the main pipeline may have missed or gotten wrong. It runs automatically after document load and populates a correction queue for human review.

Backend (/api/corrections)

Correction scanner patterns (detection/correction_scanner.py):

Frontend

• CorrectionBanner — orange alert banner on document load; "Ignore" or "Review →"
• CorrectionQueue — replaces TrustPanel in the right panel; one issue at a time with progress bar; keyboard R (redact/restore), K (keep), → (skip)
• Issues button — header badge showing total issue count; enters correction mode on click
• Document highlights — correction scanner spans get an orange pulsing dashed underline; suspicious redactions get an amber glow ring

When a reviewer acts through the queue, a userForced: true flag is set on the span, bypassing the confidence threshold slider for that span permanently.

---

Decision Modes

Token consistency. Identical text + type combinations always get the same token regardless of how many times they appear. [PERSON_1] always refers to the same individual throughout the document.

---

Document Profiles

---

Project Structure

conseal-ps1/
├── backend/
│   ├── main.py                    FastAPI app — 5 API routes, session management
│   ├── models.py                  Pydantic v2: PIISpan, ProcessResponse, FeedbackRequest
│   ├── extraction.py              Text extraction — .txt / .docx / .pdf
│   ├── calibration.json           Feedback persistence (type:method → correct/incorrect)
│   ├── sample_doc.txt             Fallback sample document
│   ├── sample_docs/
│   │   └── sample_medical.txt     Primary medical sample document
│   ├── requirements.txt           Python dependencies
│   ├── detection/
│   │   ├── rule_layer.py          Stage 1: 20+ regex detectors
│   │   ├── judgment_layer.py      Stage 2: heuristic detectors + SEVERITY_MAP
│   │   ├── presidio_layer.py      Stage 3: Presidio NER (optional)
│   │   ├── gliner_layer.py        Stage 4: GLiNER zero-shot NER (optional)
│   │   ├── merge.py               Overlap resolution, severity, risk score
│   │   └── correction_scanner.py  PS3 second-pass scanner
│   └── training/
│       ├── synthetic_generator.py  PII-annotated training data generator
│       └── train.py                GLiNER fine-tuning script
│
├── frontend/
│   ├── vite.config.js             Dev server :5173, proxies /api/* → :8000
│   ├── tailwind.config.js         Design tokens: ink, primary, teal, surface, border
│   └── src/
│       ├── App.jsx                Router: UploadPage / ReviewPage / AuditPage
│       ├── index.css              Tailwind base + span classes
│       ├── lib/spans.js           buildSegments(), applyThreshold(), tokenFor()
│       ├── pages/
│       │   ├── UploadPage.jsx     Landing — file upload + profile selector
│       │   ├── ReviewPage.jsx     Main review workspace
│       │   └── AuditPage.jsx      Compliance audit log — filter, sort, export CSV
│       └── components/
│           ├── DocumentViewer.jsx      Span highlighting, token map, AI preview
│           ├── TrustPanel.jsx          Span explanation (method, confidence, reason, feedback)
│           ├── RiskDashboard.jsx       Document-level risk badge + combination notes
│           ├── ConfidenceSlider.jsx    Global threshold (real-time, client-side)
│           ├── CustomRulesInput.jsx    Add custom entity labels → /api/custom-rule
│           ├── AIPreviewToggle.jsx     Toggle AI-visible vs annotated view
│           ├── AIAssistant.jsx         Span-aware Q&A assistant
│           ├── KeyboardShortcuts.jsx   Shortcut reference modal (?)
│           ├── CorrectionBanner.jsx    PS3 alert banner
│           ├── CorrectionQueue.jsx     PS3 one-at-a-time review queue
│           └── Dither.jsx / .css       Three.js WebGL dither wave (upload page)
│
├── Nivora Architecture.png        System architecture diagram
├── architecture.svg               Architecture diagram source
├── README.md
└── WRITEUP.md

---

Prerequisites

• Python 3.10+
• Node.js 18+
• npm 9+

Optional (for full NLP coverage):

• presidio-analyzer + spacy + en_core_web_lg — enables Stage 3
• gliner — enables Stage 4

Nivora runs without Stage 3 and 4 installed — those layers no-op and the rule + heuristic layers handle detection alone.

---

Local Setup

Terminal 1 — Backend

cd conseal-ps1/backend
pip install -r requirements.txt
python -m uvicorn main:app --host 127.0.0.1 --port 8000 --reload

FastAPI interactive docs: http://127.0.0.1:8000/docs

Terminal 2 — Frontend

cd conseal-ps1/frontend
npm install --legacy-peer-deps
npm run dev

Open http://localhost:5173. Vite proxies all /api/* to :8000 automatically.

Optional — full NLP stack

pip install presidio-analyzer presidio-anonymizer spacy
python -m spacy download en_core_web_lg
pip install gliner

Supported file types: .txt, .docx, .pdf — maximum 5 MB.

--legacy-peer-deps is required because @react-three/fiber v8 (React 18 compatible) has a peer dependency declaration that conflicts with npm 9's strict resolution. The installed versions are fully compatible at runtime.

---

API Reference

All routes are prefixed /api. Errors follow FastAPI's {"detail": "..."} shape.

POST /api/process

Upload a document and run the full 4-stage pipeline.

Request: multipart/form-data

• file — .txt, .docx, or .pdf
• profile — general / medical / financial / legal / hr (default: general)
• custom_entities — JSON array of extra entity labels (default: [])

Response:

{
  "document_text": "...",
  "spans": [
    {
      "id": "span_1",
      "text": "[email protected]",
      "start": 42,
      "end": 62,
      "type": "EMAIL",
      "decision": "redacted",
      "mode": "redact",
      "detection_method": "rule_matched",
      "confidence": 0.99,
      "severity": "HIGH",
      "stage": 1,
      "reason": "Matched standard email address pattern. Permanently removed.",
      "evidence": "regex: email pattern ([email protected])"
    }
  ],
  "risk_score": {
    "level": "HIGH",
    "score": 14,
    "critical_count": 0,
    "high_count": 3,
    "combination_notes": ["Name + Location together enable geographic re-identification"]
  }
}

Error codes: 413 (file too large) · 415 (unsupported type) · 422 (extraction failed)

---

GET /api/sample

Process the built-in sample medical document without uploading. Accepts ?profile=medical. Returns the same shape as /api/process. Use this during demos.

GET /api/corrections

Return PS3 correction review data for the most recently processed document.

{
  "potential_misses": [...],
  "suspicious_redactions": [...],
  "fresh_scan_spans": [...],
  "total_issues": 5
}

POST /api/custom-rule

Re-run detection with an additional custom entity label for GLiNER.

{ "entity_label": "employee ID", "profile": "hr" }

POST /api/feedback

Record reviewer feedback to calibrate future confidence estimates.

{ "span_id": "span_3", "type": "PERSON_NAME", "detection_method": "heuristic", "correct": false }

GET /api/audit/csv

Export all spans as a CSV file (falls back to client-side generation if pandas is unavailable).

GET /api/health

Returns {"status": "ok"}.

---

Demo Walkthrough

The following two-minute sequence demonstrates the full trust experience:

1. Landing page. The upload page opens with a Bayer-dithered wave animation. Select a document profile and drag in a file — or click "Try a sample document."

2. Processing. A pipeline progress animation plays (extracting text → rule layer → heuristics → Presidio → GLiNER → merging). The Risk Dashboard populates as the review screen appears.

3. First impression. The document renders with highlights across all four detection methods: black bars for SSN/IBAN/TAX_ID; coloured tokens for names and email; dotted underlines for location, marital status, and the HIPAA citation.

4. Click a redacted span. Click the SSN black bar. The panel shows: SSN · Rule-matched · 100% confidence · Redact — permanently removed.

5. Click a kept-visible span. Click the dotted-underlined city name. Same panel format, different content: LOCATION · Heuristic · 55% · kept visible — "a judgment call, not a certainty."

6. Click a confidently cleared span. Click the HIPAA citation. Rule-matched · 97% · kept visible — "a legal reference, not personal data." The panel makes the distinction between uncertain clearance and confident clearance visible.

7. PS3 Correction Review. The orange banner appears: "N potential misses, M uncertain redactions." Click Review → to enter the correction queue. Work through each issue one at a time with keyboard shortcuts R / K / →.

8. Adjust the threshold. Drag the slider down to 40. Low-confidence spans flip in real time. The panel surfaces overrides: "Nivora originally chose kept visible. At your current threshold, this span is redacted."

9. AI Preview. Toggle AI Preview to see exactly what a downstream AI would receive — clean text with redacted gaps and anonymization tokens. No inference, no API key, no guessing.

10. Audit Log. Navigate to Audit Log. Filter by severity, sort by confidence, export CSV as a GDPR/HIPAA compliance artifact.

The point: trust is not binary. The slider makes it tunable, and Nivora tells you exactly what changed and why.

---

Design Decisions

Symmetric treatment of kept-visible content. This is the core product decision. A span that was "considered and cleared" is explained with the same panel, the same confidence bar, and the same level of detail as a span that was redacted. Content that was never flagged at all does not receive this treatment — it is just plain text. The visual distinction matters: coloured underline vs. unstyled text communicates "this was seen" vs. "this was not looked at."

Four stages with honest confidence ranges. Regex patterns for structured PII (email, SSN, IBAN) are essentially perfect detectors — 0.97–1.00 is accurate. Heuristics for names are genuinely uncertain — 0.60–0.85 is also accurate. Presidio adds NLP-backed NER for entities that resist regex. GLiNER adds domain-specific zero-shot detection tuned per profile. Combining all four gives coverage without inflating confidence scores.

_SECTION_WORDS frozenset. Document section headers (CURRICULUM VITAE, HOLDINGS BREAKDOWN, ACCOUNT SUMMARY) are all-caps sequences that match the standalone-name detector pattern. A 200+ term frozenset lookup at O(1) blocks them cleanly across resume, financial, medical, and legal document types.

PS3 correction queue — one issue at a time. Bulk review encourages rubber-stamping. Forcing reviewers to see one issue at a time with full context and a binary choice (redact / keep) produces more deliberate decisions and a cleaner audit trail. A userForced flag on acted-upon spans bypasses the confidence slider permanently.

ISINs excluded from the correction scanner. International Securities Identification Numbers are 12-character codes starting with two uppercase letters followed by 10 digits. The lookbehind (?<![A-Z\d]) ensures the bare-10-digit scanner never fires on the digit portion of an ISIN.

Client-side threshold re-evaluation. Re-running the backend pipeline on every threshold change would add 200–500ms latency for a display-layer operation. All span data is in the browser after the initial upload; applyThreshold() re-evaluates synchronously in under one millisecond. The explanation panel surfaces threshold overrides explicitly — silence never reads as a miss.

Phone and email are permanently redacted, not anonymized. A recoverable token ([EMAIL_1]) is still a re-identification risk if the token map is exposed. These values have no utility for a downstream AI evaluating document structure. They are pure contact data and should be removed entirely.

REGULATORY_CITATION is kept_visible. References to HIPAA, GDPR, CCPA, PCI DSS are legal context, not personal data. Redacting them would degrade the meaning of compliance documents without protecting any individual's privacy.

No live LLM. Removing API-key setup, rate limits, and network failure risk from the demo path was the correct trade-off. The judgment layer is exactly where a fine-tuned NER model or LLM-based extractor would slot in a production system; the heuristics are transparent, explainable stand-ins.