Can We Trust the Faithfulness Judge?

A failure-taxonomy stress test of a real-time faithfulness verifier in voice financial RAG.

This repository contains the code, data, and experiment outputs for the paper, which asks whether the judge that gates every claim a voice assistant speaks about a company's SEC filings can be trusted in production. We compare two judges for the same task — real-time, claim-level faithfulness verification over 10-K/10-Q filings:

• GPT-OSS-120B — a 120B-parameter LLM-as-judge prompted to label each claim.
• FinFaithVerifier — a lightweight hallucination-detection classifier (HHEM fine-tuned on the financial domain via knowledge distillation).

Both are stressed against a failure taxonomy of LLM judges: numerical sensitivity (swapped units, shifted magnitudes, flipped directions), self-preference across generator families, and prompt-style sensitivity (zero-shot / few-shot / chain-of-thought).

Key findings. The small NLI judge is more trustworthy than the LLM judge on every facet measured: claim-level F1 = 0.828 vs 0.602 against human labels, at roughly two orders of magnitude lower latency (~7 ms vs ~3.6 s per claim). The LLM judge's near-perfect adversarial sensitivity (99.7%) is an artifact of a calibration failure — it labels nearly every claim unfaithful, so its robustness does not reflect genuine discrimination. There is no self-preference in the LLM judge, and chain-of-thought prompting hurts rather than helps.

---

Repository contents

.
├── pyproject.toml                # installable package (pip install -e .)
├── config.yaml                   # single source: models, thresholds, top_k, paths
├── src/faithfulvoice/            # the library package
│   ├── config.py                 # load_config() + constants + path layout
│   ├── evalio.py                 # JSONL load/save helpers
│   ├── pipeline.py               # retrieval -> generation -> verification (async)
│   ├── ingest/                   # cleaner, parser, metadata, chunker, qdrant
│   ├── retrieval/                # dense_sparse + graphrag + metadata_hybrid
│   ├── verification/             # verifier: verify_single + compute_metrics
│   ├── voice/                    # stt (Moonshine), tts (Kokoro), pipeline, rag
│   ├── graph/                    # entity extraction, Neo4j builder, checkpoint
│   └── modal/                    # images, volumes, common (the Modal app)
│
├── experiments/                  # runnable entrypoints e1..e11 + run.py + human_eval/
├── scripts/                      # data-pipeline CLIs
├── tests/                        # pytest suite
├── data/                         # raw/, cleaned/, processed/, entities/, queries/, audio/
├── results/                      # exp1..exp11 (experiment outputs)
└── docs/                         # phase-by-phase implementation + result reporting

---

Install

python -m venv venv && source venv/bin/activate     # Windows: venv\Scripts\activate
pip install -e ".[dev]"                              # installs the faithfulvoice package + deps
docker compose up -d qdrant                          # vector DB (add `neo4j` for the GraphRAG ablation)

Config lives in config.yaml. API keys are read from environment variables (GROQ_API_KEY, NVIDIA_API_KEY, HF_TOKEN, NEO4J_PASSWORD); none are stored in this repository.

---

The judge stress-test protocol

Three experiments instantiate the failure taxonomy. Each is self-contained: a dataset-building step, one run per judge, and a chart step. Outputs land in results/exp{9,10,11}/.

Judges under test. FinFaithVerifier (fine-tuned HHEM, run via Modal) and openai/gpt-oss-120b (LLM-as-judge, run via the Groq API). A DeBERTa NLI verifier (cross-encoder/nli-deberta-v3-base, threshold 0.7) serves as the NLI baseline.

# E9 — adversarial numerical perturbations
python experiments/e9_adversarial_perturb.py --dry-run
modal run experiments/e9_adversarial_perturb.py --judge-type nli --model finetuned_hhem
python experiments/e9_adversarial_perturb.py --judge-type llm --model openai/gpt-oss-120b
python experiments/e9_make_charts.py

# E10 — self-preference matrix
python experiments/e10_generate_dataset.py
modal run experiments/e10_self_preference.py --judge-type nli --model finetuned_hhem
python experiments/e10_self_preference.py --judge-type llm --model openai/gpt-oss-120b
python experiments/e10_make_charts.py --baseline-nli finetuned_hhem

# E11 — prompt-style ablation
python experiments/e11_cot_ablation.py --model openai/gpt-oss-120b
python experiments/e11_make_charts.py

Full protocol, run commands, and output schemas: docs/PHASE7_JUDGe_Workshop.md.

---

Supporting pipeline

The judge operates in a live pipeline; E1–E8 build and evaluate the system it gates, and produce the human labels against which both judges are scored.

audio → Moonshine STT → retrieval (Qdrant: BGE dense + BM25 sparse, reranked, top_k=8)
      → generation (Llama-3.1-8B) → verifier (HHEM / DeBERTa-NLI, threshold 0.7)
      → Kokoro TTS → audio

Human labels (results/exp5/majority_labels.jsonl) are the gold set used to score both judges in E3/E4/E9/E11.

---

Dataset

~953 queries over 10 companies (NVDA, AMD, INTC, AAPL, MSFT, GOOGL, META, AMZN, TSLA, JPM) × 6 archetypes (Direct, False Premise, Multi-Hop, Temporal Comparison, Qualitative, Out-of-Scope) × 5 domains (revenue, supply chain, risk factors, guidance, litigation), grounded in SEC 10-K/10-Q filings. Queries: data/queries_full_dataset/merged_queries.jsonl.

---

Reproducing results

Each experiment writes to a matching results/exp*/ directory, included here as reference output. To regenerate: install the package, start Qdrant (docker compose up -d qdrant), set the required API keys, and run the matching experiments/e*.py entrypoint (locally or via modal run).

---

Documentation

• docs/EXPERIMENTS.md — experiment map (inputs, outputs, schemas)
• docs/PHASE7_JUDGe_Workshop.md — judge stress-test protocol (E9/E10/E11)
• docs/FIGURES_Catalog.md — figures the experiments emit
• docs/ — phase-by-phase implementation notes
• docs/experiment_decisions/ — per-phase decision logs

All models are open-source and self-hostable (MIT / Apache-2.0 / permissive); no proprietary dependencies on the core path.