Trust but Verify: Financial LLM Hallucination Analysis

This repository implements the full project proposal pipeline for financial hallucination analysis: FinQA and FinRED baselines, consistency/AUROC analysis, error attribution workflow, and distractor-position sensitivity experiments.

Requirements

• Python 3.10+
• Recommended packages:
  • datasets
  • huggingface_hub
  • sentence-transformers
  • scikit-learn
  • pandas
  • matplotlib
  • requests
  • tqdm
  • nbclient
  • nbformat
  • ipykernel

Install:

python -m pip install datasets huggingface_hub sentence-transformers scikit-learn pandas matplotlib requests tqdm nbclient nbformat ipykernel

How to run

1) Full local end-to-end run (offline, all available test data)

./run_end_to_end_local.sh

This runs the entire pipeline without notebook execution, using all available FinQA/FinRED test items (--use-full-dataset) and writes all artifacts to outputs/.

1b) Final project profile (proposal-aligned)

./run_final_project.sh

This includes:

• 3-shot CoT prompting for FinQA (--finqa-shot-mode three)
• all available evaluation rows (--use-full-dataset)
• PHANTOM calibration stage (--enable-phantom-calibration)
• full consistency sampling (--consistency-k 5)

1c) PHANTOM-only local run (resume without re-running everything)

./run_phantom_only_local.sh

This skips FinQA/FinRED baseline + consistency + distractor stages and runs only PHANTOM calibration.
Use this when your previous run already produced the non-PHANTOM artifacts in outputs/.

2) Quick reproducible run (offline/simulate mode)

python analysis.py --quick --inference-mode simulate

This executes the full pipeline structure without paid model calls and is the easiest way to reproduce all outputs and figures.

3) Full proposal-scale run (live model inference)

Export your Hugging Face token:

export HF_TOKEN="YOUR_TOKEN_HERE"

Then run:

python analysis.py --inference-mode live --sample-size 200 --consistency-sample-size 200 --consistency-k 5 --distractor-sample-size 200

Optional model selection:

python analysis.py --inference-mode live --models llama3_8b_instruct,finma_7b_full

3b) Full proposal-scale run (local model inference)

Download model folders locally under models/ (or provide explicit paths via --local-model-paths), then run:

python analysis.py \
  --inference-mode local \
  --use-full-dataset \
  --finqa-shot-mode three \
  --models llama3_8b_instruct,finma_7b_full \
  --consistency-k 5 \
  --enable-phantom-calibration \
  --phantom-use-full-dataset \
  --phantom-split train \
  --require-phantom-calibration \
  --local-model-root models

If PHANTOM long contexts OOM in local mode, cap local prompt length:

python analysis.py --inference-mode local --enable-phantom-calibration --only-phantom-calibration --local-max-input-tokens 1024

Optional explicit path mapping if folder names differ:

python analysis.py --inference-mode local --local-model-paths "llama3_8b_instruct=models/Llama-3.1-8B-Instruct,finma_7b_full=models/finma-7b-full"

4) Use all available test items directly from CLI

python analysis.py --inference-mode simulate --use-full-dataset --consistency-k 5

Proposal-aligned options

python analysis.py \
  --inference-mode simulate \
  --use-full-dataset \
  --finqa-shot-mode three \
  --enable-phantom-calibration \
  --phantom-dataset-id seyled/Phantom_Hallucination_Detection \
  --phantom-split train \
  --consistency-k 5

If you run in live provider mode and need strict RQ3 token-logprob calibration enforcement:

python analysis.py --inference-mode live --require-logprobs-for-rq3

Notebook execution

python - <<'PY'
import nbformat
from nbclient import NotebookClient
nb = nbformat.read("analysis.ipynb", as_version=4)
NotebookClient(nb, timeout=1200, kernel_name="python3").execute()
nbformat.write(nb, "analysis.ipynb")
print("Notebook executed successfully.")
PY

File structure

• Project Proposal.pdf: proposal scope and methodology.
• Progress Report.pdf: implementation progress and pilot findings.
• feedback.txt: instructor feedback requirements.
• analysis.py: full end-to-end experimental pipeline.
• run_end_to_end_local.sh: one-command local end-to-end runner (no notebook).
• run_final_project.sh: proposal-aligned final run profile.
• run_phantom_only_local.sh: PHANTOM-only local runner for resume/recovery runs.
• analysis.ipynb: notebook runner and result inspection.
• paper.md: final paper draft.
• data/: downloaded dataset files (FinQA + cached HF datasets).
• outputs/: all generated tables, metrics, run config, and figures.

Key outputs in outputs/

• dataset_sizes.csv
• model_status.csv
• finqa_baseline_results.csv
• finred_baseline_results.csv
• consistency_results.csv
• consistency_auroc.csv
• distractor_position_summary.csv
• manual_error_coding_template.csv
• summary_metrics.json
• fig_dataset_sizes.png
• fig_baseline_accuracy_by_model_task.png
• fig_consistency_auroc.png
• fig_distractor_positions_by_model.png
• fig_error_types_heuristic.png
• fig_distractor_confidence_by_position.png (when token logprobs are available)
• distractor_confidence_summary.csv
• phantom_dataset_profile.csv (when PHANTOM calibration is enabled)
• phantom_baseline_results.csv (when PHANTOM calibration is enabled)
• phantom_consistency_results.csv (when PHANTOM calibration is enabled)
• phantom_consistency_auroc.csv (when PHANTOM calibration is enabled)

How to read the paper

Read paper.md first for the full narrative (Abstract to References), then inspect analysis.ipynb for executable details and generated artifacts.