faithfulness

Data Description

Scripts

Faithfulness Classifier Training (train_classifier.py)

This script trains a binary classifier (Faithful vs. Unfaithful) to evaluate the reliability of summarization models. It supports fine-tuning transformer models (e.g., RoBERTa, ModernBERT) using a weighted CrossEntropyLoss to handle class imbalance.

Features

• Data preprocessing: Filters raw JSON data, specifically excluding "Extrinsic Information" errors based on project constraints.
• Class Imbalance Handling: Implements random oversampling during training.
• Training: Uses Hugging Face Trainer with Early Stopping and best model checkpointing based on Balanced Accuracy.

Usage

1. Adjust data paths in train_classifier.py (RAW_TRAIN_PATH and CLEAN_VAL_PATH).
2. Run the script:

   python train_classifier.py
   

Evaluation Script for Faithfulness Classifiers (eval.py)

This script evaluates trained classification models (e.g., RoBERTa) on faithfulness datasets.

• Metrics: Balanced Accuracy, F1, Precision, and Recall.

Usage

1. Single Run Mode

Use this when checking one specific model against one test file

python evaluate_classifier.py \
  --model /path/to/model \
  --data /path/to/data

2. Batch Mode

python eval.py --batch

To modify the batch experiments, edit the BATCH_EXPERIMENTS list in evaluate_classifier.py:

BATCH_EXPERIMENTS = [
    {
        "model_path": "path/to/model_A",
        "test_sets": [
            ("Description 1", "path/to/test_1.json"),
            ("Description 2", "path/to/test_2.json")
        ]
    },
    # ... Add more models
]

models.md contains link to the trained models and current evaluation results.

Mitigation by Paraphrasing: Hallucination Evaluation using Log-likelihood (mitigation.py)

This file contains the evaluation pipeline for testing whether paraphrasing source documents (e.g., simplifying complex sentences) mitigates hallucination in Large Language Models (LLMs).

The script measures the Conditional Log-Likelihood of "Unfaithful" vs. "Fixed" summaries given the original source text versus a modified (paraphrased) source text.

For a given document and summary pair, we calculate the probability assigned by various LLMs (Vicuna, Llama-3, etc.) to specific target sentences:

$$\Delta \text{LogLikelihood} = \log P(\text{SummarySent} | (\text{Source}{\text{modified}} + \text{Prev SummarySents})) - \log P(\text{Summary} | (\text{Source}{\text{original}} + \text{Prev SummarySents}))$$

• Hypothesis: If the paraphrase is effective, the likelihood of the Unfaithful summary should decrease ($\Delta < 0$), and the likelihood of the Fixed summary should increase ($\Delta > 0$).

Usage

python mitigation.py \
    --data path/to/data.json \
    --out_dir ./results

Mitigation by Paraphrasing Results Compiler (results_to_csv.py)

This script processes the raw JSONL output files generated by the mitigation experiment pipeline and converts them into structured CSV reports for analysis.

1. Metric Aggregation: - Calculates the "Preference Gap" ($A = LL_{fixed} - LL_{unfaithful}$) for both original and modified sources.
   • Computes $\Delta LL$ across all models to see if the source modification successfully pushed the model toward the faithful summary.
2. Preference Mapping:
   • Generates a per-document table showing which state (Original, Modified, Both, or None) each model preferred.
3. Model-Specific Summaries: - Groups results by model (Vicuna, Llama, etc.) and provides mean values for statistical significance testing.

Metrics

The script tracks two primary values for each document/model pair:

• Value A (Baseline): How much more the model prefers the Fixed summary over the Unfaithful one given the original source.
• Value B (Mitigated): How much more the model prefers the Fixed summary given the paraphrased source.
• Improvement: Measured as $B - A$. A positive value indicates the source modification was successful.

Usage

Run the script by pointing to the log directory and providing the timestamp of the experiment run:

python analyze_results.py \
    --base_dir /path/to/log/dir \
    --timestamp "time_stamp"

Generation experiment (generation_exp.py)

This script executes an experiment to verify if modifying source documents (e.g., simplifying syntax, organizing discourse) can reduce hallucinations in abstractive summarization.

1. Baseline Generation: Summarizes the document using target models (e.g., Vicuna, Llama-3).
2. Faithfulness Check: Uses GeminiJudgeAttributor to label summary sentences as "yes" (faithful) or "no" (unfaithful) and identifies which source sentences are responsible ("attribution").
3. Targeted Modification: If hallucinations are detected, the SourceModifier uses GPT-4o to rewrite the specific implicated source sentences using a defined strategy (e.g., syntax_transform).
4. Re-Summarization: The model summarizes the modified source text.
5. Re-Evaluation: The new summary is judged again to measure the change in faithfulness rate.

Usage

1. Set API keys:

   export OPENAI_API_KEY="sk-..."
   export GEMINI_API_KEY="AIza..."
   

2. Adjust the model list and modification mathods in the if __name__ == "__main__": block.

3. Run:

   python generation_exp.py
   

Output

• template_stream_*.jsonl: A detailed line-by-line log of every summary sentence generated, its judge label, and any modifications applied.
• aggregate_*.json: High-level statistics comparing the unfaithfulness rate before (baseline) and after mitigation.