LLM_Attribution

Context attribution for large language model outputs. Given a (context, query) pair and the model's generated response, this project identifies which sentences of the context are most responsible for that response.

Attribution is computed by fitting a sparse linear surrogate model (Lasso) over many randomly ablated versions of the context — measuring how the response's likelihood changes as different sentences are removed — following the ContextCite approach.

Files

• attributor.py — core implementation: the Attributor class.
• main.py — a standalone, class-free reimplementation of the same algorithm, kept for manual testing.
• test.py — smoke test on a hardcoded example.
• evaluation.py — top-k log-probability-drop evaluation comparing Attributor, ContextCite, and leave-one-out.
• verification.py — Natural Questions evaluation: attribute, then verify the answer still follows from the top-k sources.
• attacks.py — evaluates attribution accuracy on poisoned contexts.

Usage

The scripts have no build system or CLI; edit the experiment constants at the top of a script and run it directly:

python attributor.py    # self-test on the built-in example

Most drivers require a CUDA GPU (models are loaded in 4-bit via bitsandbytes) and access to gated Hugging Face models (Llama-3, Phi-3).

Dependencies

torch, transformers, accelerate, bitsandbytes, nltk, scikit-learn, datasets, context_cite, matplotlib, tqdm.

The evaluation scripts also expect datasets in sibling directories (../context-cite/, ../cnn_dailymail/, ../NQ_dataset/).