The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality

How do I access the training data?

Available on Huggingface 🤗

How do I reproduce the experiments?

The scripts to run the experiments are in the scripts directory under the bios, medical, and plots subdirectories.

• probes.sh obtains the hidden states and trains the probe classifiers on them. Running this requires installing this fork of vllm and requires ~150GB of RAM. (If you have more or less RAM available, you can increase the inc variable in the scripts to use more memory and decrease it to use less.)
• finetune.sh runs the experiments. It performs the length control, and finetunes and evaluates the models. Evaluation metrics will be printed to stdout and to a proposition_confusion_mat.txt file in the results directory. The results directory will also be printed to stdout.
  • experiments/run.py is run by finetune.sh and handles the experiment running logic (e.g. deciding whether to filter the synthetic data, finetune models, or evaluate models) and submitting jobs using SLURM. You should change RUN_SH_DATA_TEMPLATE and RUN_SH_TEMPLATE to use your own bashrc and conda environment. Jobs are submitted with SLURM using a script called scripts/submit_job.py. You should provide your own implementation if you want to use do the same, otherwise the submit_job function will run on the same machine that finetune.sh is run on.

Please contact us for information about running the cross-domain experiments.

Notes

• For the llama 2 experiments, store the path to the Llama 2 weights in the LLAMA2_DIR environment variable (export LLAMA2_DIR=models/llama/llama-2-7b-chat-hf)
• Clone this fork of FActScore at the same directory level as this repo:

$ ls
FActScore/    epistemic_training/

Citation

@misc{newman2025curiousFactuality,
      title={The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality}, 
      author={Benjamin Newman and Abhilasha Ravichander and Jaehun Jung and Rui Xin and Hamish Ivison and Yegor Kuznetsov and Pang Wei Koh and Yejin Choi},
      year={2025},
      archivePrefix={arXiv},
      primaryClass={cs.CL}, 
}