🔥🔥🔥[ICLR 2026 Oral] Hallucination Begins Where Saliency Drops

The motivation of this paper is

The patterns of incorrect and correct tokens are as follows:

After intervention, incorrect tokens became correct tokens, and the saliency score increased significantly.

For LLaVA1.5

python demo_step1.py,python demo_step2.py

For Qwen2-VL

python Qwen_step1.py,python Qwen_step2.py

Citation

@inproceedings{
zhang-saliency,
title={Hallucination Begins Where Saliency Drops},
author={Xiaofeng Zhang, Yuanchao Zhu, Chaochen Gu, Xiaosong Yuan, Qiyan Zhao, Jiawei Cao, Feilong Tang, Sinan Fan, Yaomin Shen, Chen Shen, Hao Tang },
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=sjnErRHXf3}
}

Acknowledgement

This repo is built on LLaVA (models), Qwen2.5-VL (CHAIR evaluation) and Label words. Many thanks for their efforts. The use of our code should also follow the original licenses.