Code for paper Guidance: Sentence-Level Citation Enforcement via Prefix-Tail Guidance during LLM Decoding
Python
0
2 commits
updated Jan 29, 2026
In correctness-sensitive scenarios, it is crucial for Large Language Models (LLMs) to strictly follow the provided evidence. However, even with reference texts, models often suffer from hallucinations, especially when processing long contexts. Existing work attempts to reinforce the use of citations through Retrieval-Augmented Generation (RAG) or post-hoc methods, while citations remain a probabilistic output rather than a foundation for the generated content. To address this, we propose Guidance, which aims to correct outputs and naturally incorporate citations during the LLM decoding phase. Specifically, we first build a structured fact pool (Prefix-Tail pairs) from the documents. Then, during inference, Guidance predicts the model's intent using a lookahead strategy. When it detects a match with a context prefix, it automatically replaces the output with the verified fact and its citation. This approach is training-free and can be plugged into general-purpose or citation-fine-tuned LLMs. Experiments on LongBench-Cite demonstrate that Guidance improves the citation F1 score by 11.2% over state-of-the-art baselines.
Install the dependencies with conda and pip:
conda env create -f environment_guidance.yml
conda activate guidance
pip install flash-attn --no-build-isolation
# Download spacy modules
python -m spacy download en_core_web_sm
python -m spacy download zh_core_web_sm
Before preprocessing, please set your API token and base_url in get_prefix_tail_pairs.py
python get_prefix_tail_pairs.py --input ../LongBench-Cite-full.json --output ./prefix_output/prefix_full_qwen-max.jsonl
After setting your model path, you can direct run bash file run_and_eval.sh
Please clone the repo of LongCite by git clone https://github.com/THUDM/LongCite.git and run the GPT-4o evaluation using the following two scripts: (you have to change the json file paths and api key in the scripts)
We also provide the following command to run evaluation code, but please first set you OpenAI key in ./LongBench-Cite/utils/llm_api.py:
python eval_cite.py --pred_path [your-result-path]
python eval_correct.py --pred_path [your-result-path]
Code for paper Guidance: Sentence-Level Citation Enforcement via Prefix-Tail Guidance during LLM Decoding
Python
0
2 commits
updated Jan 29, 2026
In correctness-sensitive scenarios, it is crucial for Large Language Models (LLMs) to strictly follow the provided evidence. However, even with reference texts, models often suffer from hallucinations, especially when processing long contexts. Existing work attempts to reinforce the use of citations through Retrieval-Augmented Generation (RAG) or post-hoc methods, while citations remain a probabilistic output rather than a foundation for the generated content. To address this, we propose Guidance, which aims to correct outputs and naturally incorporate citations during the LLM decoding phase. Specifically, we first build a structured fact pool (Prefix-Tail pairs) from the documents. Then, during inference, Guidance predicts the model's intent using a lookahead strategy. When it detects a match with a context prefix, it automatically replaces the output with the verified fact and its citation. This approach is training-free and can be plugged into general-purpose or citation-fine-tuned LLMs. Experiments on LongBench-Cite demonstrate that Guidance improves the citation F1 score by 11.2% over state-of-the-art baselines.
Install the dependencies with conda and pip:
conda env create -f environment_guidance.yml
conda activate guidance
pip install flash-attn --no-build-isolation
# Download spacy modules
python -m spacy download en_core_web_sm
python -m spacy download zh_core_web_sm
Before preprocessing, please set your API token and base_url in get_prefix_tail_pairs.py
python get_prefix_tail_pairs.py --input ../LongBench-Cite-full.json --output ./prefix_output/prefix_full_qwen-max.jsonl
After setting your model path, you can direct run bash file run_and_eval.sh
Please clone the repo of LongCite by git clone https://github.com/THUDM/LongCite.git and run the GPT-4o evaluation using the following two scripts: (you have to change the json file paths and api key in the scripts)
We also provide the following command to run evaluation code, but please first set you OpenAI key in ./LongBench-Cite/utils/llm_api.py:
python eval_cite.py --pred_path [your-result-path]
python eval_correct.py --pred_path [your-result-path]