Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression

Zhengpei Hu^1,∗, Kai Li^2,∗, Dapeng Fu³, Xuechao Zou², Yuanhao Tang¹, Yue Li¹, Tengfei Cao¹, Jianqiang Huang^1,†
1School of Computer Technology and Application, Qinghai University
²Tsinghua University  ·  ³Ant Group Security and Intelligence Laboratory (SIL)
^∗Equal contribution  ·  ^†Corresponding author
🌐 Project Page | 💻 Code | 🤗 Model | 📄 Paper

English · 中文

💥 News

• [2026-07-19] We release the research code, interactive project page, and pretrained dependency detector! 🚀 (project page, code, model)

TL;DR: Hard prompt compression can preserve an answer-bearing fragment while deleting the definition or bridge fact needed to interpret it; we call this failure referential dangling and study how to diagnose and repair it.

🚀 Quick Start

Run these commands from the release root:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The following entry points expose the main analysis and training options (the --help calls only print arguments; they do not launch a full experiment):

# Required for BEAVER-based entry points after following third_party/README.md
export PYTHONPATH="$PWD/third_party/BEAVER:${PYTHONPATH:-}"

python src/diagnose.py --help
python src/metric_c.py --help
python src/exp_candidate_ablation.py --help
python src/expE_span_longdoc.py --help
python src/train_detector.py --help

Full runs additionally require the external compressor setup described in third_party/README.md, public benchmark inputs, a CUDA-capable GPU, and Hugging Face access for model checkpoints. The restoration scripts download the dependency detector from JusperLee/referential-dangling-detector by default; pass --bert_model /path/to/checkpoint to use a local checkpoint.

🔎 Why this matters

Long-context language models can process much more text than earlier systems, but prefill cost and distractor sensitivity make it attractive to shorten the input before inference. Hard compressors score tokens, sentences, or chunks independently and retain the most salient fragments under a budget.

The central question of this paper is whether individually important fragments remain jointly usable after compression. A fragment can still contain an answer string while losing the earlier definition, antecedent, or bridge fact that makes the answer interpretable. This failure is easy to miss if evaluation checks only relevance or answer-string retention.

These results show that referential dangling is widespread rather than a rare edge case, and that it causes substantial downstream errors. At a nominal compression ratio of 0.30, the diagnostic identifies incomplete answer paths in 34–54% of bridge examples; a human audit confirms genuine dependency loss with 95% precision, all six tested compressors exhibit dangling at 32–60%, and every one of the 80 LongBench-v2 documents examined contains at least one dangling reference. Under the same nominal token budget, dependency-preserving reselection improves Qwen3-8B accuracy by 28.8–34.3 points on dangling subsets. In a separate Base-versus-Full-support comparison, restoring both supporting paragraphs improves MuSiQue accuracy by 8.8 points for GPT-5.5 and 24.2 points for GLM-5.2.

🧠 Core idea

The paper introduces a dependency-oriented view of prompt compression:

1. Detect the break. Identify a retained fragment whose supporting definition, antecedent, or bridge is absent.
2. Measure the failure. Evaluate the phenomenon across compression ratios, datasets, scoring signals, output granularities, and long-document dependencies.
3. Repair the context. Reinsert missing support under a fixed or nearly fixed token budget, and test an automatic sentence-level restoration pipeline.

🔥 Main findings reported in the paper

• Under Beaver at a nominal compression ratio of 0.30, the diagnostic flags incomplete answer paths in 34–54% of bridge examples across HotpotQA, 2WikiMultiHopQA, and MuSiQue.
• A shared cross-compressor diagnostic observes dangling across six hard compressors, with reported rates from 32% to 60%.
• In a controlled content-reselection experiment, restoring omitted supporting content improves Qwen3-8B accuracy by 28.8–34.3 points on the dangling subsets.
• Automatic restoration improves HotpotQA accuracy by 4.7 points while changing the reported compression ratio from 0.30 to 0.31.
• A human audit confirms genuine dependency loss with 95% precision, and every one of the 80 LongBench-v2 documents examined contains at least one dangling reference.
• The paper also studies natural first-mention dependencies in LongBench-v2 and the sensitivity of the diagnostic to embedding choice, overlap threshold, and compression ratio.

📈 Visualizations

📊 Selected tables

Open-source and closed-source downstream model results under Base and Full support.

🖼️ Qualitative examples

The examples illustrate the recurring pattern: the compressed context retains a relevant-looking fragment, but the missing bridge prevents the downstream model from completing the intended inference.

📝 Citation

@article{hu2026relevant,
  title={Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression},
  author={Zhengpei Hu and Kai Li and Dapeng Fu and Xuechao Zou and Yuanhao Tang and Yue Li and Tengfei Cao and Jianqiang Huang},
  journal={arXiv preprint arXiv:2608.04569},
  year={2026},
  url={https://arxiv.org/pdf/2608.04569}
}

License

This package is released under the MIT License. Third-party compressors and benchmark datasets retain their original licenses.