Awesome Reliable LLMs

Factuality · Honesty · Consistency

A curated literature collection on building reliable large language models to mitigate hallucination.

Background  ·  Factuality  ·  Honesty  ·  Consistency  ·  Contribute

---

This project follows the research themes of my PhD thesis, A Framework of Building Reliable Large Language Models to Mitigate Hallucination, and curates related literature on factuality, honesty, and consistency.

Introduction

Mitigating hallucination requires understanding both why models produce unreliable responses and how to improve their behavior. Reliability encompasses factual accuracy, awareness of knowledge limitations, and coherence with relevant context.

A reliable LLM should use knowledge to produce factually accurate responses, honestly communicate uncertainty and knowledge limitations, and maintain contextual consistency throughout multi-turn interactions.

Research on hallucination, knowledge, and uncertainty is organized within these three dimensions. Papers are grouped by their primary research question, with cross-references for work spanning multiple dimensions.

Outline

Browse all sections and research topics

• Motivation and Background
  • Hallucination: Definitions and Scope
  • Causes of Hallucination
  • Three Manifestations of Unreliability
  • From Hallucination Mitigation to LLM Reliability
• Factuality
  • Knowledge Boundary
  • Hallucination Detection
  • Factuality Alignment
  • Factuality Inference
  • Factuality Evaluation
• Honesty
  • Self-Awareness and Knowledge Limits
  • Confidence and Uncertainty Estimation
  • Confidence Calibration
  • Uncertainty Expression and Abstention
  • Honesty Evaluation
• Consistency
  • Contextual Consistency and Faithfulness
  • Improving Contextual Consistency
  • Consistency Evaluation
• Open Challenges and Future Directions
• Contributing

How to read the bibliography

Papers are grouped by their primary research focus and ordered by year, newest first. Each entry links to the paper and gives its conference, workshop, or journal and year. Papers with a verified acceptance but no linked proceedings version are marked accepted. Where a formal venue has not been verified, the entry is labeled arXiv preprint with its initial submission month. Bibliographic information was checked on 2026-09-22.

---

Motivation and Background

Hallucination: Definitions and Scope

Large language models have advanced from text generation to agents that retrieve information, use tools, and interact with external environments. Yet they can still produce factually incorrect, fabricated, deceptive, or inconsistent content. These hallucination-related failures pose a central challenge to LLM reliability.

Such failures can mislead users even when the responses appear fluent and confident. The examples below illustrate this concern in health care and economics, motivating the need for reliable generation in knowledge-intensive applications.

_{Illustrative hallucinated responses in health care and economics; highlighted claims demonstrate unreliable outputs.}

Causes of Hallucination

A central challenge is the gap between learning frequent textual patterns and reliably applying factual knowledge. During pre-training, next-token prediction teaches models to produce plausible continuations from large text corpora. During post-training, models learn response patterns from instruction and feedback data. Learning these patterns does not by itself ensure factual understanding or a clear boundary between known and unknown information.

The example below illustrates how a familiar statement about the first person on the Moon can be reused to answer a superficially similar question about Mars. This pattern-matching perspective connects the training process to the unreliable behaviors discussed next.

_{An illustration of hallucination arising from reliance on familiar textual patterns.}

Three Manifestations of Unreliability

Unreliable behavior manifests at three levels: the model's expression of its own knowledge, the factual content of a single response, and the coherence of responses across an interaction.

1. Dishonesty: the model claims knowledge or expresses unwarranted certainty when it should acknowledge uncertainty or limitations.
2. Non-factual responses: an individual answer contains incorrect or unsupported factual claims.
3. Contextual inconsistency: responses contradict earlier statements or depart from relevant context during multi-turn interactions.

_{Three manifestations of unreliability: dishonesty, non-factual responses, and contextual inconsistency.}

These failure modes motivate complementary reliability requirements. Assessing answer correctness alone does not capture whether a model communicates its limitations appropriately or remains consistent throughout a conversation.

From Hallucination Mitigation to LLM Reliability

Research on hallucination mitigation spans data enhancement, fine-tuning, and retrieval-augmented generation (RAG). The causes and manifestations of hallucination motivate three complementary objectives for reliable LLMs:

• Factuality: use available knowledge to generate factually accurate responses.
• Honesty: communicate uncertainty and acknowledge knowledge limitations when faced with unsure queries.
• Consistency: maintain coherence with relevant context throughout multi-turn interactions.

_{A conceptual framework connecting hallucination causes, unreliable outputs, and the three dimensions of LLM reliability.}

Following this progression, the literature collection is organized into Factuality, Honesty, and Consistency. Each section covers the corresponding research questions, related approaches, and evaluation settings, with links across dimensions where their concerns overlap.

↑ Back to top

---

Factuality

Focus: the factual correctness of generated content, especially in knowledge-intensive responses.

Improving factuality involves understanding knowledge boundaries, detecting factual errors, improving responses through alignment and inference, and evaluating the resulting generations. The following reading lists cover these research questions, methods, and evaluation resources.

Knowledge Boundary

Studies of what a model knows, how reliably it can access that knowledge, and how prompting, retrieval, and fine-tuning affect its knowledge limits.

• Query-Level Uncertainty in Large Language Models — ICLR 2026.
• Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Tasks — ICLR 2026.
• Parametric Knowledge is Not All You Need: Toward Honest Large Language Models via Retrieval of Pretraining Data — Findings of ACL 2026.
• Understanding New-Knowledge-Induced Factual Hallucinations in LLMs: Analysis and Interpretation — Findings of ACL 2026.
• Purging the Gray Zone: Latent-Geometric Denoising for Precise Knowledge Boundary Awareness — Findings of ACL 2026.
• Knowledge Boundary Discovery for Large Language Models — arXiv preprint, 2026-01.
• Know2Guess: A Contamination-Aware Multi-Zone Benchmark for Knowledge-Boundary Evaluation in Large Language Models — ICONIP 2026 (accepted).
• Toward a Gricean Retreat: Probing LLMs for Knowledge Boundaries and Referent Specificity — arXiv preprint, 2026-08.
• KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Models — Findings of EMNLP 2025.
• Knowledge Boundary of Large Language Models: A Survey — ACL 2025.
• Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation — COLING 2025.
• Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? — EMNLP 2024.
• Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluation — ACL 2024.
• Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models — Findings of ACL 2024.
• Can AI Assistants Know What They Don’t Know? — ICML 2024.
• Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs? — NAACL 2024.
• Do Large Language Models Know What They Don’t Know? — Findings of ACL 2023.
• Language Models as Knowledge Bases? — EMNLP-IJCNLP 2019.

Hallucination Detection

Methods for identifying factual errors or estimating hallucination risk using sampled responses, semantic uncertainty, internal states, and reasoning behavior. These approaches connect hallucination detection with probing a model's knowledge and uncertainty.

• Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgments — ACL 2026.
• Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval-Augmented Generation — Findings of ACL 2026.
• FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs — Findings of EACL 2026.
• TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Models — ICLR 2026.
• HARP: Hallucination Detection via Reasoning Subspace Projection — ICLR 2026.
• Learning to Reason for Hallucination Span Detection — ICLR 2026.
• MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probing — MeLLM Workshop 2026.
• Domain-Specific Hallucination Detection in Large Language Models — arXiv preprint, 2026-09.
• Hallucination Detection in Large Language Models Using Diversion Decoding — arXiv preprint, 2026-07.
• Simple Factuality Probes Detect Hallucinations in Long-Form Natural Language Generation — Findings of EMNLP 2025.
• REFIND at SemEval-2025 Task 3: Retrieval-Augmented Factuality Hallucination Detection in Large Language Models — SemEval Workshop 2025.
• Unsupervised Hallucination Detection by Inspecting Reasoning Processes — EMNLP 2025.
• On the Universal Truthfulness Hyperplane Inside LLMs — EMNLP 2024.
• LLM Internal States Reveal Hallucination Risk Faced With a Query — BlackboxNLP Workshop 2024.
• Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models — Findings of ACL 2024.
• Detecting hallucinations in large language models using semantic entropy — Nature 2024.
• INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection — ICLR 2024.
• SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models — EMNLP 2023.
• The Internal State of an LLM Knows When It’s Lying — Findings of EMNLP 2023.

Factuality Alignment

Training approaches that improve factual responses through data construction, supervised fine-tuning, factuality preferences, reinforcement learning, or knowledge distillation. This category also includes learning to refuse questions beyond the model's knowledge.

• MR-ALIGN: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Models — Findings of ACL 2026.
• KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality — ACL 2026.
• PretrainRL: Alleviating Factuality Hallucination of Large Language Models at the Beginning — Findings of ACL 2026.
• FAITH: Factuality Alignment through Integrating Trustworthiness and Honestness — Findings of ACL 2026.
• The Unintended Trade-off of AI Alignment: Balancing Hallucination Mitigation and Safety in LLMs — Findings of EACL 2026.
• Scaling LLM Knowledge Boundaries via Distribution-Optimized Synthesis — arXiv preprint, 2026-06.
• UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models — ACL 2025.
• Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite Learning — ICLR 2025.
• InFact: Informativeness Alignment for Improved LLM Factuality — arXiv preprint, 2025-05.
• Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling — COLM 2025.
• Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation — arXiv preprint, 2025-02.
• FactAlign: Long-form Factuality Alignment of Large Language Models — Findings of EMNLP 2024.
• FLAME: Factuality-Aware Alignment for Large Language Models — NeurIPS 2024.
• Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback — COLM 2024.
• Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation — ACL 2024.
• Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigation — KnowledgeNLP Workshop 2024.
• R-Tuning: Instructing Large Language Models to Say ‘I Don’t Know’ — NAACL 2024.
• Fine-Tuning Language Models for Factuality — ICLR 2024.
• Factuality Enhanced Language Models for Open-Ended Text Generation — NeurIPS 2022.

Factuality Inference

Approaches that improve factual generation through retrieval, prompting, sampling, decoding, activation intervention, or verification and correction. Retrieval systems that also require training are included here for their generation-time use of external evidence.

• Leveraging Pretrained Knowledge at Inference Time: LoRA-Gated Contrastive Decoding for Multilingual Factual Language Generation in Adapted LLMs — ICLR 2026.
• Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders — ICLR 2026.
• CoFact: Conformal Factuality Guarantees for Language Models under Covariate Shift — ICLR 2026.
• Dynamic PMI-Guided Contrastive Decoding Reduces Hallucination in Large Language Models: A Unified Framework of Fine-Grained Input Transformations — Findings of ACL 2026.
• Enhancing Factuality through Consensus and Consistency in Summarization Using Minimum Bayes Risk Decoding — Findings of ACL 2026.
• A Stitch in Time Saves Nine: Proactive Self-Refinement for Language Models — ICLR 2026.
• TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space — ACL 2024.
• Chain-of-Verification Reduces Hallucination in Large Language Models — Findings of ACL 2024.
• Trusting Your Evidence: Hallucinate Less with Context-aware Decoding — NAACL 2024.
• DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models — ICLR 2024.
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection — ICLR 2024.
• RARR: Researching and Revising What Language Models Say, Using Language Models — ACL 2023.
• Inference-Time Intervention: Eliciting Truthful Answers from a Language Model — NeurIPS 2023.
• Active Retrieval Augmented Generation — EMNLP 2023.
• Towards Mitigating LLM Hallucination via Self Reflection — Findings of EMNLP 2023.
• A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation — arXiv preprint, 2023-07.
• Self-Consistency Improves Chain of Thought Reasoning in Language Models — ICLR 2023.
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — NeurIPS 2022.
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — NeurIPS 2020.

Factuality Evaluation

Benchmarks and evaluation methods for factual correctness, hallucination recognition, and factual support in generated text. The list includes question-answering datasets such as TriviaQA, SciQ, and Natural Questions, together with the work introducing the open-domain NQ setup. Coverage extends to short-form answers, long-form claims, and retrieval-grounded generation.

• PROBE: PROcess-Based BEnchmark for Hallucination Detection — Findings of ACL 2026.
• When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation — EACL 2026.
• KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge — Findings of EACL 2026.
• Truth or Mirage? Towards End-To-End Factuality Evaluation with LLM-Oasis — Computational Linguistics 2026.
• TruthTrap: A Bilingual Benchmark for Evaluating Factually Correct Yet Misleading Information in Question Answering — Findings of EACL 2026.
• Stress Testing Factual Consistency Metrics for Long-Document Summarization — ACL 2026.
• DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence — ICLR 2026.
• Detecting Hallucinations in Authentic LLM–Human Interactions — LREC 2026.
• IRB: Automated Generation of Robust Factuality Benchmarks — arXiv preprint, 2026-02.
• AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains — arXiv preprint, 2026-01.
• FaStFact: Faster, Stronger Long-Form Factuality Evaluations in LLMs — Findings of EMNLP 2025.
• HalluLens: LLM Hallucination Benchmark — ACL 2025.
• Beyond Factual Accuracy: Evaluating Coverage of Diverse Factual Information in Long-form Text Generation — Findings of ACL 2025.
• FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation — ACL 2025.
• Long-form factuality in large language models — NeurIPS 2024. LongFact / SAFE.
• Measuring short-form factuality in large language models — arXiv preprint, 2024-11. SimpleQA.
• RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models — ACL 2024.
• FELM: Benchmarking Factuality Evaluation of Large Language Models — NeurIPS 2023, Datasets and Benchmarks Track.
• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation — EMNLP 2023.
• HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models — EMNLP 2023.
• TruthfulQA: Measuring How Models Mimic Human Falsehoods — ACL 2022.
• Latent Retrieval for Weakly Supervised Open Domain Question Answering — ACL 2019. ORQA / NQ-Open setup.
• Natural Questions: A Benchmark for Question Answering Research — TACL 2019.
• Crowdsourcing Multiple Choice Science Questions — W-NUT Workshop 2017. SciQ.
• TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension — ACL 2017.

↑ Back to top

---

Honesty

Focus: recognizing knowledge and capability limits, and faithfully communicating uncertainty about generated responses.

Honesty involves recognizing knowledge and capability limits, estimating and calibrating confidence, and appropriately expressing uncertainty. It is model-specific: evaluating honesty requires assessing how a model's expressed certainty and answering behavior relate to its own knowledge and capabilities.

Self-Awareness and Knowledge Limits

Studies of a model's ability to assess what it knows, recognize insufficient information, false premises, or ill-posed problems, and decide when to seek external evidence. This includes identifying mathematical unsolvability and distinguishing it from the model's own capability limits. Related studies of known and unknown questions are also collected under Knowledge Boundary.

• Beyond "I Don’t Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty — ACL 2026.
• Self-Awareness before Action: Mitigating Logical Inertia via Proactive Cognitive Awareness — Findings of ACL 2026.
• Strangers to Themselves: What Language Models Say About Themselves Is Generic — arXiv preprint, 2026-09.
• Large Language Models Struggle with Unreasonability in Math Problems — AAAI 2026.
• A Survey on the Honesty of Large Language Models — TMLR 2025.
• Agentic Knowledgeable Self-awareness — ACL 2025.
• Looking Inward: Language Models Can Learn About Themselves by Introspection — ICLR 2025.
• Line of Duty: Evaluating LLM Self-Knowledge via Consistency in Feasibility Boundaries — TrustNLP Workshop 2025.
• Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home — ACL 2025.
• Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questions — NAACL 2025.
• Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill? — COLM 2025.
• VCSearch: Bridging the Gap Between Well-Defined and Ill-Defined Problems in Mathematical Reasoning — EMNLP 2025.
• Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting — NAACL 2024.
• Discovering Latent Knowledge in Language Models Without Supervision — ICLR 2023.
• Language Models (Mostly) Know What They Know — arXiv preprint, 2022-07.

Confidence and Uncertainty Estimation

Methods for estimating uncertainty from token probabilities, verbalized confidence, sampled responses, semantic variation, and internal representations. The list also includes uncertainty signals used to guide learning and agent decisions. Applications to factual-error detection are collected under Hallucination Detection.

_{Overview of four families of confidence and uncertainty estimation methods—likelihood-based, prompting-based, sampling-based, and training-based—and their limitations.}

• Confidence Estimation for LLMs in Multi-turn Interactions — Findings of ACL 2026.
• Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification — ICLR 2026.
• When Models Hesitate: Answer Instability as a Label-Free Uncertainty Signal for LLMs — ACL Student Research Workshop 2026.
• Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities — ACL 2026.
• ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps — arXiv preprint, 2026-09.
• Improving Uncertainty Estimation through Semantically Diverse Language Generation — ICLR 2025.
• Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach — arXiv preprint, 2024-04.
• Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models — TMLR 2024.
• Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities — NeurIPS 2024.
• Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs — ICLR 2024.
• Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness — ACL 2024.
• Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models — ACL 2024.
• Uncertainty Aware Learning for Language Model Alignment — ACL 2024.
• Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs — arXiv preprint, 2024-06.
• SPUQ: Perturbation-Based Uncertainty Quantification for Large Language Models — EACL 2024.
• Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation — ICLR 2023.
• Efficient Out-of-Domain Detection for Sequence to Sequence Models — Findings of ACL 2023.
• Learning Confidence for Transformer-based Neural Machine Translation — ACL 2022.
• Uncertainty Estimation in Autoregressive Structured Prediction — ICLR 2021.
• On Hallucination and Predictive Uncertainty in Conditional Language Generation — EACL 2021.

Confidence Calibration

Methods for aligning confidence estimates with observed correctness, including prompting, post-hoc recalibration, supervised learning, and reinforcement learning. Coverage extends from calibration foundations to long-form generation, confidence in reasoning steps and final answers, multi-turn conversations, and tool-using agents.

• Annotation-Efficient Honesty Alignment via Confidence Elicitation and Calibration — ICLR 2026.
• Confidence Should Be Calibrated More Than One Turn Deep — ACL 2026.
• The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents — ACL 2026.
• Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning — Findings of ACL 2026.
• BaseCal: Unsupervised Confidence Calibration via Base Model Signals — ACL 2026.
• Calibrating Verbalized Confidence with Self-Generated Distractors — ICLR 2026.
• Confidence over Time: Confidence Calibration with Temporal Logic for Large Language Model Reasoning — Findings of ACL 2026.
• DisCal: Distribution-Aware Calibration for Mathematical Reasoning Under Character-Level Noisy Inputs — ACL 2026.
• Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty — ICLR 2026.
• Calibrating Large Language Models with Sample Consistency — AAAI 2025.
• Large Language Models Must Be Taught to Know What They Don’t Know — NeurIPS 2024.
• Linguistic Calibration of Long-Form Generations — ICML 2024.
• Calibrating the Confidence of Large Language Models by Eliciting Fidelity — EMNLP 2024.
• Few-Shot Recalibration of Language Models — arXiv preprint, 2024-03.
• Calibrating Large Language Models Using Their Generations Only — ACL 2024.
• When to Trust LLMs: Aligning Confidence with Response Quality — Findings of ACL 2024.
• Preserving Pre-trained Features Helps Calibrate Fine-tuned Language Models — ICLR 2023.
• Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback — EMNLP 2023.
• Reducing Conversational Agents’ Overconfidence Through Linguistic Calibration — TACL 2022.
• How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering — TACL 2021.
• Knowing More About Questions Can Help: Improving Calibration in Question Answering — Findings of ACL-IJCNLP 2021.
• Calibration of Pre-trained Transformers — EMNLP 2020.
• On Calibration of Modern Neural Networks — ICML 2017.

Uncertainty Expression and Abstention

Research on communicating uncertainty in words or numbers, explaining uncertainty, asking for clarification, and appropriately declining to answer. The goal is to acknowledge limitations while retaining useful answers to answerable questions. Related refusal-training approaches, including R-Tuning and RL from knowledge feedback, appear under Factuality Alignment.

• When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond? — ICLR 2026.
• Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL — Findings of ACL 2026.
• High Accuracy, Less Talk (HALT): Reliable LLMs through Capability-Aligned Finetuning — ICLR 2026.
• SAFER: Risk-Constrained Sample-then-Filter in Large Language Models — ICLR 2026.
• KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoning — ICLR 2026.
• Structured Uncertainty guided Clarification for LLM Agents — Findings of ACL 2026.
• Rewarding Intellectual Humility Learning When Not To Answer In Large Language Models — arXiv preprint, 2026-01.
• Answering the Unanswerable Is to Err Knowingly: Analyzing and Mitigating Abstention Failures in Large Reasoning Models — AAAI 2026.
• Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words? — EMNLP 2024.
• Alignment for Honesty — NeurIPS 2024.
• HonestLLM: Toward an Honest and Helpful Large Language Model — NeurIPS 2024.
• Enhancing Confidence Expression in Large Language Models Through Learning from Past Experience — arXiv preprint, 2024-04.
• SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales — EMNLP 2024.
• Relying on the Unreliable: The Impact of Language Models’ Reluctance to Express Uncertainty — ACL 2024.
• "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust — FAccT 2024.
• Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learning — arXiv preprint, 2023-10.
• Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models — EMNLP 2023.
• Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge — arXiv preprint, 2023-11.
• Teaching Models to Express Their Uncertainty in Words — TMLR 2022.

Honesty Evaluation

Surveys, benchmarks, and evaluation studies for self-knowledge, confidence estimation, calibration, and faithful uncertainty expression. Evaluation considers discrimination between correct and incorrect answers, calibration, appropriate abstention, and robustness across tasks and interaction settings. Evaluation settings include mathematical solvability, theorem-proving sycophancy, and multilingual confidence estimation.

• OpenEstimate: Evaluating LLMs on Reasoning Under Uncertainty with Real-World Data — ICLR 2026.
• AwarenessBench: Assessing Cognitive Capabilities of Language Models — ACL 2026.
• CAR-bench: Evaluating the Consistency and Limit-Awareness of LLM Agents under Real-World Uncertainty — ACL 2026.
• Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs — arXiv preprint, 2026-05.
• The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems — arXiv preprint, 2025-03.
• MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models — Findings of ACL 2025.
• BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMs — arXiv preprint, 2025-10.
• ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Models — arXiv preprint, 2025-07.
• A Survey of Confidence Estimation and Calibration in Large Language Models — NAACL 2024.
• Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models — ACL 2024.
• Uncertainty is Fragile: Manipulating Uncertainty in Large Language Models — arXiv preprint, 2024-07.
• When Quantization Affects Confidence of Large Language Models? — Findings of NAACL 2024.
• Uncertainty in Language Models: Assessment through Rank-Calibration — EMNLP 2024.
• BeHonest: Benchmarking Honesty in Large Language Models — arXiv preprint, 2024-06.
• Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4 — EMNLP 2023.
• Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis — Findings of EMNLP 2022.
• Re-Examining Calibration: The Case of Question Answering — Findings of EMNLP 2022.

↑ Back to top

---

Consistency

Focus: agreement with relevant evidence, task constraints, and interaction history, including multi-turn dialogue, RAG, and search agents.

Maintaining consistency requires understanding contextual faithfulness and the effects of context interference, developing mitigation methods, and evaluating behavior across interactions. When sources conflict or new evidence becomes available, it also involves resolving conflicts and appropriately updating earlier claims.

Contextual Consistency and Faithfulness

Studies of how generated content relates to source material, task instructions, and interaction history. Coverage includes context–memory conflicts, conflicts among external sources, irrelevant-context interference, long-context information loss, and contradictions across conversation turns. These studies connect source faithfulness in summarization and dialogue to reliability in multi-turn agents.

• Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models — TMLR 2026.
• LLMs Get Lost In Multi-Turn Conversation — ICLR 2026.
• Large Language Models in Resolving Contextual Knowledge Conflicts — EMNLP 2026 (accepted).
• Do LLMs Make More Mistakes If They Do Not Believe the Input Data? — INLG 2026 (accepted).
• Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generation — arXiv preprint, 2026-08.
• Consistency of Large Reasoning Models Under Multi-Turn Attacks — arXiv preprint, 2026-02.
• A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems — ACM Computing Surveys, 2025 (online publication).
• Cats Confuse Reasoning LLM: Query-Agnostic Adversarial Triggers for Reasoning Models — COLM 2025.
• Knowledge Conflicts for LLMs: A Survey — EMNLP 2024.
• Lost in the Middle: How Language Models Use Long Contexts — TACL 2024.
• LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History — EMNLP 2024.
• Ask Again, Then Fail: Large Language Models’ Vacillations in Judgment — ACL 2024.
• Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts — ICLR 2024.
• In-context Interference in Chat-based Large Language Models — arXiv preprint, 2023-09.
• Consistency Analysis of ChatGPT — EMNLP 2023.
• Large Language Models Can Be Easily Distracted by Irrelevant Context — ICML 2023.
• Characterizing Mechanisms for Factual Recall in Language Models — EMNLP 2023.
• “Do you follow me?”: A Survey of Recent Approaches in Dialogue State Tracking — SIGDIAL 2022.
• Entity-Based Knowledge Conflicts in Question Answering — EMNLP 2021.
• On Faithfulness and Factuality in Abstractive Summarization — ACL 2020.

Improving Contextual Consistency

Approaches for improving source faithfulness and coherence across interactions. Coverage includes key-information extraction and reranking, context compression, and prompting, together with training for context adherence, conflict-aware decoding, verification, and response correction. IRCoT, ReAct, Search-o1, Search-R1, and RAG-Gym extend this coverage to iterative retrieval, reasoning, and agent training. Related work on context-aware decoding, Self-RAG, and general retrieval-augmented generation is listed under Factuality Inference.

• Mitigating Context Interference for Reliable and Efficient Search Agents — ACL 2026.
• Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learning — AAAI 2026.
• Conflict-Aware RAG: Multi-Stage Learning with Conflict Signals for Robust Retrieval-Augmented Generation — WWW 2026.
• MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversation — Findings of ACL 2026.
• Copy-Paste to Mitigate Large Language Model Hallucinations — ICLR 2026.
• PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise — ACL 2026.
• Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing — ACL 2026.
• When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generation — arXiv preprint, 2026-08.
• Improving Multi-turn Dialogue Consistency with Self-Recall Thinking — arXiv preprint, 2026-05.
• Faithfulness-QA: A Counterfactual Entity Substitution Dataset for Training Context-Faithful RAG Models — arXiv preprint, 2026-04.
• Self-Correcting RAG: Enhancing Faithfulness via MMKP Context Selection and NLI-Guided MCTS — arXiv preprint, 2026-04.
• Context-DPO: Aligning Language Models for Context-Faithfulness — Findings of ACL 2025.
• To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts — ICLR 2025.
• Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation — WWW 2025.
• MA-RAG: Multi-Agent Retrieval-Augmented Generation via Collaborative Chain-of-Thought Reasoning — arXiv preprint, 2025-05.
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning — COLM 2025.
• Search-o1: Agentic Search-Enhanced Large Reasoning Models — EMNLP 2025.
• Supervising the search process produces reliable and generalizable information-seeking agents — arXiv preprint, 2025-02. RAG-Gym.
• RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs — NeurIPS 2024.
• LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression — ACL 2024.
• LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression — Findings of ACL 2024.
• Found in the middle: Calibrating Positional Attention Bias Improves Long Context Utilization — Findings of ACL 2024.
• Discerning and Resolving Knowledge Conflicts through Adaptive Decoding with Contextual Information-Entropy Constraint — Findings of ACL 2024.
• Tug-of-War between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models — LREC-COLING 2024.
• Making Retrieval-Augmented Language Models Robust to Irrelevant Context — ICLR 2024.
• Enhancing Robustness in Large Language Models: Prompting for Mitigating the Impact of Irrelevant Information — arXiv preprint, 2024-08.
• Context-faithful Prompting for Large Language Models — Findings of EMNLP 2023.
• Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment — Findings of EMNLP 2023.
• LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models — EMNLP 2023.
• Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions — ACL 2023.
• ReAct: Synergizing Reasoning and Acting in Language Models — ICLR 2023.
• Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback — arXiv preprint, 2023-02.
• Re2G: Retrieve, Rerank, Generate — NAACL 2022.
• Retrieval Augmentation Reduces Hallucination in Conversation — Findings of EMNLP 2021.
• Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features — ACL-IJCNLP 2021.
• Improving Factual Consistency of Abstractive Summarization via Question Answering — ACL-IJCNLP 2021.
• Enhancing Factual Consistency of Abstractive Summarization — NAACL 2021.

Consistency Evaluation

Benchmarks, metrics, and human-evaluation protocols for source support, contradiction detection, knowledge-conflict handling, conversational memory, and consistency across turns. Coverage spans summarization, data-to-text generation, knowledge-grounded dialogue, RAG, and agent interactions. Question-answering datasets provide task-level evaluation resources, complemented by checks of source support and consistency across turns. Evaluation considers evidence use and context adherence alongside answer correctness, task success, and interaction cost. Related resources, including Natural Questions, TriviaQA, RAGTruth, DeepTRACE, and long-document factual-consistency stress tests, are listed under Factuality Evaluation.

• Time-To-Inconsistency: A Survival Analysis of Large Language Model Robustness to Adversarial Attacks — ICLR 2026.
• Ragability Benchmark: A Dataset and Library to Test LLMs on Inter-context Conflicts — LREC 2026.
• LIT-RAGBench: Benchmarking Generator Capabilities of Large Language Models in Retrieval-Augmented Generation — LREC 2026.
• MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations — Findings of ACL 2026.
• SemEval-2026 Task 8: MTRAGEval: Evaluating Multi-Turn RAG Conversations — SemEval Workshop 2026.
• Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence — Findings of ACL 2026.
• FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows" — ICLR 2025.
• FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs — NAACL 2025.
• MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs — Findings of ACL 2025.
• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory — ICLR 2025.
• mtRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems — TACL 2025.
• Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactions — Findings of ACL 2025.
• ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMs — NeurIPS 2024, Datasets and Benchmarks Track.
• Benchmarking Large Language Models in Retrieval-Augmented Generation — AAAI 2024.
• Evaluating Very Long-Term Conversational Memory of LLM Agents — ACL 2024.
• ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems — NAACL 2024.
• RAGAs: Automated Evaluation of Retrieval Augmented Generation — EACL 2024, System Demonstrations.
• Enabling Large Language Models to Generate Text with Citations — EMNLP 2023.
• RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledge — arXiv preprint, 2023-11.
• When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories — ACL 2023. PopQA.
• Measuring and Narrowing the Compositionality Gap in Language Models — Findings of EMNLP 2023. Bamboogle / Self-Ask.
• TRUE: Re-evaluating Factual Consistency Evaluation — NAACL 2022.
• QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization — NAACL 2022.
• SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization — TACL 2022.
• FaithDial: A Faithful Benchmark for Information-Seeking Dialogue — TACL 2022.
• Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark — TACL 2022.
• DialFact: A Benchmark for Fact-Checking in Dialogue — ACL 2022.
• ♫ MuSiQue: Multihop Questions via Single-hop Question Composition — TACL 2022.
• Don’t be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System — EMNLP 2021.
• Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation — Findings of EMNLP 2021.
• Evaluating the Factual Consistency of Abstractive Text Summarization — EMNLP 2020.
• Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference — INLG 2020.
• A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems — INLG 2020.
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps — COLING 2020. 2WikiMultiHopQA.
• Dialogue Natural Language Inference — ACL 2019.
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering — EMNLP 2018.

↑ Back to top

---

Open Challenges and Future Directions

• Joint evaluation and improvement of factuality, honesty, and consistency.
• Reliable long-form generation and extended agent interactions.
• Generalization across languages, domains, and reasoning tasks.
• Reliability in multimodal LLMs and agents.
• Balancing factual accuracy, appropriate abstention, contextual consistency, and computational cost.

Contributing

Contributions of relevant papers, surveys, benchmarks, and tools are welcome. Place each work under the subsection matching its main research question and cross-reference other dimensions when useful.

Use a linked title followed by the publication venue and year, matching the existing reading lists:

- [Paper title](PAPER_URL) — **Venue YYYY**.

Verify bibliographic details before adding an entry. Briefly describe its relevance when helpful, and include code or data links when available.

↑ Back to top