Awesome-reasoning-safety

This repo is for the trustworthy topics in reasoning-related technique, including but not limited to attacks, defenses, studies and benchmarks related to CoT, reasoning and RL. Data are mainly from arxiv.

🚀 Our Survey

📢 Check out our survey paper: A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models

📖 Citation

If you find our survey useful, please cite it as:

@article{wang2025comprehensive,
  title   = {A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models},
  author  = {Wang, Yanbo and Yu, Yongcan and Liang, Jian and He, Ran},
  journal = {arXiv preprint arXiv:2509.03871},
  year    = {2025}
}

---

🤝 Contribute

We welcome contributions from the community, including adding your papers, modifying topics, or other comments related to our survey or repo!🎉

• Found a missing paper?
• Have suggestions to improve this list?

👉 Feel free to open an issue or submit a pull request.

If you like this project, don’t forget to ⭐️ star it — it helps more people discover it! 💖

---

📚Table of contents

• Truthfulness
  • Hallucination
  • Reasoning Faithfulness
• Safety
  • Assessment
  • Jailbreak
  • Alignment
  • Backdoor
• Robustness
  • Robustness evalutaion and attack
  • Overthinking/underthinking
• Fairness
• Privacy
• Related surveys and benchmarks
• Others

---

Truthfulness: Hallucination, reasoning faithfulness

Hallucination

Click to hide/show the paper list

Title	Venue	Date	topic	Code
Uncertainty Under the Curve: A Sequence-Level Entropy Area Metric for Reasoning LLM	arxiv	08/27, 2025	hallucination study of LRM	-
Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs	arxiv	08/26, 2025	evaluation of quantized LLM	Github
Sycophancy under Pressure: Evaluating and Mitigating Sycophantic Bias via Adversarial Dialogues in Scientific QA	arxiv	08/19, 2025	hallucination mitigation with reasoning	-
Mitigating Hallucinations in Large Language Models via Causal Reasoning	arxiv	08/17, 2025	hallucination mitigation with reasoning	Github
Hop, Skip, and Overthink: Diagnosing Why Reasoning Models Fumble during Multi-Hop Analysis	arxiv	08/06, 2025	hallucination study of LRM	-
ViFP: A Framework for Visual False Positive Detection to Enhance Reasoning Reliability in VLMs	arxiv	08/06, 2025	hallucination mitigation with reasoning	-
Trustworthy Reasoning: Evaluating and Enhancing Factual Accuracy in LLM Intermediate Thought Processes	arxiv	08/02, 2025	hallucination mitigation with reasoning	-
Deliberative Searcher: Improving LLM Reliability via Reinforcement Learning with constraints	arxiv	07/22, 2025	calibration improvement with reasoning	-
Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty	arxiv	07/22, 2025	calibration improvement with reasoning	-
KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality	arxiv	06/24, 2025	hallucination mitigation with reasoning	Github
Mathematical Proof as a Litmus Test: Revealing Failure Modes of Advanced Large Reasoning Models	arxiv	06/23, 2025	evaluation on math of LRM	Github
Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?	arxiv	06/22, 2025	evaluation on uncertainty of LRM	-
Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation	arxiv	06/20, 2025	evaluation on hallucination detection	Github
CLATTER: Comprehensive Entailment Reasoning for Hallucination Detection	arxiv	06/05, 2025	evaluation on detection with reasoning	-
Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models	arxiv	06/05, 2025	evaluation on hallucination detection of LRM	Github
The Hallucination Dilemma: Factuality-Aware Reinforcement Learning for Large Reasoning Models	arxiv	05/30, 2025	evaluation of LRM	-
MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM	arxiv	05/30, 2025	evaluation and mitigation with reaosning	-
Are Reasoning Models More Prone to Hallucination?	arxiv	05/29, 2025	evaluation of LRM	-
More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models	arxiv	05/23, 2025	evaluation of MLRM	Project
Analyzing Logical Fallacies in Large Language Models: A Study on Hallucination in Mathematical Reasoning	JSAI-isAI 2025	05/23, 2025	evaluation on math with reasoning	Project
TreeCut: A Synthetic Unanswerable Math Word Problem Dataset for LLM Hallucination Evaluation	arxiv	05/20, 2025	unanswerable dataset	GitHub
The Hallucination Tax of Reinforcement Finetuning	arxiv	05/20, 2025	hallucination study with RL	Huggingface
Toward Reliable Biomedical Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models	arxiv	05/20, 2025	hallucination detection with reasoning	GitHub
Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective	arxiv	05/19, 2025	hallucination study of LRM	Anonymous
Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models	arxiv	05/19, 2025	hallucination study of LRM	Anonymous
Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem Features	arxiv	05/17, 2025	hallucination study of LRM	-
Enhancing Mathematical Reasoning in Large Language Models with Self-Consistency-Based Hallucination Detection	arxiv	04/13, 2025	hallucination detection with reasoning	-
Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoning	arxiv	04/08, 2025	hallucinationn detection with reasoning	-
Do Chains-of-Thoughts of Large Language Models Suffer from Hallucinations, Cognitive Biases, or Phobias in Bayesian Reasoning?	arxiv	03/19, 2025	hallucination study with reasoning	-
Grounded Chain-of-Thought for Multimodal Large Language Models	arxiv	03/17, 2025	hallucination study with reasoning	Github
Mitigating reasoning hallucination through Multi-agent Collaborative Filtering	ESWA 2025	03/05, 2025	hallucination mitigation with reasoning	-
Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking	arxiv	01/02, 2025	hallucination mitigation with reasoning	-
Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination	arxiv	11/15, 2024	hallucination mitigation with reasoning	Github
HalluMeasure: Fine-grained Hallucination Measurement Using Chain-of-Thought Reasoning	EMNLP 2024	11/12, 2024	hallucination measurement with reasoning	-
FG-PRM: Fine-grained Hallucination Detection and Mitigation in Language Model Mathematical Reasoning	arxiv	10/08, 2024	hallucination detection with reasoning	Anonymous
CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation	arxiv	06/17, 2024	hallucination mitigation with reasoning	Github

Reasoning Faithfulness

Click to hide/show the paper list

Title	Venue	Date	topic	Code
Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations	ICML 2025 Workshop on Actionable Interpretability	07/15, 2025	faithfulness improvement	Github
Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoning	arxiv	07/13, 2025	faithfulness improvement	Project
Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoning	arxiv	06/28, 2025	faithfulness improvement	-
Right Is Not Enough: The Pitfalls of Outcome Supervision in Training LLMs for Math Reasoning	arxiv	06/24, 2025	faithfulness understanding	-
VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories?	arxiv	06/13, 2025	MLLM faithfulness evaluation	Github
Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learning	arxiv	05/22, 2025	faitufulness improvement	-
Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models	arxiv	05/20, 2025	faithfulness measurement	Github
Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations	arxiv	05/20, 2025	faithfulness measurement	-
Reasoning Models Don’t Always Say What They Think	website	04/03, 2025	faithfulness understanding	-
Reasoning Inconsistencies and How to Mitigate Them in Deep Learning	arxiv	04/03, 2025	faithfulness improvement	-
Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models	arxiv	03/28, 2025	faithfulness understanding	Github
Policy Frameworks for Transparent Chain-of-Thought Reasoning in Large Language Models	arxiv	03/14, 2025	faithfulness improvement	-
Chain-of-Thought Reasoning In The Wild Is Not Always Faithful	arxiv	03/13, 2025	faithfulness understanding	-
A Causal Lens for Evaluating Faithfulness Metrics	arxiv	02/26, 2025	faithfulness measurement	-
Measuring faithfulness of chains of thought by unlearning reasoning steps	arxiv	02/20, 2025	faithfulness measurement	Github
Are DeepSeek R1 And Other Reasoning Models More Faithful?	ICLR 2025 Workshop	01/14, 2025	faithfulness understanding	-
Graph-Guided Textual Explanation Generation Framework	arxiv	12/16, 2024	faithfulness improvement	-
New Faithfulness-Centric Interpretability Paradigms for Natural Language Processing	arxiv	11/27, 2024	faithfulness understanding	-
On the Impact of Fine-Tuning on Chain-of-Thought Reasoning	arxiv	11/22, 2024	faithfulness understanding	-
Causal-driven Large Language Models with Faithful Reasoning for Knowledge Question Answering	ACM MM 24	10/28, 2024	faithfulness improvement	-
Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models	arxiv	10/18, 2024	faithfulness improvement	-
To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts	ICLR 2024	10/18, 2024	faithfulness improvement	Github
Enhancing Large Language Models' Situated Faithfulness to External Contexts	arxiv	10/18, 2024	faithfulness improvement	-
FLARE: Faithful Logic-Aided Reasoning and Exploration	arxiv	10/14, 2024	faithfulness improvement	-
CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning	arxiv	10/14, 2024	faithfulness improvement	Github
On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models	ICML 2024 Workshop	06/15, 2024	faithfulness understanding	-
XPrompt:Explaining Large Language Model's Generation via Joint Prompt Attribution	arxiv	05/30, 2024	faithfulness understanding	-
Faithful Logical Reasoning via Symbolic Chain-of-Thought	ACL 2024	05/28, 2024	faithfulness improvement	-
Dissociation of Faithful and Unfaithful Reasoning in LLMs	arxiv	05/23, 2024	faithfulness understanding	Github
FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answering	ACL 2025	05/22, 2024	faithfulness improvenment	Github
Faithful Reasoning over Scientific Claims	AAAI 2024	05/20, 2024	faithfulness improvement	-
Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness	ACL 2025 Findings	05/29, 2024	faithfulness improvement	Github
Markovian Transformers for Informative Language Modeling	arxiv	04/29, 2024	faithfulness improvement	-
Fact: Teaching MLLMs with Faithful, Concise and Transferable Rationales	ACM MM 2024	04/17, 2024	faithfulness improvement	-
Argumentative Large Language Models for Explainable and Contestable Claim Verification	AAAI 2025	04/11, 2024	faithfulness improvement	Github
Recent Developments on Accountability and Explainability for Complex Reasoning Tasks	Springer	04/06, 2024	faithfulness improvement	-
The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models	ACL 2024	04/04, 2024	faithfulness measurement	-
How Likely Do LLMs with CoT Mimic Human Reasoning?	arxiv	02/25, 2024	faithfulness understanding	Github
Chain-of-Thought Unfaithfulness as Disguised Accuracy	TMLR	02/22, 2024	faithfulness understanding	-
Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning	EMNLP 2024 Findings	02/21, 2024	faithfulness improvement	Github
How Interpretable are Reasoning Explanations from Prompting Large Language Models?	NAACL 2024 Findings	02/19, 2024	faithfulness understanding	Github
FaithLM: Towards Faithful Explanations for Large Language Models	EMNLP 2024 Findings	02/07, 2024	faithfulness improvement	-
Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models	arxiv	02/07, 2024	faithfulness understanding	-
Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning	ICLR 2024	10/02, 2023	faithfulness improvement	Github
Measuring Faithfulness in Chain-of-Thought Reasoning	arxiv	07/17, 2023	faithfulness measurement	-
Question Decomposition Improves the Faithfulness of Model-Generated Reasoning	arxiv	07/17, 2023	faithfulness improvement	Github
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning	EMNLP 2023 Findings	05/20, 2023	faithfulness improvement	Github
Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting	arxiv	05/07, 2023	faithfulness understanding	-
Faithful Chain-of-Thought Reasoning	IJCNLP-AACL 2023	01/31, 2023	faithfulness improvement	Github

Safety: Assessment, Jailbreak, Alignment, Backdoor

Vulnerability assessment

Click to hide/show the paper list

Title	Venue	Date	topic	Code
ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusal	arxiv	08/15, 2025	over-refusal evaluation	Github
Safe Semantics, Unsafe Interpretations: Tackling Implicit Reasoning Safety in Large Vision-Language Models	MM 2025	08/12, 2025	dataset	Github
Eliciting and Analyzing Emergent Misalignment in State-of-the-Art Large Language Models	arxiv	08/06, 2025	content-safety evaluation	Github
Evaluating LLM Agent Adherence to Hierarchical Safety Principles: A Lightweight Benchmark for Probing Foundational Controllability Components	ICML 2025 TAIG workshop	07/10, 2025	agenic safety evaluation	-
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety	arxiv	06/24, 2025	safety evaluation	Huggingface
Weakest Link in the Chain: Security Vulnerabilities in Advanced Reasoning Models	LLMSEC 2025	06/16, 2025	content-safety evaluation	-
UDora: A Unified Red Teaming Framework against LLM Agents by Dynamically Hijacking Their Own Reasoning	ICML 2025	06/06, 2025	agenic safety evaluation	Github
Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models	arxiv	05/26, 2025	content-safety evaluation	Github
RRTL: Red Teaming Reasoning Large Language Models in Tool Learning	arxiv	05/21, 2025	tool-learning safety evaluation	-
DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning	arxiv	04/02, 2025	content-safety evaluation	-
Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findings	arxiv	03/19, 2025	content-safety evaluation	-
Safety Evaluation and Enhancement of DeepSeek Models in Chinese Contexts	arxiv	03/18, 2025	content-safety evaluation	-
Red Teaming Contemporary AI Models: Insights from Spanish and Basque Perspectives	arxiv	03/13, 2025	multi-lingual safety evaluation	-
The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1	arxiv	02/27, 2025	content-safety evaluation	-
Evaluating security risk in deepseek and other frontier reasoning models	website	02/26, 2025	content-safety evaluation	-
Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuning	arxiv	02/21, 2025	content-safety evaluation	-
o3-mini vs DeepSeek-R1: Which One is Safer?	arxiv	01/30, 2025	content-safety evaluation	-
Challenges in Ensuring AI Safety in DeepSeek-R1 Models: The Shortcomings of Reinforcement Learning Strategies	arxiv	01/28, 2025	content-safety evaluation	-

Jailbreak

Jailbreak attack

Click to hide/show the paper list

Title	Venue	Date	topic	Code
Jailbreaking Commercial Black-Box LLMs with Explicitly Harmful Prompts	arxiv	08/14, 2025	jailbreak with CoT	Github
The Emotional Baby Is Truly Deadly: Does your Multimodal Large Reasoning Model Have Emotional Flattery towards Humans?	arxiv	08/05, 2025	multi-modal jailbreak attack	-
Adversarial Manipulation of Reasoning Models using Internal Representations	ICML 2025 R2-FM Workshop	07/01, 2025	White-box attack	Github
HauntAttack: When Attack Follows Reasoning as a Shadow	arxiv	06/08, 2025	jailbreak attack	-
Revealing the Intrinsic Ethical Vulnerability of Aligned Large Language Models	arxiv	05/26, 2025	jailbreak attack	-
VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Models	arxiv	05/26, 2025	jailbreak MLRM	-
Does Chain-of-Thought Reasoning Really Reduce Harmfulness from Jailbreaking?	arxiv	05/23, 2025	jailbreak analysis	-
Chain-of-Lure: A Synthetic Narrative-Driven Approach to Compromise Large Language Models	arxiv	05/23, 2025	jailbreak with CoT	-
Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers	arxiv	05/22, 2025	jailbreak attack	-
AutoRAN: Weak-to-Strong Jailbreaking of Large Reasoning Models	arxiv	05/16, 2025	jailbreak attack	Github
CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenges	arxiv	04/27, 2025	jailbreak attack	Project
When "Competency" in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers	arxiv	03/16, 2025	jailbreak with CoT	Github
Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models	arxiv	03/11, 2025	jailbreak with CoT	-
A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos	arxiv	02/19, 2025	jailbreak attack	-
H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking	arxiv	02/18, 2025	jailbreak attack	-
Adversarial Reasoning at Jailbreaking Time	arxiv	02/03, 2025	jailbreak with CoT	-
The dark deep side of DeepSeek: Fine-tuning attacks against the safety alignment of CoT-enabled models	arxiv	02/03, 2025	finetuning attack	-

Jailbreak defense

Click to hide/show the paper list

Title	Venue	Date	topic	Code
IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement	arxiv	08/27, 2025	guardrail model	-
Pragmatic Inference Chain (PIC) Improving LLMs' Reasoning of Authentic Implicit Toxic Language	arxiv	08/21, 2025	toxicity detection with CoT	-
Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questions	arxiv	08/08, 2025	Jailbreak defense	Github
ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments	arxiv	08/06, 2025	guardrail model	-
SafeAgent: Safeguarding LLM Agents via an Automated Risk Simulator	arxiv	07/18, 2025	agenic safety	project
Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Models	arxiv	07/16, 2025	CoT Monitor	-
Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety	arxiv	07/15, 2025	CoT Monitor	-
When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors	arxiv	07/07, 2025	CoT Monitor	-
AgentBreeder: Mitigating the AI Safety Impact of Multi-Agent Scaffolds via Self-Improvement	ICLR 2025	06/25, 2025	Agentic safety	Github
Detecting Harmful Memes with Decoupled Understanding and Guided CoT Reasoning	arxiv	06/10, 2025	harmful meme detection	Link
RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards	arxiv	06/09, 2025	guardrail model	Github
Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models	arxiv	05/26, 2025	guardrail model	-
ReasoningShield: Content Safety Detection over Reasoning Traces of Large Reasoning Models	arxiv	05/22, 2025	Guardrail model	Github
J4R: Learning to Judge with Equivalent Initial State Group Relative Policy Optimization	arxiv	05/20, 2025	LLM-as-a-judge	Github
ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMs	arxiv	05/20, 2025	guardrail model	-
GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning	arxiv	05/16, 2025	guardrail model	-
On the Robustness of Reward Models for Language Model Alignment	arxiv	05/12, 2025	reward model	-
LlamaFirewall: An open source guardrail system for building secure AI agents	arxiv	05/06, 2025	Guardrail model	-
MR. Guard: Multilingual Reasoning Guardrail using Curriculum Learning	arxiv	04/21, 2025	Guardrail model	-
VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimization	arxiv	04/17, 2025	multi-modal guardrail model	-
X-Guard: Multilingual guard agent for content moderation	arxiv	04/11, 2025	multi-lingual guardrail model	Github
Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control	arxiv	04/23, 2025	jailbreak defense study	Github
ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning	ICML 2025	03/26, 2025	guardrail model	Github
GuardReasoner: Towards Reasoning-based LLM Safeguards	arxiv	01/30, 2025	guardrail model	Github
GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning	ICML 2025	06/13, 2024	guardrail model	Project
$R^2$-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning	ICLR 2025 spotlight	06/08, 2024	Guardrail model	Github

Alignment

Click to hide/show the paper list

Title	Venue	Date	topic	Code
PRISM: Robust VLM Alignment with Principled Reasoning for Integrated Safety in Multimodality	arxiv	08/25, 2025	VLM alignment with reasoning	Github
Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigation	arxiv	08/21, 2025	alignment with reasoning	-
SDGO: Self-Discrimination-Guided Optimization for Consistent Safety in Large Language Models	EMNLP 2025	08/21, 2025	alignment with reasoning	Github
Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner	arxiv	08/20, 2025	Inference-time alignment with TTS	-
FuSaR: A Fuzzification-Based Method for LRM Safety-Reasoning Balance	arxiv	08/18, 2025	alignment of LRM	Github
AURA: Affordance-Understanding and Risk-aware Alignment Technique for Large Language Models	arxiv	08/08, 2025	alignment with reasoning	Github
R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledge	arxiv	08/01, 2025	alignment of LRM	Github
UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases	arxiv	07/29, 2025	alignment of LRM	Github
SafeWork-R1: Coevolving Safety and Intelligence under the AI-45$^{\circ}$ Law	arxiv	07/24, 2025	alignment of LRM	-
AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learning	arxiv	07/20, 2025	alignment with reasoning	Github
ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning	arxiv	07/14, 2025	alignment with reasoning	Project
Aligning Frozen LLMs by Reinforcement Learning: An Iterative Reweight-then-Optimize Approach	arxiv	07/03, 2025	test-time alignment with RL	Github
AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness	arxiv	07/02, 2025	VLM alignment with reasoning	Github
Reasoning as an Adaptive Defense for Safety	arxiv	07/01, 2025	alignment with reasoning	Project
MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Models	arxiv	06/23, 2025	multimodal alignment dataset	Huggingface
How Alignment Shrinks the Generative Horizon	arxiv	06/21, 2025	alignment study of LRM	Project
SafeCoT: Improving VLM Safety with Minimal Reasoning	arxiv	06/09, 2025	VLM alignment with reasoning	-
Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models	arxiv	06/09, 2025	alignment with reasoning	Github
Saffron-1: Towards an Inference Scaling Paradigm for LLM Safety Assurance	arxiv	06/06, 2025	alignment with reasoning	Github
Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment	arxiv	06/01, 2025	alignment with reasoning	-
Mitigating Deceptive Alignment via Self-Monitoring	arxiv	05/24, 2025	alignment of LRM	Project
SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning	arxiv	05/22, 2025	alignment of LRM	Project
From Evaluation to Defense: Advancing Safety in Video Large Language Models	arxiv	05/22, 2025	multimodal alignment with reasoning	-
How Should We Enhance the Safety of Large Reasoning Models: An Empirical Study	arxiv	05/21, 2025	alignment study of LRM	GitHub
Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning	arxiv	05/20, 2025	alignment of LRM	-
SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment	arxiv	05/20, 2025	alignment of LRM	Project
Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction	arxiv	05/19, 2025	agent alignment with reasoning	Huggingface
FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning	CoLM 2025	05/12, 2025	over reject mitigation with reasoning	Project
Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Model	arxiv	05/10, 2025	alignment of MLRM	Github
Energy-Based Reward Models for Robust Language Model Alignment	COLM 2025	04/17, 2025	reward model improvement	Github
Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation	arxiv	04/17, 2025	CoT monitor in alignment of LRM	-
RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability	arxiv	04/14, 2025	alignment of LRM	Huggingface
SaRO: Enhancing LLM Safety through Reasoning-based Alignment	arxiv	04/13, 2025	alignment with reasoning	Github
SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models	arxiv	04/09, 2025	alignment of MLRM	Github
ERPO: Advancing Safety Alignment via Ex-Ante Reasoning Preference Optimization	arxiv	04/06, 2025	alignment with reasoning	-
STAR-1: Safer Alignment of Reasoning LLMs with 1K Data	arxiv	04/02, 2025	alignment of LRM	Project
Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment	arxiv	03/23, 2025	reward model improvement	-
Think Before Refusal: Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior	arxiv	03/22, 2025	over-reject mitigation with reasoning	-
Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety	arxiv	03/06, 2025	alignment with reasoning	-
Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable	arxiv	03/01, 2025	alignment study of LRM	Github
Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking	arxiv	02/18, 2025	alignment with reasoning	Github
Safechain: Safety of language models with long chain-of-thought reasoning capabilities	arxiv	02/17, 2025	alignment of LRM	Project
Safety Reasoning with Guidelines	ICML 2025	02/06, 2025	alignment with reasoning	-
STAIR: Improving Safety Alignment with Introspective Reasoning	arxiv	02/04, 2025	alignment with reasoning	Github
How Effective Is Constitutional AI in Small LLMs? A Study on DeepSeek-R1 and Its Peers	arxiv	02/01, 2025	alignment study of LRM	-
Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning	arxiv	01/31, 2025	alignment with reasoning	anonymous
Backtracking Improves Generation Safety	ICLR 2025 Oral	09/22, 2024	alignment with reasoning	-

Backdoor

Click to hide/show the paper list

Title	Venue	Date	topic	Code
SLIP: Soft Label Mechanism and Key-Extraction-Guided CoT-based Defense Against Instruction Backdoor in APIs	arxiv	08/08, 2025	injection defense	Github
Defend LLMs Through Self-Consciousness	KDD 2025 Workshop	08/04, 2025	injection defense	-
BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profit	arxiv	07/24, 2025	training-phase data poisoning	Github
Thought Purity: Defense Paradigm For Chain-of-Thought Attack	arxiv	07/16, 2025	injection defense	-
Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models	arxiv	06/16, 2025	Training-time
data poisoning	Huggingface
GUARD:Dual-Agent based Backdoor Defense on Chain-of-Thought in Neural Code Generation	arxiv	05/27, 2025	backdoor defense	Github
Chain-of-Thought Poisoning Attacks against R1-based Retrieval-Augmented Generation Systems	arxiv	05/22, 2025	inference-time prompt manipulation	-
System Prompt Poisoning: Persistent Attacks on Large Language Models Beyond User Injection	arxiv	05/10, 2025	inference-time prompt manipulation	-
Practical Reasoning Interruption Attacks on Reasoning Large Language Models	arxiv	05/10, 2025	inference-time prompt manipulation	-
Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compression	arxiv	04/29, 2025	inference-time prompt manipulation	-
ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMs	arxiv	04/08, 2025	training-phase data poisoning	-
Harnessing Chain-of-Thought Metadata for Task Routing and Adversarial Prompt Detection	arxiv	03/27, 2025	injection defense	-
To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models	arxiv	02/16, 2025	training-phase data poisoning	Github
DarkMind: Latent Chain-of-Thought Backdoor in Customized LLMs	arxiv	01/24, 2025	inference-time prompt manipulation	-
Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models	ACL 2025 Findings	12/20, 2024	backdoor defense	-
SABER: Model-agnostic Backdoor Attack on Chain-of-Thought in Neural Code Generation	arxiv	12/08, 2025	training-phase data poisoning	-
BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models	ICLR 2024	01/20, 2024	inference-time prompt manipulation	-

Robustness: evaluation and attack, improvement, overthinking, underthinking

Evaluation and attack

Click to hide/show the paper list

Title	Venue	Date	topic	Code
Measuring Sycophancy of Language Models in Multi-turn Dialogues	arxiv	08/25, 2025	sycophancy evaluation	Github
Refining Critical Thinking in LLM Code Generation: A Faulty Premise-based Evaluation Framework	arxiv	08/05, 2025	evaluation on code	-
MoHoBench: Assessing Honesty of Multimodal Large Language Models via Unanswerable Visual Questions	arxiv	07/29, 2025	evaluation on unanswerable question	Github
When LLMs Copy to Think: Uncovering Copy-Guided Attacks in Reasoning LLMs	arxiv	07/22, 2025	adversarial attack	-
Reasoning Models Are More Easily Gaslighted Than You Think	arxiv	06/11, 2025	evaluation on misleading inputs	-
AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions	arxiv	06/10,2025	evaluation on unanswerable question	Github
Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generation	arxiv	06/07, 2025	evaluation on adversarial prompting	Github
Hidden in Plain Sight: Reasoning in Underspecified and Misspecified Scenarios for Multimodal LLMs	arxiv	05/30, 2025	Failure analysis	-
CodeCrash: Stress Testing LLM Reasoning under Structural and Semantic Perturbations	arxiv	05/23, 2025	evalution on code comprehension	GitHub
PolyMath: Evaluating Mathematical Reasoning in Multilingual Contexts	arxiv	04/28, 2025	evaluation on math	[Github](https://github .com/QwenLM/PolyMath)
Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Steps	arxiv	03/25, 2025	adversarial attack	-
A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1	arxiv	03/13, 2025	adversarial attack	Github
Benchmarking Reasoning Robustness in Large Language Models	arxiv	03/06, 2025	evaluation	-
Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models	COLM 2025	03/03, 2025	adversarial attack	Huggingface
Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?	NeurIPS 2024	02/27, 2025	evaluation on self-reflection	Github
A Closer Look at System Prompt Robustness	arxiv	02/15, 2025	evaluation on system prompt	Github
MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbations	arxiv	02/10, 2025	evaluation on math	-
Understanding the Dark Side of LLMs' Intrinsic Self-Correction	arxiv	12/19, 2024	self reflection study	Github
Stepwise Reasoning Disruption Attack of LLMs	ACL 2025	12/16, 2024	adversarial attack	Github
Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?	arxiv	10/31, 2024	evaluation on noisy rationales	Github
RUPBench: Benchmarking Reasoning Under Perturbations for Robustness Evaluation in Large Language Models	arxiv	06/16, 2024	evaluation	Github
Preemptive Answer "Attacks" on Chain-of-Thought Reasoning	Findings of ACL 2024	05/31, 2024	adversarial attack	-

Improvement

Click to hide/show the paper list

Title	Venue	Date	topic	Code
Error Reflection Prompting: Can Large Language Models Successfully Understand Errors?	NAACL 2025	08/22, 2025	robustness improvement	-
Any Large Language Model Can Be a Reliable Judge: Debiasing with a Reasoning-based Bias Detector	arxiv	05/21, 2025	LLM-as-a-judge	-
Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in Large Language Models against Reference Corruption	arxiv	04/29, 2025	adversarial defense	-
Assessing Judging Bias in Large Reasoning Models: An Empirical Study	arxiv	04/14, 2025	bias mitigation	-
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems?	arxiv	04/01, 2025	robustness imporvement	Huggingface
Trading Inference-Time Compute for Adversarial Robustness	arxiv	01/31, 2025	robustness improvement	-
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement	EMNLP 2023 Findings	05/23, 2023	reasoning enhancement	Github

Overthinking and underthinking

Click to hide/show the paper list

Title	Venue	Date	topic	Code
Don't Think Twice! Over-Reasoning Impairs Confidence Calibration	ICML 2025 Workshop	08/20, 2025	overthinking study	-
OptimalThinkingBench: Evaluating Over and Underthinking in LLMs	arxiv	08/18, 2025	evaluation	Github
Excessive Reasoning Attack on Reasoning LLMs	arxiv	06/17, 2025	overthinking attack	-
Mitigating Overthinking in Large Reasoning Models via Manifold Steering	arxiv	05/28, 2025	overthinking mitigation	-
ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models	arxiv	05/27, 2025	underthinking mitigation	Github
Internal Bias in Reasoning Models leads to Overthinking	arxiv	05/22, 2025	overthinking study	-
Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs	arxiv	04/30, 2025	reasoning length study	-
Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical	arxiv	04/11, 2025	overthinking study	-
Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities	arxiv	03/23, 2025	reasoning length study	Guthub
DNA Bench: When Silence is Smarter --Benchmarking Over-Reasoning in Reasoning LLMs	arxiv	03/19, 2025	overthinking evaluation	-
Output Length Effect on DeepSeek-R1's Safety in Forced Thinking	arxiv	03/02, 2025	reasoning length study	-
The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks	arxiv	02/12, 2025	overthinking study	Github
OverThink: Slowdown Attacks on Reasoning LLMs	arxiv	02/04, 2025	overthinking attack	Github
Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs	arxiv	01/30, 2025	underthinking study	-
Large Language Models Struggle with Unreasonability in Math Problems	arxiv	03/28, 2024	overthinking study	-

Privacy

Click to hide/show the paper list

Title	Venue	Date	topic	Code
Evaluating Language Model Reasoning about Confidential Information	arxiv	08/27, 2025	prompt-related privacy	Github
1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning	arxiv	08/11, 2025	privacy improvement	-
Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases	arxiv	07/01, 2025	privacy improvement	Github
Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers	arxiv	06/18, 2025	prompt-related privacy	Github
Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills	arxiv	06/15, 2025	unlearning	Github
Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Models	arxiv	06/14, 2025	unlearning attack	-
Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning	arxiv	05/23, 2025	copyright protection	-
CoTSRF: Utilize Chain of Thought as Stealthy and Robust Fingerprint of Large Language Models	arxiv	05/22, 2025	copyright protection	-
R-TOFU: Unlearning in Large Reasoning Models	arxiv	05/21, 2025	unlearning	Project
ImF: Implicit Fingerprint for Large Language Model	arxiv	05/17, 2025	copyright protection	-
Doxing via the Lens: Revealing Privacy Leakage in Image Geolocation for Agentic Multi-Modal Large Reasoning Model	arxiv	04/29, 2025	prompt-related privacy	Project
Antidistillation Sampling	arxiv	04/17, 2025	copyright protection	Project

Fairness

Click to hide/show the paper list

Title	Venue	Date	topic	Code
Bridging the Culture Gap: A Framework for LLM-Driven Socio-Cultural Localization of Math Word Problems in Low-Resource Languages	arxiv	08/22, 2025	multi-lingual bias	-
Do Biased Models Have Biased Thoughts?	COLM 2025	08/08, 2025	study	-
FairReason: Balancing Reasoning and Social Bias in MLLMs	arxiv	07/30, 2025	fairness improvement	Github
The Emperor's New Chain-of-Thought: Probing Reasoning Theater Bias in Large Reasoning Models	arxiv	07/18, 2025	fairness evaluation	-
Guiding LLM Decision-Making with Fairness Reward Models	arxiv	07/15, 2025	fairness improvement	Github
Is Reasoning All You Need? Probing Bias in the Age of Reasoning Language Models	arxiv	07/03, 2025	fairness evaluation	-
Persona-Assigned Large Language Models Exhibit Human-Like Motivated Reasoning	arxiv	06/24, 2025	persona bias	Github
Detection, Classification, and Mitigation of Gender Bias in Large Language Models	arxiv	06/14, 2025	gender bias	-
Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Tasks	arxiv	06/09, 2025	dialect fairness	Github
BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Models	ACL 2025 Findings	04/30, 2025	bias detection	-
Prompting techniques for reducing social bias in LLMs through system 1 and system 2 cognitive processes	arxiv	04/26, 2024	social bias	Github
Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs	ICLR 2024	11/08, 2023	reasoning bias	Project
Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in chatgpt	Nature Computational Science	10/05, 2023	reasoning bias	-

Related surveys and benchmarks

Click to hide/show the paper list

Title	Venue	Date	topic	Code
Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement	arxiv	07/10, 2025	Offensive Language detection benchmark	Huggingface
Towards Understanding the Cognitive Habits of Large Reasoning Models	arxiv	07/05, 2025	cognitive habits	Github
IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks	arxiv	06/19, 2025	agent safety benchmark	Github
PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehunts	arxiv	06/06, 2025	multi-modal open-ended reasoning benchmark	Github
Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systems	arxiv	05/23, 2025	evaluation benchmark	-
A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Law	arxiv	05/08, 2025	general survey	-
100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models	arxiv	05/02, 2025	r1 replication	-
Safety in Large Reasoning Models: A Survey	arxiv	04/25, 2025	safety	Gitub
Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models	arxiv	03/31, 2025	efficient reasoning	Github
Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models	arxiv	03/28, 2025	math performance	Github

Other related studies

Click to hide/show the paper list

Title	Venue	Date	topic	Code
It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics	arxiv	08/18, 2025	persuasion evaluation	-
Slow Tuning and Low-Entropy Masking for Safe Chain-of-Thought Distillation	arxiv	08/15, 2025	safe data distillation	-
Representation Bending for Large Language Model Safety	arxiv	07/14, 2025	safety	-
Defending Against Prompt Injection With a Few Defensive Tokens	arxiv	07/10, 2025	prompt injection defense	-
Truth Neurons	arxiv	07/08, 2025	interpretability study	-
The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs	arxiv	06/06, 2025	safety evaluation	-
Give Me FP32 or Give Me Death? Challenges and Solutions for Reproducible Reasoning	arxiv	06/11, 2025	reasoning reproducibility	-
If Pigs Could Fly... Can LLMs Logically Reason Through Counterfactuals?	arxiv	05/28, 2025	adversarial evaluation	Anonymous
What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning	arxiv	05/28, 2025	reasoning analysis	-
Deliberation on Priors: Trustworthy Reasoning of Large Language Models on Knowledge Graphs	arxiv	05/21, 2025	reasoning on knowledge graphs	-
Linear Control of Test Awareness Reveals Differential Compliance in Reasoning Models	arxiv	05/20, 2025	alignment study	GitHub
Persuasion and Safety in the Era of Generative AI	arxiv	05/18, 2025	persuasion study	-
AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security	ICLR 2025 Workshop	04/29, 2025	alignment study with reasoning	Github
Moral Reasoning Across Languages: The Critical Role of Low-Resource Languages in LLMs	arxiv	04/28, 2025	moral evaluation	-
Effectively Controlling Reasoning Models through Thinking Intervention	arxiv	03/31, 2025	reasoning control	-
Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models	arxiv	03/14, 2025	reasoning control	-
Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Models	arxiv	12/30, 2024	hallucination study	-
Alignment faking in large language models	arxiv	12/19, 2024	alignment study of LLM	-