Chain of Thoughts (CoT) is so hot! so long! We need short reasoning process!
72
4 commits
updated Apr 1, 2025
README
Efficient-CoT-LRMs
Chain of Thoughts (CoT) is hot now. But we can observe that CoT is also getting longer and sometimes overthinks.
Here, we track the latest news of how to keep CoT/LRMs more efficient.
Topics included:
- Prompting-based
- Budget Control
- Compress in Latent Space
- Summarization
- Skip Something
- KV Cache Management
- Reinforment Learning
- General Topic
Here we go.
Prompting-based
| Time | Title | code | published |
|---|---|---|---|
| 07/03/2025 | Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching | link | / |
| 25/02/2025 | Chain of Draft: Thinking Faster by Writing Less | link | / |
| 17/02/2025 | SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs | / | / |
| 04/06/2024 | Break the Chain: Large Language Models Can be Shortcut Reasoners | / | / |
| 28/07/2023 | Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation | link | ICLR2024 |
Budget Control
| Time | Title | code | published |
|---|---|---|---|
| 06/03/2025 | DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models | / | / |
| 24/12/2024 | Token-budget-aware llm reasoning | link | / |
| 29/07/2024 | Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost | / | / |
Compress in Latent Space
Summarization
| Time | Title | code | published |
|---|---|---|---|
| 09/03/2025 | InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models | / | / |
| 21/02/2025 | LightThinker: Thinking Step-by-Step Compression) | / | / |
| 16/12/2024 | C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness | / | AAAI2025 |
| 12/02/2024 | Anchor-based Large Language Models (related work) | link | ACL2024 |
Skip Something
| Time | Title | code | published |
|---|---|---|---|
| 18/02/2025 | Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models | / | / |
| 17/02/2025 | TokenSkip: Controllable Chain-of-Thought Compression in LLMs | link | / |
| 04/11/2024 | Can Language Models Learn to Skip Steps? | / | NeurIPS2024) |
KV Cache Management
| Time | Title | code | published |
|---|---|---|---|
| 21/02/2025 | LightThinker: Thinking Step-by-Step Compression) | / | / |
| 16/02/2025 | Efficient Long-Decoding Inference with Reasoning-Aware Attention Sparsity | / | / |
Reinforment Learning
| Time | Title | code | published |
|---|---|---|---|
| 06/03/2025 | L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning | link | / |
| 22/01/2025 | O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning | link | / |
| 22/01/2025 | Kimi k1.5: Scaling Reinforcement Learning with LLMs | / | / |
General Topic
| Time | Title | code | published |
|---|---|---|---|
| 19/02/2025 | AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence | / | / |
| 30/12/2024 | Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs | / | / |
| 10/01/2024 | The Impact of Reasoning Step Length on Large Language Models | link | ACL2024findings |
About us:
PEILab, Hong Kong University of Science and Technology (HKUST)
Contributors
Blueyee
4 commits
Similar repos
Chain of Thoughts (CoT) is so hot! so long! We need short reasoning process!
72
4 commits
updated Apr 1, 2025
README
Efficient-CoT-LRMs
Chain of Thoughts (CoT) is hot now. But we can observe that CoT is also getting longer and sometimes overthinks.
Here, we track the latest news of how to keep CoT/LRMs more efficient.
Topics included:
- Prompting-based
- Budget Control
- Compress in Latent Space
- Summarization
- Skip Something
- KV Cache Management
- Reinforment Learning
- General Topic
Here we go.
Prompting-based
| Time | Title | code | published |
|---|---|---|---|
| 07/03/2025 | Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching | link | / |
| 25/02/2025 | Chain of Draft: Thinking Faster by Writing Less | link | / |
| 17/02/2025 | SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs | / | / |
| 04/06/2024 | Break the Chain: Large Language Models Can be Shortcut Reasoners | / | / |
| 28/07/2023 | Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation | link | ICLR2024 |
Budget Control
| Time | Title | code | published |
|---|---|---|---|
| 06/03/2025 | DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models | / | / |
| 24/12/2024 | Token-budget-aware llm reasoning | link | / |
| 29/07/2024 | Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost | / | / |
Compress in Latent Space
Summarization
| Time | Title | code | published |
|---|---|---|---|
| 09/03/2025 | InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models | / | / |
| 21/02/2025 | LightThinker: Thinking Step-by-Step Compression) | / | / |
| 16/12/2024 | C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness | / | AAAI2025 |
| 12/02/2024 | Anchor-based Large Language Models (related work) | link | ACL2024 |
Skip Something
| Time | Title | code | published |
|---|---|---|---|
| 18/02/2025 | Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models | / | / |
| 17/02/2025 | TokenSkip: Controllable Chain-of-Thought Compression in LLMs | link | / |
| 04/11/2024 | Can Language Models Learn to Skip Steps? | / | NeurIPS2024) |
KV Cache Management
| Time | Title | code | published |
|---|---|---|---|
| 21/02/2025 | LightThinker: Thinking Step-by-Step Compression) | / | / |
| 16/02/2025 | Efficient Long-Decoding Inference with Reasoning-Aware Attention Sparsity | / | / |
Reinforment Learning
| Time | Title | code | published |
|---|---|---|---|
| 06/03/2025 | L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning | link | / |
| 22/01/2025 | O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning | link | / |
| 22/01/2025 | Kimi k1.5: Scaling Reinforcement Learning with LLMs | / | / |
General Topic
| Time | Title | code | published |
|---|---|---|---|
| 19/02/2025 | AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence | / | / |
| 30/12/2024 | Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs | / | / |
| 10/01/2024 | The Impact of Reasoning Step Length on Large Language Models | link | ACL2024findings |
About us:
PEILab, Hong Kong University of Science and Technology (HKUST)
Similar repos
Contributors
Blueyee
4 commits