README
OpenThought - System 2 Research Links
Here you find a collection of material (books, papers, blog-posts etc.) related to reasoning and cognition in AI systems. Specifically we want to cover agents, cognitive architectures, general problem solving strategies and self-improvement.
The term "System 2" in the page title refers to the slower, more deliberative, and more logical mode of thought as described by Daniel Kahneman in his book Thinking, Fast and Slow.
You know a great resource we should add? Please see How to contribute.
Cognitive Architectures
(looking for additional links & articles and summaries)
- SOAR (State, Operator, And Result) by John Laird, Allen Newell, and Paul Rosenbloom
- ACT-R (Adaptive Control of Thought-Rational) by John Anderson at CMU
- SPAUN (Semantic Pointer Architecture Unified Network) by Chris Eliasmith at Waterloo, SPAUN 2.0 by Feng-Xuan Choo
- ART (Adaptive resonance theory) by Stephen Grossberg and Gail Carpenter
- CLARION (Connectionist Learning with Adaptive Rule Induction ON-line) by Ron Sun
- EPIC (Executive Process/Interactive Control) by David Kieras and David Meyer
- LIDA (Learning Intelligent Distribution Agent) by Stan Franklin, 2015 Paper
- Sigma by Paul Rosenbloom
- OpenCog by Ben Goertzel
- NARS (Non-Axiomatic Reasoning System) by Pei Wang
- Icarus by Pat Langley
- MicroPsi by Joscha Bach
- Thousand Brains Theory & HTM (Hierarchical Temporal Memory) by Jeff Hawkins
- SPH (Sparse Predictive Hierarchie) by Eric Laukien
- Leabra (Local, Error-driven and Associative, Biologically Realistic Algorithm), 2016 Paper by Randall O'Reilly
- CogNGen (COGnitive Neural GENerative system) by Alexander Ororbia and Mary Alexandria Kelly, see also here and here
- KIX (KIX: A Metacognitive Generalization Framework) by A. Kumar and Paul Schrater
- ACE (Autonomous Cognitive Entity) by David Shapiro et al., gh: daveshap/ACE_Framework
- Iterative Updating of Working Memory by Jared Reser, website, Video
Agent Papers
LLM Based
- 06 Feb 2025 Multi-agent Architecture Search via Agentic Supernet
- 06 Jan 2025 Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches
- Nov 2024 LLaVA-o1: Let Vision Language Models Reason Step-by-Step
- ReAct: Synergizing Reasoning and Acting in Language Models
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
- The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery, gh: SakanaAI/AI-Scientist
- OpenDevin: An Open Platform for AI Software Developers as Generalist Agents
- TextGrad: Automatic "Differentiation" via Text
- Trace is the New AutoDiff -- Unlocking Efficient Optimization of Computational Workflows
- Agentless: Demystifying LLM-based Software Engineering Agents
- Competition-Level Code Generation with AlphaCode
- AI Agents That Matter
- Sibyl: Simple yet Effective Agent Framework for Complex Real-world Reasoning
- Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
- Self-Rewarding Language Models
- ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models
- MedAgent-Zero: Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents
- Cognitive Architectures for Language Agents
- Large Language Models Can Self-Improve At Web Agent Tasks
- AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation
- A Prefrontal Cortex-inspired Architecture for Planning in Large Language Models
- CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization
- DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search
- GPT-Swarm Language Agents as Optimizable Graphs
- Survey: Reasoning with Large Language Models, a Survey (Jul 2024)
- Survey: From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future (Aug 2024)
- ADAS: Automated Design of Agentic Systems
- IDEA:Enhancing the Rule Learning Ability of Language Agents through Induction, Deduction, and Abduction
- LAW: Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning
- GenRM: Generative Verifiers: Reward Modeling as Next-Token Prediction
- Perspective: Towards Building Specialized Generalist AI with System 1 and System 2 Fusion
- CodeAct: Executable Code Actions Elicit Better LLM Agents
- PLANSEARCH: Planning In Natural Language Improves LLM Search For Code Generation
- LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBench
- Thinking LLMs: General Instruction Following with Thought Generation
- Agent S: An Open Agentic Framework that Uses Computers Like a Human
LLM Reasoning Improvements / Training on Synthetic Data
- 06 Mar 2025 START: Self-taught Reasoner with Tools
- 02 Mar 2025 LADDER: Self-Improving LLMs Through Recursive Problem Decomposition
- 17 Feb 2025 Scaling Test-Time Compute Without Verification or RL is Suboptimal
- 13 Feb 2025 Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
- 11 Feb 2025 CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
- 07 Feb 2025 Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- 05 Feb 2025 Demystifying Long Chain-of-Thought Reasoning in LLMs
- 05 Feb 2025 LIMO: Less is More for Reasoning
- 31 Jan 2025 s1: Simple test-time scaling
- 08 Jan 2025 MetaCoT: Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
- 05 Jan 2025 Test-time Computing: from System-1 Thinking to System-2 Thinking
- 30 Dec 2024 Aviary: training language agents on challenging scientific tasks - expert iteration & rejection sampling
- 25 Dec 2024 HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
- 18 Dec 2024 A Survey on LLM Inference-Time Self-Improvement
- 12 Dec 2024 STILL-2: Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems
- 02 Dec 2024 Mastering Board Games by External and Internal Planning with Language Models
- 29 Nov 2024 Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability
- 14 Oct 2024 TPO: Thinking LLMs: General Instruction Following with Thought Generation
- 4 Oct 2024 SWAP: Deliberate Reasoning for LLMs as Structure-aware Planning with Accurate World Model
- Thinking LLMs: General Instruction Following with Thought Generation
- Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources
- Chain of Thought Imitation with Procedure Cloning
- Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
- Self-Discover: Large Language Models Self-Compose Reasoning Structures
- TRICE: Training Chain-of-Thought via Latent-Variable Inference
- Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
- Self-Taught Reasoner: STaR: Bootstrapping Reasoning With Reasoning
- Self-Notes: Learning to Reason and Memorize with Self-Notes
- From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
- LaTRO: Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding, code
Datasets
- SYNTHETIC-1 (Prime Intellect)
- Open Reasoning Data (General Reasoning)
- OpenThoughts-114k
- reasoning-gym (Open-Thought Community)
- open-r1 datastes (Huggingface)
Direct o1 Replication Efforts
- Unsloth GRPO: Train your own R1 reasoning model with Unsloth (GRPO)
- HF open-r1: A fully open reproduction of DeepSeek-R1, gh: huggingface/open-r1
- TinyZero: A reproduction of DeepSeek R1 Zero in countdown and multiplication tasks. x: thread
- DeepSeek R-1: (https://chat.deepseek.com/), Tech report: deepseek-ai/DeepSeek-R1
- OpenR: Technical Report, Project Page, code: openreasoner/openr
- GAIR-NLP/O1-Journey, O1 Replication Journey: Strategic Progress Report - Part 1
- OpenSource-O1/Open-O1
- bklieger-groq/g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains
- ack-sec/toyberry: Atlas Reasoning System (Toyberry)
- Blog: Reverse engineering OpenAI’s o1 by Nathan Lambert
Reward Models (ORM/PRM)
- 13 Jan 2025 QwQ PRM: The Lessons of Developing Process Reward Models in Mathematical Reasoning - consensus of MC estimation & LLM-as-a-judge
- 02 Jan 2025 Process Reinforcement through Implicit Rewards - implicit PRM, gh: PRIME-RL/PRIME
- 02 Dec 2024 Implicit PRM: Free Process Rewards without Process Labels, gh: PRIME-RL/ImplicitPRM
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
- Solving math word problems with process- and outcome-based feedback
- Training Verifiers to Solve Math Word Problems
- RATIONALYST: Pre-training Process-Supervision for Improving Reasoning
- 14 Dec 2023 Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
RL
- 18 Feb 2025 Competitive Programming with Large Reasoning Models (OpenAI)
- 04 Feb 2025 Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
- 22 Jan 2025 Temporal Difference Learning: Why It Can Be Fast and How It Will Be Faster
- 04 Jan 2025 REINFORCE++: A Simple and Efficient Approach for Aligning Large Language Models
- 20 Dec 2024 OREO: Offline Reinforcement Learning for LLM Multi-Step Reasoning
- 11 Oct 2024 DQO: Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization
- 02 Oct 2024 RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
- 08 Feb 2024 R^3: Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
- 05 Feb 2024 GRPO: DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- 11 Jul 2023 Secrets of RLHF in Large Language Models Part I: PPO
- ReFT: Reasoning with Reinforced Fine-Tuning
- ARES: Alternating Reinforcement Learning and Supervised Fine-Tuning for Enhanced Multi-Modal Chain-of-Thought Reasoning Through Diverse AI Feedback
MCTS
- 11 Feb 2025 Monte Carlo Tree Diffusion for System 2 Planning
- 08 Jan 2025 rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
- Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
- Improve Mathematical Reasoning in Language Models by Automated Process Supervision
- Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
- rStar: Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
- LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios, code: opendilab/LightZero
- MuZero: Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, open-source impl: werner-duvaud/muzero-general
Minecraft Agents
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models
- Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks
- STEVE Series: Step-by-Step Construction of Agent Systems in Minecraft
Massive Sampling / Generate-and-Test
- Inference survey: From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling, code, blog
- AlphaCode 2 Technical Report
World Models
- GameNGen: Diffusion Models Are Real-Time Game Engines, project page
- A Path Towards Autonomous Machine Intelligence
- GAIA-1: A Generative World Model for Autonomous Driving
- Latent space world-models: Dreamer, V2, V3, DayDreamer
- World Models, web: project page
- Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models
Neuro-Symbolic Approaches
- Deliberate Reasoning for LLMs as Structure-aware Planning with Accurate World Model
- HYSYNTH: Context-Free LLM Approximation for Guiding Program Synthesis
- SymbolicAI: A framework for logic-based approaches combining generative models and solvers, Library: ExtensityAI/symbolicai
- DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning
- A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices
- Reasoning proofs generated by Prolog: Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs, Code
- VerityMath: Advancing Mathematical Reasoning by Self-Verification Through Unit Consistency, Code
Math
- AlphaGeometry: Solving olympiad geometry without human demonstrations
- Hologram Reasoning for Solving Algebra Problems with Geometry Diagrams
- Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving
Active Inference
- From pixels to planning: scale-free active inference
- Deep active inference agents using Monte-Carlo methods
Prompting Techniques
- Surveys:
- (Jul 2024) The Prompt Report: A Systematic Survey of Prompting Techniques
- (Feb 2024) A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
- Prompt Engineering Guide Prompting Techniques
- Prompting Fundamentals and How to Apply them Effectively by Eugene Yan
- Tools:
- Chain-of-Thoughts (COT): Paper
- Tree-of-Thoughts (ToT): Paper, impl: Strategic Debate
- Graph-of-Thoughts (GoT): Paper, code
- Algorithm of Thoughts (AoT): Paper
- Chain-of-Verification (CoVe/CoV): Paper
- Mixture-of-Agents (MoA): Paper
- Tool-Integrated Reasoning (ToRA / TIR): Paper
- Program of Thoughts (PoT): Paper
- Buffer of Thoughts (BoT): Paper
- Chain of Code (CoC): Paper
- Thought of Search (ToS): Paper
- Re-Reading the question as input (RE2): Paper
- Self-Harmonized Chain of Thought (ECHO): Paper, code
- Divergent CoT (DCoT), Paper
- Iteration of Thought (IoT), Paper
- Logic-of-Thought (LoT) Paper
- Forest-of-Thought (FoT) Paper
- 04 Feb 2025 Chain-of-Associated-Thoughts (CoAT): Paper
- 25 Feb 2025 Chain of Draft (CoD): Paper
Negative results
Mechanistic Interpretability
- Anthropic: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
- Geometric Space of Hierarchical Concepts in LLM
Blog Posts / Presentations
- 09 Mar 2025 Vintage Data Actual LLM agents are coming. They will be trained
- ?? Feb 2025 Vintage Data The Model is the Product
- 06 Feb 2025 Letta Stateful Agents: The Missing Link in LLM Intelligence
- 05 Feb 2025 Sebastian Raschka Understanding Reasoning LLMs
- 06 Feb 2025 unsloth Train your own R1 reasoning model with Unsloth (GRPO)
- 02 Feb 2025 Huggingface Open-R1: Update #1
- 25 Jan 2025 NLP@HKUST: https://hkust-nlp.notion.site/simplerl-reason, gh: hkust-nlp/simpleRL-reason
- 23 Jan 2025 Philschmid: How to align open LLMs in 2025 with DPO & and synthetic data
- 08 Jan 2025 ML CMU: Optimizing LLM Test-Time Compute Involves Solving a Meta-RL Problem
- 09 Jan 2025 Notebook: Agentic RAG with Hugging Face smolagents vs Vanilla RAG
- 07 Jan 2025 Chip Huyen: Agents
- HF: Scaling Test Time Compute with Open Models
- Nebius: Leveraging training and search for better software engineering agents
- DeepMind AlphaProof and AlphaGeometry 2
- Getting 50% (SoTA) on ARC-AGI with GPT-4o, code: rgreenblatt/arc_draw_more_samples_pub
- Schmidhuber: Artificial Curiosity & Creativity
- synthesis.ai: Do Androids Dream? World Models in Modern AI
- Our Transformers Code Agent beats the GAIA benchmark!
- Lil'Log LLM Powered Autonomous Agents (Jun 2023 )
- BAIR Blog: The Shift from Models to Compound AI Systems
- Microsoft Research Tracing the path to self-adapting AI agents
- LLMs develop their own understanding of reality as their language abilities improve, Emergent Representations Paper
- LessWrong post: LLM Generality is a Timeline Crux
- Three levels of self-building autonomous agents (Tweet thread by Yohei )
- Don't Sleep on Single-agent Systems
- Video: Improving LLM Reasoning using self-generated data: RL and Verifiers, Slides by Rishabh Agarwal (DeepMind)
- Slides: Reasoning with inference-time compute by Sean Welleck, tweet
Graph Neural Networks
- Distill A Gentle Introduction to Graph Neural Networks (2021)
- Geometric Deep Learning - Grids, Groups, Graphs, Geodesics, and Gauges
Complex Logical Query Answering (CQLA)
Answering logical queries over Incomplete Knowledge Graphs. Aspirationally this requires combining sparse symbolic index collation (SQL, SPARQL, etc) and dense vector search, preferably in a differentiable manner.
- Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databases
- Adapting Neural Link Predictors for Data-Efficient Complex Query Answering
- Generalizing Knowledge Graph Embedding with Universal Orthogonal Parameterization
- Knowledge Sheaves: A Sheaf-Theoretic Framework for Knowledge Graph Embedding
- Wasserstein-Fisher-Rao Embedding: Logical Query Embeddings with Local Comparison and Global Transport
- GammaE: Gamma Embeddings for Logical Queries on Knowledge Graphs
- Soft Reasoning on Uncertain Knowledge Graphs
Inductive Reasoning over Heterogeneous Graphs
Similar to the regular CQLA, but with the emphasis on the "Inductive Setting" - i.e. querying over new, unseen during training nodes, edge types or even entire graphs. The latter part is interesting as it relies on the higher order "relations between relations" structure, connecting KG inference to Category Theory.
- Zero-shot Logical Query Reasoning on any Knowledge Graph
- Extending Transductive Knowledge Graph Embedding Models for Inductive Logical Relational Inference
- Neural-Symbolic Models for Logical Queries on Knowledge Graphs
- InGram: Inductive Knowledge Graph Embedding via Relation Graphs
Neural Algorithmic Reasoning (NAR)
Initially attempted back in 2014 with general-purpose but unstable Neural Turing Machines, modern NAR approaches limit their scope to making GNN-based "Algorithmic Processor Networks" which learn to mimic classical algorithms on synthetic data and can be deployed on noisy real-world problems by sandwiching their frozen instances inside Encoder-Processor-Decoder architecture.
- Neural Turing Machines, 2014
- A Generalist Neural Algorithmic Learner
- Transformers meet Neural Algorithmic Reasoners
- Recursive Algorithmic Reasoning
- Dual Algorithmic Reasoning
- Learning to Configure Computer Networks with Neural Algorithmic Reasoning
Grokking
- Deep Networks Always Grok and Here is Why
- Grokfast: Accelerated Grokking by Amplifying Slow Gradients, review post by Lucas Nestler
- Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
Open-Source Agents & Agent Frameworks
- QwenLM/Qwen-Agent
- meta-llama/llama-agentic-system
- gpt-researcher, docs
- open-interpreter, docs
- ADAS (Automated Design of Agentic Systems)
- AI-Scientist
- Ollama_Agents
- AgentK
- Storm, Paper
- crewAI, docs
- AutoGPT, docs
- AutoGen, docs, AutoGen Studio Paper
- Trace, docs, Paper
- motleycrew, docs
- langflow, docs
- show-me: A Visual and Transparent Reasoning Agent
- ReasonerAgent-Web, Websiet
Algorithms
Weak Search Methods
Weak methods are general but don't use knowledge (heuristics) to guide the search process.
- depth-first-search (DFS)
- breadth-first-search (BFS)
- depth-limited-search, iterative-deepening-depth-first-search (IDDFS)
- generate-and-test
- hill-climbing (borderline case between weak and strong methods)
Strong Search Methods
Books
- The Soar Cognitive Architecture, John E. Laird, MIT Press, 2019
- How to Build a Brain: A Neural Architecture for Biological Cognition Chris Eliasmith, Oxford Series on Cognitive Models and Architectures, 2013
- Active Inference: The Free Energy Principle in Mind, Brain, and Behavior, Thomas Parr, Giovanni Pezzulo, Karl J. Friston, MIT Press, 2022, MLST Interview with Thomas Parr
- Principles of Synthetic Intelligence PSI: An Architecture of Motivated Cognition, Joscha Bach, Oxford Series on Cognitive Models and Architectures Book 4, 2009
- Conscious Mind, Resonant Brain: How Each Brain Makes a Mind, Stephen Grossberg, Oxford University Press, 2021
- The Society of Mind, Marvin Minsky, Simon & Schuster, 1986
- Reinforcement Learning: An Introduction 2nd Edition, Sutton & Barto, MIT Press, 2018
- Reinforcement Learning: An Overview, Dec 2024, Kevin Murphy
- Mathematical Foundations of Reinforcement Learning, Shiyu Zhao, open course on github + video lectures
- Natural Language Cognitive Architecture, David Shapiro, 2022, open source copy
- An Introduction to Universal Artificial Intelligence, Marcus Hutter, David Quarel, Elliot Catt, CRC Press, 2024 - AIXI, Slides, Video
Biologically Inspired Approaches
Diverse approaches some of which tap into classical PDE systems of biological NNs, some concentrate on Distibuted Sparse Representations (by default non-differentiable), others draw inspiration from Hippocampal Grid Cells, Place Cells, etc. Biological systems surpass most ML methods for Continual and Online Learning, but are hard to implement efficienly on GPU.
- Ogma Sparse Predictive Hierarchies (SPH): whitepaper
- The Tolman-Eichenbaum Machine: Unifying space and relational memory through generalisation in the hippocampal formation (TEM), TEM-t
- Arousal as a universal embedding for spatiotemporal brain dynamics
- Sparse Distributed Memory is a Continual Learner
- Computation with Sequences of Assemblies in a Model of the Brain
Dense Associative Memory
Dense Associative Memory is mainly represented by Modern Hopfield Networks (MHN), which can be viewed as a generalized Transformers capable of storing queries, keys and values explicitly (as in Vector Databases) and running recurrent retrival by energy minimization (relating them to Diffusion models). Application for Continual Learning is possible when combined with uncertainty quantification and differentiable top-k selection.
- xLSTM repository
- CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory
- Energy Transformer
- Memorization and consolidation in associative memory networks
- Simplicial Hopfield networks
Continual Learning
Software Tools & Libraries
- paul-gauthier/aider
- OpenRLHF
- PRIME-RL/PRIME
- claude-engineer
- continuedev/continue
- OpenHands (formerly OpenDevin)
- princeton-nlp/SWE-agent, documentation
- stanfordnlp/dspy, DSPy awesome list: ganarajpr/awesome-dspy, paper
- InternLM/lagent - lightweight framework for building LLM-based agents
Commercial Offerings
- Software Engineering
- aide.dev + codestoryai/sidecar
- Devin
- Cursor
- Windsurf by Codeium
- GitHub Copilot & copilot-workspace
- lovable.dev
- textgrad
- Cosine Genie
- v0.dev by Vercel
- Replit AI
- bolt
- continue.dev
- Amazon Q Developer
- Codeyby Sourcegraph
- Claude Code
- Terminal
- General AI agent
- AWS Automated Reasoning checks
Competitions & Benchmarks
- DevAI: Agent-as-a-Judge: Evaluate Agents with Agents
- AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents, web: project page, gh: stonybrooknlp/appworld, Leaderboard
- CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents, gh: camel-ai/crab
- WebArena: A Realistic Web Environment for Building Autonomous Agents, web: project page, Leaderboard
- ARC-AGI: Leaderboard, On the Measure of Intelligence
- PlanBench: Paper, gh: karthikv792/LLMs-Planning
- GAIA: a benchmark for General AI Assistants: Leaderboard
- StreamBench: Towards Benchmarking Continuous Improvement of Language Agents, gh: stream-bench/stream-bench
- VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents
- ZebraLogic, Leaderboard
- Omni-MATH, gh: KbsdJames/Omni-MATH
- BatsResearch/planetarium - Dataset and benchmark for assessing LLMs in translating natural language descriptions of planning problems into PDDL
- BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
Code
- SWE-bench, SWE-bench Lite
- BigCodeBench: The Next Generation of HumanEval, Leaderboard
- SciCode: A Research Coding Benchmark Curated by Scientists, web: https://scicode-bench.github.io/
- commit-0 The challenge is to rebuild Python core libraries and pass their unit tests, Leaderboard
Related Projects
- Awesome LLM Strawberry (OpenAI o1)
- awesome-o1 literature list by Sasha Rush
- awesome-ai-agents
- Nous Research Open Reasoning Tasks, a list of reasoning tasks, gh: NousResearch/Open-Reasoning-Tasks
- ARC-AGI Resources Google table paper list by ARC price
Youtube Content
- Sasha Rush: Speculations on Test-Time Scaling (o1)
- François Chollet: It's Not About Scale, It's About Abstraction
- Evaluating, Understanding and Improving Approaches for Machine Reasoning
- Channel: David Shapiro
- Artem Kirsanov: Engrams, Building Blocks of Memory in the Brain
- Channel: Edan Meyer on AI, ML & RL, Discrete vs. Continuous RL + Paper
- MIT AGI: Cognitive Architecture (Nate Derbinsky)
- Channel: Thinking About Thinking (Mathematics of Neuroscience and AI)
- Invariance and equivariance in brains and machines
- code_your_own_AI: The CORE IDEA of AI Agents Explained
Best LLM APIs
Open-weights Reasoning Models
- DeepSeek-R1
- NovaSky-AI/Sky-T1-32B-Preview, Blog, gh: NovaSky-AI/SkyThought
- ngxson/MiniThinky-v2-1B-Llama-3.2
- SmallThinker-3B-Preview (small model trained on PowerInfer/QWQ-LONGCOT-500K)
- QwQ-32B-Preview, Blog post
- ruliad/deepthought-8b-llama-v0.01-alpha JSON format: 1. Problem understanding, 2. Data gathering, 3. Analysis, 4. Calculation (when applicable), 5. Verification, 6. Conclusion drawing, 7. Implementation
- migtissera/Tess-R1-Limerick-Llama-3.1-70B xml tags:
<thinking>tag to indicate when the model is performing CoT.<contemplation>tag when the model contemplate on its answers.<alternatively>tag for alternate suggestions.<output>for the final output
Novel model architectures
- 12 Mar 2025 Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
- 23 Feb 2025 Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking
- 14 Feb 2025 Large Language Diffusion Models
- 20 Jan 2025 Kimi k1.5: Scaling Reinforcement Learning with LLMs
- 20 Jan 2025 DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
- 09 Jan 2025 Transformer^2: Self-adaptive LLMs
- 31 Dec 2024 Titans: Learning to Memorize at Test Time
- 27 Dec 2025 Xmodel-2 Technical Report - Deep-and-Thin Architecture (1.2B, 48 layers)
- Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities
- Memory3: Language Modeling with Explicit Memory
- TTT: Learning to (Learn at Test Time): RNNs with Expressive Hidden States, Video
- TransformerFAM: Feedback attention is working memory
Philosophy: Nature of Intelligence & Consciousness
Joscha Bach
- Machine Consciousness
- Consciousness as a coherence-inducing operator Talk by Josha Bach at the Models of Consciousness Conferences
Biology / Neuroscience
- The brain simulates actions and their consequences during REM sleep
- CSCG: Clone-structured graph representations enable flexible learning and vicarious evaluation of cognitive maps
- System-1 and System-2 realized within the Common Model of Cognition (2022)
Workshops
https://s2r-at-scale-workshop.github.io (NeurIPS 2024)
Tutorials
How to contribute
To share a link related to reasoning in AI systems that is missing here please create a pull request for this file. See editing files in the github documentation.
Contributors
Similar repos
README
OpenThought - System 2 Research Links
Here you find a collection of material (books, papers, blog-posts etc.) related to reasoning and cognition in AI systems. Specifically we want to cover agents, cognitive architectures, general problem solving strategies and self-improvement.
The term "System 2" in the page title refers to the slower, more deliberative, and more logical mode of thought as described by Daniel Kahneman in his book Thinking, Fast and Slow.
You know a great resource we should add? Please see How to contribute.
Cognitive Architectures
(looking for additional links & articles and summaries)
- SOAR (State, Operator, And Result) by John Laird, Allen Newell, and Paul Rosenbloom
- ACT-R (Adaptive Control of Thought-Rational) by John Anderson at CMU
- SPAUN (Semantic Pointer Architecture Unified Network) by Chris Eliasmith at Waterloo, SPAUN 2.0 by Feng-Xuan Choo
- ART (Adaptive resonance theory) by Stephen Grossberg and Gail Carpenter
- CLARION (Connectionist Learning with Adaptive Rule Induction ON-line) by Ron Sun
- EPIC (Executive Process/Interactive Control) by David Kieras and David Meyer
- LIDA (Learning Intelligent Distribution Agent) by Stan Franklin, 2015 Paper
- Sigma by Paul Rosenbloom
- OpenCog by Ben Goertzel
- NARS (Non-Axiomatic Reasoning System) by Pei Wang
- Icarus by Pat Langley
- MicroPsi by Joscha Bach
- Thousand Brains Theory & HTM (Hierarchical Temporal Memory) by Jeff Hawkins
- SPH (Sparse Predictive Hierarchie) by Eric Laukien
- Leabra (Local, Error-driven and Associative, Biologically Realistic Algorithm), 2016 Paper by Randall O'Reilly
- CogNGen (COGnitive Neural GENerative system) by Alexander Ororbia and Mary Alexandria Kelly, see also here and here
- KIX (KIX: A Metacognitive Generalization Framework) by A. Kumar and Paul Schrater
- ACE (Autonomous Cognitive Entity) by David Shapiro et al., gh: daveshap/ACE_Framework
- Iterative Updating of Working Memory by Jared Reser, website, Video
Agent Papers
LLM Based
- 06 Feb 2025 Multi-agent Architecture Search via Agentic Supernet
- 06 Jan 2025 Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches
- Nov 2024 LLaVA-o1: Let Vision Language Models Reason Step-by-Step
- ReAct: Synergizing Reasoning and Acting in Language Models
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
- The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery, gh: SakanaAI/AI-Scientist
- OpenDevin: An Open Platform for AI Software Developers as Generalist Agents
- TextGrad: Automatic "Differentiation" via Text
- Trace is the New AutoDiff -- Unlocking Efficient Optimization of Computational Workflows
- Agentless: Demystifying LLM-based Software Engineering Agents
- Competition-Level Code Generation with AlphaCode
- AI Agents That Matter
- Sibyl: Simple yet Effective Agent Framework for Complex Real-world Reasoning
- Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
- Self-Rewarding Language Models
- ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models
- MedAgent-Zero: Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents
- Cognitive Architectures for Language Agents
- Large Language Models Can Self-Improve At Web Agent Tasks
- AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation
- A Prefrontal Cortex-inspired Architecture for Planning in Large Language Models
- CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization
- DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search
- GPT-Swarm Language Agents as Optimizable Graphs
- Survey: Reasoning with Large Language Models, a Survey (Jul 2024)
- Survey: From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future (Aug 2024)
- ADAS: Automated Design of Agentic Systems
- IDEA:Enhancing the Rule Learning Ability of Language Agents through Induction, Deduction, and Abduction
- LAW: Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning
- GenRM: Generative Verifiers: Reward Modeling as Next-Token Prediction
- Perspective: Towards Building Specialized Generalist AI with System 1 and System 2 Fusion
- CodeAct: Executable Code Actions Elicit Better LLM Agents
- PLANSEARCH: Planning In Natural Language Improves LLM Search For Code Generation
- LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBench
- Thinking LLMs: General Instruction Following with Thought Generation
- Agent S: An Open Agentic Framework that Uses Computers Like a Human
LLM Reasoning Improvements / Training on Synthetic Data
- 06 Mar 2025 START: Self-taught Reasoner with Tools
- 02 Mar 2025 LADDER: Self-Improving LLMs Through Recursive Problem Decomposition
- 17 Feb 2025 Scaling Test-Time Compute Without Verification or RL is Suboptimal
- 13 Feb 2025 Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
- 11 Feb 2025 CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
- 07 Feb 2025 Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- 05 Feb 2025 Demystifying Long Chain-of-Thought Reasoning in LLMs
- 05 Feb 2025 LIMO: Less is More for Reasoning
- 31 Jan 2025 s1: Simple test-time scaling
- 08 Jan 2025 MetaCoT: Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
- 05 Jan 2025 Test-time Computing: from System-1 Thinking to System-2 Thinking
- 30 Dec 2024 Aviary: training language agents on challenging scientific tasks - expert iteration & rejection sampling
- 25 Dec 2024 HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
- 18 Dec 2024 A Survey on LLM Inference-Time Self-Improvement
- 12 Dec 2024 STILL-2: Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems
- 02 Dec 2024 Mastering Board Games by External and Internal Planning with Language Models
- 29 Nov 2024 Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability
- 14 Oct 2024 TPO: Thinking LLMs: General Instruction Following with Thought Generation
- 4 Oct 2024 SWAP: Deliberate Reasoning for LLMs as Structure-aware Planning with Accurate World Model
- Thinking LLMs: General Instruction Following with Thought Generation
- Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources
- Chain of Thought Imitation with Procedure Cloning
- Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
- Self-Discover: Large Language Models Self-Compose Reasoning Structures
- TRICE: Training Chain-of-Thought via Latent-Variable Inference
- Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
- Self-Taught Reasoner: STaR: Bootstrapping Reasoning With Reasoning
- Self-Notes: Learning to Reason and Memorize with Self-Notes
- From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
- LaTRO: Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding, code
Datasets
- SYNTHETIC-1 (Prime Intellect)
- Open Reasoning Data (General Reasoning)
- OpenThoughts-114k
- reasoning-gym (Open-Thought Community)
- open-r1 datastes (Huggingface)
Direct o1 Replication Efforts
- Unsloth GRPO: Train your own R1 reasoning model with Unsloth (GRPO)
- HF open-r1: A fully open reproduction of DeepSeek-R1, gh: huggingface/open-r1
- TinyZero: A reproduction of DeepSeek R1 Zero in countdown and multiplication tasks. x: thread
- DeepSeek R-1: (https://chat.deepseek.com/), Tech report: deepseek-ai/DeepSeek-R1
- OpenR: Technical Report, Project Page, code: openreasoner/openr
- GAIR-NLP/O1-Journey, O1 Replication Journey: Strategic Progress Report - Part 1
- OpenSource-O1/Open-O1
- bklieger-groq/g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains
- ack-sec/toyberry: Atlas Reasoning System (Toyberry)
- Blog: Reverse engineering OpenAI’s o1 by Nathan Lambert
Reward Models (ORM/PRM)
- 13 Jan 2025 QwQ PRM: The Lessons of Developing Process Reward Models in Mathematical Reasoning - consensus of MC estimation & LLM-as-a-judge
- 02 Jan 2025 Process Reinforcement through Implicit Rewards - implicit PRM, gh: PRIME-RL/PRIME
- 02 Dec 2024 Implicit PRM: Free Process Rewards without Process Labels, gh: PRIME-RL/ImplicitPRM
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
- Solving math word problems with process- and outcome-based feedback
- Training Verifiers to Solve Math Word Problems
- RATIONALYST: Pre-training Process-Supervision for Improving Reasoning
- 14 Dec 2023 Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
RL
- 18 Feb 2025 Competitive Programming with Large Reasoning Models (OpenAI)
- 04 Feb 2025 Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
- 22 Jan 2025 Temporal Difference Learning: Why It Can Be Fast and How It Will Be Faster
- 04 Jan 2025 REINFORCE++: A Simple and Efficient Approach for Aligning Large Language Models
- 20 Dec 2024 OREO: Offline Reinforcement Learning for LLM Multi-Step Reasoning
- 11 Oct 2024 DQO: Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization
- 02 Oct 2024 RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
- 08 Feb 2024 R^3: Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
- 05 Feb 2024 GRPO: DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- 11 Jul 2023 Secrets of RLHF in Large Language Models Part I: PPO
- ReFT: Reasoning with Reinforced Fine-Tuning
- ARES: Alternating Reinforcement Learning and Supervised Fine-Tuning for Enhanced Multi-Modal Chain-of-Thought Reasoning Through Diverse AI Feedback
MCTS
- 11 Feb 2025 Monte Carlo Tree Diffusion for System 2 Planning
- 08 Jan 2025 rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
- Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
- Improve Mathematical Reasoning in Language Models by Automated Process Supervision
- Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
- rStar: Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
- LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios, code: opendilab/LightZero
- MuZero: Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, open-source impl: werner-duvaud/muzero-general
Minecraft Agents
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models
- Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks
- STEVE Series: Step-by-Step Construction of Agent Systems in Minecraft
Massive Sampling / Generate-and-Test
- Inference survey: From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling, code, blog
- AlphaCode 2 Technical Report
World Models
- GameNGen: Diffusion Models Are Real-Time Game Engines, project page
- A Path Towards Autonomous Machine Intelligence
- GAIA-1: A Generative World Model for Autonomous Driving
- Latent space world-models: Dreamer, V2, V3, DayDreamer
- World Models, web: project page
- Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models
Neuro-Symbolic Approaches
- Deliberate Reasoning for LLMs as Structure-aware Planning with Accurate World Model
- HYSYNTH: Context-Free LLM Approximation for Guiding Program Synthesis
- SymbolicAI: A framework for logic-based approaches combining generative models and solvers, Library: ExtensityAI/symbolicai
- DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning
- A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices
- Reasoning proofs generated by Prolog: Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs, Code
- VerityMath: Advancing Mathematical Reasoning by Self-Verification Through Unit Consistency, Code
Math
- AlphaGeometry: Solving olympiad geometry without human demonstrations
- Hologram Reasoning for Solving Algebra Problems with Geometry Diagrams
- Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving
Active Inference
- From pixels to planning: scale-free active inference
- Deep active inference agents using Monte-Carlo methods
Prompting Techniques
- Surveys:
- (Jul 2024) The Prompt Report: A Systematic Survey of Prompting Techniques
- (Feb 2024) A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
- Prompt Engineering Guide Prompting Techniques
- Prompting Fundamentals and How to Apply them Effectively by Eugene Yan
- Tools:
- Chain-of-Thoughts (COT): Paper
- Tree-of-Thoughts (ToT): Paper, impl: Strategic Debate
- Graph-of-Thoughts (GoT): Paper, code
- Algorithm of Thoughts (AoT): Paper
- Chain-of-Verification (CoVe/CoV): Paper
- Mixture-of-Agents (MoA): Paper
- Tool-Integrated Reasoning (ToRA / TIR): Paper
- Program of Thoughts (PoT): Paper
- Buffer of Thoughts (BoT): Paper
- Chain of Code (CoC): Paper
- Thought of Search (ToS): Paper
- Re-Reading the question as input (RE2): Paper
- Self-Harmonized Chain of Thought (ECHO): Paper, code
- Divergent CoT (DCoT), Paper
- Iteration of Thought (IoT), Paper
- Logic-of-Thought (LoT) Paper
- Forest-of-Thought (FoT) Paper
- 04 Feb 2025 Chain-of-Associated-Thoughts (CoAT): Paper
- 25 Feb 2025 Chain of Draft (CoD): Paper
Negative results
Mechanistic Interpretability
- Anthropic: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
- Geometric Space of Hierarchical Concepts in LLM
Blog Posts / Presentations
- 09 Mar 2025 Vintage Data Actual LLM agents are coming. They will be trained
- ?? Feb 2025 Vintage Data The Model is the Product
- 06 Feb 2025 Letta Stateful Agents: The Missing Link in LLM Intelligence
- 05 Feb 2025 Sebastian Raschka Understanding Reasoning LLMs
- 06 Feb 2025 unsloth Train your own R1 reasoning model with Unsloth (GRPO)
- 02 Feb 2025 Huggingface Open-R1: Update #1
- 25 Jan 2025 NLP@HKUST: https://hkust-nlp.notion.site/simplerl-reason, gh: hkust-nlp/simpleRL-reason
- 23 Jan 2025 Philschmid: How to align open LLMs in 2025 with DPO & and synthetic data
- 08 Jan 2025 ML CMU: Optimizing LLM Test-Time Compute Involves Solving a Meta-RL Problem
- 09 Jan 2025 Notebook: Agentic RAG with Hugging Face smolagents vs Vanilla RAG
- 07 Jan 2025 Chip Huyen: Agents
- HF: Scaling Test Time Compute with Open Models
- Nebius: Leveraging training and search for better software engineering agents
- DeepMind AlphaProof and AlphaGeometry 2
- Getting 50% (SoTA) on ARC-AGI with GPT-4o, code: rgreenblatt/arc_draw_more_samples_pub
- Schmidhuber: Artificial Curiosity & Creativity
- synthesis.ai: Do Androids Dream? World Models in Modern AI
- Our Transformers Code Agent beats the GAIA benchmark!
- Lil'Log LLM Powered Autonomous Agents (Jun 2023 )
- BAIR Blog: The Shift from Models to Compound AI Systems
- Microsoft Research Tracing the path to self-adapting AI agents
- LLMs develop their own understanding of reality as their language abilities improve, Emergent Representations Paper
- LessWrong post: LLM Generality is a Timeline Crux
- Three levels of self-building autonomous agents (Tweet thread by Yohei )
- Don't Sleep on Single-agent Systems
- Video: Improving LLM Reasoning using self-generated data: RL and Verifiers, Slides by Rishabh Agarwal (DeepMind)
- Slides: Reasoning with inference-time compute by Sean Welleck, tweet
Graph Neural Networks
- Distill A Gentle Introduction to Graph Neural Networks (2021)
- Geometric Deep Learning - Grids, Groups, Graphs, Geodesics, and Gauges
Complex Logical Query Answering (CQLA)
Answering logical queries over Incomplete Knowledge Graphs. Aspirationally this requires combining sparse symbolic index collation (SQL, SPARQL, etc) and dense vector search, preferably in a differentiable manner.
- Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databases
- Adapting Neural Link Predictors for Data-Efficient Complex Query Answering
- Generalizing Knowledge Graph Embedding with Universal Orthogonal Parameterization
- Knowledge Sheaves: A Sheaf-Theoretic Framework for Knowledge Graph Embedding
- Wasserstein-Fisher-Rao Embedding: Logical Query Embeddings with Local Comparison and Global Transport
- GammaE: Gamma Embeddings for Logical Queries on Knowledge Graphs
- Soft Reasoning on Uncertain Knowledge Graphs
Inductive Reasoning over Heterogeneous Graphs
Similar to the regular CQLA, but with the emphasis on the "Inductive Setting" - i.e. querying over new, unseen during training nodes, edge types or even entire graphs. The latter part is interesting as it relies on the higher order "relations between relations" structure, connecting KG inference to Category Theory.
- Zero-shot Logical Query Reasoning on any Knowledge Graph
- Extending Transductive Knowledge Graph Embedding Models for Inductive Logical Relational Inference
- Neural-Symbolic Models for Logical Queries on Knowledge Graphs
- InGram: Inductive Knowledge Graph Embedding via Relation Graphs
Neural Algorithmic Reasoning (NAR)
Initially attempted back in 2014 with general-purpose but unstable Neural Turing Machines, modern NAR approaches limit their scope to making GNN-based "Algorithmic Processor Networks" which learn to mimic classical algorithms on synthetic data and can be deployed on noisy real-world problems by sandwiching their frozen instances inside Encoder-Processor-Decoder architecture.
- Neural Turing Machines, 2014
- A Generalist Neural Algorithmic Learner
- Transformers meet Neural Algorithmic Reasoners
- Recursive Algorithmic Reasoning
- Dual Algorithmic Reasoning
- Learning to Configure Computer Networks with Neural Algorithmic Reasoning
Grokking
- Deep Networks Always Grok and Here is Why
- Grokfast: Accelerated Grokking by Amplifying Slow Gradients, review post by Lucas Nestler
- Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
Open-Source Agents & Agent Frameworks
- QwenLM/Qwen-Agent
- meta-llama/llama-agentic-system
- gpt-researcher, docs
- open-interpreter, docs
- ADAS (Automated Design of Agentic Systems)
- AI-Scientist
- Ollama_Agents
- AgentK
- Storm, Paper
- crewAI, docs
- AutoGPT, docs
- AutoGen, docs, AutoGen Studio Paper
- Trace, docs, Paper
- motleycrew, docs
- langflow, docs
- show-me: A Visual and Transparent Reasoning Agent
- ReasonerAgent-Web, Websiet
Algorithms
Weak Search Methods
Weak methods are general but don't use knowledge (heuristics) to guide the search process.
- depth-first-search (DFS)
- breadth-first-search (BFS)
- depth-limited-search, iterative-deepening-depth-first-search (IDDFS)
- generate-and-test
- hill-climbing (borderline case between weak and strong methods)
Strong Search Methods
Books
- The Soar Cognitive Architecture, John E. Laird, MIT Press, 2019
- How to Build a Brain: A Neural Architecture for Biological Cognition Chris Eliasmith, Oxford Series on Cognitive Models and Architectures, 2013
- Active Inference: The Free Energy Principle in Mind, Brain, and Behavior, Thomas Parr, Giovanni Pezzulo, Karl J. Friston, MIT Press, 2022, MLST Interview with Thomas Parr
- Principles of Synthetic Intelligence PSI: An Architecture of Motivated Cognition, Joscha Bach, Oxford Series on Cognitive Models and Architectures Book 4, 2009
- Conscious Mind, Resonant Brain: How Each Brain Makes a Mind, Stephen Grossberg, Oxford University Press, 2021
- The Society of Mind, Marvin Minsky, Simon & Schuster, 1986
- Reinforcement Learning: An Introduction 2nd Edition, Sutton & Barto, MIT Press, 2018
- Reinforcement Learning: An Overview, Dec 2024, Kevin Murphy
- Mathematical Foundations of Reinforcement Learning, Shiyu Zhao, open course on github + video lectures
- Natural Language Cognitive Architecture, David Shapiro, 2022, open source copy
- An Introduction to Universal Artificial Intelligence, Marcus Hutter, David Quarel, Elliot Catt, CRC Press, 2024 - AIXI, Slides, Video
Biologically Inspired Approaches
Diverse approaches some of which tap into classical PDE systems of biological NNs, some concentrate on Distibuted Sparse Representations (by default non-differentiable), others draw inspiration from Hippocampal Grid Cells, Place Cells, etc. Biological systems surpass most ML methods for Continual and Online Learning, but are hard to implement efficienly on GPU.
- Ogma Sparse Predictive Hierarchies (SPH): whitepaper
- The Tolman-Eichenbaum Machine: Unifying space and relational memory through generalisation in the hippocampal formation (TEM), TEM-t
- Arousal as a universal embedding for spatiotemporal brain dynamics
- Sparse Distributed Memory is a Continual Learner
- Computation with Sequences of Assemblies in a Model of the Brain
Dense Associative Memory
Dense Associative Memory is mainly represented by Modern Hopfield Networks (MHN), which can be viewed as a generalized Transformers capable of storing queries, keys and values explicitly (as in Vector Databases) and running recurrent retrival by energy minimization (relating them to Diffusion models). Application for Continual Learning is possible when combined with uncertainty quantification and differentiable top-k selection.
- xLSTM repository
- CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory
- Energy Transformer
- Memorization and consolidation in associative memory networks
- Simplicial Hopfield networks
Continual Learning
Software Tools & Libraries
- paul-gauthier/aider
- OpenRLHF
- PRIME-RL/PRIME
- claude-engineer
- continuedev/continue
- OpenHands (formerly OpenDevin)
- princeton-nlp/SWE-agent, documentation
- stanfordnlp/dspy, DSPy awesome list: ganarajpr/awesome-dspy, paper
- InternLM/lagent - lightweight framework for building LLM-based agents
Commercial Offerings
- Software Engineering
- aide.dev + codestoryai/sidecar
- Devin
- Cursor
- Windsurf by Codeium
- GitHub Copilot & copilot-workspace
- lovable.dev
- textgrad
- Cosine Genie
- v0.dev by Vercel
- Replit AI
- bolt
- continue.dev
- Amazon Q Developer
- Codeyby Sourcegraph
- Claude Code
- Terminal
- General AI agent
- AWS Automated Reasoning checks
Competitions & Benchmarks
- DevAI: Agent-as-a-Judge: Evaluate Agents with Agents
- AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents, web: project page, gh: stonybrooknlp/appworld, Leaderboard
- CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents, gh: camel-ai/crab
- WebArena: A Realistic Web Environment for Building Autonomous Agents, web: project page, Leaderboard
- ARC-AGI: Leaderboard, On the Measure of Intelligence
- PlanBench: Paper, gh: karthikv792/LLMs-Planning
- GAIA: a benchmark for General AI Assistants: Leaderboard
- StreamBench: Towards Benchmarking Continuous Improvement of Language Agents, gh: stream-bench/stream-bench
- VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents
- ZebraLogic, Leaderboard
- Omni-MATH, gh: KbsdJames/Omni-MATH
- BatsResearch/planetarium - Dataset and benchmark for assessing LLMs in translating natural language descriptions of planning problems into PDDL
- BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
Code
- SWE-bench, SWE-bench Lite
- BigCodeBench: The Next Generation of HumanEval, Leaderboard
- SciCode: A Research Coding Benchmark Curated by Scientists, web: https://scicode-bench.github.io/
- commit-0 The challenge is to rebuild Python core libraries and pass their unit tests, Leaderboard
Related Projects
- Awesome LLM Strawberry (OpenAI o1)
- awesome-o1 literature list by Sasha Rush
- awesome-ai-agents
- Nous Research Open Reasoning Tasks, a list of reasoning tasks, gh: NousResearch/Open-Reasoning-Tasks
- ARC-AGI Resources Google table paper list by ARC price
Youtube Content
- Sasha Rush: Speculations on Test-Time Scaling (o1)
- François Chollet: It's Not About Scale, It's About Abstraction
- Evaluating, Understanding and Improving Approaches for Machine Reasoning
- Channel: David Shapiro
- Artem Kirsanov: Engrams, Building Blocks of Memory in the Brain
- Channel: Edan Meyer on AI, ML & RL, Discrete vs. Continuous RL + Paper
- MIT AGI: Cognitive Architecture (Nate Derbinsky)
- Channel: Thinking About Thinking (Mathematics of Neuroscience and AI)
- Invariance and equivariance in brains and machines
- code_your_own_AI: The CORE IDEA of AI Agents Explained
Best LLM APIs
Open-weights Reasoning Models
- DeepSeek-R1
- NovaSky-AI/Sky-T1-32B-Preview, Blog, gh: NovaSky-AI/SkyThought
- ngxson/MiniThinky-v2-1B-Llama-3.2
- SmallThinker-3B-Preview (small model trained on PowerInfer/QWQ-LONGCOT-500K)
- QwQ-32B-Preview, Blog post
- ruliad/deepthought-8b-llama-v0.01-alpha JSON format: 1. Problem understanding, 2. Data gathering, 3. Analysis, 4. Calculation (when applicable), 5. Verification, 6. Conclusion drawing, 7. Implementation
- migtissera/Tess-R1-Limerick-Llama-3.1-70B xml tags:
<thinking>tag to indicate when the model is performing CoT.<contemplation>tag when the model contemplate on its answers.<alternatively>tag for alternate suggestions.<output>for the final output
Novel model architectures
- 12 Mar 2025 Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
- 23 Feb 2025 Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking
- 14 Feb 2025 Large Language Diffusion Models
- 20 Jan 2025 Kimi k1.5: Scaling Reinforcement Learning with LLMs
- 20 Jan 2025 DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
- 09 Jan 2025 Transformer^2: Self-adaptive LLMs
- 31 Dec 2024 Titans: Learning to Memorize at Test Time
- 27 Dec 2025 Xmodel-2 Technical Report - Deep-and-Thin Architecture (1.2B, 48 layers)
- Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities
- Memory3: Language Modeling with Explicit Memory
- TTT: Learning to (Learn at Test Time): RNNs with Expressive Hidden States, Video
- TransformerFAM: Feedback attention is working memory
Philosophy: Nature of Intelligence & Consciousness
Joscha Bach
- Machine Consciousness
- Consciousness as a coherence-inducing operator Talk by Josha Bach at the Models of Consciousness Conferences
Biology / Neuroscience
- The brain simulates actions and their consequences during REM sleep
- CSCG: Clone-structured graph representations enable flexible learning and vicarious evaluation of cognitive maps
- System-1 and System-2 realized within the Common Model of Cognition (2022)
Workshops
https://s2r-at-scale-workshop.github.io (NeurIPS 2024)
Tutorials
How to contribute
To share a link related to reasoning in AI systems that is missing here please create a pull request for this file. See editing files in the github documentation.