Awesome LLM-based Agent for Code Generation

🎉 Our work has been accepted to the Journal of Software 2026.

A curated list of awesome LLM-based agents for code generation, including papers, tools, frameworks, and resources.

📋 Table of Contents

• Single-Agent Methods
• Multi-Agent Methods
• Benchmarks
• Tools

🤖 Single-Agent Methods

2023

• Self-planning Code Generation with Large Language Models. (2023-03-12) [Paper]

• Self-Refine: Iterative Refinement with Self-Feedback.(2023-03-30) [Paper]

• Teaching Large Language Models to Self-Debug (2023-04-11) [Paper]

• ToolCoder: Teach Code Generation Models to use API search tools (2023-05-06) [Paper]

• Self-Edit: Fault-Aware Code Editor for Code Generation (2023-05-06) [Paper]

• Is Self-Repair a Silver Bullet for Code Generation? (2023-06-16) [Paper]

• A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis (2023-07-24) [Paper]

• CodePlan: Repository-level Coding using LLMs and Planning (2023-09-21) [Paper]

• L2MAC: Large Language Model Automatic Computer for Extensive Code Generation (2023-10-02) [Paper]

• Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models (2023-10-06) [Paper]

• Lemur: Harmonizing Natural Language and Code for Language Agents (2023-10-10) [Paper]

• CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules (2023-10-13) [Paper]

• A Self-Iteration Code Generation Method Based on Large Language Models (2023-12-17) [Paper]

2024

• CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges (2024-01-14) [Paper]

• Knowledge-Aware Code Generation with Large Language Models (2024-01-29) [Paper]

• Executable Code Actions Elicit Better LLM Agents (2024-02-01) [Paper]

• RepairAgent: An Autonomous, LLM-Based Agent for Program Repair (2024-03-25) [Paper]

• AnalogCoder: Analog Circuit Design via Training-Free Code Generation (2024-05-23) [Paper]

• Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search (2024-05-24)[Paper]

• LeDex: Training LLMs to Better Self-Debug and Explain Code (2024-05-28) [Paper]

• CodeNav: Beyond tool-use to using real-world codebases with LLM agents (2024-06-18) [Paper]

• Planning In Natural Language Improves LLM Search For Code Generation (2024-09-05) [Paper]

• SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement (2024-10-26)[Paper]

• CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models (2024-11-07) [Paper]

• ROCODE: Integrating Backtracking Mechanism and Program Analysis in Large Language Models for Code Generation (2024-11-11)[Paper]

• Tree-of-Code: A Tree-Structured Exploring Framework for End-to-End Code Generation and Execution in Complex Task Handling (2024-12-19)[Paper]

2025 (Up to June)

• Optimizing Code Runtime Performance through Context-Aware Retrieval-Augmented Generation (2025-01-28) [Paper]

• PatchPilot: A Cost-Efficient Software Engineering Agent with Early Attempts on Formal Verification (2025-02-04)[Paper]

• AIDE: AI-Driven Exploration in the Space of Code (2025-02-18) [Paper]

• DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal (2025-03-18) [Paper]

• CodeTool: Enhancing Programmatic Tool Invocation of LLMs via Process Supervision (2025-03-26) [Paper]

• CodeRAG: Supportive Code Retrieval on Bigraph for Real-World Code Generation (2025-04-14) [Paper]

• Guided Search Strategies in Non-Serializable Environments with Applications to Software Engineering Agents (2025-05-19) [Paper]

• cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Tree (2025-06-18)[Paper]

👥 Multi-Agent Methods

2023

• Self-collaboration Code Generation via ChatGPT (2023-04-15) [Paper]

• ChatDev: Communicative Agents for Software Development (2023-07-16) [Paper]

• MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework (2023-08-01) [Paper]

• GameGPT: Multi-agent Collaborative Framework for Game Development (2023-10-12) [Paper]

• AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation (2023-12-20) [Paper]

2024

• CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology (2024-02-02) [Paper]

• CleanAgent: Automating Data Standardization with LLM-based Agents (2024-03-13) [Paper]

• SOEN-101: Code Generation by Emulating Software Process Models Using Large Language Model Agents (2024-03-23) [Paper]

• MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution (2024-03-26) [Paper]

• Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization (2024-04-02) [Paper]

• AutoCodeRover: Autonomous Program Improvement (2024-04-08) [Paper]

• MARE: Multi-Agents Collaboration Framework for Requirements Engineering (2024-05-06) [Paper]

• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024-05-06) [Paper]

• MapCoder: Multi-Agent Code Generation for Competitive Problem Solving (2024-05-18) [Paper]

• AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology (2024-06-16) [Paper]

• Agentless: Demystifying LLM-based Software Engineering Agents (2024-07-01) [Paper]

• OpenHands: An Open Platform for AI Software Developers as Generalist Agents (2024-07-23) [Paper]

• VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool (2024-08-15) [Paper]

• A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinement (2024-09-08) [Paper]

• HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scale (2024-09-09)[Paper]

• AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testing (2024-09-16) [Paper]

• Self-Evolving Multi-Agent Collaboration Networks for Software Development (2024-10-22) [Paper]

• MAGE: A Multi-Agent Engine for Automated RTL Code Generation (2024-12-10) [Paper]

2025

• CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation (2025-01-14) [Paper]

• QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks (2025-01-20) [Paper]

• Cogito, ergo sum: A Neurobiologically-Inspired Cognition-Memory-Growth System for Code Generation (2025-01-30) [Paper]

• CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging (2025-02-08) [Paper]

• SyncMind: Measuring Agent Out-of-Sync Recovery in Collaborative Software Engineering (2025-02-10) [Paper]

• MARCO: Multi-Agent Code Optimization with Real-Time Knowledge Integration for High-Performance Computing (2025-05-06) [Paper]

• Adversarial Reasoning for Repair Based on Inferred Program Intent (2025-05-19) [Paper]

• SEW: Self-Evolving Agentic Workflows for Automated Code Generation (2025-05-24)[Paper]

🏆 Benchmarks

• APPS (2021-05-20) [Paper]
• HumanEval (2021-07-07) [Paper]
• MBPP (2021-08-16) [Paper]
• CodeContests (2022-02-08)[Paper]
• Aider (2023-04-04) [Link]
• SWE-Bench (2023-10-10)[Paper]
• CodeAgentBench (2024-01-14) [Paper]
• LiveCodeBench (2024-03-12)[Paper]
• EvoCodeBench (2024-03-31) [Paper]
• DevEval (2024-05-30) [Paper]

🛠️ Tools

• Github Copilot [Link]
• Devin [Link]
• Cursor [Link]
• Tongyi Lingma [Link]
• Claude Code [Link]

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Thanks to all contributors and researchers in the field
• Special thanks to the open-source community

📑 Citation

@misc{dong2025surveycodegenerationllmbased,
      title={A Survey on Code Generation with LLM-based Agents}, 
      author={Yihong Dong and Xue Jiang and Jiaru Qian and Tian Wang and Kechi Zhang and Zhi Jin and Ge Li},
      year={2025},
      eprint={2508.00083},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/2508.00083}, 
}

---

Note: This is a living document. Please help us keep it updated by contributing new papers, tools, and resources!