Agentic Software Issue Resolution with Large Language Models: A Survey
See the codeREADME
Agentic Software Issue Resolution with Large Language Models: A Survey
📰 News
- 📅 2026.05: Paper update! We add 89 papers up to 2026.05 and refresh the venue information according to the latest acceptance results. We also align the taxonomy with our survey. Now, we have included 242 papers in this survey.
- 📅 2026.01: Paper update! We add 34 papers from 2025.10 to 2025.12. Now, we have included 160 papers in this survey.
- 📅 2025.12: We release the first survey on agentic software issue resolution!
- 📅 2025.10: We summarize 126 papers about issue resolution, from 2023.10 to 2025.10!
Introduction
We classified this survey into three main parts: Benchmarks, Technologies and Empirical Studies.
Automated issue solving technologies can be mainly surveyed from 2 perspectives:
Scaffold/Method Design and Learning Strategies.
Table of Contents
- Benchmarks
- End-To-End
- Reproduction Test Generation
- Localization
- Technologies
- Scaffold/Method Design
- End-to-End Scaffold
- Agent-Based
- Pipeline-Based
- Scaffold Improvement
- Memory
- Context Management
- Self-Evolution
- Single-Phased Methods
- End-to-End Scaffold
- Learning Strategies
- Data Preparation
- Real-world Data
- Synthetic Data
- Automated Environment Infrastructure
- Training
- Supervised Fine-tuning (SFT)
- Reinforcement Learning (RL)
- Data Preparation
- Scaffold/Method Design
- Empirical Studies
- Evaluation-oriented Studies
- Evaluation Validity
- Quality and Requirement Compliance
- Efficiency and Resource Consumption
- Robustness and Process-aware Evaluation
- Technique-oriented Studies
- Performance Variation and Influencing Factors
- Agent Behavior and Resolution Process
- Failure Modes and Root Causes
- Evaluation-oriented Studies
Benchmarks
For Benchmarks, we summarized the existing benchmarks into 2 categories for their different tasks.
@End-To-End
@Reproduction Test Generation
@Localization
| Literature | Name | Scope | Journal/Conference | Time | Link |
|---|---|---|---|---|---|
| SWE-bench: Can Language Models Resolve Real-World GitHub Issues? | SWE-bench | End-To-End | ICLR'24 | 2023-10 | Paper Code |
| SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents | SWT-Bench | Reproduction Test Generation | NeurIPS'24 | 2024-06 | Paper Code |
| SWE-bench-java: A GitHub Issue Resolving Benchmark for Java | Muti-SWE-bench | End-To-End | ARXIV | 2024-08 | Paper Code |
| SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains? | SWE-bench Mutimodal | End-To-End | ICLR'25 | 2024-10 | Paper Code |
| SWE-Bench+: Enhanced Coding Benchmark for LLMs | SWE-Bench+ | End-To-End | ARXIV | 2024-10 | Paper |
| TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark | TestGenEval | Reproduction Test Generation | ICLR'25 | 2024-10 | Paper Code |
| A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models | FAUN-Eval | End-To-End | ARXIV | 2024-11 | Paper |
| TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved? | TDD-Bench | Reproduction Test Generation | ARXIV | 2024-11 | Paper Code |
| CodeV: Issue Resolving with Visual Data | Visual SWE-bench | End-To-End | ACL Findings'25 | 2024-12 | Paper Code |
| Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving | Muti-SWE-bench | End-To-End | NeurIPS'25 | 2025-04 | Paper Code |
| LiveSWEBench | LiveSWEBench | End-To-End | BLOG | 2025-04 | link Code |
| LocAgent: Graph-Guided LLM Agents for Code Localization | LocBench | Localization | ACL'25 | 2025-03 | Paper Code |
| Automated Benchmark Generation for Repository-Level Coding Tasks | SWEE-Bench/SWA-Bench | End-To-End | ICML'25 | 2025-03 | Paper |
| FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation | FEA-Bench | End-To-End | ACL'25 | 2025-03 | Paper Code |
| OmniGIRL: A Multilingual and Multimodal Benchmark for GitHub Issue Resolution | OmniGIRL | End-To-End | ISSTA'25 | 2025-05 | Paper Code |
| - | SWE-bench Multilingual | End-To-End | BLOG | 2025-05 | link Code |
| SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents | SWE-PolyBench | End-To-End | ARXIV | 2025-04 | Paper Code |
| SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents | SWE-rebench | End-To-End | NeurIPS'25 | 2025-05 | Paper Code |
| GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents | GSO | End-To-End | NeurIPS'25 | 2025-05 | Paper Code |
| SWE-bench Goes Live! | SWE-bench-Live | End-To-End | NeurIPS'25 | 2025-05 | Paper Code |
| UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench | UTBoost | - | ACL'25 | 2025-06 | Paper Code |
| SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks | SWE-Factory | End-To-End | FSE'26 | 2025-06 | Paper Code |
| SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving | Swing-Arena | End-To-End | ICLR'26 | 2025-06 | Paper Code |
| SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation | SPICE | - | ASE'25 | 2025-07 | Paper |
| SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks | SWE-MERA | End-To-End | EMNLP'25 | 2025-07 | Paper |
| SWE-Perf: Can Language Models Optimize Code Performance on Real-World Repositories? | SWE-Perf | End-To-End | ICML'26 | 2025-07 | Paper Code |
| NoCode-bench: A Benchmark for Evaluating Natural Language-Driven Feature Addition | NoCode-bench | End-To-End | ARXIV | 2025-08 | Paper Code |
| SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? | SWE-Bench Pro | End-To-End | ICML'26 | 2025-09 | Paper Code |
| A Benchmark for Localizing Code and Non-Code Issues in Software Projects | MULocBench | Localization | ARXIV | 2025-10 | Paper Code |
| SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads? | SWE-fficiency | End-To-End | ICML'26 | 2025-11 | Paper Code |
| SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories | SWE-Bench++ | End-To-End | ARXIV | 2025-12 | Paper Code |
| SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios | SWE-EVO | End-To-End | ARXIV | 2025-12 | Paper Code |
| SWE-Refactor: A Repository-Level Benchmark for Real-World LLM-Based Code Refactoring | SWE-Refactor | End-To-End | ARXIV | 2026-02 | Paper |
| ContextBench: A Benchmark for Context Retrieval in Coding Agents | ContextBench | Localization | ARXIV | 2026-02 | Paper |
| SWE Context Bench: A Benchmark for Context Learning in Coding | SWE Context Bench | Localization | ARXIV | 2026-02 | Paper |
| SWE-Bench Mobile: Can Large Language Model Agents Develop Industry-Level Mobile Applications? | SWE-Bench Mobile | End-To-End | ARXIV | 2026-02 | Paper |
| FeatureBench: Benchmarking Agentic Coding for Complex Feature Development | FeatureBench | End-To-End | ICLR'26 | 2026-02 | Paper |
| Evaluating and Improving Automated Repository-Level Rust Issue Resolution with LLM-based Agents | Rust-SWE-bench | End-To-End | ICSE'26 | 2026-02 | Paper |
| SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark | SWE-ABS | - | ICML'26 | 2026-02 | Paper |
| SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integration | SWE-CI | End-To-End | ARXIV | 2026-03 | Paper |
| Beyond Isolated Tasks: A Framework for Evaluating Coding Agents on Sequential Software Evolution | SWE-STEPS | End-To-End | ARXIV | 2026-04 | Paper |
| SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades | SWE-Chain | End-To-End | ARXIV | 2026-05 | Paper |
| SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering? | SWE-Mutation | Reproduction Test Generation | ACL'26 | 2026-05 | Paper |
Technologies
Scaffold/Method Design
From the perspective of Design Paradigms, we can classify them into 3 categories:
@End-To-End Scaffold
@Scaffold Improvement
@Single-Phased Methods
End-to-End Scaffold
For End-To-End Method, we can further classify them into 2 categories:
@Agent-Based Method
@Pipeline-Based Method
| Literature | Name | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|---|
| SWE-bench: Can Language Models Resolve Real-World GitHub Issues? | BM25 RAG | ICLR 2024 | 2023-10 | @Pipeline | Paper Code |
| SWE-agent: Agent-computer interfaces enable automated software engineering | SWE-Agent | NeurIPS 2024 | 2024-05 | @Agent | Paper Code |
| Autocoderover: Autonomous program improvement | AutoCodeRover | ISSTA 2024 | 2024-04 | @Agent | Paper Code |
| CodeR: Issue Resolving with Multi-Agent and Task Graphs | CodeR | Arxiv | 2024-06 | @Agent | Paper Code |
| Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration | LingmaAgent/RepoUnderstander | FSE Companion 2025 | 2024-06 | @Agent | Paper Code |
| MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution | MAGIS | NeurIPS 2024 | 2024-03 | @Agent | Paper Code |
| MASAI: Modular Architecture for Software-engineering AI Agents | MASAI | NeurIPS 2024 | 2024-06 | @Agent | Paper |
| Demystifying LLM-based Software Engineering Agents | Agentless | FSE 2025 | 2024-07 | @Pipeline | Paper Code |
| Specrover: Code intent extraction via llms | SpecRover (AutoCodeRover-v2) | ICSE 2025 | 2024-08 | @Agent | Paper Code |
| CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases | CodexGraph | NAACL 2025 | 2024-08 | @Agent | Paper Code |
| SuperCoder2.0: Technical Report on Exploring the feasibility of LLMs as Autonomous Programmer | SuperCoder | Arxiv | 2024-09 | @Agent | Paper |
| Hyperagent: Generalist software engineering agents to solve coding tasks at scale | HyperAgent | Arxiv | 2024-09 | @Agent | Paper |
| RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph | RepoGraph | ICLR 2025 | 2024-10 | @Pipeline | Paper Code |
| SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement | SWE-Search | ICLR 2025 | 2024-10 | @Agent | Paper Code |
| OpenHands: An Open Platform for AI Software Developers as Generalist Agents | OpenHands CodeAct | ICLR 2025 | 2024-10 | @Agent | Paper Code |
| - | Composio SWE-Kit | Blog | 2024-10 | @Pipeline | Link Code |
| Infant Agent: A Tool-Integrated, Logic-Driven Agent with Cost-Effective API Usage | Infant Agent | Arxiv | 2024-11 | @Agent | Paper |
| MarsCode Agent: AI-native Automated Bug Fixing | MarsCode Agent | Arxiv | 2024-11 | @Agent | Paper |
| Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement | SWESynInfer | Arxiv | 2024-11 | @Pipeline | Paper Code |
| - | Nebius AI | Blog | 2024-11 | @Agent | Paper |
| CodeV: Issue Resolving with Visual Data | CodeV | ACL Findings 2025 | 2024-12 | @Pipeline | Paper Code |
| - | Aide | Blog | 2024-12 | @Agent | Link |
| Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments | Learn-By-Interact | ICLR 2025 | 2025-01 | @Agent | Paper |
| PatchPilot: A Stable and Cost-Efficient Agentic Patching Framework | PatchPilot | ICML 2025 | 2025-02 | @Pipeline | Paper Code |
| CodeMonkeys: Scaling Test-Time Compute for Software Engineering | CodeMonkeys | Arxiv | 2025-02 | @Pipeline | Paper Code |
| SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution | Agentless Mini | NeurIPS 2025 | 2025-02 | @Pipeline | Paper Code |
| - | Agentless Lite | Blog | 2025-02 | @Pipeline | Code |
| - | Syntheo | Blog | 2025-02 | @Agent | Link |
| SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution | SWE-Fixer | ACL Findings 2025 | 2025-02 | @Pipeline | Paper |
| - | AgentScope | Blog | 2025-03 | @Agent | Link |
| DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal | DARS | ACL 2025 | 2025-03 | @Agent | Paper Code |
| Enhancing Repository-Level Software Repair via Repository-Aware Knowledge Graphs | KGCompass | Arxiv | 2025-03 | @Pipeline | Paper |
| - | Augment Agent v0 | Blog | 2025-03 | @Agent | Link Code |
| - | CORTEXA | Blog | 2025-03 | @Pipeline | Link |
| - | Refact.ai | Blog | 2025-03 | @Agent | Link Code |
| - | Lingxi | Blog | 2025-04 | @Agent | Link Code |
| - | Trae IDE | Blog | 2025-05 | @Agent | Link |
| - | devlo | Blog | 2025-05 | @Agent | Link |
| Putting It All into Context: Simplifying Agents with LCLMs | LCLM | Arxiv | 2025-05 | @Pipeline | Paper |
| Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks | CGM-SWE-PY | NeurIPS'25 | 2025-05 | @Pipeline | Paper |
| InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer Interaction | InfantAgent-Next | NeurIPS 2025 | 2025-05 | @Agent | Paper Code |
| Coding Agents with Multimodal Browsing are Generalist Problem Solvers | OpenHands-Versa | ICML 2026 | 2025-06 | @Agent | Paper Code |
| EXPEREPAIR: Dual-Memory Enhanced LLM-based Repository-Level Program Repair | EXPEREPAIR | FSE 2026 | 2025-06 | @Agent | Paper |
| Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing | GUIRepair | ASE'25 | 2025-06 | @Pipeline | Paper |
| SemAgent: A Semantics Aware Program Repair Agent | SemAgent | Arxiv | 2025-06 | @Pipeline | Paper |
| Nemotron-Cortexa: Enhancing LLM Agents for Software Engineering Tasks via Improved Localization and Solution Diversity | Nemotron-Cortexa | ICML'25 | 2025-06 | @Pipeline | Paper Code |
| Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving | Agent KB | Arxiv | 2025-07 | @Agent | Paper Code |
| Prometheus: Unified Knowledge Graphs for Issue Resolution in Multilingual Codebases | Prometheus | Arxiv | 2025-07 | @Agent | Paper Code |
| SWE-Exp: Experience-Driven Software Issue Resolution | SWE-Exp | Arxiv | 2025-07 | @Agent | Paper Code |
| SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution | SWE-Debate | ICSE 2026 | 2025-07 | @Agent | Paper Code |
| Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling | Trae Agent | Arxiv | 2025-07 | @Agent | Paper Code |
| SynFix: Dependency-Aware Program Repair via RelationGraph Analysis | SynFix | ACL Findings'25 | 2025-07 | @Pipeline | Paper |
| SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents | SE-Agent | NeurIPS'25 | 2025-08 | @Agent | Paper Code |
| Lita: Light Agent Uncovers the Agentic Coding Capabilities of LLMs | Lita | Arxiv | 2025-10 | @Agent | Paper |
| Lingxi: Repository-Level Issue Resolution Framework Enhanced by Procedural Knowledge Guided Scaling | Lingxi | Arxiv | 2025-10 | @Agent | Paper Code |
| SIADAFIX: issue description response for adaptive program repair | SIADAFIX | Arxiv | 2025-10 | @Pipeline | Paper Code |
| TDFlow: Agentic Workflows for Test Driven Software Engineering | TDFlow | Arxiv | 2025-10 | @Pipeline | Paper |
| InfCode: Adversarial Iterative Refinement of Tests and Patches for Reliable Software Issue Resolution | InfCode | Arxiv | 2025-11 | @Agent | Paper |
| Think-Search-Patch: A Retrieval-Augmented Reasoning Framework for Repository-Level Code Repair | TSP | EMNLP 2025 | 2025-11 | @Pipeline | Paper Code |
| Confucius Code Agent: An Open-sourced AI Software Engineer at Industrial Scale | CCA | Arxiv | 2025-12 | @Agent | Paper |
| Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering | Agyn | Arxiv | 2026-02 | @Agent | Paper |
| SVRepair: Structured Visual Reasoning for Automated Program Repair | SVRepair | Arxiv | 2026-02 | @Pipeline | Paper |
| Debug2Fix: Supercharging Coding Agents with Interactive Debugging Capabilities | Debug2Fix | Arxiv | 2026-02 | @Agent | Paper |
| SGAgent: Suggestion-Guided LLM-Based Multi-Agent Framework for Repository-Level Software Repair | SGAgent | TOSEM | 2026-02 | @Agent | Paper |
| SWE-Adept: An LLM-Based Agentic Framework for Deep Codebase Analysis and Structured Issue Resolution | SWE-Adept | Arxiv | 2026-03 | @Agent | Paper |
| RepoRepair: Leveraging Code Documentation for Repository-Level Automated Program Repair | RepoRepair | Arxiv | 2026-03 | @Pipeline | Paper |
| Architecture-Aware Multi-Design Generation for Repository-Level Feature Addition | RAIM | Arxiv | 2026-03 | @Pipeline | Paper |
| Resolving Java Code Repository Issues with iSWE Agent | iSWE Agent | Arxiv | 2026-03 | @Agent | Paper |
| Agent-Based Ensemble Reasoning for Repository-Level Issue Resolution | - | ICSE 2026 | 2026-04 | @Agent | - |
| Beyond Fixed Tests: Repository-Level Issue Resolution as Coevolution of Code and Behavioral Constraints | Agent-CoEvo | Arxiv | 2026-04 | @Agent | Paper |
| REAgent: Requirement-Driven LLM Agents for Software Issue Resolution | REAgent | Arxiv | 2026-04 | @Agent | Paper |
| ARISE: A Repository-level Graph Representation and Toolset for Agentic Fault Localization and Program Repair | ARISE | Arxiv | 2026-05 | @Pipeline | Paper |
Scaffold Improvement
For Scaffold Improvement, we discuss them in 3 categories separately:
@Memory
@Context (context management)
@Self-Evolution
| Literature | Name | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|---|
| A Self-Improving Coding Agent | SICA | ICLR 2026 | 2025-04 | @Self-Evolution | Paper |
| Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents | DGM | ICLR 2026 | 2025-05 | @Self-Evolution | Paper |
| EXPEREPAIR: Dual-Memory Enhanced LLM-based Repository-Level Program Repair | EXPEREPAIR | FSE 2026 | 2025-06 | @Memory | Paper |
| Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving | Agent KB | Arxiv | 2025-07 | @Memory | Paper Code |
| SWE-Exp: Experience-Driven Software Issue Resolution | SWE-Exp | Arxiv | 2025-07 | @Memory | Paper Code |
| SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents | SE-Agent | NeurIPS'25 | 2025-08 | @Self-Evolution | Paper Code |
| Reducing Cost of LLM Agents with Trajectory Reduction | AgentDiet | FSE'26 | 2025-09 | @Context | Paper |
| Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine | HGM | ICLR 2026 | 2025-10 | @Self-Evolution | Paper |
| TOM-SWE: User Mental Modeling For Software Engineering Agents | TOM-SWE | ICML 2026 | 2025-10 | @Memory | Paper Code |
| Self-Abstraction from Grounded Experience for Plan-Guided Policy Refinement | SAGE | Arxiv | 2025-11 | @Self-Evolution | Paper |
| Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly? | Live-SWE-agent | Arxiv | 2025-11 | @Self-Evolution | Paper Code |
| Confucius Code Agent: An Open-sourced AI Software Engineer at Industrial Scale | CCA | Arxiv | 2025-12 | @Memory | Paper |
| EET: Experience-Driven Early Termination for Cost-Efficient Software Engineering Agents | EET | ACL 2026 | 2026-01 | @Memory | Paper |
| MemGovern: Enhancing Code Agents through Learning from Governed Human Experiences | MemGovern | Arxiv | 2026-01 | @Memory | Paper |
| Outcome-Conditioned Reasoning Distillation for Resolving Software Issues | ConRAD | ICML 2026 | 2026-01 | @Context | Paper |
| SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents | SWE-Pruner | Arxiv | 2026-01 | @Context | Paper |
| Your Code Agent Can Grow Alongside You with Structured Memory | MemCoder | Arxiv | 2026-02 | @Self-Evolution | Paper |
| FailureMem: A Failure-Aware Multimodal Framework for Autonomous Software Repair | FailureMem | Arxiv | 2026-03 | @Memory | Paper |
| Compressing Code Context for LLM-based Issue Resolution | - | Arxiv | 2026-03 | @Context | Paper |
| CodeScout: Contextual Problem Statement Enhancement for Software Agents | CodeScout | Arxiv | 2026-03 | @Context | Paper |
| CODESTRUCT: Code Agents over Structured Action Spaces | CODESTRUCT | ACL 2026 | 2026-04 | @Context | Paper |
| SWE-Edit: Rethinking Code Editing for Efficient SWE-Agent | SWE-Edit | Arxiv | 2026-04 | @Context | Paper |
Single-Phased Methods
For Single-Phased Method, we discuss them in 3 categories separately:
@Localization
@Reproduction
@Selection
where, @Reproduction indicates the reproduction test generation, @Selection indicates the patch validation
and selection.
Issue Localization
| Literature | Name | Journal/Conference | Time | URL |
|---|---|---|---|---|
| BLAZE: Cross-Language and Cross-Project Bug Localization via Dynamic Chunking and Hard Example Learning | BLAZE | TSE | 2024-08 | Paper Code |
| Integrating Various Software Artifacts for Better LLM-based Bug Localization and Program Repair | DEVLoRe | TOSEM | 2024-12 | Paper |
| OrcaLoca: An LLM Agent Framework for Software Issue Localization | OrcaLoca | ICML 2025 | 2025-02 | Paper Code |
| Bridging Bug Localization and Issue Fixing: A Hierarchical Localization Framework Leveraging Large Language Models | BugCerberus | TSE | 2025-02 | Paper |
| LocAgent: Graph-Guided LLM Agents for Code Localization | LocAgent | ACL 2025 | 2025-03 | Paper Code |
| Issue Localization via LLM-Driven Iterative Code Graph Searching | CoSIL | ASE 2025 | 2025-03 | Paper Code |
| CORNSTACK: HIGH-QUALITY CONTRASTIVE DATA FOR BETTER CODE RETRIEVAL AND RERANKING | CoRNStack | ICLR 2025 | 2025-03 | Paper Code |
| SweRank: Software Issue Localization with Code Ranking | SweRank | ICLR 2026 | 2025-05 | Paper Code |
| CoRet: Improved Retriever for Code Editing | CoRet | ACL 2025 | 2025-06 | Paper |
| SACL: Understanding and Combating Textual Bias in Code Retrieval with Semantic-Augmented Reranking and Localization | SACL | EMNLP 2025 | 2025-07 | Paper |
| Meta-RAG on Large Codebases Using Code Summarization | Meta-RAG | Arxiv | 2025-08 | Paper |
| Enhancing Issue Localization Agent with Tool-Interactive Training | RepoSearcher | ICSE 2026 | 2025-08 | Paper |
| Improving Code Localization with Repository Memory | RepoMem | ICLR 2026 | 2025-10 | Paper |
| Hierarchical Reward Modeling for Fault Localization in Large Code Repositories | HiLoRM | EMNLP Findings 2025 | 2025-11 | Paper Code |
| SweRank+: Multilingual, Multi-Turn Code Ranking for Software Issue Localization | SweRank+ | Arxiv | 2025-12 | Paper Code |
| One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents | RepoNavigator | ICML 2026 | 2025-12 | Paper |
| GraphLocator: Graph-guided Causal Reasoning for Issue Localization | GraphLocator | FSE 2026 | 2025-12 | Paper |
| RGFL: Reasoning Guided Fault Localization for Automated Program Repair Using Large Language Models | RGFL | Arxiv | 2026-01 | Paper |
| Learning Adaptive Parallel Execution for Efficient Code Localization | FuseSearch | ACL 2026 | 2026-01 | Paper |
| Closing the Loop: Universal Repository Representation with RPG-Encoder | RPG-Encoder | Arxiv | 2026-02 | Paper |
| CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents | CodeScout | Arxiv | 2026-03 | Paper |
| Neurosymbolic Repo-level Code Localization | LogicLoc | Arxiv | 2026-04 | Paper |
| BLAgent: Agentic RAG for File-Level Bug Localization | BLAgent | Arxiv | 2026-05 | Paper |
Issue Reproduction
| Literature | Name | Journal/Conference | Time | URL |
|---|---|---|---|---|
| AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions | AEGIS | FSE 2025 Industry | 2024-11 | Paper |
| LLMs as Continuous Learners: Improving the Reproduction of Defective Code in Software Issues | EvoCoder | ARXIV | 2024-11 | Paper |
| Agentic Bug Reproduction for Effective Automated Program Repair at Google | BRT Agent | Arxiv | 2025-02 | Paper |
| Otter: Generating Tests from Issues to Validate SWE Patches | Otter | ICML 2025 | 2025-02 | Paper |
| Issue2Test: Generating Reproducing Test Cases from Issue Reports | Issue2Test | ICSE 2026 | 2025-03 | Paper |
| AssertFlip: Reproducing Bugs via Inversion of LLM-Generated Passing Tests | AssertFlip | ICSE 2026 | 2025-07 | Paper |
| Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection | e-Otter++ | ICSE 2026 | 2025-08 | Paper |
| Automated Generation of Issue-Reproducing Tests by Combining LLMs and Search-Based Testing | BLAST | ASE 2025 | 2025-09 | Paper Code |
| SWE-Tester: Training Open-Source LLMs for Issue Reproduction in Real-World Repositories | SWE-Tester | Arxiv | 2026-01 | Paper |
| Echo: Graph-Enhanced Retrieval and Execution Feedback for Issue Reproduction Test Generation | Echo | Arxiv | 2026-03 | Paper |
| iCoRe: An Iterative Correlation-Aware Retriever for Bug Reproduction Test Generation | iCoRe | FSE 2026 | 2026-04 | Paper |
| Reproduction Test Generation for Java SWE Issues | e-Otter++ for Java | Arxiv | 2026-05 | Paper |
Patch Selection
| Literature | Name | Journal/Conference | Time | URL |
|---|---|---|---|---|
| When Agents go Astray: Course-Correcting SWE Agents with PRMs | SWE-PRM | NeurIPS 2025 | 2025-09 | Paper |
| SWE-RM: Execution-free Feedback For Software Engineering Agents | SWE-RM | ICLR 2026 | 2025-12 | Paper |
| Agentic Rubrics as Contextual Verifiers for SWE Agents | Agentic Rubrics | Arxiv | 2026-01 | Paper |
| SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agents | SWE-Replay | Arxiv | 2026-02 | Paper |
| A Rubric-Supervised Critic from Sparse Real-World Outcomes | Critic Rubrics | Arxiv | 2026-03 | Paper |
| Beyond Verifiable Rewards: Rubric-Based GRM for Reinforced Fine-Tuning SWE Agents | GRM | Arxiv | 2026-04 | Paper |
| Scaling Test-Time Compute for Agentic Coding | RTV | Arxiv | 2026-04 | Paper |
Learning Strategies
From the perspective of Learning Strategies, we discuss them in 2 aspects:
@Data Preparation
@Training
Data Preparation
For Data Preparation, we discuss them in 3 categories separately:
@Real-world Data
@Synthetic Data
@Automated Environment Infrastructure (@Infrastructure)
| Literature | Name | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|---|
| R2E: Turning any GitHub Repository into a Programming Agent Environment | R2E | ICML 2024 | 2024-07 | @Synthetic | Paper Code |
| Training Software Engineering Agents and Verifiers with SWE-Gym | SWE-Gym | ICML 2025 | 2024-12 | @Real-world | Paper Code |
| R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents | R2E-Gym | NeurIPS 2025 | 2025-04 | @Synthetic | Paper Code |
| SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugs | SWE-Synth | ARXIV | 2025-04 | @Synthetic | Paper Code |
| SWE-smith: Scaling Data for Software Engineering Agents | SWE-smith | NeurIPS 2025 | 2025-04 | @Synthetic | Paper Code |
| SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks | SWE-Factory | FSE 2026 | 2025-06 | @Infrastructure | Paper Code |
| SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling | SWE-Dev | ACL Findings 2025 | 2025-06 | @Synthetic | Paper Code |
| SWE-Dev: Evaluating and Training Autonomous Feature-Driven Software Development | SWE-Dev | NeurIPS 2025 | 2025-06 | @Real-world | Paper Code |
| Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs | Skywork-SWE | ARXIV | 2025-06 | @Real-world | Paper |
| SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner | SWE-Flow | ICML 2025 | 2025-06 | @Synthetic | Paper |
| SWE-Mirror: Scaling Issue-Resolving Datasets by Mirroring Issues Across Repositories | SWE-Mirror | ARXIV | 2025-09 | @Synthetic | Paper |
| EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configuration | EvoConfig | ARXIV | 2026-01 | @Infrastructure | Paper |
| MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering | MEnvAgent | ICML 2026 | 2026-01 | @Infrastructure | Paper |
| SWE-Universe: Scale Real-World Verifiable Environments to Millions | SWE-Universe | ARXIV | 2026-02 | @Real-world | Paper |
| Immersion in the GitHub Universe: Scaling Coding Agents to Mastery | Scale-SWE | ARXIV | 2026-02 | @Real-world | Paper |
| SWE-MiniSandbox: Container-Free Reinforcement Learning for Building Software Engineering Agents | SWE-MiniSandbox | ICML 2026 | 2026-02 | @Infrastructure | Paper |
| Hybrid-Gym: Training Coding Agents to Generalize Across Tasks | Hybrid-Gym | ARXIV | 2026-02 | @Synthetic | Paper |
| SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale | SWE-rebench V2 | ICML 2026 | 2026-02 | @Real-world | Paper |
| SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks | SWE-Hub | ARXIV | 2026-02 | @Infrastructure | Paper |
| daVinci-Env: Open SWE Environment Synthesis at Scale | daVinci-Env | ARXIV | 2026-03 | @Infrastructure | Paper |
| SWE-Next: Scalable Real-World Software Engineering Tasks for Agents | SWE-Next | ARXIV | 2026-03 | @Real-world | Paper |
| Yet Even Less Is Even Better For Agentic, Reasoning, and Coding LLMs | STITCH | ARXIV | 2026-04 | @Real-world | Paper |
Training
For Training-Based Method, we can further classify them into 2 categories:
@SFT-Based Method
@RL-Based Method
We only display @RL if the method use both SFT and RL techniques.
| Literature | Name | Evaluation Method | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|---|---|
| Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement | Lingma SWE-GPT | SWESynInfer | ARXIV | 2024-11 | @SFT | Paper |
| Repository Structure-Aware Training Makes SLMs Better Issue Resolver | ReSAT | Agentless | ARXIV | 2024-12 | @SFT | Paper |
| SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution | SWE-Fixer | SWE-Fixer | ACL Findings 2025 | 2025-02 | @SFT | Paper |
| SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution | SWE-RL | Agentless Mini | NeurIPS 2025 | 2025-02 | @RL | Paper Code |
| SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning | SoRFT | Agentless | ACL 2025 | 2025-02 | @RL | Paper |
| SEAlign: Alignment Training for Software Engineering Agent | SEAlign | OpenHands | ICSE 2026 | 2025-03 | @RL | Paper |
| Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute | SWE-Reasoner | SWE-SynInfer+ | ASE 2025 | 2025-04 | @RL | Paper Code |
| Co-PatcheR: Collaborative Software Patching with Component(s)-specific Small Reasoning Models | Co-PatcheR | PatchPilot | NeurIPS 2025 | 2025-05 | @SFT | Paper Code |
| Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering | EvoScale | Satori-SWE | ARXIV | 2025-05 | @RL | Paper Code |
| Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards | Agent-RLVR | Agentless | ARXIV | 2025-06 | @RL | Paper |
| MCTS-Refined CoT: High-Quality Fine-Tuning Data for LLM-Based Repository Issue Resolution | MCTS-Refined | Agentless-1.0 | ASE 2025 | 2025-06 | @SFT | Paper |
| SWE-GPT: A Process-Centric Language Model for Automated Software Improvement | SWE-GPT | SWESynInfer | ISSTA 2025 | 2025-06 | @SFT | - |
| - | DeepSWE | - | Blog | 2025-07 | @RL | Link |
| - | SWE-Swiss | - | Blog | 2025-08 | @RL | Link Code |
| RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale | RepoForge | OpenHands | ARXIV | 2025-08 | @RL | Paper |
| Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning | - | - | ARXIV | 2025-08 | @RL | Paper |
| Devstral: Fine-tuning Language Models for Coding Agent Applications | Devstral-Small | OpenHands | ARXIV | 2025-08 | @RL | Paper |
| Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents | Kimi-Dev | Kimi-Dev | ICLR 2026 | 2025-09 | @RL | Paper |
| CWM: An Open-Weights LLM for Research on Code Generation with World Models | CWM | CWM | ARXIV | 2025-09 | @RL | Paper |
| Building Coding Agents via Entropy-Enhanced Multi-Turn Preference Optimization | EntroPO | R2E | ARXIV | 2025-09 | @RL | Paper Code |
| BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills | BugPilot | R2E | ARXIV | 2025-10 | @SFT | Paper |
| Training Versatile Coding Agents in Synthetic Environments | SWE-Playground | OpenHands | ARXIV | 2025-12 | @SFT | Paper Code |
| Toward Training Superintelligent Software Agents through Self-Play SWE-RL | Self-Play SWE-RL | bash+editor | ICML 2026 | 2025-12 | @RL | Paper |
| Context as a Tool: Context Management for Long-Horizon SWE-Agents | CAT/SWE-Compressor | OpenHands | Arxiv | 2025-12 | @SFT | Paper |
| SWE-RM: Execution-free Feedback For Software Engineering Agents | SWE-RM | OpenHands | ICLR 2026 | 2025-12 | @RL | Paper |
| SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving | SWE-Lego | - | ARXIV | 2026-01 | @SFT | Paper |
| daVinci-Dev: Agent-native Mid-training for Software Engineering | daVinci-Dev | - | ARXIV | 2026-01 | @SFT | Paper |
| SERA: Soft-Verified Efficient Repository Agents | SERA | - | ICML 2026 | 2026-01 | @SFT | Paper |
| SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning | SWE-Spot | - | ARXIV | 2026-01 | @SFT | Paper |
| SWE-World: Building Software Engineering Agents in Docker-Free Environments | SWE-World | - | ARXIV | 2026-02 | @RL | Paper |
| SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training | SWE-Master | - | ARXIV | 2026-02 | @RL | Paper |
| SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents | SWE-Protégé | - | ARXIV | 2026-02 | @RL | Paper |
| SWE-Fuse: Empowering Software Agents via Issue-free Trajectory Learning and Entropy-aware RLVR Training | SWE-Fuse | - | ARXIV | 2026-03 | @RL | Paper |
| From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agents | SWE-HERO | - | ARXIV | 2026-04 | @SFT | Paper |
| SWE-AGILE: A Software Agent Framework for Efficiently Managing Dynamic Reasoning Context | SWE-AGILE | - | ARXIV | 2026-04 | @RL | Paper |
| SWE-TRACE: Optimizing Long-Horizon SWE Agents Through Rubric Process Reward Models and Heuristic Test-Time Scaling | SWE-TRACE | - | ARXIV | 2026-04 | @RL | Paper |
| BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models | BoostAPR | - | ICML 2026 | 2026-05 | @RL | Paper |
| Hindsight Hint Distillation: Scaffolded Reasoning for SWE Agents from CoT-free Answers | HHD | - | ARXIV | 2026-05 | @SFT | Paper |
Empirical Studies
We categorize the empirical studies into 2 perspectives:
@Evaluation-oriented Studies
@Technique-oriented Studies
Evaluation-oriented Studies
For Evaluation-oriented Studies, we discuss them in 4 categories separately:
@Validity
@Quality
@Efficiency
@Robustness
where, @Validity indicates evaluation validity, @Quality indicates quality and requirement compliance,
@Efficiency indicates efficiency and resource consumption, and @Robustness indicates robustness and
process-aware evaluation.
| Literature | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|
| Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios | SANER | 2024-10 | @Quality | Paper |
| Large Language Model Critics for Execution-Free Evaluation of Code Changes | ARXIV | 2025-01 | @Validity | Paper |
| Ambig-SWE: Interactive Agents to Overcome Underspecificity in Software Engineering | ICLR 2026 | 2025-02 | @Robustness | Paper |
| Are "Solved Issues" in SWE-bench Really Solved Correctly? An Empirical Study | ICSE 2026 | 2025-03 | @Validity | Paper |
| SWE-Bench-CL: Continual Learning for Coding Agents | ARXIV | 2025-06 | @Robustness | Paper |
| The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason | ARXIV | 2025-06 | @Validity | Paper |
| Are AI-Generated Fixes Secure? Analyzing LLM and Agent Patches on SWE-bench | ARXIV | 2025-06 | @Quality | Paper |
| SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints | ARXIV | 2025-09 | @Efficiency | Paper |
| Saving SWE-Bench: A Benchmark Mutation Approach for Realistic Agent Evaluation | ARXIV | 2025-10 | @Validity | Paper |
| More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents | ICSE 2026 | 2025-10 | @Efficiency | Paper |
| Process-Centric Analysis of Agentic Software Systems | ARXIV | 2025-12 | @Robustness | Paper |
| SWEnergy: An Empirical Study on Energy Efficiency in Agentic Issue Resolution Frameworks with SLMs | ICSE 2026 Workshop | 2025-12 | @Efficiency | Paper |
| Does SWE-Bench-Verified Test Agent Ability or Model Memory? | ARXIV | 2025-12 | @Validity | Paper |
| What's in a Benchmark? The Case of SWE-Bench in Automated Program Repair | ICSE 2026 | 2026-02 | @Validity | Paper |
| TRAJEVAL: Decomposing Code Agent Trajectories for Fine-Grained Diagnosis | ARXIV | 2026-03 | @Robustness | Paper |
| Does Pass Rate Tell the Whole Story? Evaluating Design Constraint Compliance in LLM-based Issue Resolution | ARXIV | 2026-04 | @Quality | Paper |
| Coding Agents Don't Know When to Act | ARXIV | 2026-05 | @Robustness | Paper |
| AgentLens: Revealing The Lucky Pass Problem in SWE-Agent Evaluation | ARXIV | 2026-05 | @Robustness | Paper |
| RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations | ARXIV | 2026-05 | @Robustness | Paper |
Technique-oriented Studies
For Technique-oriented Studies, we discuss them in 3 categories separately:
@Performance
@Behavior
@Failure
where, @Performance indicates performance variation and influencing factors, @Behavior indicates agent
behavior and resolution process, and @Failure indicates failure modes and root causes.
| Literature | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|
| Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents | ICLR 2025 | 2024-08 | @Performance | Paper Code |
| An Empirical Study on LLM-based Agents for Automated Bug Fixing | ARXIV | 2024-11 | @Performance | Paper |
| Beyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarios | ICSE 2026 | 2025-03 | @Failure | Paper |
| PAGENT: Learning to Patch Software Engineering Agents | ICSE 2026 | 2025-06 | @Failure | Paper |
| Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories | ASE 2025 | 2025-06 | @Behavior | Paper |
| An Empirical Study on Failures in Automated Issue Solving | ARXIV | 2025-09 | @Failure | Paper |
| Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories | ICSE 2026 | 2025-10 | @Behavior | Paper |
| SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents | ICLR 2026 | 2025-11 | @Failure | Paper |
| Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair | FSE 2026 | 2026-01 | @Performance | Paper |
| Beyond Localization: Recoverable Headroom and Residual Frontier in Repository-Level RAG-APR | ARXIV | 2026-03 | @Performance | Paper |
| ORACLE-SWE: Quantifying the Contribution of Oracle Information Signals on SWE Agents | ICML 2026 | 2026-04 | @Performance | Paper |
| Do Agent Rules Shape or Distort? Guardrails Beat Guidance in Coding Agents | ARXIV | 2026-04 | @Performance | Paper |
| From Plan to Action: How Well Do Agents Follow the Plan? | ARXIV | 2026-04 | @Behavior | Paper |
| LLM-based Agents for Automated Bug Fixing: How Far Are We? | ICSE 2026 | 2026-04 | @Performance | - |
Contributors
Similar repos
Agentic Software Issue Resolution with Large Language Models: A Survey
See the codeREADME
Agentic Software Issue Resolution with Large Language Models: A Survey
📰 News
- 📅 2026.05: Paper update! We add 89 papers up to 2026.05 and refresh the venue information according to the latest acceptance results. We also align the taxonomy with our survey. Now, we have included 242 papers in this survey.
- 📅 2026.01: Paper update! We add 34 papers from 2025.10 to 2025.12. Now, we have included 160 papers in this survey.
- 📅 2025.12: We release the first survey on agentic software issue resolution!
- 📅 2025.10: We summarize 126 papers about issue resolution, from 2023.10 to 2025.10!
Introduction
We classified this survey into three main parts: Benchmarks, Technologies and Empirical Studies.
Automated issue solving technologies can be mainly surveyed from 2 perspectives:
Scaffold/Method Design and Learning Strategies.
Table of Contents
- Benchmarks
- End-To-End
- Reproduction Test Generation
- Localization
- Technologies
- Scaffold/Method Design
- End-to-End Scaffold
- Agent-Based
- Pipeline-Based
- Scaffold Improvement
- Memory
- Context Management
- Self-Evolution
- Single-Phased Methods
- End-to-End Scaffold
- Learning Strategies
- Data Preparation
- Real-world Data
- Synthetic Data
- Automated Environment Infrastructure
- Training
- Supervised Fine-tuning (SFT)
- Reinforcement Learning (RL)
- Data Preparation
- Scaffold/Method Design
- Empirical Studies
- Evaluation-oriented Studies
- Evaluation Validity
- Quality and Requirement Compliance
- Efficiency and Resource Consumption
- Robustness and Process-aware Evaluation
- Technique-oriented Studies
- Performance Variation and Influencing Factors
- Agent Behavior and Resolution Process
- Failure Modes and Root Causes
- Evaluation-oriented Studies
Benchmarks
For Benchmarks, we summarized the existing benchmarks into 2 categories for their different tasks.
@End-To-End
@Reproduction Test Generation
@Localization
| Literature | Name | Scope | Journal/Conference | Time | Link |
|---|---|---|---|---|---|
| SWE-bench: Can Language Models Resolve Real-World GitHub Issues? | SWE-bench | End-To-End | ICLR'24 | 2023-10 | Paper Code |
| SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents | SWT-Bench | Reproduction Test Generation | NeurIPS'24 | 2024-06 | Paper Code |
| SWE-bench-java: A GitHub Issue Resolving Benchmark for Java | Muti-SWE-bench | End-To-End | ARXIV | 2024-08 | Paper Code |
| SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains? | SWE-bench Mutimodal | End-To-End | ICLR'25 | 2024-10 | Paper Code |
| SWE-Bench+: Enhanced Coding Benchmark for LLMs | SWE-Bench+ | End-To-End | ARXIV | 2024-10 | Paper |
| TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark | TestGenEval | Reproduction Test Generation | ICLR'25 | 2024-10 | Paper Code |
| A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models | FAUN-Eval | End-To-End | ARXIV | 2024-11 | Paper |
| TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved? | TDD-Bench | Reproduction Test Generation | ARXIV | 2024-11 | Paper Code |
| CodeV: Issue Resolving with Visual Data | Visual SWE-bench | End-To-End | ACL Findings'25 | 2024-12 | Paper Code |
| Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving | Muti-SWE-bench | End-To-End | NeurIPS'25 | 2025-04 | Paper Code |
| LiveSWEBench | LiveSWEBench | End-To-End | BLOG | 2025-04 | link Code |
| LocAgent: Graph-Guided LLM Agents for Code Localization | LocBench | Localization | ACL'25 | 2025-03 | Paper Code |
| Automated Benchmark Generation for Repository-Level Coding Tasks | SWEE-Bench/SWA-Bench | End-To-End | ICML'25 | 2025-03 | Paper |
| FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation | FEA-Bench | End-To-End | ACL'25 | 2025-03 | Paper Code |
| OmniGIRL: A Multilingual and Multimodal Benchmark for GitHub Issue Resolution | OmniGIRL | End-To-End | ISSTA'25 | 2025-05 | Paper Code |
| - | SWE-bench Multilingual | End-To-End | BLOG | 2025-05 | link Code |
| SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents | SWE-PolyBench | End-To-End | ARXIV | 2025-04 | Paper Code |
| SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents | SWE-rebench | End-To-End | NeurIPS'25 | 2025-05 | Paper Code |
| GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents | GSO | End-To-End | NeurIPS'25 | 2025-05 | Paper Code |
| SWE-bench Goes Live! | SWE-bench-Live | End-To-End | NeurIPS'25 | 2025-05 | Paper Code |
| UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench | UTBoost | - | ACL'25 | 2025-06 | Paper Code |
| SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks | SWE-Factory | End-To-End | FSE'26 | 2025-06 | Paper Code |
| SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving | Swing-Arena | End-To-End | ICLR'26 | 2025-06 | Paper Code |
| SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation | SPICE | - | ASE'25 | 2025-07 | Paper |
| SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks | SWE-MERA | End-To-End | EMNLP'25 | 2025-07 | Paper |
| SWE-Perf: Can Language Models Optimize Code Performance on Real-World Repositories? | SWE-Perf | End-To-End | ICML'26 | 2025-07 | Paper Code |
| NoCode-bench: A Benchmark for Evaluating Natural Language-Driven Feature Addition | NoCode-bench | End-To-End | ARXIV | 2025-08 | Paper Code |
| SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? | SWE-Bench Pro | End-To-End | ICML'26 | 2025-09 | Paper Code |
| A Benchmark for Localizing Code and Non-Code Issues in Software Projects | MULocBench | Localization | ARXIV | 2025-10 | Paper Code |
| SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads? | SWE-fficiency | End-To-End | ICML'26 | 2025-11 | Paper Code |
| SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories | SWE-Bench++ | End-To-End | ARXIV | 2025-12 | Paper Code |
| SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios | SWE-EVO | End-To-End | ARXIV | 2025-12 | Paper Code |
| SWE-Refactor: A Repository-Level Benchmark for Real-World LLM-Based Code Refactoring | SWE-Refactor | End-To-End | ARXIV | 2026-02 | Paper |
| ContextBench: A Benchmark for Context Retrieval in Coding Agents | ContextBench | Localization | ARXIV | 2026-02 | Paper |
| SWE Context Bench: A Benchmark for Context Learning in Coding | SWE Context Bench | Localization | ARXIV | 2026-02 | Paper |
| SWE-Bench Mobile: Can Large Language Model Agents Develop Industry-Level Mobile Applications? | SWE-Bench Mobile | End-To-End | ARXIV | 2026-02 | Paper |
| FeatureBench: Benchmarking Agentic Coding for Complex Feature Development | FeatureBench | End-To-End | ICLR'26 | 2026-02 | Paper |
| Evaluating and Improving Automated Repository-Level Rust Issue Resolution with LLM-based Agents | Rust-SWE-bench | End-To-End | ICSE'26 | 2026-02 | Paper |
| SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark | SWE-ABS | - | ICML'26 | 2026-02 | Paper |
| SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integration | SWE-CI | End-To-End | ARXIV | 2026-03 | Paper |
| Beyond Isolated Tasks: A Framework for Evaluating Coding Agents on Sequential Software Evolution | SWE-STEPS | End-To-End | ARXIV | 2026-04 | Paper |
| SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades | SWE-Chain | End-To-End | ARXIV | 2026-05 | Paper |
| SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering? | SWE-Mutation | Reproduction Test Generation | ACL'26 | 2026-05 | Paper |
Technologies
Scaffold/Method Design
From the perspective of Design Paradigms, we can classify them into 3 categories:
@End-To-End Scaffold
@Scaffold Improvement
@Single-Phased Methods
End-to-End Scaffold
For End-To-End Method, we can further classify them into 2 categories:
@Agent-Based Method
@Pipeline-Based Method
| Literature | Name | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|---|
| SWE-bench: Can Language Models Resolve Real-World GitHub Issues? | BM25 RAG | ICLR 2024 | 2023-10 | @Pipeline | Paper Code |
| SWE-agent: Agent-computer interfaces enable automated software engineering | SWE-Agent | NeurIPS 2024 | 2024-05 | @Agent | Paper Code |
| Autocoderover: Autonomous program improvement | AutoCodeRover | ISSTA 2024 | 2024-04 | @Agent | Paper Code |
| CodeR: Issue Resolving with Multi-Agent and Task Graphs | CodeR | Arxiv | 2024-06 | @Agent | Paper Code |
| Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration | LingmaAgent/RepoUnderstander | FSE Companion 2025 | 2024-06 | @Agent | Paper Code |
| MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution | MAGIS | NeurIPS 2024 | 2024-03 | @Agent | Paper Code |
| MASAI: Modular Architecture for Software-engineering AI Agents | MASAI | NeurIPS 2024 | 2024-06 | @Agent | Paper |
| Demystifying LLM-based Software Engineering Agents | Agentless | FSE 2025 | 2024-07 | @Pipeline | Paper Code |
| Specrover: Code intent extraction via llms | SpecRover (AutoCodeRover-v2) | ICSE 2025 | 2024-08 | @Agent | Paper Code |
| CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases | CodexGraph | NAACL 2025 | 2024-08 | @Agent | Paper Code |
| SuperCoder2.0: Technical Report on Exploring the feasibility of LLMs as Autonomous Programmer | SuperCoder | Arxiv | 2024-09 | @Agent | Paper |
| Hyperagent: Generalist software engineering agents to solve coding tasks at scale | HyperAgent | Arxiv | 2024-09 | @Agent | Paper |
| RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph | RepoGraph | ICLR 2025 | 2024-10 | @Pipeline | Paper Code |
| SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement | SWE-Search | ICLR 2025 | 2024-10 | @Agent | Paper Code |
| OpenHands: An Open Platform for AI Software Developers as Generalist Agents | OpenHands CodeAct | ICLR 2025 | 2024-10 | @Agent | Paper Code |
| - | Composio SWE-Kit | Blog | 2024-10 | @Pipeline | Link Code |
| Infant Agent: A Tool-Integrated, Logic-Driven Agent with Cost-Effective API Usage | Infant Agent | Arxiv | 2024-11 | @Agent | Paper |
| MarsCode Agent: AI-native Automated Bug Fixing | MarsCode Agent | Arxiv | 2024-11 | @Agent | Paper |
| Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement | SWESynInfer | Arxiv | 2024-11 | @Pipeline | Paper Code |
| - | Nebius AI | Blog | 2024-11 | @Agent | Paper |
| CodeV: Issue Resolving with Visual Data | CodeV | ACL Findings 2025 | 2024-12 | @Pipeline | Paper Code |
| - | Aide | Blog | 2024-12 | @Agent | Link |
| Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments | Learn-By-Interact | ICLR 2025 | 2025-01 | @Agent | Paper |
| PatchPilot: A Stable and Cost-Efficient Agentic Patching Framework | PatchPilot | ICML 2025 | 2025-02 | @Pipeline | Paper Code |
| CodeMonkeys: Scaling Test-Time Compute for Software Engineering | CodeMonkeys | Arxiv | 2025-02 | @Pipeline | Paper Code |
| SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution | Agentless Mini | NeurIPS 2025 | 2025-02 | @Pipeline | Paper Code |
| - | Agentless Lite | Blog | 2025-02 | @Pipeline | Code |
| - | Syntheo | Blog | 2025-02 | @Agent | Link |
| SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution | SWE-Fixer | ACL Findings 2025 | 2025-02 | @Pipeline | Paper |
| - | AgentScope | Blog | 2025-03 | @Agent | Link |
| DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal | DARS | ACL 2025 | 2025-03 | @Agent | Paper Code |
| Enhancing Repository-Level Software Repair via Repository-Aware Knowledge Graphs | KGCompass | Arxiv | 2025-03 | @Pipeline | Paper |
| - | Augment Agent v0 | Blog | 2025-03 | @Agent | Link Code |
| - | CORTEXA | Blog | 2025-03 | @Pipeline | Link |
| - | Refact.ai | Blog | 2025-03 | @Agent | Link Code |
| - | Lingxi | Blog | 2025-04 | @Agent | Link Code |
| - | Trae IDE | Blog | 2025-05 | @Agent | Link |
| - | devlo | Blog | 2025-05 | @Agent | Link |
| Putting It All into Context: Simplifying Agents with LCLMs | LCLM | Arxiv | 2025-05 | @Pipeline | Paper |
| Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks | CGM-SWE-PY | NeurIPS'25 | 2025-05 | @Pipeline | Paper |
| InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer Interaction | InfantAgent-Next | NeurIPS 2025 | 2025-05 | @Agent | Paper Code |
| Coding Agents with Multimodal Browsing are Generalist Problem Solvers | OpenHands-Versa | ICML 2026 | 2025-06 | @Agent | Paper Code |
| EXPEREPAIR: Dual-Memory Enhanced LLM-based Repository-Level Program Repair | EXPEREPAIR | FSE 2026 | 2025-06 | @Agent | Paper |
| Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing | GUIRepair | ASE'25 | 2025-06 | @Pipeline | Paper |
| SemAgent: A Semantics Aware Program Repair Agent | SemAgent | Arxiv | 2025-06 | @Pipeline | Paper |
| Nemotron-Cortexa: Enhancing LLM Agents for Software Engineering Tasks via Improved Localization and Solution Diversity | Nemotron-Cortexa | ICML'25 | 2025-06 | @Pipeline | Paper Code |
| Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving | Agent KB | Arxiv | 2025-07 | @Agent | Paper Code |
| Prometheus: Unified Knowledge Graphs for Issue Resolution in Multilingual Codebases | Prometheus | Arxiv | 2025-07 | @Agent | Paper Code |
| SWE-Exp: Experience-Driven Software Issue Resolution | SWE-Exp | Arxiv | 2025-07 | @Agent | Paper Code |
| SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution | SWE-Debate | ICSE 2026 | 2025-07 | @Agent | Paper Code |
| Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling | Trae Agent | Arxiv | 2025-07 | @Agent | Paper Code |
| SynFix: Dependency-Aware Program Repair via RelationGraph Analysis | SynFix | ACL Findings'25 | 2025-07 | @Pipeline | Paper |
| SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents | SE-Agent | NeurIPS'25 | 2025-08 | @Agent | Paper Code |
| Lita: Light Agent Uncovers the Agentic Coding Capabilities of LLMs | Lita | Arxiv | 2025-10 | @Agent | Paper |
| Lingxi: Repository-Level Issue Resolution Framework Enhanced by Procedural Knowledge Guided Scaling | Lingxi | Arxiv | 2025-10 | @Agent | Paper Code |
| SIADAFIX: issue description response for adaptive program repair | SIADAFIX | Arxiv | 2025-10 | @Pipeline | Paper Code |
| TDFlow: Agentic Workflows for Test Driven Software Engineering | TDFlow | Arxiv | 2025-10 | @Pipeline | Paper |
| InfCode: Adversarial Iterative Refinement of Tests and Patches for Reliable Software Issue Resolution | InfCode | Arxiv | 2025-11 | @Agent | Paper |
| Think-Search-Patch: A Retrieval-Augmented Reasoning Framework for Repository-Level Code Repair | TSP | EMNLP 2025 | 2025-11 | @Pipeline | Paper Code |
| Confucius Code Agent: An Open-sourced AI Software Engineer at Industrial Scale | CCA | Arxiv | 2025-12 | @Agent | Paper |
| Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering | Agyn | Arxiv | 2026-02 | @Agent | Paper |
| SVRepair: Structured Visual Reasoning for Automated Program Repair | SVRepair | Arxiv | 2026-02 | @Pipeline | Paper |
| Debug2Fix: Supercharging Coding Agents with Interactive Debugging Capabilities | Debug2Fix | Arxiv | 2026-02 | @Agent | Paper |
| SGAgent: Suggestion-Guided LLM-Based Multi-Agent Framework for Repository-Level Software Repair | SGAgent | TOSEM | 2026-02 | @Agent | Paper |
| SWE-Adept: An LLM-Based Agentic Framework for Deep Codebase Analysis and Structured Issue Resolution | SWE-Adept | Arxiv | 2026-03 | @Agent | Paper |
| RepoRepair: Leveraging Code Documentation for Repository-Level Automated Program Repair | RepoRepair | Arxiv | 2026-03 | @Pipeline | Paper |
| Architecture-Aware Multi-Design Generation for Repository-Level Feature Addition | RAIM | Arxiv | 2026-03 | @Pipeline | Paper |
| Resolving Java Code Repository Issues with iSWE Agent | iSWE Agent | Arxiv | 2026-03 | @Agent | Paper |
| Agent-Based Ensemble Reasoning for Repository-Level Issue Resolution | - | ICSE 2026 | 2026-04 | @Agent | - |
| Beyond Fixed Tests: Repository-Level Issue Resolution as Coevolution of Code and Behavioral Constraints | Agent-CoEvo | Arxiv | 2026-04 | @Agent | Paper |
| REAgent: Requirement-Driven LLM Agents for Software Issue Resolution | REAgent | Arxiv | 2026-04 | @Agent | Paper |
| ARISE: A Repository-level Graph Representation and Toolset for Agentic Fault Localization and Program Repair | ARISE | Arxiv | 2026-05 | @Pipeline | Paper |
Scaffold Improvement
For Scaffold Improvement, we discuss them in 3 categories separately:
@Memory
@Context (context management)
@Self-Evolution
| Literature | Name | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|---|
| A Self-Improving Coding Agent | SICA | ICLR 2026 | 2025-04 | @Self-Evolution | Paper |
| Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents | DGM | ICLR 2026 | 2025-05 | @Self-Evolution | Paper |
| EXPEREPAIR: Dual-Memory Enhanced LLM-based Repository-Level Program Repair | EXPEREPAIR | FSE 2026 | 2025-06 | @Memory | Paper |
| Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving | Agent KB | Arxiv | 2025-07 | @Memory | Paper Code |
| SWE-Exp: Experience-Driven Software Issue Resolution | SWE-Exp | Arxiv | 2025-07 | @Memory | Paper Code |
| SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents | SE-Agent | NeurIPS'25 | 2025-08 | @Self-Evolution | Paper Code |
| Reducing Cost of LLM Agents with Trajectory Reduction | AgentDiet | FSE'26 | 2025-09 | @Context | Paper |
| Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine | HGM | ICLR 2026 | 2025-10 | @Self-Evolution | Paper |
| TOM-SWE: User Mental Modeling For Software Engineering Agents | TOM-SWE | ICML 2026 | 2025-10 | @Memory | Paper Code |
| Self-Abstraction from Grounded Experience for Plan-Guided Policy Refinement | SAGE | Arxiv | 2025-11 | @Self-Evolution | Paper |
| Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly? | Live-SWE-agent | Arxiv | 2025-11 | @Self-Evolution | Paper Code |
| Confucius Code Agent: An Open-sourced AI Software Engineer at Industrial Scale | CCA | Arxiv | 2025-12 | @Memory | Paper |
| EET: Experience-Driven Early Termination for Cost-Efficient Software Engineering Agents | EET | ACL 2026 | 2026-01 | @Memory | Paper |
| MemGovern: Enhancing Code Agents through Learning from Governed Human Experiences | MemGovern | Arxiv | 2026-01 | @Memory | Paper |
| Outcome-Conditioned Reasoning Distillation for Resolving Software Issues | ConRAD | ICML 2026 | 2026-01 | @Context | Paper |
| SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents | SWE-Pruner | Arxiv | 2026-01 | @Context | Paper |
| Your Code Agent Can Grow Alongside You with Structured Memory | MemCoder | Arxiv | 2026-02 | @Self-Evolution | Paper |
| FailureMem: A Failure-Aware Multimodal Framework for Autonomous Software Repair | FailureMem | Arxiv | 2026-03 | @Memory | Paper |
| Compressing Code Context for LLM-based Issue Resolution | - | Arxiv | 2026-03 | @Context | Paper |
| CodeScout: Contextual Problem Statement Enhancement for Software Agents | CodeScout | Arxiv | 2026-03 | @Context | Paper |
| CODESTRUCT: Code Agents over Structured Action Spaces | CODESTRUCT | ACL 2026 | 2026-04 | @Context | Paper |
| SWE-Edit: Rethinking Code Editing for Efficient SWE-Agent | SWE-Edit | Arxiv | 2026-04 | @Context | Paper |
Single-Phased Methods
For Single-Phased Method, we discuss them in 3 categories separately:
@Localization
@Reproduction
@Selection
where, @Reproduction indicates the reproduction test generation, @Selection indicates the patch validation
and selection.
Issue Localization
| Literature | Name | Journal/Conference | Time | URL |
|---|---|---|---|---|
| BLAZE: Cross-Language and Cross-Project Bug Localization via Dynamic Chunking and Hard Example Learning | BLAZE | TSE | 2024-08 | Paper Code |
| Integrating Various Software Artifacts for Better LLM-based Bug Localization and Program Repair | DEVLoRe | TOSEM | 2024-12 | Paper |
| OrcaLoca: An LLM Agent Framework for Software Issue Localization | OrcaLoca | ICML 2025 | 2025-02 | Paper Code |
| Bridging Bug Localization and Issue Fixing: A Hierarchical Localization Framework Leveraging Large Language Models | BugCerberus | TSE | 2025-02 | Paper |
| LocAgent: Graph-Guided LLM Agents for Code Localization | LocAgent | ACL 2025 | 2025-03 | Paper Code |
| Issue Localization via LLM-Driven Iterative Code Graph Searching | CoSIL | ASE 2025 | 2025-03 | Paper Code |
| CORNSTACK: HIGH-QUALITY CONTRASTIVE DATA FOR BETTER CODE RETRIEVAL AND RERANKING | CoRNStack | ICLR 2025 | 2025-03 | Paper Code |
| SweRank: Software Issue Localization with Code Ranking | SweRank | ICLR 2026 | 2025-05 | Paper Code |
| CoRet: Improved Retriever for Code Editing | CoRet | ACL 2025 | 2025-06 | Paper |
| SACL: Understanding and Combating Textual Bias in Code Retrieval with Semantic-Augmented Reranking and Localization | SACL | EMNLP 2025 | 2025-07 | Paper |
| Meta-RAG on Large Codebases Using Code Summarization | Meta-RAG | Arxiv | 2025-08 | Paper |
| Enhancing Issue Localization Agent with Tool-Interactive Training | RepoSearcher | ICSE 2026 | 2025-08 | Paper |
| Improving Code Localization with Repository Memory | RepoMem | ICLR 2026 | 2025-10 | Paper |
| Hierarchical Reward Modeling for Fault Localization in Large Code Repositories | HiLoRM | EMNLP Findings 2025 | 2025-11 | Paper Code |
| SweRank+: Multilingual, Multi-Turn Code Ranking for Software Issue Localization | SweRank+ | Arxiv | 2025-12 | Paper Code |
| One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents | RepoNavigator | ICML 2026 | 2025-12 | Paper |
| GraphLocator: Graph-guided Causal Reasoning for Issue Localization | GraphLocator | FSE 2026 | 2025-12 | Paper |
| RGFL: Reasoning Guided Fault Localization for Automated Program Repair Using Large Language Models | RGFL | Arxiv | 2026-01 | Paper |
| Learning Adaptive Parallel Execution for Efficient Code Localization | FuseSearch | ACL 2026 | 2026-01 | Paper |
| Closing the Loop: Universal Repository Representation with RPG-Encoder | RPG-Encoder | Arxiv | 2026-02 | Paper |
| CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents | CodeScout | Arxiv | 2026-03 | Paper |
| Neurosymbolic Repo-level Code Localization | LogicLoc | Arxiv | 2026-04 | Paper |
| BLAgent: Agentic RAG for File-Level Bug Localization | BLAgent | Arxiv | 2026-05 | Paper |
Issue Reproduction
| Literature | Name | Journal/Conference | Time | URL |
|---|---|---|---|---|
| AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions | AEGIS | FSE 2025 Industry | 2024-11 | Paper |
| LLMs as Continuous Learners: Improving the Reproduction of Defective Code in Software Issues | EvoCoder | ARXIV | 2024-11 | Paper |
| Agentic Bug Reproduction for Effective Automated Program Repair at Google | BRT Agent | Arxiv | 2025-02 | Paper |
| Otter: Generating Tests from Issues to Validate SWE Patches | Otter | ICML 2025 | 2025-02 | Paper |
| Issue2Test: Generating Reproducing Test Cases from Issue Reports | Issue2Test | ICSE 2026 | 2025-03 | Paper |
| AssertFlip: Reproducing Bugs via Inversion of LLM-Generated Passing Tests | AssertFlip | ICSE 2026 | 2025-07 | Paper |
| Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection | e-Otter++ | ICSE 2026 | 2025-08 | Paper |
| Automated Generation of Issue-Reproducing Tests by Combining LLMs and Search-Based Testing | BLAST | ASE 2025 | 2025-09 | Paper Code |
| SWE-Tester: Training Open-Source LLMs for Issue Reproduction in Real-World Repositories | SWE-Tester | Arxiv | 2026-01 | Paper |
| Echo: Graph-Enhanced Retrieval and Execution Feedback for Issue Reproduction Test Generation | Echo | Arxiv | 2026-03 | Paper |
| iCoRe: An Iterative Correlation-Aware Retriever for Bug Reproduction Test Generation | iCoRe | FSE 2026 | 2026-04 | Paper |
| Reproduction Test Generation for Java SWE Issues | e-Otter++ for Java | Arxiv | 2026-05 | Paper |
Patch Selection
| Literature | Name | Journal/Conference | Time | URL |
|---|---|---|---|---|
| When Agents go Astray: Course-Correcting SWE Agents with PRMs | SWE-PRM | NeurIPS 2025 | 2025-09 | Paper |
| SWE-RM: Execution-free Feedback For Software Engineering Agents | SWE-RM | ICLR 2026 | 2025-12 | Paper |
| Agentic Rubrics as Contextual Verifiers for SWE Agents | Agentic Rubrics | Arxiv | 2026-01 | Paper |
| SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agents | SWE-Replay | Arxiv | 2026-02 | Paper |
| A Rubric-Supervised Critic from Sparse Real-World Outcomes | Critic Rubrics | Arxiv | 2026-03 | Paper |
| Beyond Verifiable Rewards: Rubric-Based GRM for Reinforced Fine-Tuning SWE Agents | GRM | Arxiv | 2026-04 | Paper |
| Scaling Test-Time Compute for Agentic Coding | RTV | Arxiv | 2026-04 | Paper |
Learning Strategies
From the perspective of Learning Strategies, we discuss them in 2 aspects:
@Data Preparation
@Training
Data Preparation
For Data Preparation, we discuss them in 3 categories separately:
@Real-world Data
@Synthetic Data
@Automated Environment Infrastructure (@Infrastructure)
| Literature | Name | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|---|
| R2E: Turning any GitHub Repository into a Programming Agent Environment | R2E | ICML 2024 | 2024-07 | @Synthetic | Paper Code |
| Training Software Engineering Agents and Verifiers with SWE-Gym | SWE-Gym | ICML 2025 | 2024-12 | @Real-world | Paper Code |
| R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents | R2E-Gym | NeurIPS 2025 | 2025-04 | @Synthetic | Paper Code |
| SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugs | SWE-Synth | ARXIV | 2025-04 | @Synthetic | Paper Code |
| SWE-smith: Scaling Data for Software Engineering Agents | SWE-smith | NeurIPS 2025 | 2025-04 | @Synthetic | Paper Code |
| SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks | SWE-Factory | FSE 2026 | 2025-06 | @Infrastructure | Paper Code |
| SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling | SWE-Dev | ACL Findings 2025 | 2025-06 | @Synthetic | Paper Code |
| SWE-Dev: Evaluating and Training Autonomous Feature-Driven Software Development | SWE-Dev | NeurIPS 2025 | 2025-06 | @Real-world | Paper Code |
| Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs | Skywork-SWE | ARXIV | 2025-06 | @Real-world | Paper |
| SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner | SWE-Flow | ICML 2025 | 2025-06 | @Synthetic | Paper |
| SWE-Mirror: Scaling Issue-Resolving Datasets by Mirroring Issues Across Repositories | SWE-Mirror | ARXIV | 2025-09 | @Synthetic | Paper |
| EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configuration | EvoConfig | ARXIV | 2026-01 | @Infrastructure | Paper |
| MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering | MEnvAgent | ICML 2026 | 2026-01 | @Infrastructure | Paper |
| SWE-Universe: Scale Real-World Verifiable Environments to Millions | SWE-Universe | ARXIV | 2026-02 | @Real-world | Paper |
| Immersion in the GitHub Universe: Scaling Coding Agents to Mastery | Scale-SWE | ARXIV | 2026-02 | @Real-world | Paper |
| SWE-MiniSandbox: Container-Free Reinforcement Learning for Building Software Engineering Agents | SWE-MiniSandbox | ICML 2026 | 2026-02 | @Infrastructure | Paper |
| Hybrid-Gym: Training Coding Agents to Generalize Across Tasks | Hybrid-Gym | ARXIV | 2026-02 | @Synthetic | Paper |
| SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale | SWE-rebench V2 | ICML 2026 | 2026-02 | @Real-world | Paper |
| SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks | SWE-Hub | ARXIV | 2026-02 | @Infrastructure | Paper |
| daVinci-Env: Open SWE Environment Synthesis at Scale | daVinci-Env | ARXIV | 2026-03 | @Infrastructure | Paper |
| SWE-Next: Scalable Real-World Software Engineering Tasks for Agents | SWE-Next | ARXIV | 2026-03 | @Real-world | Paper |
| Yet Even Less Is Even Better For Agentic, Reasoning, and Coding LLMs | STITCH | ARXIV | 2026-04 | @Real-world | Paper |
Training
For Training-Based Method, we can further classify them into 2 categories:
@SFT-Based Method
@RL-Based Method
We only display @RL if the method use both SFT and RL techniques.
| Literature | Name | Evaluation Method | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|---|---|
| Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement | Lingma SWE-GPT | SWESynInfer | ARXIV | 2024-11 | @SFT | Paper |
| Repository Structure-Aware Training Makes SLMs Better Issue Resolver | ReSAT | Agentless | ARXIV | 2024-12 | @SFT | Paper |
| SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution | SWE-Fixer | SWE-Fixer | ACL Findings 2025 | 2025-02 | @SFT | Paper |
| SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution | SWE-RL | Agentless Mini | NeurIPS 2025 | 2025-02 | @RL | Paper Code |
| SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning | SoRFT | Agentless | ACL 2025 | 2025-02 | @RL | Paper |
| SEAlign: Alignment Training for Software Engineering Agent | SEAlign | OpenHands | ICSE 2026 | 2025-03 | @RL | Paper |
| Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute | SWE-Reasoner | SWE-SynInfer+ | ASE 2025 | 2025-04 | @RL | Paper Code |
| Co-PatcheR: Collaborative Software Patching with Component(s)-specific Small Reasoning Models | Co-PatcheR | PatchPilot | NeurIPS 2025 | 2025-05 | @SFT | Paper Code |
| Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering | EvoScale | Satori-SWE | ARXIV | 2025-05 | @RL | Paper Code |
| Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards | Agent-RLVR | Agentless | ARXIV | 2025-06 | @RL | Paper |
| MCTS-Refined CoT: High-Quality Fine-Tuning Data for LLM-Based Repository Issue Resolution | MCTS-Refined | Agentless-1.0 | ASE 2025 | 2025-06 | @SFT | Paper |
| SWE-GPT: A Process-Centric Language Model for Automated Software Improvement | SWE-GPT | SWESynInfer | ISSTA 2025 | 2025-06 | @SFT | - |
| - | DeepSWE | - | Blog | 2025-07 | @RL | Link |
| - | SWE-Swiss | - | Blog | 2025-08 | @RL | Link Code |
| RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale | RepoForge | OpenHands | ARXIV | 2025-08 | @RL | Paper |
| Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning | - | - | ARXIV | 2025-08 | @RL | Paper |
| Devstral: Fine-tuning Language Models for Coding Agent Applications | Devstral-Small | OpenHands | ARXIV | 2025-08 | @RL | Paper |
| Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents | Kimi-Dev | Kimi-Dev | ICLR 2026 | 2025-09 | @RL | Paper |
| CWM: An Open-Weights LLM for Research on Code Generation with World Models | CWM | CWM | ARXIV | 2025-09 | @RL | Paper |
| Building Coding Agents via Entropy-Enhanced Multi-Turn Preference Optimization | EntroPO | R2E | ARXIV | 2025-09 | @RL | Paper Code |
| BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills | BugPilot | R2E | ARXIV | 2025-10 | @SFT | Paper |
| Training Versatile Coding Agents in Synthetic Environments | SWE-Playground | OpenHands | ARXIV | 2025-12 | @SFT | Paper Code |
| Toward Training Superintelligent Software Agents through Self-Play SWE-RL | Self-Play SWE-RL | bash+editor | ICML 2026 | 2025-12 | @RL | Paper |
| Context as a Tool: Context Management for Long-Horizon SWE-Agents | CAT/SWE-Compressor | OpenHands | Arxiv | 2025-12 | @SFT | Paper |
| SWE-RM: Execution-free Feedback For Software Engineering Agents | SWE-RM | OpenHands | ICLR 2026 | 2025-12 | @RL | Paper |
| SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving | SWE-Lego | - | ARXIV | 2026-01 | @SFT | Paper |
| daVinci-Dev: Agent-native Mid-training for Software Engineering | daVinci-Dev | - | ARXIV | 2026-01 | @SFT | Paper |
| SERA: Soft-Verified Efficient Repository Agents | SERA | - | ICML 2026 | 2026-01 | @SFT | Paper |
| SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning | SWE-Spot | - | ARXIV | 2026-01 | @SFT | Paper |
| SWE-World: Building Software Engineering Agents in Docker-Free Environments | SWE-World | - | ARXIV | 2026-02 | @RL | Paper |
| SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training | SWE-Master | - | ARXIV | 2026-02 | @RL | Paper |
| SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents | SWE-Protégé | - | ARXIV | 2026-02 | @RL | Paper |
| SWE-Fuse: Empowering Software Agents via Issue-free Trajectory Learning and Entropy-aware RLVR Training | SWE-Fuse | - | ARXIV | 2026-03 | @RL | Paper |
| From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agents | SWE-HERO | - | ARXIV | 2026-04 | @SFT | Paper |
| SWE-AGILE: A Software Agent Framework for Efficiently Managing Dynamic Reasoning Context | SWE-AGILE | - | ARXIV | 2026-04 | @RL | Paper |
| SWE-TRACE: Optimizing Long-Horizon SWE Agents Through Rubric Process Reward Models and Heuristic Test-Time Scaling | SWE-TRACE | - | ARXIV | 2026-04 | @RL | Paper |
| BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models | BoostAPR | - | ICML 2026 | 2026-05 | @RL | Paper |
| Hindsight Hint Distillation: Scaffolded Reasoning for SWE Agents from CoT-free Answers | HHD | - | ARXIV | 2026-05 | @SFT | Paper |
Empirical Studies
We categorize the empirical studies into 2 perspectives:
@Evaluation-oriented Studies
@Technique-oriented Studies
Evaluation-oriented Studies
For Evaluation-oriented Studies, we discuss them in 4 categories separately:
@Validity
@Quality
@Efficiency
@Robustness
where, @Validity indicates evaluation validity, @Quality indicates quality and requirement compliance,
@Efficiency indicates efficiency and resource consumption, and @Robustness indicates robustness and
process-aware evaluation.
| Literature | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|
| Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios | SANER | 2024-10 | @Quality | Paper |
| Large Language Model Critics for Execution-Free Evaluation of Code Changes | ARXIV | 2025-01 | @Validity | Paper |
| Ambig-SWE: Interactive Agents to Overcome Underspecificity in Software Engineering | ICLR 2026 | 2025-02 | @Robustness | Paper |
| Are "Solved Issues" in SWE-bench Really Solved Correctly? An Empirical Study | ICSE 2026 | 2025-03 | @Validity | Paper |
| SWE-Bench-CL: Continual Learning for Coding Agents | ARXIV | 2025-06 | @Robustness | Paper |
| The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason | ARXIV | 2025-06 | @Validity | Paper |
| Are AI-Generated Fixes Secure? Analyzing LLM and Agent Patches on SWE-bench | ARXIV | 2025-06 | @Quality | Paper |
| SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints | ARXIV | 2025-09 | @Efficiency | Paper |
| Saving SWE-Bench: A Benchmark Mutation Approach for Realistic Agent Evaluation | ARXIV | 2025-10 | @Validity | Paper |
| More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents | ICSE 2026 | 2025-10 | @Efficiency | Paper |
| Process-Centric Analysis of Agentic Software Systems | ARXIV | 2025-12 | @Robustness | Paper |
| SWEnergy: An Empirical Study on Energy Efficiency in Agentic Issue Resolution Frameworks with SLMs | ICSE 2026 Workshop | 2025-12 | @Efficiency | Paper |
| Does SWE-Bench-Verified Test Agent Ability or Model Memory? | ARXIV | 2025-12 | @Validity | Paper |
| What's in a Benchmark? The Case of SWE-Bench in Automated Program Repair | ICSE 2026 | 2026-02 | @Validity | Paper |
| TRAJEVAL: Decomposing Code Agent Trajectories for Fine-Grained Diagnosis | ARXIV | 2026-03 | @Robustness | Paper |
| Does Pass Rate Tell the Whole Story? Evaluating Design Constraint Compliance in LLM-based Issue Resolution | ARXIV | 2026-04 | @Quality | Paper |
| Coding Agents Don't Know When to Act | ARXIV | 2026-05 | @Robustness | Paper |
| AgentLens: Revealing The Lucky Pass Problem in SWE-Agent Evaluation | ARXIV | 2026-05 | @Robustness | Paper |
| RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations | ARXIV | 2026-05 | @Robustness | Paper |
Technique-oriented Studies
For Technique-oriented Studies, we discuss them in 3 categories separately:
@Performance
@Behavior
@Failure
where, @Performance indicates performance variation and influencing factors, @Behavior indicates agent
behavior and resolution process, and @Failure indicates failure modes and root causes.
| Literature | Journal/Conference | Time | Label | URL |
|---|---|---|---|---|
| Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents | ICLR 2025 | 2024-08 | @Performance | Paper Code |
| An Empirical Study on LLM-based Agents for Automated Bug Fixing | ARXIV | 2024-11 | @Performance | Paper |
| Beyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarios | ICSE 2026 | 2025-03 | @Failure | Paper |
| PAGENT: Learning to Patch Software Engineering Agents | ICSE 2026 | 2025-06 | @Failure | Paper |
| Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories | ASE 2025 | 2025-06 | @Behavior | Paper |
| An Empirical Study on Failures in Automated Issue Solving | ARXIV | 2025-09 | @Failure | Paper |
| Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories | ICSE 2026 | 2025-10 | @Behavior | Paper |
| SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents | ICLR 2026 | 2025-11 | @Failure | Paper |
| Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair | FSE 2026 | 2026-01 | @Performance | Paper |
| Beyond Localization: Recoverable Headroom and Residual Frontier in Repository-Level RAG-APR | ARXIV | 2026-03 | @Performance | Paper |
| ORACLE-SWE: Quantifying the Contribution of Oracle Information Signals on SWE Agents | ICML 2026 | 2026-04 | @Performance | Paper |
| Do Agent Rules Shape or Distort? Guardrails Beat Guidance in Coding Agents | ARXIV | 2026-04 | @Performance | Paper |
| From Plan to Action: How Well Do Agents Follow the Plan? | ARXIV | 2026-04 | @Behavior | Paper |
| LLM-based Agents for Automated Bug Fixing: How Far Are We? | ICSE 2026 | 2026-04 | @Performance | - |