Awesome Coding Agent Papers

A research-first collection of papers, benchmarks, datasets, executable environments, open-source agents, and engineering notes for repository-level coding agents.

The focus is not generic code generation. It is the full agent loop: repository exploration, terminal/tool use, patch generation, test feedback, long-horizon recovery, verifiable training, and production-grade evaluation.

Last verified: 2026-07-19. Dates on arXiv papers refer to the first public submission. Dataset sizes and model/scaffold metadata follow the linked paper or dataset card and may change as releases are updated.

Chinese companion material: research survey · interactive benchmark/data report

Contents

• Start Here
• Surveys and Field Maps
• Agent Architectures and Open Systems
• Training and Post-Training
• Public Data and Environment Synthesis
• SWE Benchmarks
  • Core Issue Resolution
  • Long-Horizon and Production Tasks
  • Exploration and Repository Understanding
  • CI, Evolution, Review, and Process
  • Code Quality, Tests, Security, and Observability
  • Languages, Modalities, and Specialized Tasks
• Open Trajectory Datasets
• Executable SWE Environments
• Long Context and Repository Reasoning
• Adjacent: Terminal and Computer-Use Agents
• Adjacent: Tool Use and MCP
• Code Model Backbones
• Industry Engineering Notes
• Related Lists
• Contributing

Entry Format

• Paper title (first public date)
  One-sentence contribution. [Paper] [Code] [Data] [Project]
• Resource labels are intentionally explicit: a trajectory dataset is not presented as an executable environment, and a benchmark is not assumed to ship ready-to-pull images.
• The SWE sections stay software-engineering-specific. Terminal, computer-use, and general tool-use work is kept in adjacent sections.

Start Here

This short path moves from the original benchmark and agent interface to training, data scaling, and newer evaluation questions.

Foundations

1. SWE-bench · 2023-10-10
Established execution-backed, repository-level issue resolution.
Paper · Code · Leaderboard

2. SWE-agent · 2024-05-06
Showed how the agent-computer interface can be a capability multiplier.
Paper · Code

3. SWE-Gym · 2024-12-30
Opened a training-oriented task and trajectory environment for SWE agents and verifiers.
Paper · Code · Data

Training and Scale

4. SWE-smith · 2025-04-30
Scales synthetic, test-backed issue generation from real Python repositories.
Paper · Code · Project

5. R2E-Gym · 2025-04-09
Combines executable environments, procedural task generation, and hybrid verification.
Paper · Project

6. Long-Context Multi-Turn SWE Agents with RL · 2025-08-05
Studies long-horizon, multi-turn reinforcement learning with verifiable software rewards.
Paper

7. daVinci-Env / OpenSWE · 2026-03-13
Releases 45,320 executable environments with Dockerfiles and evaluation scripts.
Paper · Code · Data

8. Open-SWE-Traces · 2026-06-14
Provides 207K multilingual trajectories across two teachers and two agent scaffolds.
Paper · Data

New Evaluation Questions

9. DeepSWE · 2026-07-08
Evaluates frontier agents on 113 original, long-horizon tasks with clean verifier containers.
Paper · Code

10. SWE-Explore · 2026-06-05
Isolates repository exploration and relevant code-region retrieval under a line budget.
Paper

11. SlopCodeBench · 2026-03-25
Measures structural erosion and verbosity across iterative development checkpoints.
Paper · Code

12. Coding Agents are Effective Long-Context Processors · 2026-03-20
Frames coding agents as active context processors built around files, search, and executable tools.
Paper

Surveys and Field Maps

• Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code (2023-11)
  Broad code-LLM and software-engineering taxonomy. [Paper] [Resources]

• Software Development Life Cycle Perspective: A Survey of Benchmarks for CodeLLMs and Agents (2025-05)
  Organizes evaluation across the software lifecycle. [Paper]

• A Survey of LLM-based Automated Program Repair (2025-06-30)
  Taxonomy of LLM repair systems, design paradigms, and applications. [Paper]

• Dissecting the SWE-Bench Leaderboards (2025-06-20)
  Profiles systems, scaffolds, and architectural choices behind leaderboard submissions. [Paper]

Agent Architectures and Open Systems

• AutoCodeRover: Autonomous Program Improvement (2024-04-08)
  Uses program structure and iterative search for autonomous issue resolution. [Paper] [Code]

• SWE-agent (2024-05-06)
  Introduces the Agent-Computer Interface and an open repository repair agent. [Paper] [Code]

• Agentless: Demystifying LLM-based Software Engineering Agents (2024-07-01)
  Decomposes localization, repair, and patch selection into a simple non-interactive pipeline. [Paper] [Code]

• OpenHands
  Open platform for software development agents, evaluation, and trajectory collection. [Code] [CodeAct 2.1]

• mini-SWE-agent
  Minimal agent implementation designed for transparent SWE-bench experimentation. [Code]

• Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases (2025-12-11)
  Studies scalable scaffolding for repository-level coding. [Paper]

• OpenSWE (2026-03-13)
  Open environment construction, evaluation, and training stack compatible with SWE-agent and SWE-bench-style grading. [Paper] [Code]

Training and Post-Training

• SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution (2025-01-09)
  Trains retrieve-then-edit models from a large issue corpus. [Paper] [Code] [Model]

• Training Software Engineering Agents and Verifiers with SWE-Gym (2024-12-30)
  Trains both a policy and verifier on open SWE tasks and trajectories. [Paper] [Data]

• SWE-Dev: Evaluating and Training Autonomous Feature-Driven Software Development (2025-05-22)
  Extends agent training from bug repair to feature development with tests and trajectories. [Paper]

• Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning (2025-08-05)
  Applies verifiable RL to long multi-turn repository interaction. [Paper]

• RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale (2025-08-03)
  Combines large-scale curation, SFT, and RL for fast issue resolution. [Paper]

• Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents (2025-09-27)
  Uses non-agentic training to build repository skills later expressed through an agent. [Paper]

• Toward Training Superintelligent Software Agents through Self-Play SWE-RL (2025-12-21)
  Explores self-play to reduce dependence on human-authored SWE tasks. [Paper]

• Hybrid-Gym: Training Coding Agents to Generalize Across Tasks (2026-02-18)
  Uses synthetic training environments to transfer across issue resolution, testing, and implementation tasks. [Paper]

• Immersion in the GitHub Universe: Scaling Coding Agents to Mastery (Scale-SWE) (2026-02-10)
  Builds and filters a large real-PR task pool with multi-agent environment construction. [Paper]

• daVinci-Env / OpenSWE (2026-03-13)
  Studies environment scale, difficulty-aware filtering, and SFT across multiple scaffolds. [Paper] [Model]

• Scaling Test-Time Compute for Agentic Coding (2026-04-16)
  Compresses long trajectories into reusable experience and combines parallel selection with serial refinement. [Paper]

• Learning CLI Agents with Structured Action Credit under Selective Observation (2026-05-08)
  Improves long-horizon credit assignment for command-line agents. [Paper]

• Polar: Agentic RL on Any Harness at Scale (2026-05-22)
  Treats existing agent harnesses as black boxes, proxies model calls, reconstructs token-faithful trajectories, and serves asynchronous rollouts to independent RL trainers. [Paper] [Code]

• Inference-Time Scaling with a Critic Model
  Uses candidate generation and learned critique to select stronger SWE patches. [Blog]

• Training and Search for Software Engineering Agents
  Practical discussion of policy training, trajectory search, and execution-backed selection. [Blog]

Public Data and Environment Synthesis

This section lists published, reproducible methods only.

• SWE-smith (2025-04-30)
  Produces test-backed synthetic issue instances from real Python repositories. [Paper] [Code]

• R2E-Gym / SWE-Gen (2025-04-09)
  Procedurally generates executable tasks and combines programmatic and model-based verifiers. [Paper] [Project]

• SWE-rebench (2025-05-26)
  Automates PR-derived task collection, execution filtering, and decontaminated evaluation. [Paper] [V2 code]

• SWE-Factory (2025-06-12)
  Automated factory for issue-resolution training data and evaluation benchmarks. [Paper]

• Training Versatile Coding Agents in Synthetic Environments (SWE-Playground) (2025-12-13)
  Generates projects, task specifications, starter code, and tests from scratch. [Paper]

• SWE-World: Building Software Engineering Agents in Docker-Free Environments (2026-02-03)
  Replaces expensive physical execution with a learned surrogate environment. [Paper]

• Scale-SWE (2026-02-10)
  Scales PR mining, environment building, validation tests, and trajectory distillation. [Paper]

• CLI-Gym (2026-02-11)
  Generates environment-intensive terminal tasks by inverting environment construction histories. [Paper] [Code]

• OpenSWE / daVinci-Env (2026-03-13)
  Open-sources multi-agent environment construction, Dockerfiles, evaluation scripts, and distributed build infrastructure. [Paper] [Code]

• Terminal-World (2026-05-20)
  Uses explicit skills and environment preconditions to synthesize terminal-agent worlds. [Paper]

• REAP / Harvest (2026-04-02)
  Curates benchmarks from interactive production usage with task-testability and test-relevance filtering. [Paper]

SWE Benchmarks

Core Issue Resolution

• SWE-bench / Lite / Verified (2023-10-10)
  Real GitHub issues graded by repository tests; Verified is a 500-task human-audited subset. [Paper] [Code] [Leaderboard]

• SWE-Bench+ (2024-10-09)
  Studies leakage, weak tests, and benchmark reliability. [Paper]

• SWE-bench Goes Live! (2025-05-29)
  Continuously collects fresher issue-resolution tasks to reduce contamination. [Paper]

• SWE-Bench Pro (2025-09-21)
  Harder long-horizon tasks with public/private evaluation and prebuilt images. [Paper] [Code]

• SWE-Lancer (2025)
  Real freelance software-engineering and engineering-management tasks. [Project]

Long-Horizon and Production Tasks

• SWE-EVO (2025-12-20)
  Evaluates agents across multi-stage software evolution rather than one isolated issue. [Paper]

• SWE-Milestone: Evaluating AI Agents on Continuous Software Evolution (2026-03-13)
  Reconstructs executable milestone DAGs from commit histories and measures integrity preservation and error accumulation across continuous task streams. [Paper] [Code]

• SlopCodeBench (2026-03-25)
  Tracks correctness, structural erosion, verbosity, and clone density over 196 iterative checkpoints. [Paper] [Code]

• REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage (2026-04-02)
  The current arXiv version names the curated benchmark Harvest; evaluation uses executable fail-to-pass tests rather than an LLM judge. [Paper]

• FrontierCode / FrontierCode 1.1 (2026-06 / 2026-07)
  Maintainer-authored tasks with blocker criteria for correctness, regression, build/lint, test correctness, and scope plus weighted code-quality rubrics. [Launch] [v1.1]

• DeepSWE (2026-07-08)
  113 original tasks across 91 repositories and five languages, evaluated by applying committed diffs in clean verifier containers. [Paper] [Code] [Protocol]

Exploration and Repository Understanding

• SWE-QA: Can Language Models Answer Repository-level Code Questions? (2025-09-18)
  Multi-hop repository understanding and architecture questions. [Paper]

• A Benchmark for Localizing Code and Non-Code Issues in Software Projects (MULocBench / Multi-CoLoR) (2025-09-26)
  File, function, and line-level localization across languages. [Paper]

• SWE-Skills-Bench (2026-03-16)
  Tests whether skill documents and micro-agent context actually improve issue resolution. [Paper]

• SWE Atlas (2026-05-08)
  Expands coding-agent evaluation to codebase QA, test writing, and refactoring. [Paper]

• SWE-Explore (2026-06-05)
  Measures repository exploration and code-region ranking under constrained reading budgets. [Paper]

• SWE-Doctor (2026-07-01)
  Uses multi-faceted bug-reproduction tests to grade and guide runtime diagnosis. [Paper]

• MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization (2026-07-16)
  Provides 652 issue-PR instances across 23 languages with paired text-only/with-image evaluation and file/function localization labels. [Paper]

CI, Evolution, Review, and Process

• SWE-CI (2026-03-04)
  Evaluates codebase maintenance through continuous-integration feedback. [Paper]

• AgentLens (2026-07-07)
  Reviews trajectories for instruction following, tool use, self-verification, recovery, and communication. [Paper] [Code]

• SWE-Review (2026-07-07)
  Measures review correctness and whether review feedback improves a subsequent revision. [Paper]

Code Quality, Tests, Security, and Observability

• SWE-Perf (2025-07-16)
  Repository-level performance optimization with executable correctness and performance checks. [Paper]

• SEC-bench (2025-06-13)
  Execution-backed vulnerability reproduction and patching tasks from real projects. [Paper]

• SecureVibeBench (2025-09-26)
  Requires patches to satisfy both functionality tests and static/dynamic security oracles. [Paper] [Code]

• SlopCodeBench (2026-03-25)
  Adds longitudinal maintainability and architecture-degradation signals beyond pass/fail. [Paper]

• Can Large Language Models Generate Observability-Aware Code? (2026-07-07)
  Evaluates instrumentation placement, diagnostic variable coverage, and fault signal rate. [Paper]

• Beyond Test Presence (2026-07-13)
  Compares human and agent-generated tests by assertion strength, edge-case coverage, and flakiness potential. [Paper]

• FrontierCode 1.1 (2026-07-07)
  Combines programmatic blockers, reverse-classical test checks, scope scanners, and code-quality rubrics. [Project]

Languages, Modalities, and Specialized Tasks

• SWE-bench Multimodal (2024-10-04)
  Extends issue resolution to visual software domains. [Paper]

• Multi-SWE-bench (2025-04-03)
  1,632 issue-resolution tasks across seven programming languages. [Paper]

• SWE-PolyBench (2025-04-11)
  Multi-language repository-level benchmark with a curated PB500 image-backed subset. [Paper]

• Rust-SWE-bench (2026-02-26)
  Repository-level issue resolution in the Rust ecosystem. [Paper]

• SWE-bench-java: A GitHub Issue Resolving Benchmark for Java (2024-08-26)
  Introduces a Java benchmark and reports a Docker-based evaluation environment. [Paper]

• NL2Repo-Bench (2025-12-14)
  Evaluates long-horizon repository generation from natural-language specifications. [Paper]

Open Trajectory Datasets

These are trajectory or interaction corpora. They do not necessarily include ready-to-run Docker images.

Executable SWE Environments

Image status is intentionally explicit. Ready means the project reports prebuilt images; buildable means public recipes or Dockerfiles exist but local validation is still required.

Before using any environment family for training, independently check image pullability, base-commit checkout, pre-patch failure, gold-patch success, test determinism, and network assumptions.

Long Context and Repository Reasoning

• RepoBench (2023-06-05)
  Repository-level code completion under cross-file context. [Paper]

• RepoQA (2024-06-10)
  Long-context repository understanding and function retrieval. [Paper] [Code]

• Long Code Arena (2024-06-17)
  Suite of long-context code tasks from real repositories. [Paper] [Data]

• ExecRepoBench (2024-12-16)
  Multi-level executable evaluation for repository-aware code completion. [Paper]

• LONGCODEU (2025-03-06)
  Long-code understanding tasks spanning structural and semantic reasoning. [Paper]

• From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level (RepoReason) (2026-01-07)
  Repository-scale agentic reasoning benchmark. [Paper]

• Coding Agents are Effective Long-Context Processors (2026-03-20)
  Uses the filesystem, scripts, and search as an active long-context strategy. [Paper]

• SWE-Explore (2026-06-05)
  Measures whether agents spend a limited context budget on the right code regions. [Paper]

Adjacent: Terminal and Computer-Use Agents

These works are relevant to shell competence and environment interaction, but they are not mixed into the SWE benchmark list.

• OSWorld (2024-04-11)
  Open-ended multimodal tasks in real computer environments. [Paper] [Code]

• Terminal-Bench (2026-01-17)
  Hard command-line tasks with isolated environments, human solutions, and tests. [Paper] [Code] [Leaderboard]

• CLI-Gym (2026-02-11)
  Environment inversion for scalable generation of CLI-intensive tasks. [Paper]

• On Data Engineering for Scaling LLM Terminal Capabilities (2026-02-24)
  Studies terminal task sourcing, filtering, and data scaling. [Paper]

• OSWorld-MCP (2025-10-28)
  Evaluates MCP tool invocation for computer-use agents. [Paper]

• Terminal-World (2026-05-20)
  Scales terminal training environments through explicit agent skills. [Paper]

• CUA-Gym (2026-05-25)
  Scales verifiable computer-use tasks and environments. [Paper]

• WildClawBench (2026-05-11)
  Evaluates 60 human-authored, bilingual, multimodal, long-horizon tasks inside native CLI harnesses with Dockerized tools and hybrid grading. [Paper] [Code]

• OSWorld 2.0 (2026-06-28)
  Long-horizon real-world computer-use benchmark. [Paper]

Adjacent: Tool Use and MCP

• ToolLLM / ToolBench (2023-07-31)
  Large-scale tool-use data and evaluation across real APIs. [Paper] [Code]

• ToolACE (2024-09-02)
  Function-calling data generation, self-evolution, and execution validation. [Paper] [Data]

• ToolACE-R (2025-04-02)
  Model-aware iterative training and adaptive refinement for tool learning. [Paper]

• Berkeley Function Calling Leaderboard (BFCL)
  Multi-turn and multi-step function-calling evaluation. [Paper] [Leaderboard]

• MCP-Universe (2025-08-20)
  Evaluates agents against real Model Context Protocol servers. [Paper] [Code]

Code Model Backbones

This is deliberately not a general code-LLM list; only widely used open backbones for coding-agent research are included.

• StarCoder 2 and The Stack v2 (2024-02-29)
  Open code-model family and permissively filtered training corpus. [Paper] [Code]

• DeepSeek-Coder-V2 (2024-06-17)
  Mixture-of-experts code model with broad language and repository capability. [Paper] [Code]

• Qwen2.5-Coder (2024-09-18)
  Open coder family frequently used as the base for SWE-agent training. [Paper] [Code]

Industry Engineering Notes

Product teams do not publish complete training recipes. The following are useful public signals about agent loops, context management, sandboxes, tool design, evaluation, and long-running workflows.

OpenAI / Codex

• Introducing Codex - Cloud sandboxes, parallel tasks, repository setup, and test-backed work.
• o3/o4-mini Codex system card addendum - Security and evaluation details for Codex-style execution.
• GPT-5-Codex system card addendum - Agentic coding optimization, real coding-task RL, instruction following, and iterative testing.
• Unrolling the Codex agent loop - Model, tool schema, context construction, observations, and loop control.
• Unlocking the Codex harness - Harness-level capability and evaluation considerations.
• Harness engineering with Codex - Practical runtime and workflow design.
• Why we no longer evaluate SWE-bench Verified - Contamination and measurement limits in saturated coding benchmarks.
• Separating signal from noise in coding evaluations - Reliability and interpretation of coding evals.
• Introducing GPT-5.4 - Reports SWE-Bench Pro (Public) as the primary SWE headline.

Anthropic / Claude Code

• Claude Code best practices - Repository-aware workflows, tests, and parallel sessions.
• Building effective agents - Workflows versus agents, evaluator-optimizer patterns, and system simplicity.
• Effective context engineering for AI agents - Context selection, compaction, memory, and information density.
• Effective harnesses for long-running agents - Initializers, multiple context windows, and environment continuity.
• Harness design for long-running application development - Test feedback and state management for sustained development.
• Writing tools for agents - Tool interfaces optimized for model use.
• Code execution with MCP - Token-efficient tool discovery and execution.
• Advanced tool use - Tool search, examples, and programmatic calling.
• Claude Code sandboxing - Filesystem/network isolation and permission boundaries.
• Claude Code auto mode - Permission classification and lower-friction safe execution.
• Building a C compiler with parallel Claudes - Multi-agent coordination on a long-running engineering project.
• Claude Opus 4.6 - Reports SWE-bench Verified with repeated trials and prompt-specific evaluation notes.

Related Lists

• Awesome-Code-LLM - Broad code-language-model survey and resource list.
• Awesome Agent Papers - General LLM-agent research taxonomy.
• Awesome - Guidelines and index for curated Awesome lists.

Contributing

Contributions are welcome. Please read CONTRIBUTING.md before opening an issue or pull request.

Useful contributions include:

• a missing paper with a canonical paper link and first-public date;
• a verified code, dataset, model, image registry, or project link;
• a correction to dataset scale, teacher, scaffold, verifier, or image availability;
• a clearer categorization that avoids mixing benchmarks, traces, and environments.

This list contains public, independently verifiable resources. Claims without a paper, official technical post, repository, or dataset card are not included.

License

Released under CC0-1.0.