Awesome Co-Evolution in Agentic Systems

A curated paper list for Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design.

Co-evolution is a coupled form of self-evolution in which at least two evolving units evolve together and continually reshape each other's further evolution. The list follows a progressive taxonomy based on the expanding boundary of what a system is allowed to evolve.

Scope. We include persistent co-evolution, not systems that only exchange information, interact once, or refine a transient output. An agent is treated as one evolving unit that may evolve its model backbone, harness, or both.

Contents

• Taxonomy
• Paper Landscape
• Stage 1: Agent–Agent Co-Evolution
• Stage 2: Agent–Environment Co-Evolution
• Stage 3: Meta Co-Evolution
• Contributing
• Citation

Taxonomy

1. Agent–Agent Co-Evolution studies joint evolution among agents in an otherwise fixed environment.
2. Agent–Environment Co-Evolution extends the evolutionary process to environments containing task, feedback, and interaction spaces.
3. Meta Co-Evolution makes the evolution mechanism itself evolvable, including what, when, how, and where to evolve and how evolution is evaluated.

Paper Landscape

The dates below follow each paper's year of first public release where available. The curated list currently contains 44 Stage-1 papers, 92 Stage-2 papers, and 7 Stage-3 papers or precursors.

Stage 1: Agent–Agent Co-Evolution

Agents evolve through evolutionary pressure from other evolving agents. Adversarial and collaborative methods are distinguished by goal relations, while evolving organizations additionally change agents' roles or interaction structures.

1.1 Adversarial Agents

Pairwise Adversarial Pressure

• 2026 · AdvGRPO — Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO
• 2026 · CHASE — CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning
• 2026 · MAGIC — MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety
• 2026 · Safety Self-Play — Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replay
• 2025 · ACE-Safety — Adversarial Attack-Defense Co-Evolution for LLM Safety Alignment via Tree-Group Dual-Aware Search and Optimization
• 2025 · ARLAS — Adversarial Reinforcement Learning for Large Language Model Agent Safety
• 2025 · Active Attacks — Active Attacks: Red-teaming LLMs via Adaptive Environments
• 2025 · Self-RedTeam — Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models
• 2024 · SPAG — Self-playing Adversarial Language Game Enhances LLM Reasoning
• 2019 · Emergent Tool Use — Emergent Tool Use From Multi-Agent Autocurricula
• 2017 · Emergent Complexity — Emergent Complexity via Multi-Agent Competition
• 2017 · RARL — Robust Adversarial Reinforcement Learning

Multi-Source Adversarial Pressure

• 2026 · TriPlay-RL — TriPlay-RL: Tri-Role Self-Play Reinforcement Learning for LLM Safety Alignment
• 2025 · AdvEvo-MARL — AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning
• 2023 · GRTS — Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games
• 2019 · AlphaStar — Grandmaster level in StarCraft II using multi-agent reinforcement learning

1.2 Collaborative Agents

Parallel Collaboration

• 2026 · CORAL — CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery
• 2026 · MARS² — MARS²: Scaling Multi-Agent Tree Search via Reinforcement Learning for Code Generation
• 2026 · GEA — Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharing
• 2026 · MARTI-MARS² — MARTI-MARS²: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generation
• 2026 · MAAC — Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic
• 2025 · CoMAS — CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards
• 2025 · MAC-SPGG — Everyone Contributes! Incentivizing Strategic Cooperation in Multi-LLM Systems via Sequential Public Goods Games
• 2025 · MAGRPO — LLM Collaboration with Multi-Agent Reinforcement Learning
• 2025 · LIET — Learn as Individuals, Evolve as a Team: Multi-agent LLMs Adaptation in Embodied Environments
• 2025 · M3HF — M3HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Quality
• 2025 · MAPoRL — MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learning
• 2020 · LIO — Learning to Incentivize Other Learning Agents
• 2018 · LeCTR — Learning to Teach in Cooperative Multiagent Reinforcement Learning
• 2018 · Mean-Field MARL — Mean Field Multi-Agent Reinforcement Learning
• 2017 · MADDPG — Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• 2017 · COMA — Counterfactual Multi-Agent Policy Gradients

Role-Differentiated Collaboration

• 2026 · CoVerRL — CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution
• 2026 · EvoScientist — EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery
• 2025 · MARS — MARS: Co-evolving Dual-System Deep Research via Multi-Agent Reinforcement Learning
• 2025 · Stronger-MAS — Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs
• 2025 · WaltzRL — The Alignment Waltz: Jointly Training Agents to Collaborate for Safety
• 2025 · RL Tango — RL Tango: Reinforcing Generator and Verifier Together for Language Reasoning
• 2025 · MARFT — MARFT: Multi-Agent Reinforcement Fine-Tuning
• 2025 · SiriuS — SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning
• 2024 · CORY — Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learning

1.3 Evolving Agent Organizations

• 2026 · MetaAgent-X — MetaAgent-X : Breaking the Ceiling of Automatic Multi-Agent Systems via End-to-End Reinforcement Learning
• 2026 · SkillMAS — SkillMAS: Skill Co-Evolution with LLM-based Multi-Agent System
• 2025 · R3DM — R3DM: Enabling Role Discovery and Diversity Through Dynamics Models in Multi-agent Reinforcement Learning

Stage 2: Agent–Environment Co-Evolution

Agents co-evolve with an evolving environment. We classify methods by whether the main source of further evolutionary pressure is a changing task space, feedback space, or interaction space.

2.1 Task-Space Co-Evolution

Exposure and Selection

• 2026 · Role-Agent — Role-Agent: Bootstrapping LLM Agents via Dual-Role Evolution
• 2026 · SEAD — SEAD: Self-Evolving Agent for Multi-Turn Service Dialogue
• 2025 · HAP — Heterogeneous Adversarial Play in Interactive Environments
• 2025 · NCC — An Optimisation Framework for Unsupervised Environment Design
• 2024 · ReMiDi — Refining Minimax Regret for Unsupervised Environment Design
• 2023 · Teammate Curriculum — Curriculum Learning for Cooperation in Multi-Agent Reinforcement Learning
• 2023 · Portal — PORTAL: Automatic Curricula Generation for Multiagent Reinforcement Learning
• 2021 · PLR⊥ — Replay-Guided Adversarial Environment Design
• 2020 · PLR — Prioritized Level Replay
• 2020 · APT-Gen — Adaptive Procedural Task Generation for Hard-Exploration Problems
• 2020 · SPDL — Self-Paced Deep Reinforcement Learning
• 2019 · ALP-GMM — Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
• 2019 · SPRL — Self-Paced Contextual Reinforcement Learning
• 2017 · TSCL — Teacher-Student Curriculum Learning

Adaptive Task Generation

• 2026 · MobileForge — MobileForge: Annotation-Free Adaptation for Mobile GUI Agents with Hierarchical Feedback-Guided Policy Optimization
• 2026 · SENTINEL — SENTINEL: Failure-Driven Reinforcement Learning for Training Tool-Using Language Model Agents
• 2026 · Socratic-SWE — Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills
• 2026 · CoEvolve — CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution
• 2026 · EE-MCP — EE-MCP: Self-Evolving MCP-GUI Agents via Automated Environment Generation and Experience Learning
• 2026 · UI-Voyager — UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience
• 2026 · Autonomous Continual Learning CUA — Autonomous Continual Learning of Computer-Use Agents for Environment Adaptation
• 2026 · RLAnything — RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL System
• 2026 · Tool-R0 — Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data
• 2026 · Dr. Zero — Dr. Zero: Self-Evolving Search Agents without Training Data
• 2026 · EvoCUA — EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience
• 2025 · GenEnv — GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators
• 2025 · Agent0 — Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning
• 2025 · AgentEvolver — AgentEvolver: Towards Efficient Self-Evolving Agent System
• 2025 · AgentFrontier — AgentFrontier: Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesis
• 2025 · ASL — Towards Agentic Self-Learning LLMs in Search Environment
• 2025 · Search Self-Play — Search Self-play: Pushing the Frontier of Agent Capability without Supervision
• 2025 · SEAgent — SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience
• 2025 · Self-Challenging — Self-Challenging Language Model Agents
• 2025 · CCL — CCL: Collaborative Curriculum Learning for Sparse-Reward Multi-agent Reinforcement Learning via Co-Evolutionary Task Evolution
• 2025 · MT-ASP — Do as you teach: a multi-teacher approach to self-play in deep reinforcement learning
• 2024 · WebRL — WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning
• 2021 · Asymmetric Self-Play — Asymmetric self-play for automatic goal discovery in robotic manipulation
• 2017 · GoalGAN — Automatic Goal Generation for Reinforcement Learning Agents

2.2 Feedback-Space Co-Evolution

Preference-Driven Feedback

• 2025 · PPE — Improving Reward Models with Proximal Policy Exploration for Preference-Based Reinforcement Learning
• 2025 · DAPPER — DAPPER: Discriminability-Aware Policy-to-Policy Preference-Based Reinforcement Learning for Query-Efficient Robot Skill Acquisition
• 2025 · DUO — DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback
• 2022 · RUNE — Reward Uncertainty for Exploration in Preference-based Reinforcement Learning
• 2021 · PEBBLE — PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
• 2017 · RLHF — Deep Reinforcement Learning from Human Preferences

Outcome-Driven Feedback

• 2026 · CoEvoSkills — CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification
• 2025 · LaRes — Lares: evolutionary reinforcement learning with llm-based adaptive reward search
• 2025 · Reward Discovery — Discovery of the reward function for embodied reinforcement learning agents
• 2025 · Env-Tuning — Don't Just Fine-tune the Agent, Tune the Environment
• 2025 · RE-GoT — Reward Evolution with Graph-of-Thoughts: A Bi-Level Language Model Framework for Reinforcement Learning
• 2025 · CURE — Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning
• 2025 · AHRS — Automated Hybrid Reward Scheduling Via Large Language Models for Robotic Skill Learning
• 2024 · ROSKA — Efficient Language-instructed Skill Acquisition via Reward-Policy Co-Evolution
• 2024 · CARD — A large language model-driven reward design framework via dynamic feedback for reinforcement learning
• 2024 · REvolve — REvolve: Reward Evolution with Large Language Models using Human Feedback
• 2023 · Eureka — Eureka: Human-Level Reward Design via Coding Large Language Models
• 2023 · Self-Refined Reward Designer — Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in Robotics

Consistency-Augmented Feedback

• 2026 · ARCO — ARCO: Adaptive Rubric with Co-Evolution for Multi-Step LLM-Based Agents
• 2026 · ECHO — No More Stale Feedback: Co-Evolving Critics for Open-World Agent Learning
• 2025 · NLAC — Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space
• 2025 · R* — R*: Efficient Reward Design via Reward Structure Evolution and Parameter Alignment Optimization with Large Language Models
• 2022 · SURF — SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning

2.3 Interaction-Space Co-Evolution

Executable World Construction

• 2026 · SEAL — SEAL: Synergistic Co-Evolution of Agents and Learning Environments
• 2026 · SimWorld Studio — SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning
• 2026 · Agent-World — Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence
• 2026 · COvolve — COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Game
• 2026 · Adaptive EnvGen — Towards Adaptive Environment Generation for Training Embodied Agents
• 2026 · DiCode — Dreaming in Code for Curriculum Learning in Open-Ended Worlds
• 2026 · DEGen — Improving Regret Approximation for Unsupervised Dynamic Environment Generation
• 2025 · Driving Scenarios ACL — Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning
• 2024 · Eurekaverse — Eurekaverse: Environment Curriculum Generation via Large Language Models
• 2024 · ADD — Adversarial Environment Design via Regret-Guided Diffusion Models
• 2024 · LLM-POET — LLM-POET: Evolving Complex Environments using Large Language Models
• 2024 · OMNI-EPIC — OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code
• 2024 · EnvGen — EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents
• 2024 · EvolutionaryAgent — Agent Alignment in Evolving Social Norms
• 2023 · MAESTRO — MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning
• 2022 · ACCEL — Evolving Curricula with Regret-Based Environment Design
• 2021 · XLand — Open-Ended Learning Leads to Generally Capable Agents
• 2020 · PAIRED — Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
• 2020 · AI Economist — The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies
• 2019 · ADR — Solving Rubik's Cube with a Robot Hand
• 2019 · POET — Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions

Model-Based World Construction

• 2026 · EnvACE — EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning
• 2026 · EvolvingWorld — EvolvingWorld: An Open-Schema Framework for Co-Evolving Role-Play Agents and World Model in Interactive Literary World
• 2026 · COMAP — COMAP: Co-Evolving World Models and Agent Policies for LLM Agents
• 2026 · PaW — Policy and World Modeling Co-Training for Language Agents
• 2026 · VLAW — VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Model
• 2026 · World-VLA-Loop — World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA Policy
• 2025 · LCDrive — Latent Chain-of-Thought World Modeling for End-to-End Driving
• 2025 · DreamGym — Scaling Agent Learning via Experience Synthesis
• 2025 · WebEvolver — WebEvolver: Enhancing Web Agent Self-Improvement with Co-evolving World Model
• 2025 · EvolvingAgent — EvolvingAgent: Curriculum Self-evolving Agent with Continual World Model for Long-Horizon Tasks

Stage 3: Meta Co-Evolution

Stage 3 makes the mechanism governing later evolution itself evolvable. Single-entity meta-evolution methods are listed as precursors; full Meta Co-Evolution additionally couples mechanism evolution to a lower-level co-evolving system.

3.1 Single-Entity Meta-Evolution Precursors

• 2026 · SIA — SIA: Self Improving AI with Harness & Weight Updates
• 2026 · AEvo — Harnessing Agentic Evolution
• 2026 · HyperAgents — Hyperagents
• 2025 · Gödel Agent — Gödel Agent: A Self-Referential Agent Framework for Recursively Self-Improvement
• 2025 · MemEvolve — MemEvolve: Meta-Evolution of Agent Memory Systems
• 2023 · PromptBreeder — Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution

3.2 Meta Co-Evolution

• 2026 · RQGM — The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators

Contributing

If we have missed a relevant paper, please contact us at qzong@cse.ust.hk. Please include its title, link, first public release date, and a brief description of what co-evolves and how the evolving units influence one another over successive rounds.

We include systems in which at least two units evolve together, with changes in each shaping the other's subsequent evolution. Static multi-agent interaction, one-off refinement, and single-entity self-evolution are outside this scope. Single-entity methods that evolve their own evolution mechanisms are included only as Stage-3 precursors.

Citation

If you find this survey or paper list useful, please cite:

@article{zong2026coevolution,
  title   = {Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design},
  author  = {Zong, Qing and Liu, Jiayu and Shen, Junhao and Tang, Zecong and Wu, Linsi and Liu, Yuxuan and Wang, Rui and Wang, Zhaowei and Wang, Weiqi and Qian, Cheng and Chen, Xiusi and Song, Yangqiu},
  year    = {2026},
  eprint  = {2608.10299},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL}
}