Awesome Code as Agent Harness Papers

This repository accompanies the survey Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems. We study the emerging role of code in agentic AI: code is no longer only a generated artifact, but increasingly serves as an executable, inspectable, and stateful harness through which agents reason, act, model environments, receive feedback, and coordinate. The repository organizes representative papers around three connected layers: Harness Interface, Harness Mechanisms, and Scaling the Harness, covering directions such as coding assistants, GUI/OS automation, scientific discovery, and embodied intelligence.

[!TIP] 👋 We welcome paper suggestions, pull requests, and collaborations on code as agent harness. Please contact us at xuyingn2@illinois.edu, kt42@illinois.edu, twei10@illinois.edu, zihaoli5@illinois.edu, and bei4@illinois.edu. We will keep updating this repository with recent work on code-centric agentic systems and harness engineering.

[!NOTE] 📚 If you find this resource useful, please cite and the repo:

@article{ning2026codeasharness,
  title   = {Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems},
  author  = {Ning, Xuying and Tieu, Katherine and Fu, Dongqi and Wei, Tianxin and Li, Zihao and Bei, Yuanchen and others},
  journal = {arXiv preprint arXiv:2605.18747},
  year    = {2026}
}

🔔 News

[2026-05] 🚀 Our survey Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems is available on arXiv. Slides and project page links will be added here once available.

📋 Table of Contents

• 🔔 News
• 📋 Table of Contents
• 🧩 Harness Interface
  • 💭 Code for Reasoning
  • 🤖 Code for Acting
  • 🌍 Code for Environment Modeling
• 🛠️ Harness Mechanisms
  • 🗺️ Planning for Code Agents
  • 🧠 Memory and Context Engineering
  • 🔧 Tool Usage for Code Agents
  • 🧪 Feedback-Guided Iterative Debugging
• 👥 Scaling the Harness: Multi-Agent Code-Centric Systems
  • 🎭 Functional Role Specialization
  • 💬 Interaction Modes
  • 🕸️ Workflow Topology
  • ⚡ Execution Feedback Integration
  • 🔄 Shared-Harness Synchronization
  • 🏛️ Shared Harness Representation
  • 🎯 Harness-State Convergence
• 🚀 Applications and Emerging Fields
  • 💻 Code Assistants
  • 🖥️ GUI / OS Agents
  • 🔬 Scientific Discovery Agents
  • 🤖 Autonomous Embodied Agents

---

🧩 Harness Interface

Code as the basic interface between a model and its task environment. Programs convert model outputs into executable, inspectable, and stateful structures: code makes reasoning executable, action programmable, and environment state inspectable.

💭 Code for Reasoning

Programs externalize internal logic into verifiable computation, allowing interpreters, symbolic solvers, execution traces, or process rewards to check and refine intermediate steps.

Program-Delegated Reasoning

Hybrid Symbolic–Neural Execution

Iterative Code-Grounded Reasoning

🤖 Code for Acting

Generated programs serve as policies, tool calls, behavior trees, or reusable skills for embodied, GUI, software, and tool-use environments.

Grounded Skill Selection

Programmatic Policy Generation

Lifelong Code-Based Agents

🌍 Code for Environment Modeling

Program states, repositories, traces, simulators, and tests represent state, dynamics, and feedback signals for agent interaction.

Structured World Representations

Execution-Trace World Modeling

Code-Grounded Evaluation Environments

🛠️ Harness Mechanisms

Once code is placed inside the agent loop, the harness must decide what to execute next, preserve useful state, expose the right tools, and convert failures into corrective actions.

🗺️ Planning for Code Agents

Planning is harness control: it structures how the agent externalizes intent into executable steps, schedules interactions with code artifacts and tools, and regulates the trajectory of reasoning, execution, and revision over time.

Linear Decomposition Planning

Structure-Grounded Planning

Search-Based Planning

Orchestration-Based Planning

🧠 Memory and Context Engineering

Memory in code-as-agent-harness systems is a state-management layer: which information stays in the active context, which is compacted, and which is offloaded to durable external storage.

Working Memory

Semantic Memory

Experiential Memory

Long-Term Memory

Multi-Agent Memory

🔧 Tool Usage for Code Agents

Tool usage is the action and observation layer of the code-agent harness: agents search repositories, inspect files, edit code, run commands, execute tests, call APIs, and verify intermediate results — all under typed schemas, sandboxes, and lifecycle hooks.

Function-Oriented Tool Use

Environment-Interaction Tool Use

Verification-Driven Tool Use

Workflow-Orchestration Tool Use

🧪 Feedback-Guided Iterative Debugging

Iterative debugging closes the harness loop: development environments expose feedback (compiler diagnostics, runtime errors, tests, critique), and the agent transforms these signals into diagnosis, revision, and progressively better debugging behavior.

Development Environments for Agentic Coding

Contextual Environments for Repository-Aware Generation

Interactive Environments for Human–LLM Collaboration

Execution and Validation Environments

Engineering Platforms for Deployment and Workflow Integration

Feedback Mechanisms for Iterative Debugging

Compilation and Static-Analysis Feedback

Runtime Error and Exception Feedback

Test-Based Execution Feedback

Critique-Driven Feedback (Human or Auxiliary Agents)

Feedback-Driven Debugging and Self-Improvement

👥 Scaling the Harness: Multi-Agent Code-Centric Systems

When multiple agents operate over code, the harness must coordinate roles, share intermediate artifacts, maintain common state, and verify collective progress through repositories, tests, traces, and structured workflows.

🎭 Functional Role Specialization

Distinct agents own slices of the shared code harness — synthesis, understanding, verification, execution, and planning.

Program Synthesis Agents

Program Understanding Agents

Verification Agents

Execution Agents

Planning Agents

💬 Interaction Modes

Code-centric multi-agent interaction is artifact-mediated: agents observe and modify shared code, and grounding comes from the objective state exposed by execution.

Collaborative Synthesis

Critique and Repair

Adversarial Validation

Reasoning Debate

🕸️ Workflow Topology

Topology of agent interaction (chain, cyclic, hierarchical, star, adaptive) is one of the most consequential design decisions in multi-agent code generation.

Pre-Defined Heuristic Topologies (Waterfall / Iterative / Hierarchical / Star)

Objective-Driven and Adaptive Topologies

⚡ Execution Feedback Integration

Code is uniquely executable, producing objective oracle signals that anchor multi-agent coordination.

Compiler and Syntax Feedback

Test Pass/Fail Signals

Fuzzer Crash Traces

Static Analysis Warnings

Performance Profiling Results

Fine-Grained Simulation Feedback

🔄 Shared-Harness Synchronization

How multi-agent systems maintain a consistent shared view of program state.

Shared Blackboard

Parallel Branches with Merge

Structured Context Scheduling

Hierarchical Memory

Agent Pool Scaling

🏛️ Shared Harness Representation

Four levels of formalization for the shared substrate: implicit/file-only, repository-based, execution-based, and blackboard.

Implicit / File-Only Representation

Repository-Based Representation

Execution-Based Representation

Blackboard / Shared-State Representation

🎯 Harness-State Convergence

How a multi-agent code system decides the shared harness has reached an acceptable final state.

Correctness Convergence (Test-Gated)

Security Convergence

Performance Convergence

Score-Based Convergence

Consensus Convergence

Implicit Convergence

🚀 Applications and Emerging Fields

Code-centric agentic systems become operational in tangible domains where code defines observable state, executable actions, persistent memory, and feedback signals.

💻 Code Assistants

Repositories, tests, issue threads, and development tools form a persistent program world; assistants act over it as code-centric agents.

The Repository as a Persistent Program World

Agent Harnesses as Executable Development Interfaces

Execution Feedback as Grounded Verification

Memory and Context Management at Repository Scale

Developer Intent and Project Conventions as Latent State

From Inline Completion to Autonomous SWE Agents

From Patch Generation to Software Lifecycle Participation

Multi-Agent Code Assistance and Shared Repositories

The Harness as a Distillation Surface

Open Challenges for Code-Assistant Harnesses

🖥️ GUI / OS Agents

GUI/OS environments are program worlds in the most literal sense: every observation is rendered code, and every action is a call into another piece of code.

GUI/OS as a Partially Observable Program World

Unifying Perception, Action, and Evaluation Through Code

Memory as Persistent Program State

UI Simulators and Sandboxes as Executable Dynamics

From Simulation to Production: Executable Feedback Loops

🤖 Autonomous Embodied Agents

Code grounds embodied actions in physical feasibility, accumulates reusable skills as memory, and supports auditable real-world deployment.

Agent Harness for Grounded and Verifiable Embodied Actions

Reusable Skills as Embodied Memory

Coordinated and Auditable Real-World Deployment

🔬 Scientific Discovery Agents

Hypotheses are encoded as differential equations or generative models; protocols as XDL or Opentrons scripts; analyses as Jupyter notebooks. Code carries scientific reasoning, scientific action, and the scientific environment itself.

Scientific Discovery as a Partially Observable Program World

Unifying Ideation, Experimentation, Analysis, and Communication

Memory as Persistent Program State

Simulators as Executable Dynamics

Self-Driving Labs as Executable Feedback Loops

Toward Agentic and Instruction-Following Science

🧠 Agent Personalization

As recommendation moves from static prediction toward interactive agents, personalization systems must reason over latent and evolving user preferences through structured, editable preference states and executable feedback pipelines.

From Static Recommendation to Interactive Personalization

Preference State as an Editable Artifact

Feedback as Policy Adaptation

Controllable and Instruction-Following Personalization

---

✨ Acknowledgements

We thank the broader community for the contributions surveyed here. If your paper should be added or moved, please open a pull request or issue.

📄 License

This repository is released under the MIT License.