PaperlayerView exact revision on arXiv

Source-linked AI summary

Code as Agent Harness

Xuying Ning, Katherine Tieu, Dongqi Fu, Tianxin Wei, Zihao Li, Yuanchen Bei, Jiaru Zou, Mengting Ai, Zhining Liu, Ting-Wei Li, Lingjie Chen, Yanjun Zhao, Ke Yang, Bingxuan Li, Cheng Qian, Gaotang Li, Xiao Lin, Zhichen Zeng, Ruizhong Qiu, Sirui Chen, Yifan Sun, Xiyuan Yang, Ruida Wang, Rui Pan, Chenyuan Yang, Dylan Zhang, Liri Fang, Zikun Cui, Yang Cao, Pan Chen, Dorothy Sun, Ren Chen, Mahesh Srinivasan, Nipun Mathur, Yinglong Xia, Hong Li, Hong Yan, Pan Lu, Lingming Zhang, Tong Zhang, Hanghang Tong, Jingrui He

arXiv:2605.18747v1cs.CLcs.AI

TL;DR

Most agentic systems lack a formal shared code substrate, limiting a unified account of how code supports reasoning, action, and state. This survey frames code as an agent harness, synthesizes interfaces and mechanisms across applications, and identifies coordination, verification, evaluation, and safety challenges.

  • Problem

    Most surveyed multi-agent systems lack a formal shared harness substrate, relying instead on agents to reason about code quality through natural language.

  • Method

    The survey frames code as agent harness and organizes prior work into interfaces, mechanisms, and scaling, including tool use and multi-agent coordination.

  • Results

    The synthesis connects executable code-based harnesses to representative methods and applications across coding, automation, embodied agents, science, DevOps, and enterprise workflows.

  • Takeaways & Limitations

    Code can serve as an executable, verifiable, and stateful substrate for agent systems, while shared-state coordination remains an open design problem.

  • Takeaways & Limitations

    Implicit shared-state representations prevent agents from reliably detecting divergence between their internal understanding and the true program state.

Abstract

from arXiv · show

Recent large language models (LLMs) have demonstrated strong capabilities in understanding and generating code, from competitive programming to repository-level software engineering. In emerging agentic systems, code is no longer only a target output. It increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification. We frame this shift through the lens of agent harnesses and introduce code as agent harness: a unified view that centers code as the basis for agent infrastructure. To systematically study this perspective, we organize the survey around three connected layers. First, we study the harness interface, where code connects agents to reasoning, action, and environment modeling. Second, we examine harness mechanisms: planning, memory, and tool use for long-horizon execution, together with feedback-driven control and optimization that make harness reliable and adaptive. Third, we discuss scaling the harness from single-agent systems to multi-agent settings, where shared code artifacts support multi-agent coordination, review, and verification. Across these layers, we summarize representative methods and practical applications of code as agent harness, spanning coding assistants, GUI/OS automation, embodied agents, scientific discovery, personalization and recommendation, DevOps, and enterprise workflows. We further outline open challenges for harness engineering, including evaluation beyond final task success, verification under incomplete feedback, regression-free harness improvement, consistent shared state across multiple agents, human oversight for safety-critical actions, and extensions to multimodal environments. By centering code as the harness of agentic AI, this survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.

1. Introduction

Recent LLMs show strong code understanding and generation across competitive programming and repository-level software engineering. In agentic systems, code is expanding beyond a generated artifact into infrastructure that connects inference, actions, and execution-based verification.

  • Motivation: LLMs demonstrate strong code understanding and generation across competitive programming and repository-level software engineering.The cited capabilities span tasks from competitive programming to repository-level software engineering.
  • Motivation: Code is increasingly used in agentic systems as infrastructure rather than only as a target artifact.This marks a shift in code’s role within agentic systems.
  • Harness Functions: As an agent harness, code can externalize inference, support intermediate reasoning, interact with tools, and verify behavior through execution.The harness connects reasoning and verification with tool interaction, code-to-action interfaces, and embodied robots.

Code for Reasoning

Code serves as an executable, inspectable substrate for reasoning: it externalizes intermediate computation and enables interpreters, symbolic solvers, execution traces, and process rewards to check and refine agent reasoning.

  • Code for Reasoning: Code externalizes intermediate computation into executable programs that support verifiable reasoning.Program-aided reasoning uses executable code, while code-centered systems organize reasoning around inspectable programs.
  • Code for Reasoning: Code is used not only to produce solutions, but also to execute reasoning, ground actions, maintain state, and expose feedback.The survey contrasts this harness role with views that treat code only as an LLM’s end product.
  • Code for Reasoning: Interpreters, symbolic solvers, execution traces, and process rewards check and refine reasoning.These mechanisms make intermediate reasoning executable and inspectable within the agent loop.

2. Harness Interface: Code for Reasoning, Acting, and Environment Modeling · 2.1. Code for Reasoning · 2.2. Code for Acting

The survey presents code as the interface that turns stateless language models into executable, stateful, and verifiable agents. It develops this view through code for reasoning and code for acting, connecting model intent to computation, feedback, and environment-constrained operations.

  • 2. Harness Interface: Code for Reasoning, Acting, and Environment Modeling: Code transforms a stateless language model into a functional agent through external execution, persistent state, and verifiable feedback.The harness interface uses executable, inspectable, and stateful artifacts to connect models with task environments.
  • 2. Harness Interface: Code for Reasoning, Acting, and Environment Modeling: The interface assigns code three roles: externalizing reasoning, translating intent into actions, and representing environment state, dynamics, and feedback.These roles span symbolic computation, embodied or software operations, and program states, repositories, simulators, tests, and logs.
  • 2.1. Code for Reasoning: Code-for-reasoning separates high-level model procedures from low-level computation executed and evaluated by runtimes, interpreters, solvers, or verification modules.The harness observes runtime behavior, stores intermediate states, and feeds execution results into subsequent reasoning.
  • 2.1. Code for Reasoning: Execution artifacts—including traces, variable states, control-flow structures, and function-level tests—become reusable intermediate signals that the harness verifies, scores, and feeds back.The survey organizes this interface into program-delegated reasoning, formal verification and symbolic reasoning, and iterative code-grounded reasoning.
  • 2.2. Code for Acting: Code-for-acting converts model outputs into executable tool calls, robot-control policies, GUI actions, or software commands that produce grounded environmental effects.Unlike reasoning, action execution must handle partial observability, dynamic evolution, embodiment limits, APIs, environment dynamics, and safety requirements.
  • 2.2. Code for Acting: Executable action code interfaces with perception, feasibility models, planners, controllers, parsers, accessibility structures, backend APIs, and safety layers rather than replacing them.The harness can execute, monitor, validate, reuse, and refine actions through interaction feedback.
  • 2.2. Code for Acting: Action interfaces range from reusable skills and generated policies to persistent skill memory, GUI/API protocols, and validation harnesses that filter invalid actions before execution.Grounded skill selection emphasizes invoking, composing, retrying, relabeling, and persistently refining executable capabilities under environmental constraints.
  • 2.2. Code for Acting: Programmatic policy generation and lifelong code-based agents use executable programs as control logic and persistent memory for perception-conditioned behavior, feedback loops, reusable skills, traces, and environment knowledge.These approaches support multimodal policy generation, long-horizon interaction, capability accumulation, and skill reuse or recomposition.

2.3. Code for Environment

Code for environment makes executable programs the interface through which agents explicitly represent, inspect, modify, and verify environment state. This supports structured world modeling, execution-based evaluation, and scalable harnesses with runnable dynamics, feedback channels, and verification oracles.

  • Code for Environment: Code-for-environment materializes environment structure and dynamics as executable artifacts, enabling agents to store, inspect, execute, and modify state during interaction.Artifacts include simulators, repositories, tests, execution traces, logs, and state-transition programs.
  • Structured World Representations: Structured world representations encode world state, object relations, spatial layouts, and interaction dynamics as programmatic structures rather than only latent embeddings or textual descriptions.These structures provide interfaces that agents can execute, inspect, and manipulate.
  • Execution-Trace World Modeling: Execution-trace world models treat runtime transitions in executable interaction traces as representations of environment dynamics, enabling refinement and coordination across tasks and knowledge sources.The cited approaches use reinforcement learning and align, fuse, or coordinate multiple executable world models.
  • Code-Grounded Evaluation Environments: Code-grounded evaluation exposes runtime state transitions, execution feedback, and verifiable interaction outcomes, extending assessment beyond static textual outputs.InterCode, SWE-bench, AgentBench, and subsequent benchmarks apply executable evaluation across coding, software repositories, embodied tasks, and digital tasks.
  • Executable Harness Artifacts: Executable environments can serve as synthesizable, scalable, and programmatically validated harness artifacts for long-horizon agents, providing runnable state, transition dynamics, feedback channels, and verification oracles.This direction is illustrated by SWE-smith’s construction of repository-level tasks and execution environments.

3. Harness Mechanisms: Planning, Memory, Tool Use, Control, and Optimization

Harness mechanisms make code-harnessed agents reliable beyond single-step generation by coordinating model judgment, mutable task state, and human-designed infrastructure. The section organizes these mechanisms around planning, memory and context engineering, tool usage, and feedback-guided control.

  • Harness Mechanisms: Harness mechanisms coordinate model judgment, mutable task state, and human-designed infrastructure to support reliable execution beyond a single generation step.The model decomposes goals, selects actions, interprets feedback, and decides when to revise, while mutable state records task information.
  • Harness Mechanisms: The section reviews five interacting mechanism categories spanning planning, memory and context engineering, tool usage, harness control, and optimization.Planning organizes long-horizon execution; memory manages mutable state; tool usage connects agents to executable interfaces; and control and optimization support reliability and adaptation.
  • Planning: Planning organizes long-horizon task execution through goal decompositions, structural constraints, search trajectories, and workflow-level orchestration.These mechanisms externalize aspects of the task so execution can extend beyond a single generation step.
  • Memory and Context Engineering: Memory and context engineering manage mutable state across long interactions by preserving working context, retrieving repository evidence, storing reusable experience, and supporting shared histories.They also offload state beyond the active context window.
  • Tool Use and Control: Tool usage connects agents to governed executable interfaces, while the Plan-Execute-Verify loop frames feedback-guided debugging as a broader control process.Interfaces include APIs, repositories, terminals, sandboxes, verification tools, and workflow orchestrators; plans form contracts over intended changes and execution applies them in sandboxed, permissioned environments.

3.1. Planning for Agent Harness

Planning is a core agent-harness control layer that converts intent into executable steps, structures interaction with code and tools, and regulates reasoning and execution. It has evolved from linear decomposition toward structured grounding, inference-time search, and orchestration for complex software-engineering workflows.

  • Planning for Agent Harness: Planning externalizes intent into executable steps, schedules interactions with code artifacts and tools, and regulates the agent’s reasoning and execution trajectory.It is especially important when tasks involve large action spaces, sparse feedback, and deeply interdependent subproblems.
  • Planning for Agent Harness: Early systems used linear natural-language decomposition, while later harnesses grounded plans in repository structure or external knowledge to constrain action generation.ReAct interleaves thoughts, actions, and observations; WebAgent derives executable Python actions from successive web sub-instructions; KareCoder uses an external knowledge library.
  • Planning for Agent Harness: Structured representations such as dependency, repository, circuit, and knowledge graphs expose entities and relations that guide subtask generation, revision, and verification.These representations serve as harness scaffolds rather than relying solely on free-form natural-language prompts.
  • Planning for Agent Harness: Search-based planning explores and evaluates multiple candidate paths, while orchestration coordinates roles, stages, feedback routing, candidate pruning, and iterative refinement.Together, these mechanisms determine what the agent should do next while keeping decisions constrained, interpretable, and coordinated across multi-stage workflows.

3.2. Memory and Context Engineering for Agent Harness

Memory and context engineering form a unified state-management layer for code-agent harnesses, enabling agents to preserve, retrieve, compress, and reuse information across long-horizon software-engineering workflows. The section organizes this layer around working, semantic, and experiential memory, together with governance, context offloading, and multi-agent synchronization.

  • Motivation: Code agents need memory because software-engineering tasks require sustained, interdependent steps across requirements, localization, editing, testing, debugging, and verification.Without stateful support, agents risk losing critical clues, repeating completed analyses, and breaking consistency across later modifications.
  • Memory Types: Working memory maintains the current coding trajectory by prioritizing next-action information within limited context budgets.Typical representations include structured prompts, state summaries, failed-test records, file lists, and stack information; CodeMem uses budgeted memory slots to stabilize multi-step edits.
  • Memory Types: Semantic memory retrieves repository-specific evidence, while experiential memory preserves reusable repair trajectories, failure cases, debugging records, and strategy patterns across tasks.Together, these memories extend the harness beyond conversation history toward queryable codebase evidence and cross-task experience transfer.
  • Memory Governance: Memory governance determines when to write, compress, retrieve, and control long-term state as trajectories grow and compression can distort evidence or cause drift.Systems such as MemGPT and MemoryOS shift attention from storage capacity toward retrieval planning and memory control.
  • Context Engineering and Multi-Agent Memory: Context compaction and state offloading regulate the boundary between active model context and durable task state, while multi-agent memory supports information sharing, intention passing, and consistency across roles.These mechanisms manage high-volume artifacts such as logs, traces, diffs, test outputs, and intermediate plans, and extend memory from individual state to a shared harness.

3.3. Tool Use for Agent Harness

Tool use forms the action-and-observation layer of code-agent harnesses, expanding agents’ action spaces while exposing external feedback for executable, inspectable behavior. The literature organizes tools into function-oriented, environment-interaction, verification-driven, and workflow-orchestration mechanisms.

  • Harness role of tool use: Tools let agents search repositories, edit code, execute tests, call APIs, query documentation, and verify intermediate results within the agent loop.This makes code-agent harnesses executable and inspectable by connecting generation to actions and observations.
  • Function-oriented tool use: Function-oriented tools ground implementation choices in APIs, libraries, documentation, and external coding utilities when models lack programming knowledge.ToolCoder addresses API hallucinations and inappropriate function selection by integrating API search into code generation.
  • Environment-interaction tool use: Environment-interaction tools provide interfaces for acting over repositories, development artifacts, and execution environments rather than merely retrieving missing functions.These approaches target repository-level software engineering tasks involving broader environment operation.
  • Verification-driven tool use: Verification-driven tools use tests, execution results, compiler errors, runtime traces, type checkers, static analyzers, and verifier feedback as structured observations for iterative improvement.AgentCoder closes the loop with programmer, test designer, and test executor agents that generate, test, execute, and refine code.
  • Workflow-orchestration tool use: Workflow orchestration coordinates retrieval, bug localization, file modification, testing, failure inspection, memory updates, approvals, permissions, and repeated execution across long-horizon tasks.The harness must determine when tools run, with what permissions and context, and how results update harness state.

3.4. Harness Control through the Plan, Execute, and Verify Loop

The Plan–Execute–Verify loop turns agent intentions into bounded, observable, and revisable state transitions. It externalizes plans, executes changes in sandboxed and permissioned environments, and verifies outcomes with deterministic sensors and human-review gates.

  • Harness Control through the Plan, Execute, and Verify Loop: PEV externalizes intended changes and validation criteria, executes them in a sandboxed environment, and verifies resulting state through deterministic sensors and human-review gates.The loop provides a unified control structure for planning, action, and state validation.
  • Planning: Planning creates an explicit transition contract covering implementation steps, relevant files, invariants, validation commands, rollback points, and risky operations.These artifacts constrain the subsequent action space rather than remaining unobserved reasoning traces.
  • Execution: Execution realizes plans as bounded, observable state transitions within an isolated substrate that includes filesystems, dependencies, runtimes, interfaces, and resource boundaries.Stable execution substrates make verification signals interpretable and reduce failures caused by environment drift.
  • Execution: Permission tiers separate read-only inspection, sandbox editing, and full-access operations such as network access, credentials, deployment, publishing, and destructive filesystem changes.This tiered model governs actions according to their risk.
  • Verification: Verification combines static and runtime signals to determine whether the state is acceptable and whether the harness should diagnose, retrieve context, patch, delegate, or abandon a branch.The loop treats repair, reflection, and termination as consequences of verification, while human gates preserve accountability at safety boundaries.

3.5. Agentic Harness Engineering for Adaptive Harness Optimization

Agentic Harness Engineering treats the software substrate that operates coding agents as an object of measurement, diagnosis, and revision. Deep telemetry and Evolution Agents make harness improvement iterative and measurable, while verification and human approval govern changes.

  • Agentic Harness Engineering: Agentic Harness Engineering analyzes and revises the operating environment that turns a language model into a coding agent.Its scope includes tool schemas, planning artifacts, memory policies, retrieval strategies, sandbox configuration, verification sensors, and permissions.
  • Telemetry-Driven Diagnosis: Deep telemetry connects model decisions, harness actions, environment states, and outcomes, exposing failures across prompts, tools, memory, sandboxes, validators, permissions, and workflows.Relevant traces include tool arguments, permission requests, edited files, sandbox snapshots, command outputs, tests, stack traces, branch decisions, human interventions, and outcomes.
  • Telemetry-Driven Diagnosis: Telemetry turns harness revision from anecdotal debugging into comparative diagnosis by revealing wasted costs, unproductive tool choices, repeated failure loops, and recurring execution problems.Examples include retrieval or reflection stages that consume budget without improving verification outcomes, hallucinated APIs, flaky sandboxes, and over-permissive tool calls.
  • Evolution Agents: An Evolution Agent uses trajectory telemetry to diagnose harness-specific failure modes and propose revisions to prompts, retrieval, tools, validators, permissions, and workflow topology.It edits the operating conditions for later task agents rather than the target repository.
  • Governance: Harness mutations require sandboxed evaluation, fixed regression suites, auditable rationales, and stronger governance for changes affecting permissions, network access, credentials, deployment, or human review.The discussion frames harness design as an iterative, measurable engineering process governed by verification and human approval.

4. Scaling the Harness: Multi-Agent Orchestration over Code

This section argues that multi-agent systems can scale coding harnesses beyond single-agent limitations by coordinating specialized agents over shared code-centric substrates. It surveys this direction and advances shared code-centric harnesses as a foundation for AI-agent collaboration.

  • Motivation: Single agents face context-window constraints that limit simultaneous access to an entire codebase, interaction history, and execution trace.These constraints motivate scaling coding harnesses beyond a single agent.
  • Motivation: Single-agent coding is inefficient when planning, synthesis, testing, review, and debugging require specialized capabilities.The section identifies specialization requirements as a fundamental limitation of single-agent systems.
  • Section scope and contribution: The section surveys the rapidly growing use of multi-agent systems to scale coding harnesses and proposes shared code-centric harness substrates for AI agents.The proposed position centers shared code-centric infrastructure as the basis for multi-agent orchestration over code.

4.1. Improved Coding Support through Multi-agent Collaboration

Multi-agent coding harnesses improve complex software development by distributing planning, synthesis, execution, and verification across specialized agents coordinated through shared artifacts and feedback. The surveyed systems increasingly adapt their agent pools and workflow topologies in response to task complexity and execution outcomes.

  • Role Specialization: Multi-agent systems decompose coding harnesses into specialized, coordinated components that share code artifacts and feedback signals across software-development tasks.This division distributes planning, synthesis, execution, and verification rather than combining them in one agent loop.
  • Role Specialization: Common roles include program synthesis, program understanding, verification, execution, and planning agents, each handling a distinct slice of the shared code harness.Execution may use deterministic scripts or shells, while planning agents decompose tasks and assign subtasks to synthesis agents.
  • Adaptive Topologies: EvoMAC uniquely adapts the harness itself: a Gradient Agent attributes failures from execution logs, and an Updating Agent revises prompts and restructures the workflow DAG.More broadly, FlowReasoner and BOAD treat multi-agent organization as an object to generate, search, or optimize per task.
  • Interaction Modes: Code-centric collaboration uses shared artifacts and execution results as coordination channels, including code, APIs, files, diffs, tests, logs, schemas, blackboards, and workflow states.Collaborative synthesis pairs agents such as Navigator and Driver, while critique-and-repair has a verifier provide feedback that guides synthesis revisions.
  • Interaction Modes: Critique-and-repair is the dominant interaction mode, whereas collaborative synthesis and adversarial validation provide complementary ways to construct or stress-test code.Adversarial validation actively seeks failures through inputs such as crash-inducing fuzzing seeds rather than passively reviewing artifacts.
  • Adaptive Topologies: Dynamic agent-pool scaling grows the number of agents with task complexity while keeping each agent’s context bounded through hierarchical decomposition.SoA uses Mother agents to decide how many subfunctions to create and spawns corresponding Child agents at runtime.

4.2. Execution Feedback and Shared-Harness Synchronization

Code-centric multi-agent systems use executable shared harnesses to generate objective feedback and coordinate agents around a consistent program state. Execution signals range from structural checks and tests to localized simulation feedback, while synchronization uses shared state, context scheduling, and scalable agent organization.

  • Execution Feedback and Shared-Harness Synchronization: Executable shared harnesses provide objective oracle signals while supporting consistent program-state views across agents.The section organizes synchronization around execution-feedback types and shared-state coordination.
  • Execution Feedback: Test pass/fail signals dominate execution feedback, while compiler checks, static warnings, crash traces, and profiling provide complementary diagnostics.AgentCoder iterates until all tests pass or reaches a 5-iteration budget; QualityFlow’s Imagined Execution exceeds 98% precision and recall on MBPP.
  • Execution Feedback: MAGE records signal values at every clock edge and provides waveform windows around the first failing cycle, enabling sub-test-granularity repair.Its State Checkpoint mechanism delivers the Debug Agent targeted simulation feedback rather than only pass/fail outcomes.
  • Shared-Harness Synchronization: Sequential handoffs are common but can create invisible state divergence when agents modify a codebase in parallel or iteratively.In sequential pipelines, program state is represented only by the latest artifact, which suits simple linear workflows but limits coordination in multi-agent settings.
  • Shared-Harness Synchronization: Shared blackboards, structured context scheduling, and agent-pool scaling coordinate shared harness state by controlling persistence, visibility, and context load.L2MAC persists and revises file state while resetting contexts around targeted summaries; SoA bounds individual context by distributing task state across more agents.

4.3. Position: The Shared Code-Centric Harness Substrate

The section positions a shared code-centric harness substrate as the foundation for scalable multi-agent intelligence, defining it as the evolving program environment that agents collectively query, modify, and evaluate. It organizes existing systems by substrate formalization and argues that executable program behavior enables more objective convergence than conversational agreement alone.

  • The Central Gap: Most multi-agent systems remain in the implicit/file-only category, lacking formal persistent representations of shared harness state across agent iterations.The section identifies this absence as the central gap and argues that a formal substrate is feasible and necessary for robust, scalable multi-agent intelligence.
  • Shared Harness Substrate: A shared harness substrate comprises the artifacts, execution contexts, and quality signals that agents collectively evolve while producing, revising, and evaluating code.The section distinguishes four levels of formalization for representing this shared program environment.
  • Levels of Formalization: Existing systems represent shared code state implicitly through files, repositories, execution behavior, or explicit blackboards and shared-state structures.File-only systems reconstruct state from conversational history, repository systems add navigable structure and dependencies, execution-based systems encode behavior, and blackboards provide persistent queryable state.
  • Convergence: Executable substrates support convergence criteria grounded in objective behavior, including passing tests, eliminating vulnerabilities and crashes, and satisfying runtime or memory thresholds.These criteria replace reliance on conversational agreement alone with observable program outcomes.
  • Convergence: Implicit convergence remains prevalent, relying on fixed stages, iteration budgets, or repeated outputs rather than objective quality signals.The section links this prevalence directly to the absence of formal shared substrates and principled representations of program state.

4.4. Patterns and Trends

The surveyed systems reveal recurring coordination bottlenecks around implicit shared state, partial code-mediated channels, and increasingly complex topologies. More robust harnesses combine formal repository and execution representations with context management and execution-grounded verification.

  • Implicit shared state: Most surveyed systems lack explicit shared-harness representations and reconstruct code state implicitly from conversational history at each invocation.This approach works for function-level tasks with simple, nonfragmented program state.
  • Coordination channels: Code-mediated channels improve coordination but remain partial, trading off state fidelity, latency, and scope across files, APIs, diffs, tests, logs, schemas, blackboards, and workflow states.These channels encode, transmit, and reconstruct task state rather than eliminating coordination bottlenecks.
  • Execution feedback: Execution-grounded systems use runtime behavior as ground truth, detecting crashes, resource exhaustion, boundary errors, and performance regressions that linguistic simulation may miss.Linguistic reasoning can still suffice for many correctable bugs, motivating a hybrid fast-path and verification-oracle design.
  • Shared-harness representations: Repository-based representations capture code structure and dependencies, while execution-based representations capture runtime behavior, state evolution, and emergent failures.The surveyed systems therefore expose two conceptually orthogonal views of the shared harness.
  • Topology and state formality: Systems with formal shared substrates tend to use simpler topologies, whereas implicit-state systems adopt increasingly complex topology patterns as structural workarounds.L2MAC pairs a persistent file store and explicit context scheduling with a simple sequential chain and sophisticated state management.
  • Context and specialization: Sophisticated context-management mechanisms arise as a tax of implicit shared state, helping agents maintain coherent views of codebases too large for one context window.Examples include L2MAC’s Control Unit, MetaGPT’s publish-subscribe pool, SoA’s agent-pool scaling, and Cogito’s three-tier memory.

5. Emerging Fields and Open Problems

The section examines how code as an agent harness appears across concrete application domains and identifies the open problems these applications expose. Across these domains, code functions as an operational substrate for agent state, action, memory, feedback, and governance.

  • Code as an agent harness is examined across coding assistants, GUI/OS agents, scientific discovery, personalization, and embodied agents.
  • Across these domains, code serves not only as model output but also as an operational substrate for state representation and action execution.
  • The section uses these application domains to expose open problems surrounding memory, feedback, and governance in agent harnesses.

5.1. Emerging Fields and Tangible Applications

Code-centric agent harnesses are already driving tangible applications in software engineering, GUI/OS automation, and scientific discovery. Across these domains, executable code connects perception, action, evaluation, simulation, and persistent workflows.

  • Code Assistants: Code assistants operate in persistent engineering workflows, using execution and test feedback for autonomous repair, review, CI/CD, and production issue resolution.AlphaCodium shows that test-driven flow engineering improves competitive-programming performance over single-shot prompting.
  • Code Assistants: Production harnesses increasingly serve as training-data sources, with real usage traces tightening the loop between deployed agent behavior and model updates.The passage identifies Cursor’s Composer as using continuous online reinforcement learning on real Cursor usage traces.
  • GUI/OS Automation: GUI/OS agents treat environments as partially observable program worlds, with code defining observations, executable actions, evaluation, and increasingly the world model.Agents can emit Python or JavaScript snippets for UI control, while evaluator scripts inspect post-action system state.
  • GUI/OS Automation: GUI benchmarks operationalize executable evaluation across browsers, operating systems, mobile devices, and enterprise tools, including 116 AndroidWorld tasks and 23,150 ServiceNow instances.Spider2-V extends OSWorld to BigQuery, dbt, and Airbyte pipelines; BrowserGym and WorkArena provide a common Gym-style API.
  • Scientific Discovery: Scientific agents use executable simulators and laboratory APIs to roll out experiments, achieving over 87% task success on six chemistry benchmarks and synthesizing 41 novel compounds from 58 targets in 17 days.El Agente Q wraps computational chemistry tools as callable functions, while Berkeley’s A-Lab combines learned recipes with autonomous robotics.

5.2. Open Problems

The open problems of code-as-harness systems concern evaluating and verifying the complete execution loop, evolving harnesses reliably, coordinating shared state, compressing multimodal context, and governing long-horizon autonomy. Progress requires executable, inspectable, stateful, verifiable, and governed systems.

  • Evaluation: Harness evaluation must measure the operational substrate—not only end-task success—including reliability, feedback quality, context sustainability, safety, coordination, reproducibility, and oracle adequacy.Suggested dimensions include trajectory efficiency, verification strength, recovery ability, state consistency, safety compliance, and replayability.
  • Verification: Execution feedback can create false correctness when tests, analyzers, GUI checkers, or scientific scripts fail to capture the intended task.Future harnesses should compose verification artifacts with explicit scope, treating verification as an evolving, inspectable contract rather than a single terminal gate.
  • Evolution and coordination: Harnesses need reliable evolution and shared-state coordination as fixed designs become suboptimal and multiple agents modify overlapping artifacts.The shared substrate must support synchronization across planners, coders, testers, reviewers, security agents, and humans.
  • Governance and harness engineering: Future harnesses should be executable, inspectable, stateful, and governed through code, tools, tests, provenance, durable human feedback, permissions, verification, and accountability.Human approvals, rejections, policy exceptions, and reviewer corrections should become auditable harness state transitions that update rules and future behavior.
  • Multimodal environments: Multimodal harnesses need compression mechanisms that preserve task-relevant visual evidence rather than merely reducing token cost.Screenshots and embodied trajectories may contain many irrelevant elements or frames while only a few reveal task-critical relations or failure causes.
Loading 2605.18747v1…