Code as Agent Harness

Conceptual framing

• Code as agent harness treats agent-created code artifacts as operational tools for reasoning, acting, maintaining state, and adapting through execution feedback, alongside model capabilities and system infrastructure.The artifacts can be created, executed, observed, revised, persisted, and shared within task loops.Source: §1 · pp. 2–3

Harness Interface

• The harness interface makes reasoning executable, actions programmable, and environment state inspectable through code.The survey covers code for reasoning, acting, and environment modeling, including programs, policies, repositories, traces, simulators, and tests.Source: §1 · p. 3

Harness Mechanisms

• Harness mechanisms sustain long-horizon execution by combining planning, memory, tool use, and feedback-driven control and optimization.They organize execution, preserve useful state, expand the action space, and use failures and feedback to drive revision.Source: §1 · p. 3

Scaling the Harness

• In multi-agent systems, shared repositories, tests, traces, and structured artifacts enable role coordination, common-state management, review, and collective verification.The survey includes centralized, distributed, and streaming collaboration across programming, repair, debate, and other workflows.Source: §1 · p. 4

Applications and open challenges

• The survey connects its taxonomy to coding assistance, GUI and OS automation, scientific discovery, and personalization and embodied control, while identifying challenges in evaluation, verification, regression, shared state, oversight, and multimodal environments.The application examples include repository patches and tests, executable interface commands, hypothesis-testing pipelines, and revisable policies and skill libraries.Source: §1 · p. 4

2. Harness Interface: Code for Reasoning, Acting, and Environment Modeling

• Code serves as an agent harness interface through executable, inspectable, and persistent artifacts, while keeping perception, physical state, human intent, and latent reasoning distinct from code.The survey includes programs, scripts, specifications, tests, repositories, simulators, and execution traces within its scope when executable systems produce or consume them.Source: §2 · p. 7

2.1.1. Program-Delegated Reasoning

• Program-delegated reasoning lets models propose executable procedures while runtimes perform computation, expose traces, and return results for later reasoning.Methods including Program-of-Thoughts [6], POET [67], and MathCoder [68] develop this approach; CodeI/O uses executable verification to expose procedural reasoning primitives.Source: §2.1 · pp. 7–8, §2.1.1 · p. 8

2.1.2. Formal Verification and Symbolic Reasoning Interfaces; 2.1.3. Iterative Code-Grounded Reasoning

• Formal and symbolic interfaces make intermediate reasoning checkable, while iterative code-grounded methods use execution feedback to refine reasoning trajectories.Proof assistants such as Lean, Isabelle, and Coq support checked derivations; methods including CodePRM, CodeRL [100], and R1-Code-Interpreter use runtime feedback, optimization, or interactive execution.Source: §2.1.2 · pp. 9–10, §2.1.3 · p. 10

2.2. Code for Acting

• Code for acting turns model intent into executable operations, but must ground actions in environment constraints and account for partial observability, delayed feedback, and silent failures.Action code mediates perception, grounding, planning, controllers, APIs, and safety layers; AutoHarness [109] synthesizes a harness that filters invalid actions before execution.Source: §2.2 · pp. 10–12

2.2.1. Grounded Skill Selection; 2.2.3. Lifelong Code-Based Agents

• Grounded skill interfaces select, compose, or expand reusable executable capabilities, while lifelong agents retain and refine skills and interaction knowledge across tasks.SayCan [9] selects actions using semantic relevance and embodiment feasibility; LYRA [121] converts human corrections into reusable skills and retrieval-augmented memory.Source: §2.2.1 · pp. 12–13, §2.2.3 · p. 13

2.2.2. Programmatic Policy Generation

• Programmatic policy generation materializes control logic, perception-conditioned branches, feedback loops, and API interactions as executable programs for embodied and multi-agent systems.Systems such as Code-BT compile generated programs into behavior-tree controllers, while ALRM [115] and RACAS [120] integrate code generation, execution, monitoring, and interaction in closed-loop architectures.Source: §2.2.2 · p. 13

2.3. Code for Environment; 2.3.1. Structured World Representations; 2.3.2. Execution-Trace World Modeling

• Environment code makes state and dynamics persistent, modifiable, and inspectable through artifacts such as structured world programs and editable execution-trace models.Structured representations include compositional room programs in FactoredScenes [126] and language-to-simulation programs in Code2Worlds [139]; WorldCoder [36] lets agents write and update executable world models.Source: §2.3 · p. 14, §2.3.1 · p. 15, §2.3.2 · p. 15

2.3.3. Code-Grounded Evaluation Environments; 2.3.4. Verifiable Environment Construction

• Code-grounded evaluation uses runtime interaction and verifiable outcomes, while environment-construction methods synthesize runnable states, transitions, tools, and validators.SWE-bench [5] evaluates repository changes through unit tests; SWE-smith [137] constructs repository-level tasks and environments, and EnvScaler synthesizes tool-interactive environments with trajectory validators.Source: §2.3.3 · pp. 15–16, §2.3.4 · p. 16

3. Harness Mechanisms: Planning, Memory, Tool Use, Control, and Optimization

• Agent-authored executable interfaces adapt the environment to a task, while human-designed policies, sandboxes, permissions, verification, audit logs, and review gates govern execution.The model judges, decomposes goals, selects actions, interprets feedback, and revises; the harness persists state, routes feedback, and verifies state transitions.Source: §3 · p. 16
• The section reviews five interacting mechanisms: planning, memory and context engineering, governed tool use, Plan-Execute-Verify control, and agentic harness engineering.These cover long-horizon task organization; preserving, retrieving, and offloading state; governed executable interfaces; feedback-guided acceptance or revision; and telemetry-driven, replay-evaluated harness improvement.Source: §3 · pp. 16–17

3.1. Planning for Agent Harness

• Planning externalizes intent into executable steps, schedules interactions with code and tools, and revises the trajectory when feedback reveals errors or missing dependencies.It is a form of harness control for long-horizon problem solving, not merely an internal reasoning capability.Source: §3.1 · p. 17

Ø Structure-grounded Planning

• Structure-grounded planning is framed as linear, single-path planning with explicit structure.Source: Ø Structure-grounded Planning · p. 18

3.1.1. Linear Decomposition Planning

• Linear decomposition planning turns a task into an executable sequence that guides successive actions, sometimes refreshed as observations arrive.Persistent plans and status logs support multi-session progress and auditability, but a single initial trajectory offers limited exploration when the decomposition is misaligned.Source: §3.1.1 · pp. 18–19

3.1.2. Structure-grounded Planning

• Structure-grounded planning uses explicit task or repository representations to expose dependencies and guide subtask generation, revision, and verification.Graphs, architecture notes, specifications, and testing guides make relevant project or domain knowledge inspectable and support coherent long-horizon control.Source: §3.1.2 · p. 19

3.1.3. Search-based Planning

• Search-based planning explores and evaluates alternative thought, action-trajectory, or code paths, using feedback to revise choices or backtrack from weak decisions.Some systems preserve candidates, patches, tests, logs, and traces as durable artifacts, extending search from model-side sampling to harness-level state management.Source: §3.1.3 · pp. 19–20

3.1.4. Orchestration-based Planning

• Orchestration-based planning coordinates specialized roles, stages, feedback, and verification to control what agents do next in long-horizon coding workflows.Its reliability depends on execution conditions and evaluation that stress long-range coordination; open challenges include adaptive commitment, meaningful planning states, and efficient exploration and revision.Source: Ø Orchestration-based Planning · p. 18, §3.1.4 · pp. 20–21

3.2. Memory and Context Engineering for Agent Harness

• Memory is a state-management layer that selects what stays in active context, what is summarized, and what is stored durably across long-horizon software tasks.Its functional categories are working, semantic, experiential, long-term, and multi-agent memory, with context compaction and state offloading as cross-cutting mechanisms.Source: §3.2 · pp. 21–22, §3.2.6 · p. 25

3.2.1. Working Memory

• Working memory curates state for the next action, while semantic memory retrieves repository-structured evidence to inform the current coding decision.Structured prompt regions and execution records can reduce repeated localization; AST-based chunking, query rewriting, and localization-conditioned retrieval improve the utility of retrieved context.Source: §3.2.1 · pp. 22–23, §3.2.2 · p. 23

3.2.3. Experiential Memory

• Experiential memory transfers curated debugging experience across tasks, while long-term memory governs writing, compression, and retrieval to limit noise, drift, and redundancy.Validated commits and solutions can preserve reusable experience; uncontrolled historical records risk semantic noise, error propagation, and stale or misleading retrieval.Source: §3.2.3 · p. 24, §3.2.4 · p. 24

3.2.5. Multi-Agent Memory

• Multi-agent memory shares decisions and execution traces across roles; compaction and offloading retain concise active summaries while preserving full-fidelity artifacts externally.The harness must control sharing granularity and preserve provenance so compact context remains auditable and can support execution-time verification.Source: §3.2.5 · p. 24, §3.2.6 · pp. 24–25

3.3. Tool Use for Agent Harness

• A reliable harness governs tool access and execution, using permission checks, validation, human approval, output sanitization, memory updates, and verification as lifecycle controls.Tools expand the agent’s action space while lifecycle hooks make their use a monitored transition in the execution loop.Source: §3.3 · pp. 25–26

3.3.1. Function-Oriented Tool Use

• Function-oriented tools ground code generation in external APIs and documentation, reducing reliance on parametric knowledge but not replacing repository-wide reasoning or runtime debugging.ToolCoder [19] learns when to query API search and how to select retrieved APIs; retrieval is most useful when reliable knowledge of a function or library is missing.Source: §3.3.1 · pp. 26–27

3.3.2. Environment-Interaction Tool Use

• Environment-interaction tools let agents navigate repositories, modify files, and test outcomes; CodeAgent [185], SWE-agent [57], and RepairAgent [183] exemplify this approach.These methods support repository-level generation, issue resolution, and open-ended software engineering tasks.Source: §3.3.2 · p. 27

3.3.3. Verification-Driven Tool Use

• Verification-driven tools use tests, execution results, compiler errors, traces, type checkers, and static analyzers as deterministic feedback for iterative code improvement.AgentCoder [50] closes the loop with programmer, test designer, and test executor agents; the harness should route parsed or summarized observations while retaining full traces for audit and replay.Source: §3.3.3 · pp. 27–28

3.3.4. Workflow-Orchestration Tool Use

• Workflow orchestration coordinates tools, roles, memory, and control policies across long-horizon tasks, with SDKs, guardrails, sandboxes, and lifecycle hooks supporting safe, auditable execution.MapCoder assigns agents to recall, planning, generation, and debugging; hooks can validate permissions before use and sanitize outputs, compact logs, or trigger verification afterward.Source: §3.3.4 · p. 28

3.4.1. From Debugging to Harness-Level Control

• Feedback-guided debugging reframes repository-level code repair as control over executable state, with validation signals deciding whether agents continue, revise, or stop [243] [244] [245] [246] [23].The harness observes deterministic signals such as diagnostics, traces, test outcomes, coverage gaps, and security warnings, then can revise a patch, request context, route work, reduce permissions, or escalate to a human reviewer.Source: §3.4.1 · p. 29

3.4.2. Planning as Contract Formation

• Planning forms a harness contract that specifies implementation steps, relevant files, invariants, validation commands, rollback points, and risky operations [40] [42].Repository instructions, typed tool schemas, adapters, and protocol gateways make permitted actions inspectable before execution [261]; failed verification updates the plan.Source: §3.4.2 · p. 30

3.4.3. Sandboxed Execution and Permissioned State Transition

• Sandboxed execution realizes the plan as an observable, reproducible state transition while isolating filesystem, dependency, runtime, interface, and resource state [263] [22].Substrates range from coding sandboxes to browser or IDE environments and durable microVM or WASM runtimes [251, 252, 264, 265, 266, 250, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278]; stable replay conditions help distinguish program defects from environment drift [250] [217]. Actions are tiered from read-only inspection to sandbox editing and full access, with high-risk actions gated by human approval and governance policies [250] [281] [178].Source: §3.4.3 · pp. 30–31

3.4.4. Verification through Deterministic Sensors

• Verification compares resulting state with explicit constraints using compilation, static analysis, runtime signals, tests, and repeatable evaluation harnesses [246, 292, 293, 294, 295, 245, 243, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309].When checks fail, sensor evidence grounds diagnosis, context retrieval, localized repair, task routing, or abandonment; termination depends on passed checks, lack of improvement, risk changes, or required human review [166] [50] [52] [253]. Human or agentic critique should interpret evidence rather than replace deterministic sensors [244].Source: §3.4.4 · pp. 31–32

3.5. Agentic Harness Engineering for Adaptive Harness Optimization

• AHE makes the agent’s operating environment—not just its prompts or context—the object of analysis, targeting infrastructure failures such as brittle tools, weak validators, and mismatched permissions.The harness may include tool schemas, planning artifacts, memory and retrieval policies, sandbox configuration, verification sensors, routing, multi-agent workflows, and human-review gates.Source: §3.5 · p. 32
• Prior work spans AutoHarness harness synthesis [14], Meta-Harness infrastructure optimization [13], and observability-driven diagnosis of agent-loop failures [281].These strands support a systems view in which changing the scaffold can change agent behavior without retraining the base model.Source: §3.5 · p. 32

3.5.1. Deep Telemetry as the Optimization Substrate

• Deep telemetry links model decisions, harness actions, environment states, and outcomes, enabling comparative diagnosis and replay across harness versions.It can include prompts, retrieved context, costs, tool arguments, permissions, edits, command outputs, tests, stack traces, human interventions, and outcomes; evaluators expose task regressions, tracing stacks trajectory causes, and policy gateways boundary violations.Source: §3.5.1 · pp. 32–33

3.5.2. The Evolution Agent

• An Evolution Agent uses trajectory telemetry to diagnose failures and propose revisions to harness components rather than editing the task repository.Its five-stage loop observes executions, attributes failures to components, proposes changes, evaluates them on held-out tasks or replayed traces, and promotes only verified improvements that avoid regressions.Source: §3.5.2 · p. 33

3.5.3. Governed Harness Mutation

• Governed harness mutation requires sandboxed evaluation, fixed regression suites, auditable rationales, and human approval for changes affecting risky boundaries or actions.The Evolution Agent itself follows a plan-execute-verify loop, escalating changes to permissions, network access, credentials, deployment, or human-review requirements for HITL approval.Source: §3.5.3 · pp. 33–34

Motivation

• Single agents face context-window limits, inefficient generalist workloads, and a lack of independent coordination and verification for long-horizon execution.These constraints make it difficult to hold a full codebase, interaction history, and execution trace in working memory while detecting and correcting errors.Source: §4 · p. 34

Multi-Agent Orchestration

• ChatDev, MetaGPT [55], and AgentCoder [50] distribute software-development work among specialized roles coordinated through communication protocols and shared code artifacts.Roles include architect, programmer, tester, reviewer, and executor; code serves as the shared substrate for planning, action, verification, and improvement.Source: §4 · p. 34

Section Contribution

• The section surveys multi-agent systems as a way to scale coding harnesses and proposes shared code-centric harness substrates for AI agents.Source: §4 · p. 34

4.1.1. Functional Role Specialization and Human-Guided Planning

• Functional specialization divides harness work among agents; EvoMAC adds Gradient and Updating meta-roles that use execution feedback to revise prompts and workflow structure.These roles operate at the multi-agent-system level rather than directly on the program.Source: §4.1.1 · pp. 35, 38

4.1.2. Diverse Interaction Modes Grounded in Shared Program State

• Agents communicate through shared artifacts—including APIs, files, diffs, tests, logs, schemas, blackboards, and workflow states—grounding coordination in objective program state and execution results.The harness typically defines these channels, while agents write to or modify the artifacts passing through them.Source: §4.1.2 · p. 38
• Critique and repair dominates surveyed systems, while collaborative synthesis is relatively rare; PairCoder uses Navigator–Driver co-construction, whereas AgentCoder [50] and Self-Collaboration [56] illustrate feedback-driven repair.Critique may use simulated or execution-grounded feedback, with signal richness and repair limits varying across systems.Source: §4.1.2 · p. 38
• Beyond repair loops, agents validate with adversarial execution counterexamples (AutoSafeCoder [52], MAGE [339]) or debate specifications and decisions (ChatDev, FlowGen, CANDOR [342], MAGIS).Fuzzing can demonstrate crash-inducing failures, while debate ranges from clarifying questions and shared-context discussion to majority-vote verdicts.Source: §4.1.2 · p. 39

4.1.3. Optimized Workflow Topology for Agentic Coordination

• Fixed topologies mirror software-development lifecycles through sequential chains, cyclic feedback, manager-worker hierarchies, and hub-centered stars (ChatDev, MetaGPT [55], AgentCoder [50], CANDOR [342]).These designs are generally fixed at design time; FlowGen compares alternative lifecycle-mirroring topologies.Source: §4.1.3 · pp. 39–40
• Adaptive designs scale pools by task complexity in SoA [212] and candidate-file count in MAGIS, discover hierarchies in BOAD [337], and restructure workflows from execution feedback in EvoMAC.SEW [312] evolves workflow specifications, while FlowReasoner [338] generates a tailored multi-agent system per input under external execution feedback.Source: §4.1.3 · p. 40

4.2.1. Execution Feedback Integration

• L2MAC [344] blocks pipeline progress on syntax errors after each file write, while ChatDev uses compiler errors for one-off corrections and AutoSafeCoder [52] supplies crash inputs and exit codes.AutoSafeCoder’s type-aware fuzzing yields concrete failing inputs that localize bugs to input categories.Source: §4.2.1 · pp. 41–42
• Test pass/fail is the most common execution feedback: AgentCoder [50] iterates until tests pass or five iterations, while QualityFlow [253] predicts outcomes with 98%+ precision and recall on MBPP.QualityFlow’s Imagined Execution simulates Python step by step without running code; Self-Collaboration [56] reports near-identical performance for simulated and real-compiler testers.Source: §4.2.1 · p. 41
• AutoSafeCoder [52] uses CWE-mapped static warnings for remediation, MACRO [345] profiles time, memory, and FLOPS, and MAGE [339] targets repair with waveforms around the first failing clock cycle.MACRO also searches research literature for optimization techniques; MAGE records signal values at every clock edge, enabling sub-test-granularity debugging.Source: §4.2.1 · p. 42

4.2.2. Shared-Harness Synchronization

• Sequential handoff shares only the latest artifact and can hide state divergence; persistent blackboards provide a shared view, but finite channels, stale views, and authority conflicts remain constraints.Code-rich coordination does not remove distributed-systems limits such as compression loss and noisy logs.Source: §4.2.2 · p. 42
• Shared blackboards let agents read and update persistent program state; L2MAC [344], MAGIS, and Self-Collaboration [56] implement this pattern.L2MAC’s Control Unit controls reads and writes, while MAGIS stores version-linked summaries and Self-Collaboration names the blackboard architecture explicitly.Source: §4.2.2 · p. 42
• Parallel branches let MAGIS merge independent file edits and HyperAgent [333] merge parallel agent results, while QualityFlow [253] can revert to its initial artifact if debugging degrades quality.QualityFlow is the only surveyed system explicitly described as managing state history rather than always moving forward.Source: §4.2.2 · pp. 42–43
• L2MAC [344] schedules targeted context with summaries and file-selection instructions, while MetaGPT [55] filters shared-state messages by agent role; hierarchical memory preserves both immediate and accumulated knowledge.ChatDev separates phase dialogue from carried-over solutions, Cogito [346] adds evolving abstractions, and HyperAgent [333] summarizes logs before storage.Source: §4.2.2 · p. 43
• SoA [212] scales agent pools to keep each agent’s context bounded by partitioning task state, but agents lose global consistency and cannot reason over the full program.This distributes context load rather than providing a shared representation for all agents.Source: §4.2.2 · p. 43

4.3.1. Shared Harness Representation

• Repository-based harnesses expose structure and history for navigation; SyncMind formally models ground-truth repository state and agent-belief divergence.Examples include MAGIS updating file summaries from git diffs, HyperAgent [333] providing repository-navigation tools, and Lingma SWE-GPT using AST skeletons.Source: §4.3.1 · pp. 44–45
• Execution-based harnesses represent shared state through behavior such as tests, vulnerabilities, performance, and specification matching, providing objective signals beyond linguistic evaluation.MAGE [339] provides fine-grained feedback through clock-edge-level State Checkpoint waveform snapshots.Source: §4.3.1 · p. 45
• Blackboard or shared-state harnesses persist data that agents can query and update, with L2MAC [344] using a persistent file store and a Control Unit to select visible state.Other examples include Self-Collaboration [56], GameGPT [210], and Cogito [346].Source: §4.3.1 · p. 45
• Most multi-agent coding systems use implicit or file-only state, leaving code state reconstructed from context and divergence from the true state undetected.The paper identifies this reliance on linguistic reasoning, rather than objective execution signals, as the central gap motivating the shared harness substrate.Source: §4.3.1 · pp. 44–45

4.3.2. Harness-State Convergence

• Harness convergence can use objective behavioral criteria: test-passing correctness, vulnerability-free security, or user-defined runtime and memory thresholds.AgentCoder [50], L2MAC [344], SyncMind, and CANDOR [342] use test-gated convergence; AutoSafeCoder [52] requires no flagged CWE vulnerabilities or fuzzer-induced crashes; MACRO [345] uses runtime and memory thresholds.Source: §4.3.2 · pp. 45–46
• Score-based convergence ranks or prunes candidate outputs, while Trae Agent [336] frames repository issue resolution as search, pruning, and selection among patches.MAGE [339] iterates until its maximum simulation score reaches 1.0; CodeCoR [166] selects the highest-ranked code and may submit the best available result without perfect correctness.Source: §4.3.2 · p. 46
• Consensus convergence accepts outputs through reviewer judgments, including CANDOR's [342] majority vote and single-agent gates in MAGIS and QualityFlow [253].QualityFlow's Code Quality Checker is both convergence oracle and controller, allowing early exit; the section reports 75–84% of problems converge after the first generator call.Source: §4.3.2 · p. 46
• Implicit convergence—termination by fixed stages, iteration limits, or repeated code without an objective quality criterion—is prevalent and reflects the lack of formal shared substrates.Examples include ChatDev, MetaGPT [55], Self-Collaboration [56], and EvoMAC.Source: §4.3.2 · pp. 46–47

• Most surveyed systems reconstruct shared harness state from conversation, leaving agents unable to reliably detect divergence from the true program state.This implicit approach can work for simple function-level tasks, but the paper identifies it as a technical root of brittleness rather than a scalability convenience.Source: §4.4 · p. 47
• Code-mediated channels such as files, tests, logs, and blackboards encode only partial task state, trading off fidelity, latency, and scope.The design challenge is to establish which artifacts are authoritative, how they are compressed, and how conflicts among channels are resolved.Source: §4.4 · p. 47
• LLM-simulated execution predicts actual outcomes with 98%+ precision and recall [56] [253], but execution feedback is needed for failures linguistic simulation cannot imagine.Execution helps detect runtime crashes, resource exhaustion, boundary-condition errors, and performance regressions; the proposed mature harness uses linguistic reasoning as a fast path and execution as a verification oracle when needed.Source: §4.4 · p. 47
• Repository-based and execution-based representations capture complementary harness views, but no surveyed system unifies static code structure with dynamic behavior.MAGIS and HyperAgent [333] primarily use repository structure, while AgentCoder [50] and MAGE [339] primarily use runtime signals; a unified substrate could connect call graphs and profiling data or static analysis and dynamic testing.Source: §4.4 · p. 48
• Formal shared substrates support simpler topologies, while implicit-state systems use complex workflows to compensate; context-management mechanisms likewise address the lack of queryable shared state.L2MAC [344] pairs a persistent file store and explicit context scheduling with a sequential chain, whereas EvoMAC and SEW [312] use adaptive topologies. L2MAC [344], MetaGPT [55], SoA [212], and Cogito [346] use context-management mechanisms; a mature substrate could provide task state on demand.Source: §4.4 · p. 48
• As agent role diversity grows, shared-state divergence can cause planning, execution, and verification errors, making a unified substrate increasingly critical.EvoMAC monitors failure attribution at the multi-agent-system level, while SyncMind formalizes belief divergence as |Bk −Sk| and proposes synchronization protocols.Source: §4.4 · p. 48

• Code serves as an operational substrate across coding assistants, GUI/OS agents, scientific discovery, personalization, and embodied agents.Source: §5 · p. 49
• In these domains, code supports state representation, action execution, memory, feedback, and governance.Source: §5 · p. 49
• The applications reveal unresolved challenges in evaluation, verification, safety, coordination, multimodal grounding, and harness evolution.Source: §5 · p. 49

5.1.1. Code Assistants

• Managed development harnesses control repository access, execution, permissions, context, logging, and validation, while production traces and harness components increasingly support optimization and model training.Examples include sandboxed workspaces, reusable harness components, and coding interactions used for online reinforcement learning or long-horizon training.Source: §5.1.1 · pp. 50, 52
• LingmaAgent resolves 16.9% of in-house issues fully autonomously and 43.3% with manual intervention [373].This deployment reflects a shift toward persistent engineering workflows involving pull-request review, CI/CD feedback, and production issue resolution.Source: §5.1.1 · p. 52
• Open problems include verification beyond unit tests, weak step-level failure attribution, scarce capability-based execution governance, stable harness self-evolution, live-repository synchronization, and calibrated human oversight.The survey identifies these as coding-specific challenges for safely scaling harness-driven autonomy.Source: §5.1.1 · pp. 52–53

5.1.2. GUI/OS Agents as a Program World

• GUI/OS agents treat observations, executable actions, state-checking evaluators, and increasingly world models as code, making interfaces program worlds with executable feedback.Code-defined sandboxes provide reproducible environment dynamics; emerging directions include end-to-end VLA agents and world models that predict renderable code.Source: §5.1.2 · pp. 53–56

5.1.3. Autonomous Embodied Agents

• Embodied harnesses translate high-level intent into physically admissible, verifiable actions, while reusable executable skills support long-horizon memory, coordination, auditability, and cross-embodiment transfer.Open challenges include closing the sim-to-real gap, scaling multi-agent coordination, and maintaining safety as environments evolve.Source: §5.1.3 · pp. 56–57

5.1.4. Agents for Scientific Discovery as Program Worlds

• Scientific research forms a partially observable program world where structured project memory, typed code actions, and executable environments connect hypotheses, experiments, observations, and revision.Execution may run through interpreters, proof kernels, scientific packages, robotic systems, or experiment managers.Source: §5.1.4 · p. 57
• Scientific agents turn heterogeneous research workflows into flows of executable program artifacts, with persistent memory spanning scratchpads, durable files, literature indexes, and shared preprints.ResearchAgent and SciAgents-style systems structure hypotheses for downstream planners; Agent Laboratory [445] and AgentRxiv [446] divide research into literature review, experimentation, and report writing. PaperQA / PaperQA2 [450] and the AI co-scientist [439] retrieve literature through tools, while Biomni populates a tool registry across 25 biomedical subfields.Source: §5.1.4 · pp. 57–58
• More than 87% task success on six university-level benchmarks was achieved by El Agente Q [449], while the Virtual Lab designed 92 nanobodies, two with validated binding to JN.1 and KP.3.El Agente Q wraps computational chemistry tools as callable functions and logs simulation traces; the Virtual Lab composes protein-design tools into a Python pipeline. AlphaProof verifies proof steps in Lean, and AlphaEvolve [451] executes and scores code edits.Source: §5.1.4 · p. 58
• 41 of 58 target compounds were synthesized in 17 days by Berkeley’s A-Lab; Coscientist planned, executed, and analyzed organic couplings, demonstrating code-driven laboratory feedback loops.Self-driving labs expose instruments through code APIs, while Chemputer and XDL express syntheses as hardware-independent code that compiles to compliant robotic platforms. Biomni, the AI co-scientist [439], and MatPilot [454] extend these workflows to biology and materials.Source: §5.1.4 · p. 59
• 16.9% of 75 Kaggle competitions reached bronze-medal level with the best-performing MLE-bench scaffold at release, OpenAI o1-preview with Weco AIDE [457]; AIDE achieved roughly three times the next agent’s medal rate.Other benchmarks include MLAgentBench [455] (13 ML research tasks), ScienceAgentBench (102 program-output tasks), and DiscoveryBench [459] (264 hypothesis-search tasks, with the best system score around 25%). Systems also translate scientists’ natural-language goals into typed program specifications enforced by runtime execution.Source: §5.1.4 · p. 59
• Code-centric scientific agents unify hypotheses, experiments, analyses, and lab actions as executable program objects, enabling a closed generate–execute–feedback loop while leaving autonomous trust as an open challenge.The AI Scientist [63] [438] represents an ML paper as an executable trace; the AI co-scientist [439], Virtual Lab, Biomni, Coscientist, and AlphaEvolve [451] provide related program-world systems. Benchmarks including MLAgentBench [455], MLE-bench, ScienceAgentBench, and DiscoveryBench [459] help assess whether agents can drive the full loop autonomously.Source: §5.1.4 · pp. 59–60

5.1.5. Agent Personalization

• Agentic recommendation shifts personalization from one-shot ranking to adaptive interaction, using tool calls and structured intermediate states to retrieve, filter, rerank, explain, and collect feedback.Agent4Rec and iAgent simulate recommendation sessions with synthetic users for offline evaluation; user preferences remain latent and may change over time.Source: §5.1.5 · p. 60
• Structured preference memory lets personalization agents maintain, inspect, correct, and reuse user information, including short-term context and longer-term interests and constraints.AMem and related systems [199] [470] use editable documents or structured records, while MemRec [471] studies collaborative memory management; users can correct stored preferences in natural language.Source: §5.1.5 · pp. 60–61
• Personalization harnesses process sparse, delayed, and ambiguous feedback, but simulated behavior may not match real users; explicit constraints and explanations can make policy adaptation more controllable and auditable.User simulators [464] support controlled offline testing, while conversational recommenders translate instructions into policy specifications and constraint-based systems enforce fairness, diversity, and exposure requirements at serving time. Open challenges include unreliable satisfaction proxies, privacy and governance of preference memory, and conflicts among user, creator, and platform objectives.Source: §5.1.5 · p. 61

5.2.1. Harness-Level Evaluation and Oracle Adequacy

• Harness evaluation should measure execution, feedback, context, safety, coordination, and reproducibility, while addressing whether evaluators capture intended tasks rather than narrow executable proxies.Candidate dimensions include trajectory efficiency, verification strength, recovery, state consistency, safety compliance, and replayability.Source: §5.2.1 · p. 62

5.2.2. Semantic Verification Beyond Executable Feedback

• Executable feedback can create false confidence when tests, analyzers, GUI checkers, scientific scripts, or simulators fail to capture correctness or safety.Future harnesses should compose verification artifacts with explicit scope and confidence, and attach evidence bundles describing checks, assumptions, untested regions, and remaining risks to accepted actions.Source: §5.2.2 · pp. 62–63

5.2.3. Self-Evolving Harnesses without Regression

• Harnesses may adapt to new environments, but self-evolution must avoid overfitting, weakened safety, hidden failures, excessive cost, and regression on rare tasks.Treat each mutation as a safety-critical code change with a contract, preserved invariants, falsifiable evaluation, rollback, held-out regression suites, and evidence for its effects.Source: §5.2.3 · p. 63

5.2.4. Transactional Shared Program State and Semantic Conflict Resolution

• Multi-agent harnesses need transactional shared state that tracks read and write sets, assumptions, versions, verifier obligations, and semantic conflicts across agents and humans.Conflict resolution should support semantic merge, rollback, reconciliation, and re-verification, while determining when external judgment is required.Source: §5.2.4 · p. 64

5.2.5. Human-in-the-Loop Safety and Accountability as Harness State

• Safety-critical harnesses should govern actions through risk-sensitive permissions, human approval for consequential transitions, and durable, auditable records of human feedback.Permissions should depend on arguments, environment state, data sensitivity, and expected side effects, while approvals and corrections update future rules and memory.Source: §5.2.5 · pp. 64–65

5.2.6. Multimodal Code-Harness Systems

• Multimodal harnesses must preserve task-relevant observations, ground actions in evidence, and treat visual or physical feedback as scoped, uncertain evidence rather than binary success.A useful memory design retains raw observations, structured annotations, and compact summaries; reusable skills should link multimodal preconditions, executable actions, and expected postconditions while retaining grounding evidence.Source: §5.2.6 · pp. 65–66

5.2.7. Toward a Science of Harness Engineering

• Harness engineering studies the complete closed-loop system and needs benchmarks, telemetry, component-isolating metrics, and principles for safe operation in persistent program worlds.The proposed frontier combines executability, inspectability, statefulness, and governance through permissions, verification, and accountability.Source: §5.2.7 · p. 66