Source-linked AI summary
The Empire, Long Divided, Must Unite: Architectural Convergence in Three LLM Agent Harnesses
Dai Jiahong
TL;DR
The paper asks where coding-agent harness architectures are heading as the harness increasingly constrains long-horizon agent behaviour. Through a source-level, multi-case study of three opposing harnesses, it finds convergence on five elements, while external verifiability remains absent and predicts the next architectural distinction.
Problem
Existing studies leave the general harness layer unread, its architectural trajectories uncharacterised, and convergence neither claimed nor tested.
Method
The authors conduct a source-level, multi-case study of three coding-agent harnesses, reading pinned revisions and following commit histories, then check the convergent form against a third harness.
Results
Three harnesses converged on a middle form comprising a commoditised loop, append-only replayable record, quirks as data, progressive disclosure, and explicit extension seams, through parallel discovery, diffusion, and literal reuse.
Takeaways & Limitations
Harness engineering should adopt the settled shapes, differentiate on unsettled dimensions, and treat externally verifiable records as the likely next requirement where provenance is demanded.
Takeaways & Limitations
The cases are not fully independent because dsh depends on pi, and selection favours well-known, source-available projects, introducing survivorship bias.
Abstract
from arXiv · showhide
An agent harness is what turns a language model into an autonomous agent: the surrounding code that builds the model's context, mediates its tools, runs the loop, and persists state across a long-horizon run. This layer, not the model it wraps, is increasingly the binding constraint on agent behaviour. We present a source-level, multi-case study of three open coding-agent harnesses built from deliberately opposing philosophies: LangChain's deepagents (batteries-included), Earendil's pi (radical minimalism), and DeepSeek's dsh (everything-is-a-plugin). Reading each at a pinned commit and following its commit history, we find that the two mature harnesses have travelled in opposite directions (deepagents subtracting authored scaffolding, pi accreting durable infrastructure), yet converged toward one architectural middle form of five recurring elements: a commoditised loop, an append-only replayable session record, model quirks kept as data, progressive disclosure of context, and explicit extension seams. A third harness, read afterward as a held-out check, exhibits all five, and in one seam reuses another's implementation outright. We therefore do not claim independent invention, and decompose the convergence into parallel discovery, diffusion, and literal reuse. Finally, one load-bearing dimension shows no convergence, and indeed no presence: external verifiability, a tamper-evident record an outside party can check without trusting the runtime. We read this absence not as an oversight but as a predictive gap, the next axis on which harnesses for provenance-sensitive domains will differ.
1 Introduction
The paper studies three opposing agent-harness architectures and argues that they converge on a five-element middle form while remaining divided on external verifiability.
- Motivation: The harness, rather than the wrapped model, often governs a larger share of performance variance on long-horizon tasks.With the model held fixed, single-harness changes move Terminal-Bench 2 pass@1 by several points and SWE-bench Verified by up to fifteen.
- Research Questions: The study addresses three gaps: limited source-level coverage, uncharacterised architectural trajectories, and untested convergence in the harness layer.It compares three open harnesses selected for maximal philosophical spread and reads their implementations at pinned revisions.
- Central Finding: Three harnesses that began from opposing philosophies converged on one architectural middle form under shared pressure from long-horizon autonomous operation.The form comprises a commoditised loop, append-only replayable record, quirks as data, progressive disclosure, and explicit extension seams.
- Central Finding: The convergence is not fully independent: the paper decomposes it into parallel discovery, diffusion, and literal reuse.The third harness was read subsequently as a check, and one case literally reuses another harness’s implementation.
- Boundary: External verifiability remains absent from all three harnesses, despite prescriptive protocols placing verifiable lineage and auditability at the centre.The paper defines this boundary as a record an outsider can check without trusting the runtime.
- Scope: The claims are bounded by non-independence among cases, directional rather than systematic longitudinal evidence, and a decision to characterise rather than solve the verifiability gap.The comparison also notes that the append-only record is only partially realised in deepagents.
2 Background and Related Work
Prior work establishes vocabulary and partial maps of agent harnesses but does not comparatively read implementations across trajectories; this paper fills that gap through a source-level multiple-case study.
- Existing Work: Research on harnesses includes formal taxonomies, surveys, reading lists, and automatic optimisation, but these works do not comparatively read implementations.The cited vocabulary describes components such as the execution loop, tool registry, context manager, state store, lifecycle hooks, and evaluation interface.
- Nearest Neighbour: The nearest source-code taxonomy studies thirteen coding-agent scaffolds at pinned commits, whereas this paper studies the general harness layer beneath coding-agent applications.It explicitly excludes deepagents and does not include pi or dsh.
- Contribution: This paper differs by tracing architectural trajectories, making convergence its central thesis, and treating auditability as a first-class dimension.The related longitudinal study tracks release velocity and quality, not the architectural form produced by evolution.
- Prescriptive Counterpart: Complementary work specifies protocol capabilities top-down, including standardised tool invocation, agent messaging, and versioned, traceable, reversible resources.The paper instead documents what shipped harnesses provide through source-level reading.
- Method Lineage: The study follows established multiple-case and open-source architecture methods while applying them to the agent-harness layer.Its cases are read at source level rather than treated as abstract framework descriptions.
3 Study Design
The study uses an explanatory, theory-building multiple-case design to compare three maximally varied open-source harnesses at pinned revisions, combining source reading, history, reproduction, and upstream confirmation.
- Design: The study applies literal replication across maximum-variation cases, using deepagents and pi to develop a model before reading dsh as a held-out check.Because dsh reuses pi’s provider catalogue, its confirmation is limited to independently instantiated elements.
- Case selection: Cases span opposing philosophies, organisational forms, and programming languages, with all projects open-source and readable at pinned revisions.The candidate pool consisted of actively developed, source-readable coding-agent harnesses selected for philosophical spread.
- Data sources: Evidence combines pinned source code, commit and issue archaeology, hands-on reproduction, and upstream confirmation of specific claims.The study independently re-derived two defects in a sandbox and tested two plugins against dsh’s runtime.
- Data sources: Upstream interactions confirmed individual claims rather than the five-element model or seam taxonomy, and their evidential weight is explicitly scoped.One documentation fix was accepted, assigned, and merged, while one defect was filed as issue #5640.
- Analytic framework: Comparison dimensions were fixed before close reading, adding auditability and evolution while refining session-record strength and recovery semantics.The refined dimensions move beyond binary presence toward graded architectural forms.
- Reliability: AI agents assisted the source sweep, but every claim was verified by hand; deepagents and pi received line-by-line reading, while dsh received one thorough pass and an experiment.The authors report asymmetric reading depth and identify reproduction artefacts as a durable reliability check.
4 Divergent Origins (RQ1)
The three harnesses began at opposing points on an automation–observability spectrum but followed distinct trajectories toward the middle: subtraction, accretion, and reuse.
- Shared trajectory: The harnesses occupy a single spectrum and travel it in three ways: deepagents subtracts scaffolding, pi accretes infrastructure, and dsh enters near the centre.Figure 2’s positions are interpretive, and its arrows indicate commit-history direction rather than speed.
- deepagents: deepagents is a batteries-included assembly layer whose planning, filesystem, subagents, memory, permissions, and other capabilities are composed as middleware.Its runtime uses a three-phase middleware stack, with optional storage backends and four automatic context-management layers.
- deepagents: deepagents’s trajectory removes authored defaults, including the base prompt, built-in tool-usage prose, and default todo/planning middleware.Planning becomes an opt-in, per-model need rather than a harness default.
- pi: pi retains a small, readable loop while accumulating substantial infrastructure for durable state, crash recovery, subagent lanes, compaction, and model-compatibility data.Its formal specification defines immutable entries, mutable registers, append-only usage rows, and a durable program counter.
- dsh: dsh implements its plugin absolutism through replaceable services, declarative load order, reversible registration, and a configurable agent loop.Its sandbox fails closed with SANDBOX_UNAVAILABLE rather than running unconfined.
- dsh: dsh enters near the middle and reuses pi’s published provider package through llm-pi-ai, described in-tree as a design-verification twin.This is the clearest literal-reuse instance in the trajectories.
5 The Convergent Form (RQ2)
Across opposing philosophies, the three harnesses share five recurring architectural elements, while differing mainly in composition style and how much automation remains observable to their audiences.
- 5 The Convergent Form (RQ2): The convergent form is a commoditised loop surrounded by four recurring elements, with one dashed slot reserved for the dimension that remains unconverged.Each element appears in three distinct instantiations rather than as a single shared implementation.
- 5.1 Five recurring elements: All three provide an append-only or replay-oriented session record, ordered from deepagents’s weak view, through pi’s append-only compaction entry, to dsh’s strongest event-sourced form.The record forms a strength ladder rather than a simple cluster, although deepagents only partially realises the append-only form.
- 5.1 Five recurring elements: Model quirks are externalised as data in three serialisations: deepagents uses registered profiles, pi a compatibility catalogue, and dsh a runtime adapter resolver.dsh is the most dynamic and correspondingly the least inspectable at rest.
- 5.1 Five recurring elements: Progressive disclosure keeps only a name, description, and path in prompts, loading capability bodies on demand across all three harnesses.dsh additionally records each plugin’s token and KV-cache cost in a written per-capability contract.
- 5.1 Five recurring elements: Explicit extension seams recur across all three, although deepagents uses middleware, backends, and profiles, pi uses events and registries, and dsh uses service-definition triples.The shared architecture is the presence of seams, not a common composition mechanism.
- 5.1 Five recurring elements: The five elements recur across all three columns despite opposing philosophies, supporting convergence of ideas rather than identical code.The implementations are respectively class hierarchy, data directory, and runtime resolver for one model-quirk idea.
- 5.2 The residual disagreement is audience, not architecture: Residual disagreement concerns automation versus observability: deepagents automates context management for product embedding, while pi leaves it manual for terminal users.The paper interprets this as an audience difference rather than an architectural difference.
6 Convergent Fault Lines
Defects recur at four architectural seams across the studied harnesses, but this taxonomy is illustrative and suggests rather than establishes that the seams track the underlying problem.
- Scope: The reproduced or identified defects cluster on four recurring seams, with two defects re-derived in a sandbox and the remainder found through source reading.The authors present the taxonomy as illustrative rather than an independent test.
- S1: Sync/async drift: S1 is sync/async drift: hand-written asynchronous and synchronous tool paths diverge, causing an unhandled KeyError in one checkpointed deepagents caller while the other degrades cleanly.The seam is Python-specific because pi uses a single-threaded model.
- S2: Normalisation trust gaps: S2 is a normalisation trust gap, where path-shape disagreement lets a redundant leading slash bypass component-based filesystem checks.deepagents’s normalisation preserves two leading POSIX slashes while the filesystem resolves both spellings to one file.
- S3: Semantics by string-matching: S3 is semantics by string-matching: pi’s regex-based provider-error classification can mis-handle Bedrock throttles, skip retry, and trigger irreversible compaction.The proposed rule is to use typed error kinds and fail toward the safe side when regex remains necessary.
- S4: Silent degradation versus loud failure: S4 is silent degradation versus loud failure, where unrecognised values continue through the system instead of stopping at the boundary.The authors recommend treating an unrecognised shape as an error rather than a default in long-running autonomous harnesses.
- Cross-seam findings: The seams cut across design philosophies, and three of four specimens end in unrecoverable loss while the sync/async specimen merely crashes.The damage concentrates where the seam is irreversible, including recursive deletion and overwritten conversation history.
7 The Boundary: A Dimension Without Convergence (RQ3)
All three harnesses stop below external verifiability: none produces a tamper-evident record an outsider can check without trusting the runtime. The paper treats this absence as a predictive gap rather than an oversight.
- Every harness stops below the external-verifiability line, leaving the two higher rungs unreached.Figure 4 charts the gap without proposing how to fill it.
- dsh climbs highest through durable approval records, visible-implies-logged behavior, and recovery that closes rather than truncates.Even dsh targets operational reconstructability rather than external verifiability.
- deepagents checkpoints support resumption rather than evidence, while its resume path rewrites message history to patch dangling calls.
- pi uses append-only sessions on its newer path, but its process-owned files remain mutable and unverifiable by third parties.
- The paper identifies external verifiability as a predictive gap, not an oversight, because no harness produces a tamper-evident outsider-checkable record.The two unreached rungs also concern provenance, permission, and cost-ceiling questions.
8 Discussion and Implications
The discussion argues that harness builders should adopt the converged shapes while differentiating on unsettled dimensions. It qualifies the “less harness” thesis as thinner coaching but thicker durability for long autonomous runs.
- The five elements function as a checklist for attended, long-horizon terminal sessions, but the advice is scoped to that audience.Unattended benchmark scaffolds may not draw on the same durability and disclosure.
- deepagents subtracts scaffolding while pi accretes session records, recovery, and quirk catalogues needed to survive long autonomous runs.
- The resulting design principle is “thinner in coaching, thicker in durability,” with thickness set by audience.
- If the harness is the binding constraint, residual performance differences increasingly lie in recovery semantics, quirk coverage, and verifiability rather than the shared loop.
- Future harnesses should adopt settled shapes, differentiate on unsettled dimensions, and treat verifiable records as a likely requirement where provenance is demanded.
9 Threats to Validity
The study’s validity is bounded by limited external scope, non-independent cases and selection bias, construct risk in the induced five-element framework, and single-analyst AI-assisted coding.
- With N = 3 coding-agent harnesses read at one point in time, generalisation to other agent shapes and future revisions is limited.The study mitigates this through maximal-spread selection and pinned revisions.
- The cases are not fully independent because dsh depends on pi, so some convergence reflects diffusion rather than rediscovery.Case selection also favors well-known, source-available projects, creating survivorship bias.
- The five elements were induced from two grounding cases and checked against one more, making construct risk the primary construct concern.The inclusion rule required line-level evidence in both grounding cases.
- The verifiability dimension aligns with the first author’s research interest, a disclosed consideration for weighing the framework.
- A single analyst read the source with AI assistance, introducing single-coder and instrument risk; dsh was also read less deeply than the other two.Claims lacking line evidence are marked documentation-level, while pinned revisions and executable artifacts support re-checking.
10 Conclusion
Three philosophically opposed harnesses evolved in opposite directions yet converged on five architectural elements, through parallel discovery, diffusion, and literal reuse. External verifiability remains absent across all three, marking the next point of differentiation.
- The convergent middle form combines a commoditised loop, append-only replayable record, quirks as data, progressive disclosure, and explicit seams.
- Convergence is real but not independent: the study identifies parallel discovery, diffusion, and one case of literal reuse.
- External verifiability—a record an outsider can check without trusting the runtime—is absent in all three harnesses.The conclusion presents this as the dimension on which the next harness will differ.
Data Availability
Reproduction artefacts and the full per-dimension evidence table will be archived with a DOI on release, while defect reproductions are released under coordinated disclosure.
- The supplementary material contains two dsh plugins, two sandbox defect reproductions, and the full per-dimension evidence table with file:line references.
- Source claims are verifiable at the pinned revisions and repositories specified in Section 3.