PaperlayerView exact revision on arXiv

Source-linked AI summary

Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures

Benjamin Rombaut

arXiv:2604.03515v2cs.SEcs.AIcs.ET

TL;DR

Coding-agent scaffold architectures remain poorly understood because capability-based surveys and trajectory studies do not examine the implementation choices that distinguish systems. This paper analyzes 13 open-source scaffolds at pinned commits across 12 source-code dimensions, finding continuous architectural spectra and composable control primitives rather than discrete agent types.

  • Problem

    Existing surveys classify coding agents by abstract capabilities, while trajectory studies observe behavior without examining the scaffold code that determines why systems differ.

  • Method

    The paper presents a source-code-level taxonomy of 13 open-source coding-agent scaffolds, organized into 12 dimensions across control architecture, tool and environment interface, and resource management.

  • Results

    Scaffold architectures occupy continuous spectra rather than discrete types, and 11 of 13 agents layer multiple loop primitives instead of relying on one control structure.

  • Takeaways & Limitations

    The taxonomy provides a source-grounded reference for distinguishing scaffold designs and relating agent behavior to specific architectural choices.

  • Takeaways & Limitations

    The study is taxonomic only, with no performance benchmarking or claims linking scaffold design to task success rates.

Abstract

from arXiv · show

LLM-based coding agents can localize bugs, generate patches, and run tests with diminishing human oversight, yet the scaffolding code that surrounds the language model (the control loop, tool definitions, state management, and context strategy) remains poorly understood. Existing surveys classify agents by abstract capabilities (tool use, planning, reflection) that cannot distinguish between architecturally distinct systems, and trajectory studies observe what agents do without examining the scaffold code that determines why. This paper presents a source-code-level architectural taxonomy derived from analysis of 13 open-source coding agent scaffolds at pinned commit hashes. Each agent is characterized across 12 dimensions organized into three layers: control architecture, tool and environment interface, and resource management. The analysis reveals that scaffold architectures resist discrete classification: control strategies range from fixed pipelines to Monte Carlo Tree Search, tool counts range from 0 to 37, and context compaction spans seven distinct strategies. Five loop primitives (ReAct, generate-test-repair, plan-execute, multi-attempt retry, tree search) function as composable building blocks that agents layer in different combinations; 11 of 13 agents compose multiple primitives rather than relying on a single control structure. Dimensions converge where external constraints dominate (tool capability categories, edit formats, execution isolation) and diverge where open design questions remain (context compaction, state management, multi-model routing). All taxonomic claims are grounded in file paths and line numbers, providing a reusable reference for researchers studying agent behavior and practitioners designing new scaffolds.

1 Introduction

Coding-agent scaffolds are poorly understood because capability taxonomies and trajectory studies do not expose the implementation choices behind agent behavior. This paper addresses the gap with a source-code taxonomy of 13 open-source scaffolds, finding compositional architectures rather than discrete types.

  • Existing surveys classify agents by abstract capabilities, while trajectory studies observe behavior without examining the scaffold code that produces it.
  • Architectural differences matter because they affect cost, reliability, and failure modes, yet current analyses confound scaffold design with model capability.
  • The paper analyzes 13 open-source scaffolds at pinned commits across 12 dimensions organized into control, interface, and resource-management layers.
  • The taxonomy finds continuous architectural spectra rather than discrete categories, with control strategies ranging from fixed pipelines to Monte Carlo Tree Search.
  • Loop primitives act as composable building blocks, while tool sets range from zero LLM-callable tools to 37 action classes.
  • The study provides a reusable, source-grounded reference with taxonomic claims tied to file paths and line numbers.

2 Related Work

Related work provides conceptual taxonomies, runtime trajectory analyses, individual system descriptions, configuration studies, and benchmark evaluations, but lacks comparative source-code analysis of coding-agent scaffolds. The paper positions its taxonomy as an implementation-level complement to these approaches.

  • Conceptual taxonomies: Conceptual agent taxonomies organize systems by capabilities such as reasoning, planning, tool use, memory, and reflection, but cannot distinguish production scaffolds with different implementations.
  • Trajectory analysis: Trajectory studies identify behavioral regularities such as bottlenecks, shorter successful traces, and repetitive failure cycles, while treating agents as black boxes.
  • Trajectory analysis: The present study examines scaffold code to connect observed behavior with implementation choices that trajectory analysis alone cannot identify.
  • Individual systems: Individual system papers reveal design choices spanning fixed pipelines, custom tool interfaces, and tree-structured search, but do not provide comparative coverage across agents.
  • Configuration studies: Developer configuration studies concern instructions received by agents, whereas this paper analyzes how scaffold architecture processes those instructions.
  • Benchmark evaluation: The paper deliberately avoids performance benchmarking because scores confound scaffold architecture with model capability, prompting, and configuration choices.

3 Methodology

The study conducts a qualitative source-code analysis of 13 selected open-source coding agents, using pinned commits and an iteratively developed framework that expands into 12 taxonomy dimensions. Its scope excludes performance benchmarking and is limited to inspectable open-source scaffolds.

  • Analysis procedure: The analysis derives categories from implementation patterns, using qualitative case-study analysis and source-code evidence rather than documentation claims alone.
  • Agent selection: Agents were selected for coding relevance, readable open-source scaffolding, and architectural distinctness, with the corpus spanning interactive CLI and automated SWE-bench systems.
  • Agent selection: The 13-agent selection is not exhaustive and aims to cover architectural strategies in the open-source coding-agent ecosystem as of early 2026.
  • Analysis dimensions: The framework began with six candidate dimensions and added three after pilot analysis revealed architectural variation not captured initially.
  • Analysis dimensions: Dimensions cover control loops, tools, discovery, state, retrieval, execution isolation, context compaction, and multi-model routing.
  • Analysis dimensions: The resulting taxonomy contains 12 dimensions, including independently analyzed loop driver, edit and patch format, and control-flow implementation.
  • Analysis procedure: All analyses were pinned to commits and verified against source code, while uncertainty was recorded explicitly when code was ambiguous.
  • Scope and limitations: The study is purely taxonomic, excludes performance benchmarking, and is limited to open-source agents with readable source code.

4 Results

The results organize scaffold architectures into three layers and find continuous variation within each layer rather than discrete architectural categories.

  • The taxonomy covers control architecture, tool and environment interface, and resource management.
  • Architectural choices within each layer fall along continuous gradients rather than into discrete categories.
  • The three layers distinguish how agents decide what to do, interact with code and execution environments, and manage context, state, and models.

4.1 Layer 1: Control Architecture

Control architectures span multiple orthogonal dimensions rather than discrete categories: agents vary in loop topology, loop driver, and code-level implementation, while loop primitives can be nested and combined.

  • Control loop strategies: Control loops range from fixed pipelines without feedback to tree-structured search with backpropagation, and loop types are not mutually exclusive.Agents may nest one loop type inside another, so a single label can understate actual control complexity.
  • Control loop strategies: Agentless independently samples candidate patches and selects by majority vote, whereas DARS-Agent branches over alternative actions and uses an LLM critic for local choices.DARS-Agent lacks numeric rewards and backpropagation, unlike full Monte Carlo Tree Search.
  • Control loop strategies: Moatless Tools implements full Monte Carlo Tree Search with numeric rewards, visit counts, exploration–exploitation balancing, and reward backpropagation.Its richer search requires maintaining execution state across branches through shadow-mode execution.
  • Control loop strategies: Aider is user-driven externally but runs an autonomous generate-test-repair cycle that re-prompts the model after lint or test failures.The cycle can continue for up to max reflections iterations.
  • Loop driver: 9 of 13 agents give the LLM full autonomy over tool selection, while Aider leaves navigation to the user and Agentless and AutoCodeRover sequence phases around within-phase model decisions.The loop driver determines whether bug localization is an agent responsibility or a user-supplied input.
  • Control flow implementation: Control flow uses four implementation mechanisms: imperative while loops, recursion, graph-as-control-flow, and exception-based signaling.8 of 13 agents use imperative while loops; graph-based control is inspectable, serializable, and checkpointable.

4.2 Layer 2: Tool and Environment Interface

Tool and environment interfaces vary widely in implementation, yet converge around shared capability categories, patch interfaces, and retrieval paradigms. The taxonomy also exposes architectural choices that constrain tool use, discovery, and code-context acquisition.

  • Tool set design: Tool counts range from 0 for Aider to 37 action classes for Moatless Tools, while read, search, edit, and execute recur across all LLM-driven agents.A dedicated validate category appears only in Moatless Tools; other agents subsume validation under execution.
  • Tool set design: Prometheus scopes tools per decision node, while AutoCodeRover restricts localization to read-only search tools and separates patch generation into another phase.Prometheus assigns five tools to EditNode, one read-file tool to BugReproducingWriteNode, and only run-command to BugFixVerifyNode.
  • Edit and patch format: Five of 13 agents use exact string replacement interfaces, reflecting convergence toward editing patches through old-string and new-string matching.The shared interface appears independently in OpenHands, SWE-agent, Codex CLI, Agentless, and Moatless Tools.
  • Edit and patch format: SWE-agent supports 10 output parsers, while Aider provides 13 model-specific edit formats, addressing tool parsing and edit-format adaptation at different layers.Aider selects formats according to model capabilities, including unified diffs or SEARCH/REPLACE blocks.
  • Edit and patch format: Agentless uses tool calls as structured-output extraction rather than execution, whereas other agents apply variants including XML-like patches, whole-file writes, and shell-based edits.In Agentless’s Anthropic path, every simulated edit receives the same hardcoded success response before changes are applied post-hoc; mini-swe-agent edits directly through shell commands.
  • Tool discovery strategy: Tool discovery spans static initialization, configuration-dependent bundles, and per-turn rebuilding, with Codex CLI reconstructing tools for every sampling request.Codex CLI can incorporate MCP changes, newly enabled connectors, or tools added during a session.
  • Context retrieval paradigm: Context retrieval comprises seven strategy types and divides into LLM-driven navigation with general-purpose shell tools versus scaffold-driven preprocessing using graphs, narrowing, or ASTs.Eight agents treat the LLM as a navigator, while Aider, Agentless, AutoCodeRover, Moatless Tools, and Prometheus add progressively more structured repository analysis.
  • Context retrieval paradigm: Retrieval paradigm correlates with loop driver: scaffold-driven agents tend to invest in retrieval infrastructure, whereas LLM-driven agents rely on on-demand navigation.The correlation follows from which component controls sequencing and therefore has the opportunity to preprocess the repository.

4.3 Layer 3: Resource Management

Resource management varies widely across state representation, context compaction, model routing, and persistent memory. The agents alternate between structurally preventing context growth and compressing it on demand, while routing commonly assigns cheaper models to mechanical work.

  • State management: State representations range from destructive message overwrite to event sourcing, with graph-scoped and tree-structured variants between them.Aider replaces summarized messages, OpenHands preserves immutable events, Prometheus scopes state per graph node, and tree-search agents store branch metadata.
  • Context compaction: Two context-management philosophies emerge: prevention bounds growth structurally, while cure compresses accumulated context when token thresholds are reached.Prevention includes graph scoping, search-round limits, and trajectory-depth limits; cure uses LLM-based summarization.
  • Context compaction: SWE-agent’s polling stabilizes message prefixes, preserving prompt-cache reuse by slowing observation-truncation changes.The mechanism links compaction frequency to API cost through cache invalidation.
  • Context compaction: Gemini CLI uniquely verifies LLM-generated summaries with a probe turn, trading an additional LLM call for checking whether critical information was lost.The probe addresses lossy compression of technical details after summarization.
  • Multi-model routing: Multi-model routing spans single-model operation to seven-layer classifier chains, with cost optimization commonly assigning cheaper models to mechanical subtasks.Prometheus routes models per graph node, while Gemini CLI resolves simple cases through progressively more sophisticated classifiers.
  • Multi-model routing: Moatless Tools can use an actor-critic arrangement in which one model generates actions and another evaluates them, unlike DARS-Agent’s same-model critic.DARS-Agent’s critic shares generator biases, whereas Moatless Tools can separate generation and evaluation models.

4.4 Cross-Cutting Themes

Sampling, iteration, delegation, and platform integration cut across individual control-loop designs. Agents differ in whether they generate independent candidates, refine one trajectory through feedback, delegate explicitly or implicitly, and trade platform coupling for richer context.

  • Sampling vs. iteration: Agentless independently generates multiple patches and selects among them by majority voting, so no patch benefits from another patch’s failure.This is the paper’s clearest sampling-based strategy.
  • Sampling vs. iteration: Six of nine LLM-driven agents use pure iteration, refining a single attempt through feedback from prior steps.When tests fail, the agent receives the error message and adjusts.
  • Sampling vs. iteration: SWE-agent combines iteration within complete attempts with sampling across attempts, then uses a reviewer model to select the best trajectory.Different retry attempts may also use different model configurations.
  • Sub-agent delegation: Five agents support sub-agent spawning, enabling parallelism and specialization through mechanisms that differ in who controls delegation and which permissions are enforced.Delegation ranges from LLM-controlled tools to scaffold-defined roles, event streams, separate ReAct loops, and graph nesting.
  • Search and selection: DARS-Agent and Moatless Tools separate online search guidance from offline final selection, but DARS-Agent extracts the leftmost path while Moatless Tools re-evaluates completed trajectories.Moatless’s discriminator may select a trajectory that was not most visited during search.
  • Ecosystem structure: DARS-Agent forks SWE-agent’s codebase, whereas mini-swe-agent reuses its execution environments through structural interfaces, illustrating unsettled extension practices.The contrast suggests different assumptions about reuse and modularity.
  • IDE as architecture: Cline’s IDE integration supplies diagnostics, terminal output, and file-change tracking unavailable to typical CLI agents, but its richest capabilities depend on VS Code APIs.This creates platform lock-in despite ongoing decoupling efforts.

5 Discussion

The discussion argues that scaffold architectures are better understood as compositional spectra than discrete categories. It identifies convergence where external software-engineering constraints dominate and divergence where design choices remain open, with implications for architecture-aware evaluation and scaffold design.

  • Architectural spectra: Across 12 dimensions, agents resist discrete classification because composable loop primitives produce combinatorial architectures.The primitives include ReAct, generate-test-repair, plan-execute, multi-attempt retry, and tree search.
  • Implications for evaluation: Evaluating scaffold dimensions independently can attribute behavioral differences more precisely than labeling whole agents as, for example, ReAct or pipeline systems.Whole-agent labels conflate loop topology, loop driver, tool design, and context management.
  • Convergence and divergence: Tool capability categories converge on reading, searching, editing, and executing code because software-engineering tasks require these operations across architectures.Edit formats also trend toward string replacement because exact matching is more reliable for LLM-generated edits.
  • Convergence and divergence: Context compaction exhibits seven strategies, state management spans overwrite through event sourcing, and routing ranges from one model to seven-layer classifier chains.These dimensions diverge more widely than externally constrained tool capabilities and edit formats.
  • Implications for evaluation: The taxonomy provides architectural variables for controlled agent comparisons because scaffold design mediates how models interact with tools, context, and loop structure.Examples include tool counts from 0 in Aider to 35 in SWE-agent and differing context representations between mini-swe-agent and OpenHands.
  • Loop composition: Eleven of 13 agents layer multiple loop primitives, while Agentless and mini-swe-agent are near-minimal single-loop examples.The paper presents composition as a common design strategy rather than an exception.
  • Tool design: Tool counts range from 0 to 37 despite convergence on four capability categories, exposing a tradeoff between tool expressiveness and LLM confusion.The four categories suggest a minimum viable capability baseline for autonomous coding agents.
  • Context compaction: Every sustained-autonomy agent must address context growth through prevention or cure, while mini-swe-agent lacks compaction and crashes when its context window is exceeded.Prevention avoids information loss but requires anticipating growth; cure is flexible but risks lossy compression.

6 Threats to Validity

The validity analysis identifies threats from single-author interpretation, framework and prompt confounds, evolving code, correlated dimensions, limited corpus coverage, and static analysis. Commit-pinned evidence improves reproducibility, but the taxonomy remains a snapshot of publicly inspectable agents and architectural capability.

  • Construct validity: Single-author analysis threatens construct validity because classifications, evidence selection, and comparisons reflect one researcher’s interpretation.A verification pass checked 296 claims, confirming 267, correcting 19, and accepting 10 as minor simplifications, but was also performed by the same researcher.
  • Construct validity: The dimension framework cannot assess prompt differences’ architectural impact without runtime experimentation, leaving an important aspect of scaffold design outside the analysis.The excluded prompt factors include wording, length, few-shot examples, and persona instructions.
  • Internal validity: Pinned commits ensure reproducibility but make the taxonomy a snapshot that may diverge from agents’ current architectures after active development.Readers can use the listed commit hashes to assess subsequent evolution.
  • Internal validity: Some dimensions may be correlated, including loop driver with retrieval strategy and potentially tool discovery with tool count or state management with compaction.Presenting dimensions as independent axes may therefore simplify architectural relationships.
  • External validity: The corpus excludes proprietary or non-inspectable agents, introducing survivorship bias because their designs may reflect business constraints or proprietary model access.The scope is limited to open-source agents with readable source code.
  • External validity: The 13-agent corpus is not exhaustive and targets analytical rather than statistical generalizability.New or excluded agents may exhibit patterns absent from the taxonomy.
  • External validity: The corpus is dominated by Python repositories, and the study does not systematically evaluate how architecture varies across target languages.Ten of 13 agents are Python-implemented, while language-specific constraints may differ elsewhere.
  • Reliability: Static source analysis captures architectural capability rather than runtime behavior, so deployed use of configurable features may remain unobserved.Runtime MCP discovery and typical use of pluggable selectors are examples of potentially missed behavior.

7 Conclusion

The taxonomy characterizes coding-agent scaffolds across layered dimensions and shows that their architectures are compositional, continuously varied, and grounded in verifiable source evidence. It also identifies controlled experimentation and architecture-aware evaluation as direct extensions of this framework.

  • 13 open-source scaffolds are organized across three layers and 12 dimensions, with claims tied to pinned commits, file paths, and line numbers.The evidence base is designed to be independently verified and extended as the ecosystem evolves.
  • 11 of 13 agents layer multiple loop primitives, making the design space combinatorial rather than categorical.The primitives include ReAct, generate-test-repair, plan-execute, multi-attempt retry, and tree search.
  • Control strategies range from fixed pipelines to Monte Carlo Tree Search, while tool counts span 0 to 37 and context compaction covers seven strategies.These spectra show why single capability labels cannot distinguish architecturally different systems.
  • The taxonomy supports controlled comparisons that hold models or tool sets constant while isolating loop and compaction strategies.Such experiments are proposed to attribute performance differences to scaffold design rather than model differences.
  • Its specific architectural positions remain to be tested across proprietary agents, other languages, and later ecosystem snapshots.The framework is intended to be language- and platform-agnostic, but generalization remains open.
  • The framework enables architecture-aware metrics linking loop strategy, compaction, and tool design to task success, token cost, and trajectory length.This would extend evaluation beyond system-level leaderboards toward component-level understanding.

A Candidate Agent Corpus

The study considered a pool of 22 candidate agents and included 13, with excluded agents grouped by the inclusion criterion they failed.

  • 22 candidate agents were considered, with the 13 included agents listed before 9 excluded agents.The full pool is organized by disposition in Table 14.
  • The nine excluded agents are grouped according to the inclusion criterion each failed.The three inclusion criteria are defined in Section 3.1.

B Pinned Commit Hashes

The study records pinned repository commits for all 13 analyzed agents so that cited source evidence can be reproduced or verified.

  • File paths and line numbers in Section 4 refer to these specific commits, enabling readers to clone repositories and verify claims.The pinned references support reproducibility of the source-code analysis.
Loading 2604.03515v2…