PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Coding Agents Reuse Existing Code or Reinvent the Wheel?

Dongsheng Ma, Sizhe Wang, Xinyi Huang, Zhengren Wang, Yuhan Wang, Luyang Si, Xincheng Wei, Wentao Zhang

arXiv:2609.35357v1cs.SEcs.AIcs.CL

TL;DR

Existing code-generation evaluation centers on functional correctness, leaving code reuse in realistic multi-turn repository development undermeasured despite the maintenance costs of duplicated implementations. RepoReuse audits repository and agent-history reuse turn by turn, finding that agents explore less, reuse their own code less, and accumulate cross-turn duplicates even while pass rates barely move.

  • Problem

    Existing code-generation evaluation centers on functional correctness and gives little attention to reusability in realistic multi-turn repository development, where duplicated implementations require fixes and interface changes to be applied twice.

    Source: §1 · p. 1

  • Method

    RepoReuse is a multi-turn benchmark that reveals requirements over an accumulating workspace and measures reuse of both repository code and agents’ historical code.

    Source: §1 · pp. 2–3

  • Results

    Agents progressively explore less repository code and reuse their own history less, while cross-turn reimplementations reach 50.8% of task chains by turn 5 even as pass rates barely move.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Pass rates alone do not reveal cross-turn duplication, so code-generation evaluation also needs to track reuse and structural redundancy.

    Source: §1 · p. 3, §3.3.2 · p. 5

    The benchmark covers five mature pure-Python libraries and 75 five-turn task chains; generalization to other languages, domains, and longer horizons remains open.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Coding agents are increasingly deployed for iterative development on real repositories, yet existing evaluation barely answers a basic question: \emph{do coding agents reuse existing code or reinvent the wheel?} The question matters: every duplicated implementation is a fix applied twice and agents produce code far faster than humans can audit, so redundancy accumulates unsupervised. Thus, we present \textbf{RepoReuse}, a multi-turn benchmark for auditing code reuse in real repositories, where requirements are revealed turn by turn and the workspace accumulates across turns. It is built by a fully automated pipeline combining AST-based dependency graphs, guided evidence collection, and execution-verified task synthesis, and scales readily to new repositories. Beyond pass rates, we measure the reuse rate together with recall and cross-turn structural redundancy. An audit over 3{,}000 turns shows that agents progressively stop exploring relevant repository code, reuse their own history less even when it is fully in the workspace, and leave duplicated logic in 50.8\% of task chains by turn~5---all while pass rates barely move. Such deficiencies are invisible to pass rates, underscoring the need to evaluate code generation beyond functional correctness.

1 Introduction

Existing code-generation evaluation centers on functional correctness, while realistic multi-turn repository-scale reuse remains undermeasured. RepoReuse measures reuse of repository and agent-history code across turns, and its audit finds declining exploration and self-reuse alongside rising cross-turn duplication even as pass rates barely move.

1 Introduction

  • Duplicated implementations require bug fixes and interface changes to be applied twice, while parallel copies can drift and increase code later turns must navigate.Existing benchmarks largely score functional correctness and give little attention to reusability in realistic multi-turn, repository-scale development.Source: §1 · p. 1
  • RepoReuse reveals requirements turn by turn in an accumulating workspace to measure reuse of both repository code and agents’ historical code.Its automated pipeline combines AST-based dependency graphs, guided evidence collection, and execution-verified task synthesis, and scales readily to new repositories.Source: §1 · pp. 2–3
  • Its metrics trace reuse decisions from code exploration to execution and downstream redundancy, separating repository targets from self-produced targets.Reuse is based on AST-identified call edges; upstream recall tracks target source read, while Cdup counts bypassed targets whose logic is mostly rewritten.Source: §1 · p. 2
  • Agents read 83.6% of relevant repository code at turn 1 but 35.4% at turn 5, while self-reuse falls from 83.9% to 69.1% despite nearly saturated recall of historical code.Disclosing complete historical implementations barely helps, performing on par with providing no memory at all.Source: §1 · p. 2
  • 50.8% of task chains contain a cross-turn reimplementation by turn 5, while pass rates barely move.This structural redundancy is invisible to pass-rate-centered evaluation.Source: §1 · p. 3

2 Related Work

Prior code-generation evaluation emphasizes task success, while work on iterative development and reuse examines narrower settings or different outcomes. RepoReuse fills this gap by auditing reuse of repository code and agents’ own historical code in realistic multi-turn repository development.

2 Related Work

  • Harness and interaction design can be performance variables alongside backbone models; SWE-agent highlights tool-interface design, while Agentless (Xia et al., 2024) uses a localization–repair–validation pipeline without an autonomous loop.Harnesses span research frameworks and industrial products.Source: §2 · p. 3
  • Code-generation evaluation ranges from function-level tests to repository issue resolution and completion, including HumanEval (Chen et al., 2021), MBPP (Austin et al., 2021), SWE-bench (Jimenez et al., 2024), and RepoBench (Liu et al., 2023).These evaluation lines largely center on functional correctness or task success rather than reuse behavior.Source: §2 · p. 3
  • Related iterative benchmarks use narrower settings: CodeFlowBench iterates within one file, MaintainCoder (Wang et al., 2025b) remains single-turn, and SWE-Explore (Zhang et al., 2026b) uses static snapshots without execution.Other work studies iterative interaction, package evolution, or artifacts such as bloated patches and rising redundancy, without auditing reuse behavior in repository-level multi-turn development.Source: §2 · p. 3
  • RepoReuse measures, turn by turn, agents’ reuse of both pre-existing repository code and their own historical code in a repository-level multi-turn setting.The related-work passage identifies this behavior as unaudited by prior work in realistic repository-level multi-turn development.Source: §2 · p. 3

3 RepoReuse

RepoReuse constructs multi-turn repository tasks with verifiable reuse targets, then evaluates how agents explore and reuse repository and prior-turn code alongside correctness and redundancy.

3.1 Benchmark Construction

  • RepoReuse tasks come from mature Python libraries with internal implementation layers, pinned commits, and reproducible environments.The pipeline produces task chains and reuse ground truth through evidence collection, task synthesis, and execution verification without human involvement.Source: §3.1 · pp. 3–4

Dependency Graph and Evidence Collection

  • AST-based dependency maps guide evidence-pack collection and task synthesis; execution checks verify tests and genuine reuse before packaging each turn.Later walks build on the previous turn’s artifact and add an unused nearby repository symbol; expected test values come from reference-solution execution.Source: Dependency Graph and Evidence Collection · p. 4

3.2 Benchmark Overview

  • Each turn averages 10.8 execution-verified tests; from turn 2 onward, reference solutions reuse 1.9 repository symbols and 2.4 earlier-turn functions per turn.Both kinds of reuse targets are present in every turn.Source: §3.2 · p. 4, Table 1 · p. 4

3.3.1 Task Formulation

  • At each turn, the agent receives a requirement and current workspace, submits code, and installs its own submission into the workspace carried forward.Reuse targets are split between original-repository functions and functions produced by the agent in earlier turns; self targets begin at turn 2.Source: §3.3.1 · p. 5

3.3 Evaluation Framework

  • The evaluation follows agents’ workflow from exploration and reuse decisions to functional correctness and the redundancy left by bypassed targets.Files viewed and source lines viewed are descriptive statistics used to interpret recall differences, not evaluative metrics.Source: §3.3.2 · p. 5

3.3.2 Metric Definition

  • Reuse rate counts directly called targets, recall distinguishes read targets from unread ones, and duplicated-chain rate tracks chains with re-implemented logic.A target is re-implemented when bypassed while the submission calls at least 80% of its non-builtin callees, for targets with at least three such callees; duplicated-chain rate is cumulative.Source: §3.3.2 · p. 5

4 Experiments

The experiments compare reuse across model–harness configurations, turns, and historical-memory settings. They find that reuse and redundancy shift substantially even when correctness changes little.

4.1 Experiment Setup

  • The setup crosses two harnesses—a bash-only mini-SWE-agent and tool-rich OpenCode—with four backbone models; each turn starts a fresh session while the workspace persists.Earlier function interfaces are listed from turn 2 onward, but their source is not shown; all configurations share the same per-turn budget.Source: §4.1 · p. 6

4.2 Main Results

  • Across two harnesses and four models, configurations with greater repository recall tend to reuse more repository code, while the highest-reuse DeepSeek configurations have the lowest Cdup.Even the strongest configuration misses 24.3% of repository targets and 13.6% of its own earlier functions.Source: §4.2 · p. 6
  • Every configuration reads at least 98.9% of its own earlier targets, yet self reuse ranges from 67.0% to 86.4%.The reported gap is in choosing whether to build on code found, rather than finding it.Source: §4.2 · p. 6

4.3 Reuse Across Turns

  • Repository recall drops from 76–91% of targets at turn 1 to 16–51% by the last turn, while later reuse increasingly occurs without reading targets.The authors suggest agents may use call sites encountered in their own earlier code.Source: §4.3 · p. 6
  • Self reuse falls 8–24 points from turn 2 to the last turn, while Cdup rises from 9–21% of task chains at turn 1 to 51–69% by the last turn.The decline in repository reuse tracks falling recall; self-target recall remains saturated across turns.Source: §4.3 · pp. 6–7

4.4 Effect of Historical Memory

  • Interface memory raises self reuse from 30.0% with no memory to 67.8%, whereas full source memory leaves it at 29.2%.Interface memory's self reuse still declines from 82.9% to 58.9%; the comparison is qualified because interface memory explicitly encourages reuse and source memory does not.Source: §4.4 · pp. 7–8
  • Without memory, agents open 27.7 files per turn versus 13.5 under interface memory, with higher repository recall (55.5% versus 35.1%) and reuse (58.9% versus 50.5%).Interface memory shifts reuse toward the agent’s own code partly at the expense of repository code not covered by the note.Source: §4.4 · p. 8
  • Across memory settings, correctness is nearly identical at 27.7–30.4% resolved and 70.2–72.4% passed, but Cdup ranges from 46.7% under interface memory to 70.7% under source memory.By the last turn, 94.7% of source-memory task chains contain a re-implemented target.Source: §4.4 · p. 8

5 Conclusion

The conclusion answers whether coding agents reuse existing code or reinvent it: RepoReuse makes this behavior measurable, and its audit finds redundancy that pass rates do not register.

5 Conclusion

  • Agents progressively stop exploring repository code, reuse their own history less even when it is in view, and accumulate cross-turn duplication that pass rates never register.The conclusion also states that handing agents the complete source does not help.Source: §5 · p. 8
  • RepoReuse measures turn-by-turn reuse of pre-existing repository modules and agents’ historical code in real repositories with accumulating workspaces and revealed requirements.The benchmark is fully automated and scales readily to new repositories.Source: §5 · p. 8
  • The authors hope RepoReuse will provide a sustainable instrument for auditing and ultimately improving how agents build on existing code.Source: §5 · p. 8

Limitations

RepoReuse currently evaluates five mature pure-Python libraries across 75 five-turn task chains, two harnesses, and four backbone models; generalization beyond this scope remains open.

Limitations

  • Whether the observed deficiencies generalize to other languages, domains, or longer development horizons remains open.The current benchmark covers five mature pure-Python libraries and 75 five-turn task chains, evaluated with two harnesses and four backbone models.Source: Limitations · p. 9

A.1 Evaluation Protocol

Each task chain runs sequentially in its own accumulating workspace, while repository preparation removes history and tests that could bypass exploration.

A.1 Evaluation Protocol

  • Each turn starts with the original repository and all earlier submissions, runs in a fresh agent session, and scores the file left at the required path.A turn without that file fails all tests, and a failed turn does not end the chain.Source: §A.1 · p. 11
  • Before the first turn, the protocol deletes Git history and test files but retains runtime test helpers and verifies that the package still imports.This removes later-change information and repository tests that may reveal task conventions.Source: §A.1 · p. 11
  • Repositories are pinned to benchmark commits and installed in editable mode in dedicated conda environments shared across configurations.The setup follows the SWE-bench installation recipe.Source: §A.1 · p. 11

A.2 Harness and Model Configuration

The appendix specifies two harness configurations and the requirement and memory formats used to present successive tasks.

A.2 Harness and Model Configuration

  • mini-SWE-agent uses bash commands in fresh subshells, reports command usage, and ends a turn with an explicit submit command.A turn also stops after three consecutive malformed responses; Table 6 lists its budgets and settings.Source: §A.2 · p. 11, Table 6 · p. 11
  • OpenCode runs headlessly with external plugins disabled and auto-approved tool calls, with each turn bounded by a 2,400-second wall time.Runs use isolated configurations with a single model provider.Source: §A.2 · p. 11
  • Agent errors, timeouts, and budget overruns are not retried; scoring uses whatever file remains at the required path.Such failures count against correctness rather than being excluded.Source: §A.2 · p. 11

A.3 Requirement and Memory Formats

  • Each requirement specifies task behavior, file path and module name, function signatures, behavior, and return-value contracts; repository reuse targets are never named.Both harnesses use identical delivery instructions that encourage reuse across memory settings.Source: §A.3 · p. 11
  • The memory setting changes only the historical note: it is absent, lists earlier interfaces, or embeds the exact earlier source files.Interface memory is the default and encourages reuse.Source: §A.3 · p. 12

B Prompt Templates

The prompts guide evidence collection through repository call graphs, then turn collected symbols and prior-round artifacts into executable programming tasks. Evaluation prompts specify deliverables and provide either earlier source files or public interfaces as memory.

B.1 Prompts for RepoReuse Pipeline

  • Evidence collection walks the call graph one hop at a time and seeks symbols that combine into one coherent programming task.The prompt supplies collected symbols, the current position, and a menu of symbols to expand.Source: §B.1 · p. 12
  • Evidence collection and task-generation prompts use structured outputs, including a strict-JSON response for the evidence-pack step.The task-generation prompt also specifies a six-block output format.Source: §B.1 · p. 12
  • Task-generation prompts require implementations to read evidence-symbol source for internal conventions, rejecting tasks solvable from docstrings alone.The requested tasks should exploit conventions such as zero representation, coefficient ordering, or implicit domain transforms.Source: §B.1 · p. 12
  • Generated tasks must include six blocks, including function specifications, a solution that calls evidence symbols, executable test calls, and source locations developers must read.The calls include edge cases and predicted representations, which are executed and checked against actual returns.Source: §B.1 · p. 12

B.2 Prompts for RepoReuse Evaluation

  • Evaluation requires a new importable module with exactly the requested functions and signatures, and asks agents to reuse suitable existing modules without modifying existing files.Agents must check that the module imports and behaves as required; a round without the required file scores zero.Source: §B.2 · p. 13
  • The seaborn example specifies categorical color lookup behavior, including category order, Python-scalar labels, and RGB or RGBA color tuples.The prompt also names map_color_values and describe_palette as related functions.Source: §B.2 · p. 13
  • The evaluation prompt can provide exact source files from earlier rounds or encourage reuse of public interfaces implemented in earlier rounds.The source-memory prompt identifies prior submissions from the same harness and task.Source: §B.2 · p. 13

C Case Study

A two-turn case study shows an agent passing tests while duplicating its own earlier implementation instead of reusing it. At turn 5, it reads the turn-4 code in full but rewrites the centering logic from scratch.

C Case Study

  • At turn 5, the agent has recall = 1.0 of its turn-4 interface but does not import it, instead rewriting the centering logic and passing 10/10 tests.The requirement explicitly lists the earlier interface.Source: C Case Study · p. 13, Figure 3 · p. 15
  • The turn-5 rewrite spans 262 lines, while the ground-truth solution uses 68 lines by calling the turn-4 function and aggregating its output.The rewrite reproduces dtype coercion, group masking, categorical ordering, and mean-centering.Source: C Case Study · p. 13
  • At turn 4, the agent implements center_within_groups to center a numeric column within seaborn’s categorical groups and passes all tests.The turn-4 implementation is the interface explicitly listed in the next requirement.Source: C Case Study · p. 13, Figure 2 · p. 14

Related papers

Loading 2609.35357v1…