PaperlayerView exact revision on arXiv

Source-linked AI summary

Runnable Commit Untangling for Coding Agents

Jinfeng Jiang, Dongsun Kim, Dayi Lin, Zhou Yang

arXiv:2610.11593v1cs.SEcs.AI

TL;DR

Existing commit-untangling research does not check runnability after each commit or directly demonstrate maintenance benefits. RucTangle and TangleEval address these gaps; with RucTangle histories and git bisect results, repair pass@1 rose by 16.6% on average across two models.

  • Problem

    Existing studies neither check whether code remains runnable after each commit nor directly demonstrate untangling’s practical benefits for software development.

    Source: §1 · p. 3

  • Method

    RucTangle uses execution feedback to revise commit groupings and order while checking runnability, and TangleEval evaluates regression repair using untangled histories and git bisect results.

    Source: §1 · pp. 2–3, §3.1 · p. 6

  • Results

    16.6% average pass@1 increase across two models when repair agents receive RucTangle histories and git bisect results, relative to repair without untangled commits.

    Source: §8 · p. 17

  • Takeaways & Limitations

    The findings provide empirical evidence that untangled histories can improve coding agents’ bug-fixing performance and motivate using rich histories in software development.

    Source: §1 · p. 3

    Experiments cover Python projects from SWE-CI; benefits for code review, refactoring, human developers, and other languages or deployment requirements remain unevaluated.

    Source: §6.3 · p. 16

Abstract

from arXiv · show

Coding agents produce large, tangled patches that mix multiple development purposes, making the code hard to review and maintain. Commit untangling offers the promise of organizing such large patches into untangled, manageable commits. This paper emphasizes two important limitations in existing commit untangling studies. First, they do not consider that untangled commits are ordered and should leave the code runnable. In practice, maintainers are unlikely to accept commits that prevent the code from running. Second, existing studies claim that commit untangling helps software maintenance. However, they conduct syntactic comparisons between the untangled commits and developers' original commits without directly showing the claimed maintenance benefits. To address these gaps, this paper makes two novel contributions: (1) RucTangle, the first agentic method that untangles commits while keeping the code runnable after each commit; and (2) TangleEval, the first evaluation framework that quantifies how untangled, manageable commit histories help coding agents repair bugs. We compare RucTangle against four untangling methods on 131 agent-generated patches. All histories produced by RucTangle are runnable, while baselines produce 20.6%-37.4% unrunnable commit histories. We further collect 453 agent-generated patches that introduce regressions (i.e., causing previously passing tests to fail) and ask two other coding agents to repair regressions. Augmenting agent context with RucTangle-produced histories yields 5.2% absolute improvement in pass@1. We also analyze agent trajectories to learn how they use untangled commits to navigate and fix bugs. Our findings demonstrate the value of adopting established software engineering practices in the era of coding agents, which broaden the future research agenda: how can agents actively use software history to make better development decisions?

1 Introduction

The paper addresses whether untangled commits remain runnable and whether they provide practical maintenance benefits for coding agents. It introduces RucTangle and TangleEval, reporting runnable histories and improved regression repair with untangled history.

1 Introduction

  • Agent-generated patches mix development purposes and can be substantially larger than human patches, increasing the work needed to review and maintain them.On SWE-bench Verified, agent-written patches are twice as large as human-written patches.Source: §1 · p. 1
  • Prior untangling methods do not check that intermediate versions remain runnable, and existing evaluations do not directly test claimed maintenance benefits.Existing evaluations compare generated groupings with developers’ original commits rather than establishing practical benefits for subsequent software development.Source: §1 · p. 2
  • RucTangle histories passed runnability checks after every commit on 131 patches, while 20.6%–37.4% of baseline histories had at least one version that failed.With RucTangle histories, pass@1 rose from 26.9% to 31.3% for Qwen and from 30.8% to 36.0% for Nemotron; average relative improvement was 16.6%.Source: §1 · p. 2
  • Case studies found agents use commit diffs and messages to identify behavior a repair should preserve, though a misleading message led one attempt to reconsider a correct repair.Source: §1 · p. 2
  • RucTangle groups changes by purpose, orders commits, and uses runnability checks and execution feedback to revise the history.When checks fail, it adjusts ordering, revises grouping, or merges adjacent commits while preserving the patch’s final code.Source: §1 · pp. 2–3
  • TangleEval evaluates whether untangled histories help coding agents repair regressions in agent-generated patches.Its repair experiment compares agents given RucTangle histories with agents repairing without untangled commits.Source: §1 · pp. 2–3

2 Motivation and Background

Commit untangling separates changes by purpose, but ordered commits must preserve runnability at each intermediate version. Existing grouping-based evaluations do not directly establish maintenance benefits, motivating evaluation of both runnable histories and their use in repair.

2.1 Commit Untangling and Runnability

  • Separating changes by development purpose can make related changes easier to inspect, but each ordered commit should leave the code runnable.Intermediate versions matter because testing and debugging tools may inspect them; runnability is distinct from functional correctness.Source: §2.1 · p. 3
  • In the backend example, separating the dictionary update from the default selection causes a KeyError, while grouping those changes avoids the mismatch.The logging-level change can remain separate.Source: §2.1 · p. 3, Figure 1 · p. 4

2.2 Task Formulation

  • The task reorganizes every patch change into ordered commits while preserving the final code and keeping together changes that cannot be separated without breaking runnability.For the Python projects, runnability is assessed by whether pytest can discover and load tests; functional tests need not pass, and a final patch that fails these checks cannot satisfy both requirements.Source: §2.2 · pp. 3–4

2.3 Evaluating Untangled Commits

  • Grouping-accuracy scores do not assess runnability, commit order, or whether histories help developers review changes or repair bugs.Reference groupings provide a comparison, but do not necessarily represent the only reasonable split.Source: §2.3 · p. 4
  • TangleEval evaluates whether histories reconstruct the patch and keep every resulting version runnable, then compares regression repair with and without untangled commits.Repair agents start from the same failing code and tests under the same repair budget; success requires fixing failing tests without causing other previously passing tests to fail.Source: §2.3 · pp. 4–5

3 Methodology

RucTangle iteratively constructs and validates ordered commits, revising or merging them when intermediate versions fail runnability checks. TangleEval measures history properties and evaluates whether untangled histories and regression-localization context help agents repair regressions.

3.1 RucTangle: Runnable Commit Untangling

  • RucTangle checks agent-proposed histories and revises them to keep each version runnable, then uses git bisect to identify a candidate commit for regression repair.The final code must remain equal to the original patch’s final version, and the method assumes that version passes the runnability checks.Source: §3.1 · pp. 5–6, §3 · p. 5
  • The agent groups changes by purpose and proposes an order; RucTangle validates assignments and intermediate versions, tries reordering or agent revisions after failures, and merges commits if needed.The merge fallback removes an unrunnable intermediate version at the cost of fewer commits; the plan is accepted only after all checks pass.Source: §3.1 · pp. 5–6

3.2.1 Evaluating Untangling Results

  • TangleEval measures patch reconstruction, per-version runnability, temporary pass-to-fail test transitions, and commit counts to characterize history validity and granularity.It also uses test-gain distribution as a proxy for how functionality is established across commits: MTP measures concentration in one commit, while larger TPE indicates a more even distribution.Source: §3.2.1 · pp. 6–7

3.2.2 Evaluating Regression Repair

  • For regression repair, TangleEval supplies repair agents with reconstructed histories and, for RucTangle, git-bisect test outcomes; it evaluates repaired tests and repeated attempts.A successful repair fixes all tests that failed after the patch without causing other tests to fail; pass@1 is the proportion of attempts achieving that outcome.Source: §3.2.2 · p. 7
  • TangleEval also records partial and full repair rates and patch-generation rate to characterize progress beyond repair success.Patch-generation rate is the proportion of attempts whose final version differs from the failing version.Source: §3.2.2 · p. 7

4.1 Datasets

The datasets use SWE-CI runs to construct patches for evaluating commit splitting and regression repair. They include completed patches for RQ1 and repairable regressions for RQ2.

4.1.1 SWE-CI

  • SWE-CI tasks start from an earlier repository version and use later-version tests, with architect and programmer agents iterating until tests pass or the limit is reached.The evaluation extracts cumulative patches by comparing the initial code with recorded later versions.Source: §4.1.1 · p. 8

4.1.2 Dataset Construction

  • The RQ1 dataset evaluates whether splitting completed patches preserves final code and runnable intermediate commits, while RQ2 tests whether untangled commits help repair regressions.Source: §4.1.2 · p. 8
  • 131 RQ1 patches from 100 Python repositories meet completion, deduplication, and test-outcome filters for evaluating generated commit sequences.Selected runs completed SWE-CI; retained tests passed in three final-version runs, with at least one test failing initially and passing finally.Source: §4.1.2 · p. 8
  • 453 RQ2 patches come from 108 tasks in 90 repositories and contain regressions that later recover in the generating runs.The patches span multiple models or iterations, and a task may contribute more than one patch.Source: §4.1.2 · p. 8
  • A stratified 150-patch sample supports four repair models and commit-merging analysis, drawn from 434 eligible candidates with at least four RucTangle commits.All 150 sampled patches are included in the 453-patch repair dataset.Source: §4.1.2 · p. 8
  • 350 and 26 are the RQ1 median added and deleted lines, versus 265 and 39 for RQ2; both datasets have seven median modified files, with 21 versus 24 median hunks.Source: §4.1.2 · p. 8, Figure 3 · p. 9

4.2 Compared Methods and Repair Conditions

The study compares structural, dependency-aware, and agentic untangling methods, then tests whether histories from selected methods help agents repair regressions.

4.2.1 Commit Untangling Methods

  • RucTangle plans ordered commits and revises their grouping and order using runnability checks, compared with FileSplit, HunkSplit, Armchr, and Atomizer [42].Source: §4.2.1 · p. 9
  • NoUntangle serves as a reference that retains the full development patch as one commit for the untangling-results evaluation.Source: §4.2.1 · p. 9
  • FileSplit, HunkSplit, and Atomizer groups are ordered by dependencies, with the first applicable group selected when dependencies do not determine the next commit.Atomizer is adapted to Python using a change-graph pipeline and merges groups above 0.6 cosine similarity.Source: §4.2.1 · p. 9

4.2.2 Regression Repair Conditions

  • Regression repair compares NoHistory with histories from Armchr, Atomizer [42], and RucTangle; FileSplit and HunkSplit are excluded because they lack purpose-based grouping and commit messages.Source: §4.2.2 · p. 9

4.3 Implementation

The implementation fixes agent, compute, and execution settings for untangling and repair, and evaluates additional models on a shared sample.

4.3.1 Agents and Execution Environment

  • Untangling and repair use mini-swe-agent, DeepSeek-V4-Flash for untangling, and locally served Qwen3-Coder-Next and Nemotron-3.5-Lightning-30B-A3B for the main repair evaluation.Agents operate in SWE-CI containers; histories are applied and tested, and each untangled commit or repair attempt uses a separate container.Source: §4.3.1 · pp. 9–10
  • Untangling and repair sessions are limited to 80 agent steps, repair agents cannot change the checked-out revision, and the main experiment runs three attempts per patch and condition.RucTangle uses one planning session and up to two revisions; agents use temperature 0 and top-p 1.Source: §4.3.1 · p. 10

4.3.2 Additional Models

  • Four additional models are evaluated under NoHistory and RucTangle on the shared 150-patch sample, with one attempt per patch and condition.The models are MiMo-V2.6-Flash, Qwen3.7-Flash, DeepSeek-V4-Flash, and GPT-6-Luna.Source: §4.3.2 · p. 10

4.4 Trajectory Analysis

The trajectory analysis examines how coding agents use untangled histories when repairing regressions. It compares cases where history availability corresponds with different repair outcomes.

4.4 Trajectory Analysis

  • Untangled histories pair code changes with purpose-describing commit messages that may help agents identify regression-relevant changes and behavior a repair should preserve.Source: §4.4 · p. 10
  • Case studies select patches where NoHistory and RucTangle produce different outcomes for the same model and attempt, then trace inspected code, message interpretation, repair decisions, and outcomes.Source: §4.4 · p. 10
  • Repeated attempts for the same patch, model, and condition are compared to examine how agents interpret the same commit messages differently.Source: §4.4 · p. 10

5 Results Analysis

RucTangle produces runnable, multi-commit histories, and those histories improve repair success for most evaluated models. Agents use diffs and messages to infer which behavior to preserve, although misleading messages can make them question a correct repair.

5.1 RQ1. How well do different untangling methods split AI-generated patches?

  • RucTangle distributes test gains less unevenly across commits than baselines: mean MTP is 0.552 versus 0.610–0.739, and mean TPE is 0.562 versus 0.361–0.419.HunkSplit averages more than three times as many commits as RucTangle but has a higher mean MTP of 0.610.Source: §5.1 · p. 11, Table 1 · p. 11
  • 100% of RucTangle histories reconstruct all 131 patches while remaining runnable after every commit, with a median of eight commits per patch.Among final-passing tests, an average of 1.7% temporarily fail during RucTangle’s commit sequence, compared with 16.4%–32.4% for baselines.Source: §5.1 · p. 11, Table 1 · p. 11

5.2 RQ2. How does commit untangling affect regression repair by coding agents?

  • RucTangle raises Qwen3-Coder-Next’s pass@1 from 26.9% to 31.3% and Nemotron’s from 30.8% to 36.0%, while GPT-6-Luna’s falls from 50.0% to 48.7%.RucTangle also raises MiMo-V2.6-Flash’s pass@1 from 29.3% to 36.0% and Qwen3.7-Flash’s from 50.0% to 53.3%; DeepSeek-V4-Flash remains at 61.3%.Source: §5.2 · p. 12, Table 2 · p. 12, Table 3 · p. 13
  • RucTangle achieves the highest pass@3 for Qwen3-Coder-Next and Nemotron, increasing their rates from 40.4% to 43.5% and from 41.1% to 47.7%, respectively.For Qwen3-Coder-Next, Armchr and Atomizer raise pass@1 but leave pass@3 unchanged at 40.4%.Source: §5.2 · p. 12, Table 2 · p. 12
  • RucTangle histories more often help all six models fix target regressions, but five models also more often introduce new failures in previously passing tests.For GPT-6-Luna, full repair rises from 70.7% to 79.3%, while new regressions rise from 34.0% to 43.3%, and pass@1 falls from 50.0% to 48.7%.Source: §5.2 · p. 12, Table 3 · p. 13
  • RucTangle improves pass@1 for four of six models, leaves one unchanged, and yields a slightly lower rate for one; it achieves the highest pass@3 for both models with repeated attempts.Across all six models, target failures are fixed more often, but new test failures also become more frequent for most models, limiting improvement in overall repair success.Source: §5.2 · pp. 12–13, Table 2 · p. 12, Table 3 · p. 13

5.3 RQ3. How do coding agents utilize untangled histories?

  • Misleading commit messages can make agents repeatedly question a correct repair, even after they locate the faulty change.The paper identifies accurate explanations of untangled changes as a future research direction.Source: §5.3 · p. 14
  • Agents use commit diffs and earlier versions to identify what changed and what needs repair, and consult messages to decide which behavior to preserve.In the XML-parsing example, the message leads the agent to retain consume() but move it after reading the element contents, fixing the target failure without new failures.Source: §5.3 · pp. 13–14

6 Discussion

The discussion finds that halving RucTangle histories is associated with lower complete-repair success and more new regressions, while outlining implications for agent workflows and limits on the evidence.

6.1 How Do Coarser Histories Affect Repair?

  • Halving preserves the original order and runnable retained versions, but removes intermediate versions; the original histories have a median of eight commits versus five for Armchr and six for Atomizer.The study randomly retains enough boundaries to produce ⌈k/2⌉ commits and evaluates Nemotron on 150 patches with three attempts per patch and an 80-step limit.Source: §6.1 · p. 14
  • 34.9% to 31.8%: pass@1 falls with halved histories, while full repair drops from 46.9% to 45.1% and new regressions rise from 25.6% to 28.4%.Partial repair changes little, from 65.1% to 64.7%; patch generation rises from 83.3% to 86.4%. The comparison combines changes to candidate-commit scope and messages as well as commit count.Source: §6.1 · p. 15

6.2 Implications and Research Opportunities

  • Agent workflows should prepare changes for later review and maintenance, since passing tests alone does not establish that changes are easy to inspect and maintain.The discussion connects this concern to training pipelines that use test outcomes to identify successful coding trajectories.Source: §6.2 · p. 15
  • History-aware agents could learn to identify relevant commits, compare earlier and current code, and check historical explanations against code and tests.The proposed training and evaluation should include useful and misleading history and measure repair decisions while preserving previously working behavior.Source: §6.2 · p. 15
  • Recorded agent runs could help later agents understand changes, but repeated exploration and mistaken explanations may obscure useful information.The discussion proposes linking selected actions and test results to commits, retrieving details as needed, and distinguishing explanations from observed edits and test outcomes.Source: §6.2 · p. 15
  • The effect of commit-message quality on repair remains open because message quality was not independently controlled.Future evaluation could test whether messages help agents identify relevant changes, preserve required behavior, and avoid new regressions.Source: §6.2 · p. 15

6.3 Threats to Validity

  • The evidence is bounded by Python projects from SWE-CI and agent-based regression repair; the comparisons also do not isolate RucTangle components.The Atomizer comparison depends on a Python adaptation, and the additional-model experiment uses one attempt per patch; human review, refactoring, other languages, and deployment requirements remain unevaluated.Source: §6.3 · p. 16

7 Related Work

Related work spans purpose-based and dependency-aware methods for separating changes, while evaluation has often emphasized agreement with reference groupings. TangleEval adds ordered-history checks and regression repair by coding agents.

7.1 Commit Untangling

  • Untangling methods group changes by edit patterns, code dependencies, surrounding context, or inferred development purpose.Examples include EpiceaUntangler [6], Flexeme, UTango, CoLaUntangle [14], and Atomizer [42].Source: §7.1 · p. 16
  • Dependency-aware methods order required changes before dependents, but prior tools still do not check the paper’s emphasized runnability requirement after every commit.SmartCommit constrains grouping to avoid compilation errors; Armchr also groups and orders patches using dependencies. RucTangle checks whether tests can be collected after each commit while preserving the input patch’s changes.Source: §7.1 · p. 16

7.2 Evaluating Untangled Commits

  • Common evaluations compare generated groups with original commits or expert annotations, although the reference grouping need not be the only reasonable split.Developer studies provide another form of evidence about whether separated changes help code-review understanding.Source: §7.2 · p. 16
  • Recent benchmarks assess agent-created or reconstructed histories, while TangleEval also checks ordered versions and whether agents can use histories to repair regressions.GitGoodBench [21] restricts iterative committing to one file; AtomicCommitBench [20] checks application order and tests after reversing commits. TangleEval checks patch reconstruction, runnability, temporary test failures, and regression repair.Source: §7.2 · p. 17

8 Conclusion

RucTangle produces runnable ordered histories, and TangleEval tests their value in regression repair. On 453 regressing patches, histories with bisect results improved pass@1 on average across two models; case studies show both useful and misleading uses of history.

8 Conclusion

  • 16.6% on average across two models: pass@1 rises when repair agents receive RucTangle histories and git bisect results rather than no untangled commits.RucTangle histories passed runnability checks after every commit on 131 agent-generated patches, with a median of eight commits per patch.Source: §8 · p. 17
  • Case studies show agents using diffs to locate relevant changes and messages to preserve behavior, but a misleading message can lead an agent to question a correct repair and exhaust its budget.The authors conclude that software history merits further study as a guide for coding-agent decisions.Source: §8 · p. 17
Loading 2610.11593v1…