PaperlayerView exact revision on arXiv

Source-linked AI summary

Orchestrating AI-Assisted Code Remediation: Socio-Technical Bottlenecks in a Large Industrial Repository

Andreas Bexell, Lo Gullstrand Heander, Emma söderberg

arXiv:2609.29172v1cs.SE

TL;DR

Code degradation is costly to remediate through manual refactoring and opportunistic clean-ups, while the impact of AI-assisted remediation on industrial workflows remains underexplored. In a 15-day field study, an experienced developer used an AI coding buddy in a large industrial repository; the resulting work exposed CI capacity, review effort, and change orchestration as primary bottlenecks.

  • Problem

    Code degradation accumulates in long-lived codebases, and the effects of AI-assisted remediation on industrial CI, code review, and team coordination remain underexplored.

    Source: §1 · p. 2

  • Method

    A 15-day exploratory field study tracked one experienced developer using an AI coding buddy to remediate code in a large industrial repository, combining Gerrit metadata, diary, and team-interaction data.

    Source: §1 · p. 2, §3.2 · p. 6

  • Results

    CI capacity, reviewer attention, and change orchestration became primary constraints, as naive per-file commits overloaded CI and created unmanageable review queues.

    Source: §7 · p. 15

  • Takeaways & Limitations

    Treating semantic change sets as primary objects and deriving separate slices for review, CI, integration, and history is a proposed approach to change orchestration.

    Source: §6 · p. 14

    The study covers one developer, one AI coding buddy, one large C++ repository, and 15 working days, so its findings are not statistically generalizable.

    Source: §5 · p. 11

Abstract

from arXiv · show

Background: Code degradation in large, long-lived codebases is costly to remediate through manual refactoring and opportunistic clean-ups. LLM-based coding assistants can perform mechanical remediation at scale, but their impact on industrial workflows is underexplored. Objective: We investigate how massive AI-assisted code remediation affects build-on-commit continuous integration (CI), code review, and team coordination in a large industrial repository, and which socio-technical bottlenecks constrain such remediation when source editing becomes cheap through AI assistance. Method: We report on a 15-day exploratory single-case field study in which an experienced developer used a command-line AI coding buddy to remediate widespread issues in a closed-source industrial C++ repository. We triangulate Gerrit metadata with a developer diary and team chat, analyzed through descriptive statistics and qualitative coding. Results: AI-assisted remediation rapidly generated hundreds of commits touching thousands of lines, saturating CI and reviewer attention. Na\"ive per-file commits overloaded build-on-commit CI; Switching to directory-based batching and capping the number of files per change restored throughput, but still required explicit review solicitation, negotiation of acceptable commit granularity, and iterative follow-up to resolve build and static-analysis failures. Conclusion: When mechanical editing is cheap, CI capacity, review effort, and change orchestration become primary bottlenecks. Sustainable AI-assisted remediation in very large repositories requires deliberate control of commit, review, and CI batch granularity and treating semantic change sets, such as ``fix all instances of warning X'', as first-class units of work that can be sliced differently for developers, reviewers, and CI.

1 Introduction

Code degradation accumulates as standards evolve, while opportunistic clean-ups are slow and difficult to distinguish from feature work. This study examines how AI-assisted remediation affects industrial CI, review, and coordination, and what constrains it when editing becomes cheap.

1 Introduction

  • Code degradation creates a backlog of refactoring needs, while static analysis typically prevents new issues rather than repairing existing ones.Opportunistic clean-ups are slow and difficult to distinguish from feature work.Source: §1 · p. 2
  • Higher code health is associated with substantially lower break rates when LLMs refactor existing code, motivating maintainability for both human comprehension and AI use.Low-quality code may be unfit for reliable AI intervention [5].Source: §1 · p. 2
  • AI coding assistants enable mechanical remediation at scale without specialized scripts or knowledge, prompting questions about industrial workflow effects and remaining bottlenecks.The study asks how remediation affects CI, code review, and team coordination, and what constrains it when editing becomes cheap.Source: §1 · p. 2
  • A 15-day field study followed an experienced developer using a command-line assistant in a large industrial repository and triangulated review metadata, diary, team interactions, and assistant logs.The study contributes evidence that large-scale clean-ups shift bottlenecks from editing to CI capacity, review attention, and change orchestration.Source: §1 · p. 2

2 Related Work

Prior work treats refactoring as costly work and studies large-scale restructuring, automated edits, and developers’ use of generative AI. This study focuses on workflow bottlenecks when AI makes many small clean-ups cheap to propose.

Technical debt, code degradation, and refactoring practice

  • Refactoring is perceived as costly and risky, while practitioners’ work often extends beyond behavior-preserving changes and interleaves with feature or bug-fix work.The paper uses floss refactoring for small-scale clean-ups interleaved with feature or bug-fix development.Source: §2 · p. 3

Large-scale refactoring in industry

  • Large-scale refactoring is common but often postponed for short-term delivery goals; this study instead examines rapid local clean-ups in a 10-million-line repository.Prior work defines large-scale refactoring as non-functional or architectural restructuring over months to years.Source: §2 · p. 3

Automated and AI-assisted refactoring

  • Prior automated-refactoring work includes scripted tools, offline LLM evaluations, and an IDE-integrated renaming agent, but does not examine CI capacity and review workflows in large industrial systems.Offline LLM benchmarks report test success and smell reductions alongside failures and unintended behavioral changes; CoRenameAgent uses interactive review and static analysis.Source: §2 · pp. 3–4

Developer experience with GenAI tools

  • Field studies report perceived productivity benefits alongside cognitive-load costs, while an enterprise AI mandate saw review load rise as AI-authored changes approached 90% of volume.This study examines day-to-day AI-assisted remediation in a closed-source industrial C++ repository and its interaction with CI and human review [9].Source: §2 · p. 4

3 Method

The study uses a single-case field exploration of AI-assisted code cleaning in a long-lived industrial repository. Gerrit metadata, diary entries, and team chat are combined to describe remediation activity and workflow bottlenecks.

3 Method

  • The exploratory single-case study examines bottlenecks arising when one developer’s AI-assisted code cleaning becomes very cheap.The developer had substantial software engineering experience and used an AI coding buddy.Source: §3 · p. 5

3.1 Context

  • The closed-source repository contains around 10 million lines and uses Gerrit review plus CI verification; a full commit analysis typically takes 45–90 minutes.Many lines remain untouched for years, and static-analysis warnings are typically triggered only on new or modified lines, so propagation may be slow.Source: §3.1 · pp. 5–6, Figure 1 · p. 5

3.2 Data Collection and Analysis

  • Over 15 working days, one developer selected remediation targets using assistant suggestions, prior knowledge, and team requests, while the study collected diary, chat, and Gerrit data.The researchers derived commit status and line-change counts from Gerrit, linked sources by timestamps and change identifiers, and analyzed metadata descriptively and diary/chat material inductively.Source: §3.2 · pp. 6–7

4 Results

The exploration shows that high-volume AI-assisted clean-ups can saturate CI and review capacity, requiring changes to batching, review solicitation, and workflow coordination. Build and static-analysis failures also required iterative follow-up and redesign.

4.1 Scale and Progression of AI-assisted Remediation

  • The initial wave of 639 per-file struct-init commits saturated build-on-commit CI, prompting maintainers to purge the queue and manually retrigger jobs.The developer had first uploaded 20 directory-based duplicate-include commits; Figure 2 tracks open commits by category and daily review requests and merges.Source: §4.1 · p. 7, Figure 2 · p. 8
  • Replacing 639 per-file struct-init commits with 24 directory batches reduced the open-commit wave, but reviewer feedback led to a cap of 60 files per change.The cap aimed to keep trivial reviews near five minutes; subsequent boolean-comparison cleanup produced 56 commits under the batching rules.Source: §4.1 · pp. 8–9
  • Later work included race-related fixes and 13 interdependent module-refactoring commits that required carefully ordered merges.CI and static-analysis remediation continued late in the exploration, including redesigning a struct-init change after it broke assumptions in legacy builds.Source: §4.1 · p. 9
  • Thousands of lines across hundreds of files were changed, making CI capacity, reviewer attention, and change orchestration first-order constraints.The remediation included 639 per-file struct-init commits followed by 24 directory-based commits; the developer’s commit cadence was an order of magnitude above the median per developer.Source: §4.1 · p. 9

4.2 Qualitative Dynamics From the Codebook

  • Review solicitation dominated the coded diary and chat material; the developer actively asked colleagues to review rather than relying on them to discover changes in Gerrit.The merged-commit curve closely tracked explicit review requests.Source: §4.2 · p. 10
  • Teams negotiated commit granularity to balance CI load against reviewer effort, settling on a 60-file cap after discussing alternative batch sizes.The discussion connected the cap to a target of about five minutes for a trivial review.Source: §4.1 · p. 9, §4.2 · p. 10
  • Delivery stops coincided with periods when many changes were ready but could not be merged, while continuing cleanup ideas helped explain sustained activity.The authors conclude that CI capacity, review effort, batching, commit granularity, and tool integration became primary constraints.Source: §4.2 · pp. 10–11
  • The developer used CI error messages and local checks to help the assistant amend failing changes, including locally verifying and amending 26 commits.The assistant was instructed to run available static analysis before upload and update commits through new patch sets.Source: §4.2 · p. 10

5 Threats to Validity

Validity is bounded by insider qualitative evidence, judgment-dependent filtering, a closed-source case, and the narrow scope of a single developer's 15-day study.

Credibility

  • The coauthor-explorer's diary is not independent; timestamps, team chat, and separate initial coding provide triangulation, but selective reporting or narrative framing cannot be ruled out.Source: §5 · p. 11

Analyzability

  • Gerrit-derived counts are reproducible, but filtering rules involve judgment; one-author inductive coding was initially independent of the explorer, though a second coder could strengthen inter-rater reliability.Source: §5 · p. 11

Transparency

  • The closed-source repository prevents full replication, although documented Gerrit queries and counting methods can be applied in other Gerrit-based environments.Source: §5 · p. 11

Usefulness

  • This one-developer, one-buddy, one-C++-repository, 15-day case is not statistically generalizable; recurrence is scoped to industrial settings with build-on-commit CI, static analysis, and large legacy systems.Related developer-centered studies report similar tensions between productivity, cognitive load, and review capacity [9].Source: §5 · pp. 11–12

6 Discussion

The discussion shows that systematic remediation quickly creates changes too numerous for opportunistic cleanup workflows, shifting constraints toward CI capacity, review effort, and coordination. It argues for semantic change sets whose commit, review, and CI views can be managed separately.

6 Discussion

  • Widespread code smells can make systematic cleanup necessary, but thousands of lines across hundreds of files demand deliberate commit and review granularity.One-line commits risk overwhelming CI and reviewers, while a few large commits are harder to review, merge, and selectively revert.Source: §6 · p. 12
  • Build-on-commit CI can become a first-order constraint as AI-generated cleanup increases commit cadence; predictive skipping is risky when failures may emerge later in the CI chain.Dedicated cleanup batches and latency expectations distinct from feature work are proposed as ways to manage CI load.Source: §6 · pp. 12–13
  • Limited access to build logs, analysis outputs, and issue tracking left the developer manually transferring results and coordinating follow-up fixes.The authors propose structured assistant access to those systems, with safeguards, to support scaling beyond single-developer explorations.Source: §6 · p. 13
  • For repetitive cleanup patterns, review of every commit may yield diminishing returns after representative examples establish the pattern.The discussion argues that tests and static analysis suit defect detection in mechanical refactorings, while knowledge transfer and team awareness may saturate after a few reviews.Source: §6 · p. 13
  • Developers, reviewers, and CI need not use the same change unit: bulk commits, representative review samples, and carefully chosen build batches can be separated.Disjoint file sets may allow changes to be built together while retaining traceability to the offending commit.Source: §6 · pp. 13–14
  • The proposed orchestration model treats a semantic change set, such as fixing every instance of a warning, as the primary work object and derives consumer-specific slices.The authors identify an orchestration layer above version control, with commits as outputs, as a direction for future research.Source: §6 · p. 14

7 Conclusions and Implications

The conclusions report that AI-assisted cleanup strained CI and review capacity, requiring batched changes and explicit coordination. They recommend semantic change sets and independently tuned commit, review, and CI granularity, while noting that the study data are not shared.

7 Conclusions and Implications

  • High-cadence AI-generated cleanup saturated build-on-commit CI, while hundreds of small commits strained review capacity and required explicit review solicitation.The team shifted from per-file commits to directory-based batches and capped the number of files per change.Source: §7 · p. 15
  • CI capacity, reviewer attention, and change orchestration became primary constraints; carefully batched commits and file limits were needed to sustain remediation at scale.The conclusion contrasts these practices with naive per-file commits, which overloaded CI and created unmanageable review queues.Source: §7 · p. 15
  • Practitioners are advised to represent bulk remediation as semantic change sets, cap commit size, decouple CI batching from review granularity, and actively solicit reviews.The recommendations also call for structured assistant access to CI systems and static-analysis outputs.Source: §7 · p. 15
  • The research agenda includes evaluating orchestration layers, predictive CI skipping, commit merging, automated review, and longitudinal studies beyond single-developer explorations.Source: §7 · p. 15
  • Raw study data are not shared because they may contain sensitive information.Source: §7 · p. 15
Loading 2609.29172v1…