PaperlayerView exact revision on arXiv

Source-linked AI summary

AI Agent Pull Requests on GitHub: Frequency, Structure, and Merge Conflict Rates

George Xu, Arjun Subramanian, Nithilan Karthik

arXiv:2607.04697v2cs.SEcs.AIcs.MA

TL;DR

The paper addresses limited evidence about concurrent work by AI coding agents on shared repositories and conflicts between their PRs. It examines AIDev-pop and replays merges of sampled co-active pairs; co-activity is widespread, and cross-agent pairs have higher textual-conflict rates than intra-agent pairs.

  • Problem

    Evidence is limited on how often agent PRs are concurrently developed in the same repository and how often those PRs conflict with one another.

    Source: §1 · p. 1

  • Method

    The study analyzes AIDev-pop PRs, measures co-activity across time-window thresholds, and replays stratified samples of co-active pairs using Git merges.

    Source: §1 · p. 1, §4.1 · p. 3, §4.2 · p. 3

  • Results

    79.4% of agent PRs were open concurrently with another agent PR; textual conflict rates were 19.8% for intra-agent pairs and 41.7% for cross-agent pairs.

    Source: §5.1 · p. 3, §5.3 · p. 3

  • Takeaways & Limitations

    Cross-agent co-active pairs had higher textual-conflict rates than intra-agent pairs, while the measured rates cover textual rather than deeper build or semantic conflicts.

    Source: §5.3 · p. 3, §7 · p. 6

    The study measures textual conflicts only, leaving build and semantic conflicts untracked; its reported rates are therefore conservative lower bounds.

    Source: §7 · p. 6

Abstract

from arXiv · show

AI coding agents may generate and submit Pull Requests (PRs) to the same repository at the same time. However, research concerning the extent of concurrent submission by AI coding agents to a common repository does not exist. This paper uses the AIDev-pop dataset (33,596 PRs in 2,807 repositories) to provide the first empirical examination of the prevalence of concurrent submission using PRs authored by agents. We report that when considering exact temporal overlap, 40.2% of repositories contain co-active agent-authored PR pairs; further, the co-active pairs account for 79.4% of all PRs generated by an AI agent. When we examine co-activity within a one week collaboration window, the percentages are increased to 53.4% and 95.0%, respectively. For the majority of the co-active PR pairs (underlying the vast majority of which are intra-agent authored), both PRs were authored by the same agent, while only 0.5% of co-active pairs were cross-agent, and occurred in only 122 out of 2807 total repositories examined (or approximately 4.3%). Additionally, we replayed actual three way git merges on 747 unique co-active pairs (one per repository), and computed the percentage of textual conflict encountered during the merge operation to combine the two PRs in each pair. We observed that the percentage of textual conflict encountered was significantly higher for cross-agent pairs compared to intra-agent pairs: 41.7% vs. 19.8%, respectively, with non-overlapping 95% confidence intervals. Lastly, we developed a classification system based on the detection of conflict reported by git, and determined that the majority of conflicts resulted from modifications to source code files (84.4% of conflicted files) and not dependency manifest files; further, nearly 42% of conflicts we observed were structural (i.e., modify/delete or add/add).

1 Introduction

Autonomous agents can submit overlapping pull requests, but little is known about concurrent agent workflows and conflicts between them. This study examines agent-authored PRs in AIDev-pop and asks about co-activity, pair composition, merge collisions, and associations with repository and agent characteristics.

1 Introduction

  • Git can identify line-level merge conflicts, but it cannot determine conflicting intent among developers.The paper describes coordination issues arising as autonomous source-code development increases.Source: §1 · p. 1
  • The study addresses a gap in evidence about how often concurrently developed agent PRs occur and how often they conflict with one another.Prior work examined human merge conflicts, individual agents' PR success, and conflicts between one agent PR and its base branch.Source: §1 · p. 1
  • The analysis covers 33,596 agent-authored pull requests across 2,807 repositories and uses automated merge operations to measure actual conflict rates.The process uses headless git merge-tree functionality.Source: §1 · pp. 1–2
  • The paper frames four questions: how frequently agent PRs are co-active, how co-active pairs are composed, how often merges collide, and which repository or agent characteristics associate with concurrency.The questions cover time horizons, same- versus different-agent pairs, conflict structure, programming languages, repository sizes, and agent types.Source: §1 · p. 2

2 Background: The Three Layers of Conflict

The paper distinguishes textual, build, and semantic conflicts as sequential layers of integration friction. Its empirical analysis focuses on textual conflicts, which it treats as a conservative lower bound on total system-wide friction.

Conflict Layer 1

  • Textual conflicts arise when PRs overlap on source-file lines or when one deletes code or a file modified by the other, leaving the merge engine unable to determine a merge path.The engine flags the failed merge, adds conflict markers, and stops processing.Source: §2 · p. 2

Conflict Layer 2

  • Build conflicts occur when textual merging succeeds but compilation or static analysis fails, such as when a changed function signature conflicts with another PR's call.Each change may be individually correct while the combined program has a faulty abstract syntax tree.Source: §2 · p. 2

Conflict Layer 3

  • Semantic conflicts occur after a valid textual merge and compilation, when the PRs' execution assumptions or state invariants logically contradict one another.Source: §2 · p. 2

Scope

  • The study measures textual conflicts rather than build or semantic conflicts, making its measured conflict rate a conservative lower bound on total system-wide friction.Textual conflict detection is described as more reproducible and scalable because it does not require language runtimes, environment variables, or external mock database configurations.Source: §2 · p. 2

3 Previous Research

Prior work has examined human merge conflicts, agent output quality, and conflicts between an individual agent PR and its base branch. The paper identifies inter-PR conflict among concurrently open agent PRs as an unaddressed area.

Human merge-conflict experience research

  • Human merge-conflict research classified text-based collisions in open-source Java projects and developed speculative merging tools to identify potential conflicts in advance [9].Those studies make assumptions about human review increments, shift schedules, and typing or cognitive processing limits.Source: §3 · p. 2

Quantitative assessments of output quality of autonomous agents

  • Agent-output evaluations have assessed patch accuracy and PR rejection likelihood, while other studies reported code churn and message–code inconsistencies in LLM-rewritten code [4] [5][6, 7].Source: §3 · p. 2

Agent-based merge conflicts

  • AgenticFlict reported that 27.67% of agent PRs conflicted with their base branch [3], whereas this paper targets conflicts between concurrently open agent PRs.The authors state that prior work had not evaluated inter-PR conflict, intra- versus cross-agent pair proportions, or a taxonomy of those conflicts.Source: §3 · p. 2

4 Evaluation Pipeline

The evaluation pipeline analyzes agent-authored PRs from AIDev-pop, formalizes co-activity using padded activity windows, and uses stratified merge replay to assess conflicts between co-active pairs.

4 Evaluation Pipeline

  • The evaluation pipeline extracts 33,596 unique agentic pull requests across 2,807 repositories from AIDev-pop.Source: §4 · p. 2

4.1 Formalizing Co-activity (RQ1–RQ2)

  • Co-activity is determined by overlap between same-repository PR activity windows, with thresholds k ∈ {0, 1, 3, 7} days; pairs are intra-agent when authored by the same model and cross-agent otherwise.Unresolved PRs at data cutoff have their closing time bounded by the global data-cutoff epoch.Source: §4.1 · p. 3

4.2 Large-Scale Merge Replay (RQ3)

  • The merge-replay process samples one co-active pair per repository to limit skew from highly active repositories, using 625 intra-agent repositories and all 122 repositories with cross-agent activity.This yields 747 pairs; the process fetches each pair's PR head commits into a bare Git workspace.Source: §4.2 · p. 3

5 Results

Across 33,596 agent-authored PRs in 2,807 repositories, the results show broad, predominantly same-agent concurrency; replayed merges found higher textual-conflict rates for cross-agent pairs and co-activity varied across repository throughput, agents, and languages.

5.2 RQ2 – Composition Profile

  • At k=0, cross-agent interactions were 0.50% of co-active pairs across 122 repositories; the most frequent cross-agent pairings involved OpenAI Codex.The paper describes the concurrency burden as primarily multiple PRs from the same agent.Source: §5.2 · p. 3, Figure 1 · p. 4

5.1 RQ1 – Concurrency Prevalence

5.3 RQ3 — The Rates & Taxonomies of Textual Conflicts

  • 41.7% of cross-agent pairs and 19.8% of intra-agent pairs had textual conflicts, with non-overlapping 95% confidence intervals.These textual-layer rates are a lower-bound estimate of overall inter-PR conflict.Source: §5.3 · pp. 3–4, Figure 2 · p. 5
  • 57.6% of conflict reports were content conflicts; modify/delete and add/add conflicts accounted for 26.8% and 15.1%, respectively.Source: Table 2 · p. 4, §5.3 · p. 4, Figure 3 · p. 5
  • 84.4% of conflicted files were source code, versus 3.9% dependency manifests and lockfiles; 19.2% of conflicting pairs involved dependency files.Source: Table 2 · p. 4, §5.3 · p. 4, Figure 3 · p. 5

5.4 RQ4 – Associations with Concurrency

  • PR-level co-activity rose with repository throughput, from approximately 53% in repositories with fewer than 6 agent PRs to approximately 90% in those with over 100.Intermediate-throughput repositories (6–20 PRs) were approximately 68%, and high-throughput repositories (21–100 PRs) approximately 85%.Source: §5.4 · p. 5
  • PR-level co-activity ranged from Devin at 82.5% to Claude Code at 39.7%; differences appeared more influenced by deployment than inherent platform properties.Source: §5.4 · p. 5
  • Observed co-activity ranged from approximately 92% for Go repositories to approximately 44% for HTML repositories, but these rounded associations were not significance-tested or confounder-adjusted.Source: §5.4 · p. 5

6 Discussion

The discussion finds that same-agent concurrency is the dominant operational pattern, while independent PR workflows create textual conflicts. It also frames downstream costs as plausible consequences, not measured outcomes.

6.1 The Reality of Autonomous Software Engineering

  • Same-agent concurrency is the dominant composition mode, and identical-agent PRs have a 19.8% textual conflict rate amid agents’ lack of horizontal awareness.The discussion identifies managing one framework’s high-frequency output—not cross-vendor coordination—as the central operational issue.Source: §6.1 · p. 5
  • Source code accounts for most conflicts, while approximately 42% of conflict signals are structural modify/delete or add/add disagreements.Structural conflicts concern whether a file should exist, not only competing edits to the same lines.Source: §6.1 · p. 5

6.2 The Hidden Operational Costs of Agent Churn

  • The proposed downstream costs—wasted CI/CD compute, token use for conflict resolution, and maintainer reconciliation—are hypothesized rather than directly measured.The paper does not measure CI/CD logs, token usage, or maintainer time; it identifies quantifying these costs as future research.Source: §6.2 · pp. 5–6

7 Threats to Validity

The study’s validity is bounded by its operational definitions, an uneven distribution of activity across repositories, and the dataset’s coverage of open-source projects and agent profiles.

Construct Validity

  • Temporal co-activity is a proxy for concurrent work, and git three-way merges capture textual conflicts but not deeper build or semantic conflicts, making results a conservative lower bound.The study reports sensitivity from zero through seven days to avoid relying on an arbitrary co-activity threshold.Source: §7 · p. 6

Internal Validity

  • The top ten repositories account for 91.4% of raw co-active pairs; the cross-agent merge-replay stratum is smaller than the intra-agent stratum (N=115 versus N=601).The study normalizes prevalence and samples one co-active pair per repository to address repository activity bias.Source: §7 · p. 6

External Validity

  • Findings cover open-source projects and five agent profiles from December 2024–July 2025; patterns may differ in proprietary enterprise codebases with stricter branching rules.The authors also note that AIDev mined each agent independently and that multi-vendor use had only begun during the study period.Source: §7 · p. 6

8 Conclusion and Next Steps

The conclusion reports widespread temporal overlap among agent PRs and textual merge conflicts in replayed pairs, with conflicts concentrated in source files. It calls for orchestration and runtime-aware measurement of consequences not measured here.

8 Conclusion and Next Steps

  • 79.4% of agent PRs overlap temporally; merge replay found textual conflicts in 19.8% of homogeneous and 41.7% of cross-agent scenarios, with 84.4% of conflicted files being source code.The conclusion also reports that 42% of conflicts were structural and 58% content-based.Source: §8 · p. 6
  • The authors call for orchestration, live workspace communication, and sandboxed runtime replay to measure build failures, semantic failures, and operational costs that remain hypothesized.The proposed cost measures include CI compute and token spend.Source: §8 · p. 6

Availability of Data

A replication package provides pair-level merge results, analysis tables, and scripts, while the original PR corpus is the publicly available AIDev-pop dataset.

Availability of Data

  • The permanently archived replication package includes pair-level results for 747 sampled pairs, rate and taxonomy tables, and scripts; the source corpus is AIDev-pop.The package supports recalculation of the numerical values in section 5.3, Table 2, and Figures 2–3.Source: Availability of Data · p. 7
Loading 2607.04697v2…