PaperlayerView exact revision on arXiv

Source-linked AI summary

What Will Replace Pull Requests? Rethinking Software Collaboration for AI Agents

Christoph Treude, Sebastian Baltes

arXiv:2610.34933v1cs.SEcs.HC

TL;DR

Coding-agent traffic is pressuring pull-request workflows, while project restrictions and agent-based review challenge records researchers rely on. We reviewed 50 gray-literature documents; most responses retained pull requests, and only Zed reported using a replacement.

  • Problem

    Coding agents are putting pull-request workflows under pressure, while restrictions and reviews conducted in agent conversations complicate the pull-request records used in empirical research.

    Source: §I · p. 1

  • Method

    We reviewed 50 gray-literature documents and used a coded search-and-screening process with independent model coding and human resolution.

    Source: §I · p. 1, §III · p. 2

  • Results

    Most responses retained pull requests; only Zed reported using a replacement, while two documents outlined non-pull-request external contribution and neither reported adoption.

    Source: B. Responses · p. 3

  • Takeaways & Limitations

    Pull-request datasets can miss agent-conversation reviews and projects that disable pull requests, challenging the records and assumptions used in empirical research.

    Source: §I · p. 1, §VII · p. 4

    When session records stay private, researchers will depend on vendor partnerships or studies that record the data themselves.

    Source: §V · p. 4

Abstract

from arXiv · show

The emergence of artificial intelligence (AI) coding agents has put pull requests under pressure: agents open millions of pull requests per month, while some open-source projects have begun to restrict or disable external contributions. We investigate which alternatives to pull requests practitioners are proposing or adopting and what these changes mean for empirical software engineering research. We reviewed 50 gray-literature documents from 31 organizations or individuals published between October 2025 and October 2026, coding the problems each named, the mechanisms it proposed or adopted, which of five roles of pull requests (integration, review, rationale, accountability, and external contribution) each mechanism takes over, and the evidence offered. Most responses retained pull requests in some form, whether by adding tooling, attaching a new artifact, or restricting access, and restriction was the response most often reported as adopted. Several proposed alternatives moved review and rationale into agent conversations and session records, but only Zed reported using a replacement in practice. Only two documents outlined a replacement for external contribution, and neither reported adoption. Pull requests are changing in ways that pull-request datasets will not capture: reviews conducted in agent conversations leave no pull-request comments, and projects that disable pull requests disappear from samples of newly opened ones. We outline a research agenda on the effectiveness of the emerging alternatives, their consequences for open-source participation, and the adaptation of empirical methods.

I. INTRODUCTION

Coding-agent activity is putting pull-request workflows under technical and social pressure as projects restrict contributions and practitioners test alternatives. These changes also challenge the records used in empirical software engineering research.

I. INTRODUCTION

  • In June 2026, agents opened more than 17 million pull requests monthly; high-adoption periods coincided with 441.5% higher median review time and 31.3% more pull requests merged without review.Faros AI compared each organization’s periods of lowest and highest AI adoption.Source: §I · p. 1
  • Ghostty, Ladybird, and OpenAI’s Codex repository restricted or closed external contributions, while GitHub added controls to cap or disable pull requests.Source: §I · p. 1
  • Zed’s Delta supports implementation, review, and integration in agent conversations without pull requests, while Entire stores prompts and tool actions alongside commits.Zed disabled pull requests on Delta’s repository; by September 16, 2026, 33 people had committed 570 changes there.Source: §I · p. 1
  • The paper examines which problems alternatives address, which pull-request roles they take over, and which projects have adopted them, informing a research agenda based on 50 gray-literature documents.Source: §I · p. 1
  • Agent-conversation reviews leave no record in pull requests, and projects that disable them disappear from samples of newly opened pull requests.Pull requests otherwise record how contributions are proposed, reviewed, and accepted; prior work used that record to study acceptance and assemble the AIDev dataset.Source: §I · p. 1

II. FIVE ROLES OF PULL REQUESTS

The paper distinguishes five roles of pull requests to compare proposed responses, while noting that these roles do not exhaust pull requests’ functions. Most responses take over or restrict one role and leave the others to the pull request.

II. FIVE ROLES OF PULL REQUESTS

  • The review uses five roles to compare responses: integration, review, rationale, accountability, and external contribution.The roles are not claimed to exhaust every function of pull requests.Source: §II · p. 1, Table I · p. 2
  • A pull request combines a proposed branch and diff with discussion and a merge button; contributors without commit rights can fork and submit changes for maintainer review and integration.Source: §II · p. 1
  • Applying a coherence criterion that separates concepts serving multiple purposes, the review treats pull requests’ roles as distinct; most responses take over or restrict one role and leave others to the pull request.The cited criterion had identified Git commits as combining work-saving and logical grouping purposes.Source: §II · p. 1

III. METHOD

The authors reviewed gray literature using link-following and web searches, then coded documents for problems, mechanisms, pull-request roles, stance, and evidence. Two models coded independently, with human adjudication and an audit of a sample of agreements.

III. METHOD

  • The search followed links from an initial Delta announcement until two consecutive documents added no new material, then used web searches and a follow-up pass.The second pass added 13 documents but no new problem or mechanism category.Source: §III · p. 2
  • The review included 50 documents published from October 2025 to October 6, 2026, representing 31 sources; vendors supplied 22 documents.The corpus also included maintainer policies, practitioner essays, press or analyst items, and community discussions.Source: §III · p. 2
  • The codebook guided coding of stated problems, proposed or adopted mechanisms, assigned pull-request roles, stance toward pull requests, and strongest evidence.A role was assigned only when a document said a mechanism took over or supported it; documents and mechanisms could receive multiple codes.Source: §III · p. 2
  • Two models agreed on 88% of 819 codes; one author resolved disagreements and audited 15% of codes on which all three agreed, changing one.Cohen’s κ ranged from 0.46 for inclusion to 0.96 for mechanism.Source: §III · p. 2
  • The corpus is not representative of software projects, and one author resolved disagreements without a second human coder; model agreement cannot establish correct classifications.The search may favor English-language sources that link to one another; reported frequencies describe reviewed documents, not industry-wide adoption.Source: §III · p. 2

A. Problems

The reviewed documents named different problems with pull requests, with review capacity the most frequent and the only one appearing across every source category. Maintainers emphasized contribution norms and accountability, while vendors emphasized provenance.

A. Problems

  • Review capacity was named most often and was the only problem named in every source category.Source: A. Problems · p. 2
  • Lost provenance was named mainly by vendors, including every document from Zed and Entire, whose products store agent sessions.Source: A. Problems · p. 2
  • All nine maintainer documents named contribution norms, seven named accountability, and none named volume or infrastructure.The Ghostty policy attributed reduced effort-based backpressure to agentic programming.Source: A. Problems · p. 2

B. Responses

Most responses kept pull requests, often with added tooling or artifacts, while restriction was the most frequently reported adopted response. Proposals shifted review and rationale into agent conversations and records, but only Zed reported using a replacement, and alternatives for external contribution remained unadopted.

Responses and mechanisms

  • Across 44 documents, nine proposed keeping pull requests with tooling, nine extending them with an artifact, 11 replacing the unit of work, and 12 restricting submitters.Two would leave GitHub while retaining pull requests; one would dispense with pull requests and human review. The review separately identified six mechanisms across 42 documents.Source: B. Responses · p. 3

Retaining pull requests

  • In 20 of 44 position-taking documents, pull requests remained the unit of work with tooling, an attached artifact, or a different host.Another 12 documents restricted access while retaining pull requests; risk-prioritized review appeared in 14 documents.Source: B. Responses · p. 3

Replacement in practice

  • Only Zed reported using a pull-request replacement; among platform vendors, it alone shipped a redesigned collaboration workflow as a public beta.Cursor’s Origin and Harness’s migration rebuilt hosting for agent traffic while retaining pull requests and their workflow.Source: B. Responses · p. 3

Review and rationale

  • Threads and session checkpoints carried most review and rationale proposals, while six documents proposed reviewing agent instructions rather than generated code.Zed’s Delta uses agent subthreads and code-attached comments; Entire records prompts, transcripts, files, token usage, and tool calls. Linux and LLVM adopted Assisted-by tags, and Propel proposed structured agent-context alongside diff review.Source: B. Responses · p. 3

Restriction

  • Restriction was the most often adopted response: seven of 12 restricting documents described project policies already in use.The other four were GitHub announcements or essays about contribution controls, and one was a practitioner essay.Source: B. Responses · p. 3

External contribution

  • Only two documents proposed external contribution without pull requests, both through prompt requests, and neither reported adoption.One proposed supporters lending tokens to fund submitted prompts; the other proposed maintainers refining prompts and generating implementations under controlled conditions.Source: B. Responses · pp. 3–4

C. Evidence

The evidence offered for the proposed responses was limited: many documents relied on arguments, anecdotes, or repeated figures, and none tested whether the mechanisms improved review.

Evidence

  • None of the reported measurements tested whether a proposed mechanism improved review or whether session records helped reviewers find defects.Among 44 position-taking documents, 15 offered only arguments, eight anecdotes, and six repeated numbers reported elsewhere; Rust was one of two maintainers reporting its own project data, counting 1,281 open pull requests.Source: C. Evidence · p. 4

V. IMPLICATIONS

Changes to collaboration can make pull-request data mismeasure review and omit projects or changes from samples. Researchers need records that capture off-pull-request activity and dated policy changes, though access varies.

What pull-request data measures

  • Reviews conducted in agent conversations or session checkpoints may leave no pull-request comments, causing comment-based studies to classify reviewed changes as unreviewed.The pull-request-based 31.3% rise in changes merged without review could appear larger as review moves elsewhere, even if every change were reviewed.Source: §V · p. 4

Who is in the sample

  • Projects that disable pull requests and changes integrated through Delta threads disappear from pull-request datasets, potentially biasing samples of newly opened pull requests.Projects that cap or limit pull requests remain represented, but with less external contribution.Source: §V · p. 4

What data researchers need

  • Potential records differ in accessibility: Assisted-by trailers are in commit messages, Entire stores checkpoints on a repository branch, and Delta threads are hosted outside GitHub and its API.GitHub’s API gives current pull-request settings but not when they changed; researchers may need policy histories, announcements, vendor partnerships, or data they record themselves.Source: §V · p. 4

VI. FUTURE PLANS

The paper proposes research on whether alternative review workflows work, how contribution mechanisms affect participation, and how empirical methods should adapt to collaboration outside pull requests.

Effectiveness of alternatives

  • An experiment will compare review of agent-generated changes using a diff alone, a diff with its prompt, a rationale summary, or the full session record.Outcomes include defects found, false positives, acceptance-decision correctness, and review time; summary and full-record conditions address concerns about long conversations.Source: §VI · p. 4

Open-source participation

  • A matched-repository event study and interviews will examine changes in contribution and review activity after projects restrict or disable pull requests.The proposed work also asks whether newcomers find other routes in and whether existing non-pull-request workflows accommodate agent-generated changes.Source: §VI · p. 4

Empirical methods

  • Researchers should measure how much review occurs outside pull requests and revisit classic findings in projects that moved review into threads.Mining studies also need dated policy-change records and review measures that do not depend on pull-request comments.Source: §VI · p. 4

VII. CONCLUSION

The question of what will replace pull requests remains open: reviewed alternatives redistribute their roles, but none of the implemented replacements demonstrates how external contributors would participate. Changes are already underway, challenging both pull requests as a research mechanism and assumptions behind pull-request datasets.

  • No implemented replacement in the corpus has demonstrated how external contributors would participate.Source: §VII · p. 4
  • Platforms have rebuilt infrastructure for agent traffic, large projects have restricted contribution or introduced AI-code disclosure requirements, and Zed has moved Delta development out of pull requests.Source: §VII · p. 4
  • The alternatives reviewed redistribute the roles that pull requests combine, leaving the replacement question open.Source: §VII · p. 4
  • Researchers face changes to both the mechanism they study and the assumptions behind pull-request datasets.Source: §VII · p. 4

Related papers

Loading 2610.34933v1…