Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests

1 Introduction

• The study tracks 6,774 merged agent PRs across 891 repositories and five AI coding agents, against 5,044 merged human PRs from the same repositories, verifying candidate fixes with annotators and an LLM judge (Cohen’s κ = 0.78; direct-fix precision 90%).Source: §1 · pp. 2–3
• 1.62 times the human odds: merged agent PRs had verified fixes more often within 30 days, with half of the 30-day incidence accruing in the first week.The comparison uses human PRs from the same repositories and observation window.Source: §1 · p. 2
• 69.6% of verified fixes on agent merges came from the same agent, while 89% of verified fixes on human merges came from humans.The findings describe self-fixing in both authoring populations.Source: §1 · p. 2
• 76.4% of verified-fix PRs on agent merges were agent-authored throughout all commits, with commits averaging 87.4% agent authorship per PR.These measures cover commits between first push and merge.Source: §1 · p. 3
• Agent PRs with and without follow-up fixes had nearly identical overall merge-time signal distributions, but ten times the commits within a repository corresponded to 6.1 times the verified-fix odds.The overall effects were negligible to small.Source: §1 · p. 3

2.1 AI Coding Agents in Software Development

• Agent coding has spread across open-source and industry settings, while prior research on adoption motivates examining what remains after an agent PR merges.The paper situates its study amid rapid adoption and notes earlier work on agent use and PRs.Source: §2.1 · pp. 3–4

2.2 The Study of AI Coding Agent’s Pull Requests

• Prior PR studies cover first-push characteristics, review, merge outcomes, and post-merge changes, but this study mines follow-up fixes and validates candidates with humans and an LLM judge.Earlier post-merge work examined surviving static-analysis issues, modification patterns, and explicitly referenced corrective PRs.Source: §2.2 · p. 4

2.3 Code Ownership in the Age of AI Coding Agents

• Earlier ownership research examines quality and self-fixing, while this paper measures who authors a follow-up fix after an agent PR merges and whether the merged PR’s author class returns to fix it.Prior human self-fixing studies report developers fixing roughly half of their own defects.Source: §2.3 · p. 5

3.1 Overview

• The study links merged PRs to candidate fixes, verifies candidate pairs, then analyzes who opened each fix PR, who authored its commits, and merge-time signals.The analyses use merged agent and human PRs, with the human cohort serving as a baseline for follow-up fixes.Source: §3.1 · p. 5
• Candidate fixes must merge in the same repository within 30 days, carry a fix task-type tag, share edited files with the original PR, and include a non-boilerplate file.The overview also identifies human validation and an LLM judge as the added verification condition.Source: §3.1 · p. 6

3.2 Dataset

• 6,774 merged agent PRs from 891 repositories and five agents form the agent cohort, alongside 5,044 merged human PRs from the same popular-repository cohort.Repositories were filtered to more than 500 GitHub stars; the cross-cohort comparison uses the human cohort’s end date as a shared cutoff.Source: §3.2 · p. 6

4.1.1 Approach of RQ1

• Candidate fixes were later same-repository PRs within 30 days, tagged as fixes, sharing edited files with the merged PR and including a non-boilerplate file.Candidate fixers could come from either the agent or human cohort, and partially observed merges were excluded.Source: §4.1.1 · p. 7
• A candidate pair counted as a verified fix only when annotators labeled it a direct fix; the LLM judge reached κ = 0.78 and 90% direct-fix precision on revalidation.The annotators distinguished direct fixes from related touches and unrelated pairs; the judge used the same PR information and was revalidated on sampled agent and human pairs.Source: §4.1.1 · pp. 7–8

4.1.2 Results of RQ1

• 4.5% of fully observed agent merges had a verified fix within 30 days, while 22.9% had a candidate fix.The full agent cohort included 4,505 PRs with a complete 30-day observation window; Table 1 covers candidate, verified, and precision-adjusted rates.Source: §4.1.2 · p. 8, Table 1 · p. 8
• 1.62 times the human odds of a verified fix were observed for agent merges within the same repositories and shared observation window.The repository-stratified odds ratio was 1.62 (95% CI=1.10–2.39, p=0.015).Source: Table 2 · p. 8, §4.1.2 · p. 9
• Half of the 30-day verified-fix incidence accumulated within the first week in both cohorts, with cumulative incidence reaching 4.5% for agents versus 2.6% for humans by day 30.The cumulative-incidence curves separate from the first day; each merge enters the time-to-event view with its own observation time.Source: §4.1.2 · p. 9, Figure 3 · p. 9

4.2.1 Approach of RQ2

• The study classifies each verified-fix PR author as human, self-agent, or other-agent based on who opened the fixing PR and whether it was the same agent as the original PR.The same classification is applied to verified fixes of the human cohort as a baseline.Source: §4.2.1 · p. 10

4.2.2 Results of RQ2

• 69.6% of verified fixes on agent merges are by the same agent, versus 27.4% by a human and 3.0% by another agent; 89.1% of verified fixes on human merges are by humans.The self-fixing pattern holds for every agent with a non-trivial count; Codex merges are most often human-fixed (38.4%), while Copilot self-fixes about 95%.Source: §4.2.2 · p. 10

4.3.1 Approach of RQ3

• The commit-level analysis examines who authored commits within verified-fix PRs, rather than only who opened the PR.It aggregates per PR, reporting both average agent-authored share and the percentage of PRs whose commits are all agent-authored.Source: §4.3 · p. 11, §4.3.1 · p. 11
• Commits are classified as agent, automation, or human using author-login patterns and agent-generated commit-message patterns.Because Codex does not sign commits and fewer than 2% carry a marker, it is excluded from the commit-share measure; unmarked agent commits may be counted as human, so agent shares are a conservative lower bound.Source: §4.3.1 · p. 11

4.3.2 Results of RQ3

• Verified-fix agent PRs average 87.4% agent-authored commits per PR, and 76.4% are agent-authored throughout all commits.Across four agents and all merged PRs, the corresponding values are 78.8% average agent-authored share and 54.1% all-agent-authored PRs, lower than on verified-fix PRs.Source: §4.3.2 · pp. 11–12

4.4.1 Approach

• The study compares agent PRs with and without verified fixes to identify merge-time characteristics associated with later fixes.The within-repository analysis tests each signal while accounting for repository differences and PR size.Source: §4.4 · p. 12, §4.4.1 · p. 13
• Merge-time comparisons use review time, commits per PR, post-first-push churn, and review items, with pooled tests and within-repository conditional logistic regressions.The regression controls for repository and the number of non-boilerplate files shipped; its odds ratio compares same-repository merges ten times apart on a signal.Source: §4.4.1 · pp. 12–13

4.4.2 Results

• A merge with ten times the commits carries 6.1 times the odds of a verified fix within the same repository; review items carry 2.4 times, churn 1.3 times, and review time has no significant effect.In the pooled comparison, signal distributions are nearly identical, with negligible-to-small effects; the within-repository analysis holds repository and shipped-file count fixed.Source: §4.4.2 · pp. 13–14, Table 6 · p. 13, Figure 5 · p. 14

5.1 The merge is not the end of the work, and code review is needed.

• 4.5% of agent merges carry a verified fix within 30 days, which is 1.62 times more than human merges; at the aggregate level, merge-time signals show only minor differences.The study indicates that PRs should be monitored beyond merge because signals may live in code rather than merge metadata. It says verification requires reading code changes.Source: §5.1 · p. 15

5.2 Owners fix their own code, but now the owner is an agent.

• Agent merges are fixed by the same agent 69.6% of the time, paralleling humans fixing human merges 89% of the time, but a deprecated agent leaves no experienced insider behind.The discussion identifies future questions about model-level ownership and whether projects should adopt explicit ownership policies.Source: §5.2 · p. 15

Construct Validity

• Verified-fix rates are floors because file co-location within 30 days can miss cross-file, later, or non-fix-typed repairs.Under 2% of PRs reach the 30-commit fetch cap, which the authors accept.Source: §6 · p. 16

Internal Validity

• Unmarked agent PRs may be counted as human-authored, and fixer identity captures authorship rather than ownership through code review.The authors report that findings remain when agents consistently fix more than half of their code.Source: §6 · p. 16

External Validity

• Findings apply to five AIDev agents in repositories with at least 500 stars, not to other agents, low-activity repositories, or behavior after the data window.The authors verified that their snapshot reproduces Li et al. (2026)’s ≥101-star curated set exactly.Source: §6 · p. 16

Conclusion Validity

• Verification is more reliable for binary Direct-fix labels than three-way labels, and the judge’s Direct-fix precision is 90% in both cohorts.Cross-cohort comparisons use shared observation windows and repository-stratified estimates; own-window cohort rates are not compared.Source: §6 · p. 16

7 Conclusion

• Agent merges attract verified fixes at 1.62 times the odds of human merges, while 69.6% of verified fixes come from the same agent.On average, 87.4% of commits to verified-fix PRs for agent merges are agent-authored.Source: §7 · p. 17
• Merges later needing a fix have nearly the same merge-time signals as other merges; within repositories, commit count is the clearest signal.Agents largely finish their own work, but their merges need fixing modestly more often than human merges, with little visibility up front.Source: §7 · p. 17

Statements and Declarations

• The work was supported by the Australian Composites Manufacturing CRC under Grant HPN003 – ForgeX.ai, AI factory management (Phase 1).Source: Statements and Declarations · p. 17
• Wannita Takerngsaksiri led study design, data collection, analysis, and drafting; Nhat Duong joined materials preparation and human validation.Scott Barnett reviewed the work and acquired funding; all authors commented on and approved the manuscript.Source: Statements and Declarations · p. 17
• The replication package is available on GitHub, and the study uses public AIDev data without private or personally identifiable data.Source: Statements and Declarations · p. 17
• The author declared no conflict of interest, and ethical approval, informed consent, and a clinical trial number were not applicable.Source: Statements and Declarations · p. 17

A The Labeler for Verified Fix Annotation

• Figure 6 shows the web labeler screen used by human annotators to label an original PR and candidate fix pair.Source: A The Labeler for Verified Fix… · p. 20, Figure 6 · p. 21
• Annotators assign one of four labels: Direct fix, Related touch, Unrelated/mislabeled, or Indeterminate.Source: A The Labeler for Verified Fix… · p. 21
• Only a Direct fix is verified: the candidate repairs a defect, regression, or wrong or incomplete behavior introduced or left behind by the original PR.Related touches extend or refactor the same area without repairing a defect; unrelated shared-file changes and insufficient evidence receive separate labels.Source: A The Labeler for Verified Fix… · p. 21

• Author login and commit message are the two evidence sources used to map author class in RQ3.Source: B The Rule List of Commit Author-Class… · p. 21
• Table 7 presents the complete set of commit author-class mapping rules for each agent.Source: B The Rule List of Commit Author-Class… · p. 21, Table 7 · p. 21