Source-linked AI summary
Repairnator patches programs automatically
Martin Monperrus, Simon Urli, Thomas Durieux, Matias Martinez, Benoit Baudry, Lionel Seinturier
TL;DR
The paper addresses repair of CI build failures, especially test failures, by generating patches for developers to review. Repairnator processed thousands of failures, and five of its 12 drafted patches were accepted and merged.
Problem
The paper asks whether automatically generated patches for CI build failures can be accepted by developers as valid contributions.
Source: Automatic Repair and Continuous… · p. 2Method
Repairnator analyzes CI logs, reproduces failures locally, runs repair prototypes, and has a team member check and propose any patch to developers.
Source: Repairnator in a Nutshell · p. 2Results
Five of 12 drafted patches were accepted and merged after Repairnator processed 6,173 failures.
Source: Expedition Achievements · p. 3Takeaways & Limitations
The project demonstrated that program repair can be human-competitive and provided bug and CI data to the research community.
Source: Expedition Achievements · p. 3, The Future · p. 4Bugs requiring deep algorithmic or architectural understanding are expected to remain out of reach for automatic repair for the foreseeable future.
Source: The Future · p. 4
Abstract
from arXiv · showhide
Repairnator is a bot. It constantly monitors software bugs discovered during continuous integration of open-source software and tries to fix them automatically. If it succeeds in synthesizing a valid patch, Repairnator proposes the patch to the human developers, disguised under a fake human identity. To date, Repairnator has been able to producepatches that were accepted by the human developers and permanently merged into the code base. This is a milestone for human-competitiveness in software engineering research on automatic program repair.
Automatic Repair and Continuous Integration
Repairnator targets CI build failures by generating patches for developers to review, with acceptance as evidence of human-competitive repair. Its comparison setup notifies the bot and a human developer at the same time.
Automatic Repair and Continuous Integration
- About one in four CI builds fails, most commonly because of test failure.CI runs compilation and tests for each commit.Source: Automatic Repair and Continuous… · p. 2
- Repairnator generates patches for build failures, and developer acceptance of a patch as a valid contribution counts as success.Such successes would be evidence of human-competitiveness in program repair.Source: Automatic Repair and Continuous… · p. 2
- In a Turing-test-like setup, the human developer and Repairnator are notified simultaneously when a test failure occurs.Repairnator proposes Java source-code patches, unlike CGC defender bots, which applied binary matches in machine-level code.Source: Automatic Repair and Continuous… · p. 2
Repairnator in a Nutshell
Repairnator monitors Travis CI builds and launches frequent repair attempts to find patches before human developers. Its pipeline analyzes logs, reproduces failures, and runs repair prototypes before a team member checks and proposes a patch.
Repairnator in a Nutshell
- Repairnator monitors thousands of Travis builds and launches new repair attempts every minute to fix bugs before the human developer.It specializes in build failures during continuous integration, focusing on broken tests.Source: Repairnator in a Nutshell · p. 2
- Its three-stage pipeline analyzes CI logs, locally reproduces failures, and runs program-repair prototypes; a team member sanity-checks any patch before proposing it to developers.The bot also collects data for future program-repair research.Source: Repairnator in a Nutshell · p. 2
- Repairnator focuses on Java, Maven, GitHub, and Travis CI, using Nopol [9], Astor, and NpeFix [11] to generate fixes.Nopol uses dynamic analysis and SMT-based code synthesis; Astor is based on GenProg [12], while NpeFix analyzes Java null pointer exceptions.Source: Repairnator in a Nutshell · p. 2
Expedition Achievements
Repairnator’s expeditions produced patches accepted and merged by human developers, demonstrating human-competitive program repair. The results also expose limits in patch quality and in the bot’s ability to respond to requested changes.
Expedition Achievements
- The project operated through a January 2017 pilot, Expedition #1 from February through December 2017, and Expedition #2 from January through June 2018, which monitored Travis CI in real time.The pilot prototype performed approximately 30 repair attempts per day.Source: Expedition Achievements · pp. 2–3
- In Expedition #2, Repairnator drafted 102 patches and suggested 12 to developers, who accepted five as human-competitive contributions.The first accepted patch was merged on January 12, 2018.Source: Expedition Achievements · p. 3
- In Expedition #1 [13], none of the patches were human-competitive because they arrived too late or were low quality, often repairing only one buggy input.The patches could make CI builds successful, but the patch analysis found none human-competitive.Source: Expedition Achievements · p. 3
- After six more months, Repairnator had processed 6,173 failures and drafted 12 patches, five of which were accepted and merged.One Ditto patch added a precondition guarding a crash-causing statement; Nopol [9] synthesized the condition.Source: Expedition Achievements · p. 3
- The other seven proposed patches were not accepted because some owners did not respond or requested further changes beyond Repairnator’s capabilities.The authors suggest some projects may not accept external contributions.Source: Expedition Achievements · p. 3
- The authors conclude that Repairnator demonstrated that program repair can be human-competitive, delivering good-quality patches before humans found patches.The project also provided bug and continuous-integration data to the research community.Source: Expedition Achievements · p. 3
The Future
Repairnator’s early experiment is presented as proof-of-concept evidence that human-competitive patches can be generated automatically, though the authors identify limits and describe broader potential.
The Future
- Five accepted patches from 6,173 failures did not provide serious relief to human developers.The authors nevertheless frame the experiment as proof-of-concept evidence that human-competitive patches can be generated automatically.Source: The Future · p. 3
- At an estimated 12 million bugs globally and $400 per fix, the observed acceptance rate represents potential savings of $3.8 million globally.Source: The Future · p. 4
- Bugs requiring deep algorithmic or architectural understanding are expected to remain out of reach for automatic repair for the foreseeable future.Source: The Future · p. 4
- The authors expect repair bots to improve and fix easier bugs, while humans handle the toughest repairs and may complete partial bot fixes.They also describe humans making initial fixes that bots examine and validate, and view bots as removing tedious software-development work.Source: The Future · p. 4
- The Repairnator project also made information about bugs and continuous integration available to the research community.Source: The Future · p. 4
Authors
The biographies describe the authors’ academic and industry roles and research interests, while the included table caption identifies five developer-accepted Repairnator patches.
Authors
- Martin Monperrus is a KTH professor whose research focuses on automatic program repair, program hardening, and chaos engineering.Source: Authors · p. 5
- Simon Urli helped create Repairnator while researching automatic software repair and became a software engineer at XWiki in 2018.Source: Authors · p. 5
- Thomas Durieux researches automation of software-engineering tasks, including patch generation and fault localization using dynamic and static techniques.Source: Authors · p. 5
- Matias Martinez’s research interests include software evolution, automated software repair, software testing, and mobile applications.Source: Authors · p. 5
- Benoit Baudry is a KTH software-technology professor who leads the CASTOR software research center.Source: Authors · p. 5
- Lionel Seinturier is a Lille computer-science professor whose research interests include distributed systems and self-adaptive software systems.Source: Authors · p. 5
- Table 1 covers five Repairnator patches accepted by the developer community.Source: Table 1 · p. 6