Source-linked AI summary
Energy Consumption of Automated Program Repair
Matias Martinez, Silverio Martínez-Fernández, Xavier Franch
TL;DR
Energy use in automated program repair (APR) has not previously been studied, despite its relevance to greener software development. This paper measures energy use across traditional repair tools and fine-tuned language models, finding that some tools achieve more correct patches while using less energy than others.
Problem
No previous APR research had focused on the energy consumption of APR tools, leaving the energy impact of their use unknown.
Source: §1.1 · p. 1Method
The study measures energy used to find the first plausible patch with 10 traditional APR tools and 10 fine-tuned LLMs on Defects4J bugs.
Source: §1.2 · p. 1Results
Some tools combined lower energy use with more correct first patches; among the studied LLMs, CodeGen350M had the lowest median energy and the most correct first patches.
Source: §1.3 · p. 2Takeaways & Limitations
The initial study is a first step toward understanding APR energy use, which the authors consider essential to green software development.
Source: §1.1 · p. 1, §1.4 · p. 2The two APR families were reported separately because their setups differ in fault localization, GPU requirements, and candidate-patch evaluation rules.
Source: §1.2 · p. 2
Abstract
from arXiv · showhide
Automated program repair (APR) aims to automatize the process of repairing software bugs in order to reduce the cost of maintaining software programs. Moreover, the success (given by the accuracy metric) of APR approaches has increased in recent years. However, no previous work has considered the energy impact of repairing bugs automatically using APR. The field of green software research aims to measure the energy consumption required to develop, maintain, and use software products. This paper combines, for the first time, the APR and Green software research fields. We have as main goal to define the foundation for measuring the energy consumption of the APR activity. We measure the energy consumption of ten traditional program repair tools for Java and ten fine-tuned Large-Language Models (LLM) on source code trying to repair real bugs from Defects4J, a set of real buggy programs. The initial results from this experiment show the existing trade-off between energy consumption and the ability to correctly repair bugs: Some APR tools are capable of achieving higher accuracy by spending less energy than other tools.
1 EXTENDED ABSTRACT
The paper establishes an initial study of energy use in automated program repair and finds trade-offs between energy consumption and the ability to repair bugs. It examines traditional APR tools and fine-tuned LLMs on Defects4J.
1.1 The problem addressed
- Energy use in APR had not previously been studied, and the authors argue it should be considered alongside accuracy when designing repair tools.They cite repeated correctness-oracle executions [8], complementary tools needed to address limited accuracy [5] [11], and potential repair attempts triggered by failed CI/CD builds [9]; Moritz et al. reported a 2.9% median and 10.3% mean rate of builds with failed tests in Java projects.Source: §1.1 · p. 1
1.2 The approach taken
- The paper’s goal is to measure and analyze APR energy trade-offs from software developers’ perspective, beginning with energy consumed to find plausible and correct patches.The research question asks how much energy Java APR tools consume to find the first plausible patch.Source: §1.2 · p. 1
- The study measures first-plausible-patch energy for 10 traditional APR tools and 10 fine-tuned LLMs on Defects4J.Repair attempts ran on Linux with 8 GPUs, and an external wattmeter sampled power every 20 milliseconds; the tool families used different fault-localization, hardware, and candidate-evaluation setups.Source: §1.2 · pp. 1–2
1.3 Preliminary Results
- SimFix showed a favorable energy-repairability trade-off, with a lower median energy than the highlighted template-based tools and 21 correct patches among 40 plausible patches.TBar had the highest median energy and the most correct first patches among the traditional tools: 28 of 71 plausible patches; the authors suspect its large template search space contributes to its energy use.Source: §1.3 · p. 2, Table 1 · p. 2
- CodeGen350M had the lowest median first-patch energy and the most correct first patches among the studied LLMs, with 18 of 39 patches correct.Larger models found more plausible patches but were not necessarily more correct and consumed more energy while using 8 GPUs.Source: §1.3 · p. 2
1.4 Conclusion and Future work
- Future work will study energy to the first correct patch, total repair-process energy, bug- and project-specific results, and relationships with efficiency metrics.The proposed efficiency metrics include repair time and number of patch candidates.Source: §1.4 · p. 2