Source-linked AI summary
AVATAR : Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations
Kui Liu, Anil Koyuncu, Dongsun Kim, Tegawendé F. Bisyandé
TL;DR
Pattern-based APR needs reliable and diverse fix ingredients, but mining relevant bug-fix changes from repositories is challenging. AVATAR uses fix patterns from patches addressing static-analysis violations and generated 49 plausible patches on Defects4J, including 35 manually confirmed correct patches.
Problem
Pattern-based APR depends on reliable, precise, and diverse fix ingredients, yet collecting suitable bug-fix changes from repositories is challenging.
Method
AVATAR mines developer patches that fix static-analysis violations and uses their categorized fix patterns as ingredients for patch generation.
Results
49 plausible patches were generated on Defects4J, including 35 manually confirmed correct patches.
Takeaways & Limitations
Fix patterns from static-analysis violation patches can provide relevant ingredients for APR targeting semantic bugs.
Takeaways & Limitations
The evaluation assumes perfect fault localization and covers Java programs with fix patterns from only FindBugs and PMD violations.
Abstract
from arXiv · showhide
Fix pattern-based patch generation is a promising direction in Automated Program Repair (APR). Notably, it has been demonstrated to produce more acceptable and correct patches than the patches obtained with mutation operators through genetic programming. The performance of pattern-based APR systems, however, depends on the fix ingredients mined from fix changes in development histories. Unfortunately, collecting a reliable set of bug fixes in repositories can be challenging. In this paper, we propose to investigate the possibility in an APR scenario of leveraging code changes that address violations by static bug detection tools. To that end, we build the AVATAR APR system, which exploits fix patterns of static analysis violations as ingredients for patch generation. Evaluated on the Defects4J benchmark, we show that, assuming a perfect localization of faults, AVATAR can generate correct patches to fix 34/39 bugs. We further find that AVATAR yields performance metrics that are comparable to that of the closely-related approaches in the literature. While AVATAR outperforms many of the state-of-the-art pattern-based APR systems, it is mostly complementary to current approaches. Overall, our study highlights the relevance of static bug finding tools as indirect contributors of fix ingredients for addressing code defects identified with functional test cases.
I. INTRODUCTION
APR targets semantic bugs detected by tests, but incomplete test suites make overfitting and incorrect patches common. This section motivates AVATAR, which mines static-analysis violation fixes as reliable and diverse ingredients for semantic-bug repair.
- I. INTRODUCTION: Incomplete test suites make APR prone to generating patches that pass available tests but violate intended behavior.Such overfitted patches are reported to be common.
- I. INTRODUCTION: Mining fix templates from human-written patches can improve correctness, but collecting substantial, precise, and diverse training fixes remains challenging.Manual template construction is limiting and error-prone, while mined patterns require relevant bug-fix changes.
- I. INTRODUCTION: Existing mining strategies trade coverage against reliability: simple bug types limit variety, while linked developer changes may include irrelevant patterns.Developer changes are not necessarily atomic, complicating extraction of useful fix changes.
- I. INTRODUCTION: Static-analysis fixes offer tool-based verification and violation categories that can reduce collection effort and support consistent patterns.Some semantic bugs in Defects4J have also been identified by static-analysis tools, while violations may signal more severe defects.
- I. INTRODUCTION: AVATAR investigates whether static-analysis violation fix patterns can serve as patch-generation ingredients for semantic bugs.The approach focuses on developer patches fixing statically detected violations.
- I. INTRODUCTION: AVATAR is presented as a novel pattern-based APR system and is empirically assessed on the Defects4J benchmark.Its fix ingredients come from patches verified by bug-detection tools as true bug-fix patches.
II. BACKGROUND
Pattern-based APR abstracts code changes into reusable templates that guide patch generation, but mining must balance coverage, actionability, precision, and data quality. Static analysis supplies categorized violations and a basis for identifying targeted fixes, illustrated through FindBugs examples.
- A. Automated Program Repair with Fix Patterns: Fix-pattern APR abstracts code changes into templates whose context constraints match faulty code and guide candidate generation.For example, Method Replacer substitutes a compatible method call for a potentially faulty one.
- A. Automated Program Repair with Fix Patterns: Manual mining can produce precise patterns but requires substantial human effort, whereas automatic mining scales through common modification rules or redundant sub-patches.Automatic mining remains vulnerable to noisy input from tangled changes.
- A. Automated Program Repair with Fix Patterns: Coarse-grained patterns cover more bug types but may be unactionable, while fine-grained patterns are actionable but cover fewer defects.Pattern granularity is therefore an intrinsic mining trade-off.
- B. Static Analysis Violations: Static analysis tools inspect source code or bytecode without dynamic execution and can cover more paths than dynamic analyses requiring test cases.They target errors including syntactic defects, security vulnerabilities, performance issues, and bad practices.
- B. Static Analysis Violations: Static-analysis warnings are called violations because they identify code fragments that fail analysis rules.Common Java tools include FindBugs, PMD, and Error-Prone.
- B. Static Analysis Violations: A FindBugs example flags an equals implementation that assumes a specific type instead of supporting all object types.The associated fix adds an instanceof check.
- B. Static Analysis Violations: Not every static-analysis violation is an actual defect because limited analysis information can produce false positives or irrelevant warnings.Developer acceptance is treated as evidence that a violation is actionable in the described setting.
- B. Static Analysis Violations: Static-analysis violation mining uses data collection, preprocessing, and fix-pattern mining, with changes retained when a violation disappears and affected lines lie within the commit diff.This criterion distinguishes an actual violation fix from coincidental removal through broader deletions.
III. MINING FIX PATTERNS FOR STATIC VIOLATIONS
Static-analysis violation fixes are mined from repository histories and transformed into AST-based fix patterns for APR. The process identifies actual violation-fixing changes, derives fine-grained edit scripts, and groups similar scripts into patterns.
- Static-analysis fix-pattern mining was introduced for quick fixes and maintenance, while AVATAR applies it to semantic bugs repaired and validated with test cases.
- The mining workflow comprises data collection, data preprocessing, and fix-pattern mining.
- Violation-fixing changes are retained when a violation disappears after a commit and the affected lines lie within that commit’s diff.
- Because line-based diffs do not directly represent semantic code entities, mining approaches convert buggy and fixed revisions into AST edit scripts with GumTree.
C. Fix Pattern Mining
AVATAR mines common edit actions from AST edit scripts and uses the resulting static-analysis fix patterns as ingredients for semantic-bug repair. Earlier live studies indicate that such mined patterns can produce patches accepted by developers.
- Similar AST edit scripts are grouped to infer a consistent common subset of repair actions, producing fix patterns.
- 67 of 116 FindBugs patches and 6 of 16 PMD pull requests generated from mined patterns were merged by developers.
- AVATAR is presented as the first reported APR approach using fine-grained static-analysis violation patterns to address semantic bugs revealed by test cases.
A. Fault Localization
AVATAR localizes suspicious statements, matches them against mined AST edit-script patterns, generates candidates, and validates them with tests. Candidate selection prioritizes fully fixing patches and then control-flow-changing partially fixing patches.
- A. Fault Localization: GZoltar executes tests and uses Ochiai suspiciousness scores to rank likely faulty statements.
- B. Fix Pattern Matching: AVATAR iteratively matches suspicious locations with mined patterns, whose repair actions are defined over specific AST node types.
- C. Patch Generation: A matching pattern that removes one sub-condition from a three-predicate IfStatement generates three patch candidates.
- D. Patch Validation: Patch candidates are assessed with test cases as fully-fixing or partially-fixing patches.
- D. Patch Validation: When no fully-fixing patch exists, partially-fixing candidates changing control flow are prioritized over candidates changing only data flow.
- D. Patch Validation: Control-flow-changing partial patches are ordered by AST edit distance, with smaller distances preferred and generation order breaking ties.
V. ASSESSMENT
The assessment evaluates AVATAR on Defects4J through research questions about effectiveness, target and pattern diversity, and comparison with state-of-the-art APR systems. It distinguishes localization-dependent repair effectiveness from patch-generation performance under known fault locations.
- Evaluation Setting: AVATAR is evaluated on Defects4J, a benchmark widely used by APR systems targeting Java programs.
- Research Questions: RQ1 measures how many semantic bugs can be repaired with static-analysis fix patterns, including scenarios with static-analysis localization and known bug locations.
- Research Questions: RQ2 examines the diversity of bugs repaired and the violation-pattern types successfully leveraged by AVATAR.
- Research Questions: RQ3 compares AVATAR’s repair performance with state-of-the-art systems and assesses its complementarity with existing APR approaches.
B. Experimental Setup
AVATAR is evaluated under multiple fault-localization configurations on Defects4J, including statically localized bugs, all bugs with perfect localization, and comparisons with existing APR tools. For statically detectable bugs, it correctly repairs 3 SpotBugs, 2 Infer, and 1 ErrorProne cases, while its mined patterns also repair four bugs overall.
- Experimental configurations: AVATAR evaluates statically localized bugs, all bugs with perfect localization, and comparisons against state-of-the-art APR systems.The configurations use static-analysis localization, developer-patch ground truth, and replicated APR fault-localization scenarios.
- Static-analysis evaluation: 6 bugs are correctly repaired among those localized by static analysis: 3 SpotBugs, 2 Facebook Infer, and 1 Google ErrorProne.The corresponding pools contain 14, 4, and 7 bugs, respectively, in Defects4J version 1.2.0.
- Static-analysis evaluation: Four distinct bugs are correctly fixed using patterns mined from patches addressing FindBugs violations.The four bugs involve distinct violation types, and two are detected by SpotBugs, FindBugs’ successor.
- Observed limitations: Many supposedly statically detectable bugs remain ineffective targets because their contexts lack matching patterns, detections are false positives, or fixes are domain-specific.Figure 8 illustrates a false positive whose violation is unrelated to the failing test case.
- Patch assessment: AVATAR verifies generated patch candidates with test cases, targeting plausible patches that pass the available tests.The evaluation distinguishes plausible patches from manually confirmed correct patches.
D. Applying AVATAR to All Defects4J Bugs
With faulty locations supplied from developer patches, AVATAR generates plausible and correct patches for numerous Defects4J bugs. Its fix ingredients cover several violation types and 11 of the benchmark’s 60 documented bug patterns.
- Results: 49 bugs receive plausible patches under perfect localization, including 35 manually confirmed as correct.Fourteen additional plausible patches fix part of some bugs, with five manually judged partially correct.
- Results: AVATAR can correctly address multiple faulty code locations, including four partially correct fixes among the additional multi-location cases.The paper describes this as the first APR tool to partially but correctly fix such Defects4J bugs.
- Fix ingredients: All correctly fixed bugs use patterns mined from FindBugs-violation fixes, while PMD-based patterns target issues unlikely to appear as semantic bugs.The examined Defects4J bugs were not associated with the performance, practice, or style issues targeted by the cited PMD pattern example.
- Fix ingredients: 36 of 40 fully or partially correctly fixed bugs, or 90%, use patterns from four violation types.These are NP_NULL_ON_SOME_PATH, DLS_DEAD_LOCAL_STORE, UC_USELESS_CONDITION, and UCF_USELESS_CONTROL_FLOW.
- Bug-pattern diversity: AVATAR addresses 11 of the 60 bug patterns enumerated in the Defects4J dissection study.This indicates coverage across a subset of the benchmark’s documented bug-pattern taxonomy.
F. Comparing against the State-of-the-Art
The state-of-the-art comparison classifies APR systems by fault-localization assumptions, including restricted localization to known faulty methods. These comparisons use the corresponding repair-operation and bug-pattern summaries.
- Fault-localization configurations: APR comparisons distinguish normal, restricted, and supplemented fault-localization configurations.Restricted localization assumes faulty methods are known, whereas supplemented approaches improve localization with additional heuristics.
- Restricted localization: Restricted fault localization substantially improves the accuracy of the target list by limiting repair to statements within known faulty methods.This setting is used to compare AVATAR with systems evaluated under the same strong assumption.
1) Comparison against a Restricted FL-based APR System:
Under restricted fault localization, AVATAR substantially outperforms HDRepair on Defects4J. Under normal fault localization, it also exceeds the compared systems in plausible and correct fixes while complementing them.
- Restricted FL comparison: AVATAR correctly fixes 31+4 bugs versus HDRepair’s 6 under the restricted fault-localization comparison.AVATAR also has a higher probability of generating correct patches among plausible patches.
- Restricted FL comparison: 34 of the 35 bugs fixed by AVATAR, except Lang-6, are not addressed by HDRepair.AVATAR additionally correctly fixes seven bugs that HDRepair fixes only plausibly and incorrectly, and partially fixes 11 multi-location bugs.
- Restricted FL comparison: AVATAR is reported to substantially outperform HDRepair on the Defects4J benchmark.This is the paper’s RQ3 conclusion for the restricted fault-localization comparison.
- Normal FL comparison: AVATAR outperforms all compared normal-FL APR systems in both plausibly fixed and correctly fixed bugs.Its probability of producing correct patches among plausible patches is also higher than those tools except FixMiner.
- Normal FL comparison: 18 of the 30 bugs correctly fixed by AVATAR are not correctly fixed by the compared normal-FL tools.The paper uses this overlap result to characterize AVATAR as complementary to existing APR systems.
3) Comparison against Supplemented FL-based APR Systems:
AVATAR compares favorably with several supplemented fault-localization APR systems while also repairing bugs untouched by existing approaches. Its broader relevance is tempered by evaluation assumptions and a focus on particular static-analysis tools.
- Comparison against Supplemented FL-based APR Systems:: AVATAR is inferior only to SimFix but outperforms other supplemented fault-localization APR systems by correctly fixing more bugs.The comparison is based on the number of correctly fixed bugs.
- Comparison against Supplemented FL-based APR Systems:: 14 out of 31 bugs never addressed by any supplemented fault-localization system are correctly fixed by AVATAR.The count combines 11 and 3 bugs reported in the comparison.
- Comparison against Supplemented FL-based APR Systems:: AVATAR correctly fixes 11 out of 31 bugs never correctly fixed by any state-of-the-art APR system.It also outperforms state-of-the-art tools on bugs in Chart, Closure, and Mockito.
- Comparison against Supplemented FL-based APR Systems:: AVATAR underperforms some recent APR systems but remains complementary because it fixes bugs that state-of-the-art systems cannot.This positions AVATAR as complementary rather than uniformly superior.
- Comparison against Supplemented FL-based APR Systems:: The study assumes perfect fault localization and evaluates only Java Defects4J bugs with fix patterns for FindBugs and PMD violations.Other static-analysis tools, including tools for C programs, are not investigated.
VIII. CONCLUSION
AVATAR uses developer patches for static-analysis violations as fix ingredients for semantic-bug repair. The approach is effective on Defects4J, complements existing APR systems, and motivates evaluation on larger datasets.
- VIII. CONCLUSION: AVATAR uses concise developer patches addressing static-analysis bugs as fix ingredients for generating patches for semantic bugs.Static detectors systematically assess whether the targeted violations have been removed.
- VIII. CONCLUSION: AVATAR outperforms several state-of-the-art APR approaches and fixes some Defects4J bugs that no prior APR system had fixed.The conclusion characterizes the approach as complementary to other systems.
- VIII. CONCLUSION: Future work will evaluate AVATAR on larger bug datasets and apply static-analysis-based patterns to method-refactoring research.These are identified directions beyond the reported evaluation.