Source-linked AI summary
Getafix: Learning to Fix Bugs Automatically
Johannes Bader, Andrew Scott, Michael Pradel, Satish Chandra
TL;DR
Static-analysis warnings are common and their fixes remain largely manual, despite recurring patterns within bug categories. Getafix learns hierarchical, context-sensitive fix patterns from human-written fixes and ranks suggestions without costly candidate exploration. Across 1,268 Java fixes, it exactly matched human fixes as the top suggestion 12%–91% of the time, and Facebook developers accepted around 42% of its production suggestions.
Problem
Getafix addresses the problem of automatically fixing common static-analysis bugs by learning from past fixes, targeting bug categories with non-trivial yet repetitive solutions.
Method
Getafix decomposes fixes into AST-level edits, hierarchically clusters generalized patterns with surrounding context, and ranks context-appropriate suggestions.
Results
Across 1,268 fixes in six Java warning categories, Getafix predicted the exact human fix as the top suggestion 12%–91% of the time and covered 526 bugs within five suggestions.
Takeaways & Limitations
Getafix was deployed at Facebook, where developers accepted around 42% of suggested fixes, contributing to software stability and saving developer time.
Takeaways & Limitations
Getafix targets bug categories with non-trivial yet repetitive fixes and can miss application-specific details such as custom exception messages or surrounding exception-handling adaptations.
Abstract
from arXiv · showhide
Static analyzers help find bugs early by warning about recurring bug categories. While fixing these bugs still remains a mostly manual task in practice, we observe that fixes for a specific bug category often are repetitive. This paper addresses the problem of automatically fixing instances of common bugs by learning from past fixes. We present Getafix, an approach that produces human-like fixes while being fast enough to suggest fixes in time proportional to the amount of time needed to obtain static analysis results in the first place. Getafix is based on a novel hierarchical clustering algorithm that summarizes fix patterns into a hierarchy ranging from general to specific patterns. Instead of a computationally expensive exploration of a potentially large space of candidate fixes, Getafix uses a simple yet effective ranking technique that uses the context of a code change to select the most appropriate fix for a given bug. Our evaluation applies Getafix to 1,268 bug fixes for six bug categories reported by popular static analyzers for Java, including null dereferences, incorrect API calls, and misuses of particular language constructs. The approach predicts exactly the human-written fix as the top-most suggestion between 12% and 91% of the time, depending on the bug category. The top-5 suggestions contain fixes for 526 of the 1,268 bugs. Moreover, we report on deploying the approach within Facebook, where it contributes to the reliability of software used by billions of people. To the best of our knowledge, Getafix is the first industrially-deployed automated bug-fixing tool that learns fix patterns from past, human-written fixes to produce human-like fixes.
1 Introduction
Getafix learns recurring, context-sensitive fix patterns from past human fixes to automate static-analysis bug repair. It targets repetitive but non-trivial fixes, producing human-like suggestions quickly without exploring a large candidate space.
- Motivation: Static analyzers identify recurring bug categories, but fixing their warnings remains mostly manual and hampers adoption.
- Problem and approach: Getafix learns from past fixes for a specific warning category to predict future fixes similar or equal to human developers’ fixes.
- Motivation: Null-dereference fixes may add an if-condition conjunct, use a conditional expression, or return early, depending heavily on existing code.
- Approach: Getafix splits example fixes into AST-level edits, clusters generalized patterns hierarchically, ranks context-appropriate candidates, and validates suggestions once against the static analyzer.
- Evaluation: 1,268 fixes across six Java warning categories yielded exact top-ranked human fixes for 12%–91% of cases, while top-five suggestions covered 526 bugs.
- Deployment: At Facebook, developers accepted around 42% of Getafix’s suggested fixes, addressing bugs with a single click.
2 Overview
Getafix learns fix patterns from past bug fixes and applies ranked patterns to unseen code, using context to choose suggestions and validating only the predicted fix(es) against the original signal.
- Learning and prediction: Getafix learns from pairs of bugs and fixes during training, then applies learned patterns to previously unseen code with the same signal.Training data may consist of past human code changes tied to static analysis warnings, type errors, lint messages, or similar signals.
- Learning and prediction: The tree differencer decomposes each fix into AST-level edits, providing concrete changes that can be replayed on other code.Concrete edits are pairs of before and after sub-ASTs.
- Learning and prediction: Hierarchical clustering organizes fix patterns from specific to general, while surrounding context helps select among multiple applicable patterns.The hierarchy and context are learned from concrete edits.
- Prediction: Getafix ranks candidate fixes using contextual information, then validates the top-ranked suggestion or suggestions against the tool that produced the warning.Validation is performed before suggesting a fix and can use a static analyzer, type checker, or linter.
3 Tree differencer
Getafix extracts structurally meaningful, fine-grained edits from AST differences rather than relying on coarse line changes, while emitting multiple granularities for later pattern learning.
- Edit representation: Getafix represents edits as before and after ASTs plus mappings that identify corresponding nodes and whether mapped subtrees are modified.Mappings distinguish modified from unmodified subtree pairs.
- Edit extraction: AST-based edits preserve structural information that line-based diffs lose, including moves and changes inside moved methods.Line-based diffing may mark entire methods as removed and reinserted.
- Edit extraction: The tree differencer uses AST structure to extract deletion, insertion, move, and update operations from paired trees.Unmapped nodes yield deletions or insertions; mapped nodes with changed parent relationships yield moves.
- Edit granularity: Because nested modifications make edit granularity ambiguous, Getafix extracts an entire spectrum of concrete edits rather than choosing one grouping strategy.It also adds edits rooted at parents of contiguous modified regions to capture changes near other modifications.
- Edit granularity: The approach intentionally extracts too many edits, relying on clustering to prioritize the most useful granularity when similar edits recur.For example, frequently recurring insertions can outrank less frequent moves or updates.
4 Learning Fix Patterns
Getafix learns recurring edit patterns for a specific bug category by organizing concrete edits into a hierarchy that spans multiple levels of generality.
- Pattern learning: The learning phase converts fine-grained tree edits into recurring edit patterns for a particular bug category.Edit patterns generalize multiple concrete fixes and use holes to abstract differing tree parts.
- Pattern hierarchy: A novel hierarchical clustering algorithm arranges patterns from concrete leaf edits to increasingly abstract root patterns.The hierarchy contains variants ranging from patterns matching few concrete edits to patterns matching many.
- Pattern hierarchy: The hierarchy is built using anti-unification, which generalizes edit patterns, and is augmented with contextual information for prediction.The paper presents the generalization operation before describing clustering and context augmentation.
4.1 Generalizing Edit Patterns via Anti-Unification
Anti-unification generalizes ASTs and edits by preserving shared structure while replacing incompatible parts with indexed holes, then reconstructs edit context and mappings where possible.
- Tree patterns: Tree patterns extend ASTs with indexed holes that match arbitrary subtrees, optionally constrained by a root label.Repeated hole indices require the matched subtrees to be identical.
- Tree patterns: Patterns can express both label constraints and equality constraints between matched subtrees, producing different matching scopes.A pattern with identical indexed holes matches only trees whose corresponding subtrees are identical.
- Tree anti-unification: Anti-unification recursively preserves nodes when labels, values, and child counts match, and otherwise replaces subtrees with holes.When merging holes, a matching hole label is retained.
- Edit anti-unification: Getafix generalizes edits by anti-unifying before and after trees with shared substitutions, so corresponding holes represent the same AST node across the change.It first drops unmodified subtrees, restores mappings between generalized nodes, and repopulates unmodified nodes where possible.
- Edit anti-unification: The edit-pattern operation abstracts recurring changes while retaining surrounding structure needed to represent how the fix applies in context.The examples generalize edits that insert conditional behavior while preserving or restoring nearby statements and mappings.
4.2 Hierarchical Clustering of Edit Patterns
Getafix builds a hierarchy of edit patterns by repeatedly anti-unifying concrete edits, preserving increasingly general and specific representations while approximating costly clustering decisions for scalability.
- Hierarchy representation: A dendrogram represents concrete edits merged into patterns at multiple abstraction levels, from specific subsets to a pattern generalizing all edits.Intermediate patterns retain useful variants rather than only the most general cluster representation.
- Clustering procedure: Getafix initializes a working set with one pattern per concrete edit and repeatedly replaces selected pairs with their anti-unification until one hierarchy remains.Each new generalized pattern becomes the parent of the merged pair.
- Merge order: The clustering algorithm selects pairs whose anti-unification loses the least concrete information, using the induced partial order of edit patterns.Anti-unification provides the generalization operation and orders patterns by abstraction level.
- Anti-unification: Anti-unification reuses holes across before and after trees by memoizing calls, enabling consistent generalization of edit patterns.The operation first generalizes the before trees and then the after trees using shared substitutions.
- Efficiency approximations: For scalability, Getafix approximates the partial order, reuses nearest-neighbor chains, partitions edits by modified-node labels, and then generalizes partition roots into one dendrogram.These approximations address expensive order comparisons, repeated merge comparisons, and quadratic clustering in the number of concrete edits.
4.3 Additional Context in Edit Patterns
Getafix augments edit patterns with code and error context, making matches more selective and binding holes that would otherwise remain unspecified.
- Motivation: Context makes edit patterns more specific, reducing matching locations and the chance of invalid fixes, while also binding otherwise unbound holes.Getafix uses both code context and error context.
- Code context: Code context comes from unmodified AST surroundings included with concrete edits and helps determine applicability when applying patterns to new code.Additional before-part context can bind a hole that is unbound in the after part.
- Error context: Error context propagates information from the static-analysis warning, allowing a hole to represent the error variable even when it is absent from the before part.For null dereferences, the blamed expression can guide where and how a pattern is applied.
4.4 Comparison with Prior Work on Inferring Edit Patterns
Unlike greedy clustering with one context-free cluster representation, Getafix’s hierarchy preserves intermediate, context-rich patterns that can better match human fixes.
- Prior approach: Greedy clustering keeps one representation per cluster and cannot generally preserve context shared by only some training edits.Adding context to such a representation would retain only context present in all edits.
- Getafix comparison: Hierarchical clustering retains a specific early-return pattern for calls using a potentially null variable, beyond the general null-check pattern.The added context helps predict a human-like fix in the illustrated example.
5 Applying and Ranking Fix Patterns
Getafix applies learned patterns by matching and instantiating AST holes, then ranks multiple candidates using statistics learned from human fixes to select a likely relevant suggestion.
- Applying patterns: Getafix matches tree patterns to buggy code, instantiates their holes consistently, and replaces the matching subtree with the instantiated after part.This produces concrete fixes from abstract edit patterns.
- Candidate generation: Multiple patterns can produce several candidates, making the choice of which fix to suggest a separate ranking problem.Less-specific patterns may match unintended locations, whereas context-rich patterns match more selectively.
- Ranking model: Getafix ranks candidates by a product of prevalence, location, and specialization scores learned from human-written fixes.The ranking estimates candidate relevance from how often patterns occur, where they are applied, and how selectively they match.
- Ranking example: In the worked example, candidate p31 ranks above p21 and p11 with overall scores 3.6, 1.9, 0.3 and 0, respectively.The specialized pattern benefits from matching fewer AST nodes and occurring near the warning location.
- Comparison with other ranking techniques: The ranking is category-specific and AST- and line-based, unlike Prophet’s generic naturalness model and its language-specific feature-adaptation requirements.The authors note that alternative ranking techniques could replace or complement Getafix’s current approach.
6 Evaluation
Getafix is evaluated on six Java bug categories using cross-validation, exact matching to human fixes, and comparisons with simpler learning and ranking baselines. Its effectiveness varies by bug category, while hierarchical learning and multi-score ranking generally improve candidate selection.
- The evaluation addresses six research questions covering prediction effectiveness, ranking, baselines, training data, efficiency, and industrial deployment.
- Getafix evaluates exact human-fix matching, a strict measure that treats comments, whitespace differences, and semantically equivalent fixes as mismatches.The measure may underapproximate developer acceptance because multiple fixes can be acceptable.
- Getafix is evaluated on 1,268 fixes across six Java bug categories detected by Infer and Error Prone.Experiments generally use 10-fold cross-validation, dropping patterns representing less than 1% of training data.
- 12% to 91% of fixes are predicted exactly as the top-most suggestion, while top-5 suggestions cover 526 of 1,268 fixes.Top-1 predictions exactly match 381 fixes; top-5 predictions exactly match 526.
- Accuracy varies with fix diversity: repetitive categories such as BoxedPrimitiveConstructor are easier than NullPointerExceptions, which have many strategies.Examples include adding conditional checks, ternary operations, and early returns for null dereferences.
- Missed fixes include application-specific exception messages, adaptations to existing exception handling, renamings, code removal, and deeper root-cause changes.The paper notes that larger training sets could help with popular exception-handling scenarios.
- Ranking pushes many fixes that Getafix can find toward the top of its candidate lists, enabling only a small number of candidates to undergo validation.Figure 11 reports coverage as the number of top-k suggestions increases, including all candidates at k = ∞.
- For five of six bug categories, hierarchical learning and multi-score ranking improve results over simpler baselines.The largest accuracy drop occurs without hierarchical clustering; DefaultCharSet is especially affected because the baseline cannot combine learned patterns with imports.
6.5 Influence of Available Training Data
The paper examines training-data effects, prediction cost, and deployment outcomes for Getafix. In production, validated suggestions were frequently accepted and auto-fixes improved warning resolution rates and speed for additional warning types.
- Training-data experiments use subsets of 804 null-dereference fixes and validate on samples excluded from training.Each training fraction is repeated so every fix appears at least eight times in both training and validation.
- Getafix predicts individual fixes in 1.7 to 9.2 seconds on average.The timing experiment used 10-fold training and prediction on one 24-core machine with 114GB of RAM.
- In Facebook deployment, 84% of validation runs produced a top-ranked fix that removed the reported Infer warning.Only the top-ranked predicted fix was validated and shown to developers.
- Developers directly accepted 106 of roughly 250 suggested null-dereference fixes within three months, an acceptance rate of 42%.The paper attributes the higher acceptance rate than experimental top-1 accuracy to developers accepting multiple possible fixes.
- Among unaccepted suggestions, 10% corresponded to fixes Getafix knew but failed to rank highest, and about 9% were replaced by semantically equivalent fixes.Other cases involved assertions, code removal, or custom fixes not expressed as recurring patterns.
- For Field Not Nullable and Return Not Nullable warnings, suggested fixes were accepted around 60% of the time.Displaying auto-fixes increased fix rates by 4% and 12%, respectively, yielding around 80 additional fixed warnings per month.
- The deployment experience identifies integration into existing development tools and fast suggestions as practical requirements for usefulness.The paper reports integrating auto-fixes into code review and keeping suggestion time close to the static-analysis wait.
7 Related Work
Getafix occupies a learning-based program-repair design point: it learns fix patterns for specific warning categories rather than broadly generating and validating mutations. Related approaches differ in search-space construction, runtime information, transformation languages, or neural representations.
- Unlike generate-and-validate systems such as GenProg, Getafix learns patterns from past fixes for specific warning categories.It does not seek generic solutions from arbitrary ingredients or generic code mutations.
- Genesis also learns transformations between before-and-after ASTs, but its templates contain generators that enlarge the search space.
- SketchFix minimizes validated candidates but relies on runtime information from repeated test executions, which Getafix does not use.
- Refazer learns repetitive edits using PROSE and a domain-specific language, while Revisar uses anti-unification and greedy clustering without extracting surrounding context.Revisar learns from arbitrary code changes, whereas Getafix targets fixes for specific bug categories.
- Neural repair approaches include manually generated candidates ranked by learned models, embedding-based reference search, and end-to-end repair models.
- Other code-learning work suggests identifiers, literals, and learned representations of edits could further guide Getafix’s learning and ranking phases.
8 Conclusion
Getafix learns hierarchical fix patterns from human-written commits and ranks fixes for new bug occurrences. Its evaluation and Facebook deployment show accurate human-like repair suggestions, including bugs detected by automated testing.
- Getafix learns fix patterns from past human-written commits and ranks appropriate fixes for new occurrences.Its hierarchical clustering summarizes fixes from general to specific patterns before ranking suggestions.
- 1,268 real-world bug fixes supported accurate prediction of human-like fixes across various bug categories.The evaluation is reported together with deployment experience within Facebook.
- Getafix can suggest fixes for null pointer exceptions detected by Sapienz through SapFix.This extends its reported use beyond static-analysis warnings when the bug category, location, and training fixes are available.