PaperlayerView exact revision on arXiv

Source-linked AI summary

RefDiff: Detecting Refactorings in Version Histories

Danilo Silva, Marco Tulio Valente

arXiv:1704.01544v1cs.SE

TL;DR

Refactoring information is valuable for understanding software evolution and supporting tasks involving code changes, but existing automatic detectors have precision and recall limitations. RefDiff combines static analysis with code similarity to detect 13 refactoring types, achieving the best evaluated results with 1.000 precision and 0.877 recall. Its evaluation is based on injected refactorings in seven open-source Java projects, so performance may not generalize to different projects or developer-applied refactorings.

  • Problem

    Existing automatic refactoring detectors still need improved precision and recall, despite the value of identifying refactorings in version histories.

  • Method

    RefDiff combines static-analysis heuristics and code similarity to detect 13 well-known refactoring types between code revisions.

  • Results

    RefDiff achieved the best overall results among evaluated tools, with 1.000 precision and 0.877 recall.

  • Takeaways & Limitations

    RefDiff provides an automated way to identify refactorings in Java version histories for software-evolution research and practical code-change tasks.

  • Takeaways & Limitations

    Evaluation used injected refactorings in seven open-source Java projects, so equivalent precision and recall cannot be claimed for different projects or developer-applied refactorings.

Abstract

from arXiv · show

Refactoring is a well-known technique that is widely adopted by software engineers to improve the design and enable the evolution of a system. Knowing which refactoring operations were applied in a code change is a valuable information to understand software evolution, adapt software components, merge code changes, and other applications. In this paper, we present RefDiff, an automated approach that identifies refactorings performed between two code revisions in a git repository. RefDiff employs a combination of heuristics based on static analysis and code similarity to detect 13 well-known refactoring types. In an evaluation using an oracle of 448 known refactoring operations, distributed across seven Java projects, our approach achieved precision of 100% and recall of 88%. Moreover, our evaluation suggests that RefDiff has superior precision and recall than existing state-of-the-art approaches.

I. INTRODUCTION

RefDiff is an automated approach for identifying refactorings in version histories, motivated by the value of refactoring information for software evolution and practical development tasks. It combines static-analysis and code-similarity heuristics, and its evaluation reports the best precision and recall among the compared approaches.

  • I. INTRODUCTION: Knowing refactoring activity supports software-evolution research and can assist code review, merge-conflict resolution, and diff visualization.The paper also identifies practical uses such as replaying API refactorings on client code.
  • I. INTRODUCTION: RefDiff identifies refactorings in version histories using static-analysis and code-similarity heuristics.The approach detects 13 well-known refactoring types and is implemented for Java code in git repositories.
  • I. INTRODUCTION: RefDiff achieved 100% precision and 88% recall, with the evaluation comparing it against three state-of-the-art approaches.The paper compares RefDiff with Refactoring Miner, Refactoring Crawler, and Ref-Finder.
  • I. INTRODUCTION: The paper contributes a publicly available implementation and evaluation data for RefDiff.The paper states that the implementation and data are publicly available through GitHub.
  • I. INTRODUCTION: 448 known refactoring operations across seven Java systems provide a benchmark for evaluating refactoring-detection approaches.The oracle is presented as a publicly available evaluation contribution.

II. RELATED WORK

Prior refactoring-detection work uses IDE monitoring, version-control metadata, behavior-preservation testing, and static analysis. The paper positions RefDiff among static-analysis approaches and reviews Refactoring Miner and Refactoring Crawler, including their detection strategies and reported evaluation results.

  • II. RELATED WORK: Earlier studies identify refactoring activity through IDE instrumentation, commit metadata, behavior-preserving tests, and static analysis.These techniques infer refactoring activity at different levels, from user actions to source-code differences.
  • II. RELATED WORK: Static-analysis approaches compare source-code differences and can identify individual refactoring operations, enabling direct comparison among tools in this category.RefDiff is presented as one of these source-code-difference approaches.
  • A. Refactoring Miner: Refactoring Miner matches classes, methods, and fields across revisions using names and signatures before matching added and removed elements.Its lightweight algorithm infers entities added, deleted, or moved between object-oriented models.
  • A. Refactoring Miner: Refactoring Miner’s reported precision and recall vary across studies, ranging from 63% precision in 285 repositories to 93% precision and 98% recall on another benchmark.The studies reported different evaluation settings and oracle construction methods.
  • B. Refactoring Crawler: Refactoring Crawler combines abstract-syntax-tree analysis, shingles-based similarity, and reference-graph analysis to refine refactoring candidates.It detects seven high-level refactoring types.

C. Ref-Finder

Ref-Finder uses logic programming to detect 63 refactoring types, while RefDiff builds before-and-after code models and analyzes entity relationships to identify refactorings.

  • C. Ref-Finder: Ref-Finder identifies 63 refactoring types by expressing structural constraints as template logic rules and inferring concrete instances from extracted program facts.It can query lower-level refactorings to identify higher-level composite refactorings.
  • III. PROPOSED REFACTORING DETECTION ALGORITHM: RefDiff analyzes modified source files by building models of types, methods, and fields before and after a change.The detection algorithm has Source Code Analysis and Relationship Analysis phases.
  • A. Matching Relationships: RefDiff constructs a bipartite graph whose vertices are code entities before and after the change, with edges representing relationships between them.A Rename Method relationship connects corresponding methods with different names when the specified conditions hold.
  • A. Matching Relationships: Matching relationships require entity pairs to satisfy relationship-specific conditions, such as different method names, matching containers, and similarity above threshold τ for Rename Method.Potential relationships are added as triples containing the entities and their similarity score, then conflicting matches are resolved by selecting one relationship per entity.
  • A. Matching Relationships: Table I lists the relationship types RefDiff can identify between types, methods, and fields.The algorithm searches relationship types in the order presented in the table.

B. Non-matching Relationships

Non-matching relationships allow one entity to participate in multiple relationships, so RefDiff identifies every pair satisfying the relevant conditions.

  • B. Non-matching Relationships: Non-matching relationships do not conflict, allowing multiple extractions from the same original method.A method can have separate code extracted into multiple new methods.
  • B. Non-matching Relationships: For Extract Method, RefDiff requires a newly added method, a retained original method, a call from the retained method to the new method, and similarity above threshold τ.The approach also supports Inline Method and Extract Supertype detection.

C. Computing Similarity

RefDiff represents code entities as token multisets and computes weighted Jaccard similarity, emphasizing tokens that better distinguish entities.

  • C. Computing Similarity: RefDiff represents source code as multisets of tokens, preserving repeated occurrences through each token’s multiplicity.Figure 1 illustrates this transformation for the methods sum, min, and power.
  • C. Computing Similarity: Weighted Jaccard similarity compares two entities using the minimum and maximum token weights across their shared token universe.The coefficient generalizes the Jaccard coefficient for weighted token representations.
  • 1) Weight of a token for a code entity:: TF-IDF weighting assigns greater importance to tokens that are more discriminative across code entities.The method token Math is a better similarity indicator than return because Math appears in only one of the example methods.
  • 1) Weight of a token for a code entity:: A token’s weight combines its multiplicity in an entity with its inverse document frequency across all code entities.The inverse document frequency decreases as the token occurs in more entities.

2) Similarity of fields:

For fields without bodies, RefDiff constructs a virtual body from statements that read or write the field, enabling token-based similarity computation.

  • 2) Similarity of fields:: RefDiff defines a field’s virtual body as all source-code statements that access the field by reading or writing it.This representation lets the approach extract a token multiset for the field and compute similarity.

3) Similarity for non-matching relationships:

RefDiff uses containment-sensitive token similarity for refactorings where one entity’s code should be contained in another, and calibrates relationship-specific thresholds against known refactorings. The calibration optimizes the precision–recall trade-off and reports aggregate performance for the selected thresholds.

  • 3) Similarity for non-matching relationships:: Containment-sensitive similarity handles Extract Supertype, Extract Method, and Inline Method relationships where source entities may differ substantially.For extraction, the extracted entity’s source should be contained within the original; for inlining, the inlined method should be contained within the target.
  • 3) Similarity for non-matching relationships:: Similarity is maximized when every token in e1 appears in e2, making the measure directional rather than symmetric.The paper explicitly notes that simp(e1, e2) can differ from simp(e2, e1).
  • D. Calibration of similarity thresholds: Relationship-specific thresholds determine whether entity pairs are considered potential matches, so threshold selection directly affects precision and recall.RefDiff calibrates one minimum-similarity threshold for each relationship type.
  • D. Calibration of similarity thresholds: Thresholds are calibrated by testing values from 0.1 to 0.9 on commits covering every refactoring type and selecting the value that optimizes F1.The calibration compares detected refactorings with an oracle of known refactorings.
  • D. Calibration of similarity thresholds: 85.7% precision and 94.1% recall were achieved overall with the calibrated thresholds.The results summarize true positives, false positives, and false negatives across the oracle’s refactoring relationships.

IV. EVALUATION

The evaluation compares RefDiff with three existing refactoring-detection approaches using an oracle of known refactoring instances. The comparison targets precision and recall.

  • IV. EVALUATION: RefDiff’s evaluation compares its output with an oracle of known refactoring instances and with Refactoring Miner, Refactoring Crawler, and Ref-Finder.The approaches are evaluated against the same refactoring-detection task.
  • IV. EVALUATION: The evaluation uses selected open-source Java projects as the repository setting for comparing refactoring-detection approaches.The selected projects are summarized in Table IV.

1) Construction of the oracle:

The evaluation oracle was constructed from controlled refactorings injected into seven Java repositories, producing 448 refactoring relationships across 12 types. This design improves recall measurement but leaves limitations around approach coverage and reporting accuracy.

  • 1) Construction of the oracle:: Recall is difficult to compute from naturally occurring commits because an oracle may miss refactorings among complex, multi-file changes.The paper notes that manual inspection is time-consuming and error-prone, making recall from such an oracle potentially unreliable.
  • 1) Construction of the oracle:: The authors therefore built a controlled oracle by having graduate students apply specified refactorings to selected GitHub-hosted Java repositories and document them.Students worked on separate repositories and were instructed to apply at least three refactorings of each listed type.
  • 1) Construction of the oracle:: Some applied refactorings were omitted from student spreadsheets, including one of two method inlinings performed by a student.This creates a reporting discrepancy between the controlled changes and the documented oracle.
  • 1) Construction of the oracle:: Seven students completed the tasks on projects including Google Guava, Spring Boot, and OrientDB.The repositories contained relevant Java projects and varied in their reported Java-code size.
  • 1) Construction of the oracle:: The resulting oracle contains 448 refactoring relationships covering 12 well-known refactoring types.A single operation can produce multiple relationships, such as extracting one method from two different source methods.
  • 1) Construction of the oracle:: Not all approaches support every oracle type, so unsupported entries were excluded from false-negative counts and could allow perfect recall without full type coverage.The paper gives Move Attribute as an example of a refactoring unsupported by Refactoring Crawler.

3) Results and discussion:

RefDiff achieved the strongest overall precision and recall among the evaluated refactoring-detection approaches, while maintaining acceptable execution time and scalability. Results also expose refactoring-specific weaknesses and show that mixed real-world commits are more challenging than refactoring-only commits.

  • Overall results: 1.000 precision and 0.877 recall made RefDiff the best overall approach, ahead of Refactoring Miner, Refactoring Crawler, and Ref-Finder.The overall comparison is reported in Table VII.
  • Refactoring-specific results: RefDiff’s recall reached only 0.269 for Move Field, suggesting that its threshold for this refactoring may be too restrictive.
  • Refactoring-specific results: 284 of Ref-Finder’s 382 Move Method false positives, or 74%, arose from missing Move Type or Rename Type detection support.Moved classes can cause member relationships to be mistakenly reported as Move Method and Move Field operations.
  • Adjusted comparison: After removing false positives from that scenario, precision improved from 0.419 to 0.582 for Refactoring Crawler and from 0.264 to 0.489 for Ref-Finder, but both remained behind RefDiff and Refactoring Miner.
  • Calibration: Precision was lower during calibration than evaluation because calibration used real commits that could interleave refactorings with other code changes.
  • Execution time: RefDiff averaged 1.96 seconds per commit versus 0.89 seconds for Refactoring Miner, although both approaches were considered acceptable for large code bases.RefDiff’s total analysis time was 3,893 seconds, compared with 1,779 seconds for Refactoring Miner.

V. THREATS TO VALIDITY

The validity discussion limits the precision and recall conclusions to the evaluated setting, while describing planned expansion to broader real-world commit corpora. The paper also highlights publicly released artifacts and future applications of RefDiff.

  • External validity: Precision and recall were evaluated on injected refactorings in seven popular open-source Java projects, so equivalent results are not established for different projects or actual developer refactorings.
  • Internal validity: The evaluation oracle may contain human errors because refactorings were manually applied and documented, although the authors inspected refactored source code to validate them.
  • Future work: The authors plan to assess RefDiff’s precision in a large corpus of commits from open-source repositories.
  • Implications: The publicly available implementation and experimental data support reuse of RefDiff for future applications and empirical studies of refactoring practices.Future applications include side-by-side visualization of refactored elements and their matches in previous versions.
Loading 1704.01544v1…