PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning Anti-patterns from Code Metrics History

Antoine Barbez, Foutse Khomh, Yann-Gaël Guéhéneuc

arXiv:1910.07658v1cs.SE

TL;DR

Anti-pattern detectors that rely only on structural metrics or change history can miss complementary information. CAME combines historical structural metrics in a CNN-based classifier, and its God Class evaluations report improved detection performance over existing tools and static classifiers.

  • Problem

    Approaches using structural metrics or change history alone miss complementary information, limiting their detection performance.

    Source: §I · p. 1

  • Method

    CAME mines structural code metrics across revisions and feeds their historical values to a CNN classifier to detect anti-patterns.

    Source: §I · p. 1, A. Input · p. 3

  • Results

    For God Class detection, CAME achieved an F-measure of 0.77, improving precision by 196% and recall by 51% over the best competing technique.

    Source: §I · p. 2

  • Takeaways & Limitations

    The authors suggest considering CAME for software maintenance tasks, based on its God Class evaluation and comparisons with detection tools.

    Source: §I · p. 2

    The evaluation covers one anti-pattern, God Class, and the small sample of systems may limit generalizability; broader evaluation is desirable.

    Source: §VII · p. 10

Abstract

from arXiv · show

Anti-patterns are poor solutions to recurring design problems. Number of empirical studies have highlighted the negative impact of anti-patterns on software maintenance which motivated the development of various detection techniques. Most of these approaches rely on structural metrics of software systems to identify affected components while others exploit historical information by analyzing co-changes occurring between code components. By relying solely on one aspect of software systems (i.e., structural or historical), existing approaches miss some precious information which limits their performances. In this paper, we propose CAME (Convolutional Analysis of code Metrics Evolution), a deep-learning based approach that relies on both structural and historical information to detect anti-patterns. Our approach exploits historical values of structural code metrics mined from version control systems and uses a Convolutional Neural Network classifier to infer the presence of anti-patterns from this information. We experiment our approach for the widely known God Class anti-pattern and evaluate its performances on three software systems. With the results of our study, we show that: (1) using historical values of source code metrics allows to increase the precision; (2) CAME outperforms existing static machine-learning classifiers; and (3) CAME outperforms existing detection tools.

I. INTRODUCTION

The paper addresses limitations in anti-pattern detectors that use structural metrics or change history alone by introducing CAME, which combines both. In God Class evaluations, it reports gains over static classifiers and existing detection techniques.

I. INTRODUCTION

  • Existing structural and historical detection approaches can identify different occurrences, while each misses information available from the other source.Structural techniques use a single version; the historical technique discussed does not account for structural properties or the nature of changes.Source: §I · p. 1
  • CAME mines structural code metrics at each revision, arranges the values in a two-dimensional input, and uses a CNN to classify components.The approach is presented as combining structural and historical information.Source: §I · p. 1
  • The paper presents CAME as the first approach, to the authors’ knowledge, to use structural and historical information simultaneously for anti-pattern detection.Source: §I · p. 1
  • With 500 revisions of metric history, F-measure on three test systems increased by 33% ± 6% over using one revision.The study used an oracle of God Class occurrences from eight open-source Java projects.Source: §I · pp. 1–2
  • For God Class detection, CAME improved F-measure by 38% over the best-performing static machine-learning classifier.Static approaches use only the current system revision.Source: §I · p. 2
  • CAME detected God Class with an F-measure of 0.77, improving precision by 196% and recall by 51% over the best competing technique.The comparison included two static code-analysis techniques and one approach using change history.Source: §I · p. 2
  • The paper evaluates CAME for God Class detection on three software systems and compares it with static classifiers and existing detection tools.The paper also describes its sections on case-study design, history length, comparisons, validity threats, and future work.Source: §I · p. 2

II. BACKGROUND AND RELATED WORK

This section defines God Class and surveys rule-, refactoring-, history-, and machine-learning-based detection approaches. It situates CAME among prior metric-based and learning-based work.

A. Definition of God Class

  • A God Class grows rapidly as functionality is added, accumulates many responsibilities, and delegates only trivial operations while accessing other classes’ data.It violates uniform distribution of system intelligence and is associated with reduced comprehension and reusability.Source: A. Definition of God Class · p. 2

B. Anti-patterns detection

  • Detection approaches use metric thresholds and formal rules, identify refactoring opportunities, or analyze co-changes between code components.Examples include detection strategies, DECOR, Extract Class and Move Method suggestions, and HIST.Source: B. Anti-patterns detection · pp. 2–3

C. ML-based Anti-patterns detection

  • Prior machine-learning detectors include decision trees, Bayesian networks, and SVMs; later work applied CNNs to structural metrics and lexical information for Feature Envy.A large-scale study evaluated 16 machine-learning algorithms and boosting techniques across four anti-patterns and 74 systems.Source: C. ML-based Anti-patterns detection · p. 3

D. Anti-patterns Through Systems’ History

Prior research examined how anti-patterns arise and evolve through system histories. CAME then uses revision-level metric histories as fixed-size inputs for component classification.

D. Anti-patterns Through Systems’ History

  • Studies tracked anti-pattern change behavior and evolution across systems; one study of 200 projects found many instances were introduced with files or in the month before deadlines by experienced developers.Source: D. Anti-patterns Through Systems’… · p. 3

III. CAME: CONVOLUTIONAL ANALYSIS OF CODE METRICS EVOLUTION

  • CAME’s method section describes its input, CNN architecture, and training procedure for detecting anti-patterns from historical source-code metrics.Source: §III · p. 3

A. Input

  • For each component, CAME recomputes selected structural metrics across mined revisions and forms a real-valued matrix of history length Lh by Nm metrics.Lh is a configurable history limit; shorter histories are zero-padded to provide fixed-size input.Source: A. Input · p. 3

B. Architecture

CAME uses a CNN to classify components from evolving structural metrics, with convolution and pooling layers feeding dense layers and a sigmoid output. Its training objective addresses class imbalance by optimizing a differentiable approximation of F-measure.

B. Architecture

  • Convolution and max-pooling layers process metric histories before dense layers and a sigmoid output predict whether a component is affected.The architecture is motivated by the conjecture that successive layers identify metric changes and build representations of anti-pattern patterns.Source: B. Architecture · pp. 3–4

C. Training

  • Training examples pair each component's input matrix with its true label, and the model outputs a predicted probability using a sigmoid function.The passage defines the training set, labels, model weights, and probability output.Source: C. Training · p. 4

1) Loss Function:

  • Because anti-pattern instances are a minority and threshold-based F-measure is discontinuous, training minimizes the negative of a differentiable F-measure approximation.The approximation is attributed to Jansche.Source: 1) Loss Function · p. 4

2) Regularization:

CAME uses L2 regularization to discourage overfitting by adding a penalty that encourages smaller model weights.

2) Regularization:

  • L2 regularization adds a term based on the weight matrices' Euclidean norm to the loss, encouraging smaller weights to help prevent overfitting.The regularization coefficient λ is adjusted during cross-validation.Source: 2) Regularization · p. 4

IV. STUDY DESIGN FOR GOD CLASS

The study evaluates CAME against static machine-learning algorithms and existing detection techniques for God Class. It uses eight open-source Java projects, with project selection shaped by available oracle labels and repository histories.

IV. STUDY DESIGN FOR GOD CLASS

  • The study asks whether metric-history length improves detection, how CAME compares with static ML algorithms, and how it compares with existing detection techniques.The first question also seeks an effective history length before the comparisons.Source: §IV · p. 5

A. Studied Systems

  • The evaluation uses eight open-source Java projects, reuses God Class occurrences from HIST and DECOR, and excludes test directories and systems lacking full history or with invalid oracle occurrences.The projects span Android APIs, Apache projects, ArgoUML, and Jedit; only core-feature directories were analyzed.Source: A. Studied Systems · p. 5

B. Source Code Metrics

For God Class detection, CAME analyzes seven structural metrics for top-level classes, excluding nested-class members. RepositoryMiner extracts metric histories by walking system revisions and produces revision-specific metric files.

B. Source Code Metrics

  • God Class candidates are top-level classes; nested-class members are excluded because inner classes are assumed possibly to result from Extract Class refactoring.The study retrieves seven structural metrics for each candidate class.Source: B. Source Code Metrics · p. 5
  • The selected metrics include ATFD, LCOM5, and LOC, covering foreign-data access, method cohesion, and method lines of code.The passage relates these measures to God Class characteristics and notes LCOM5's use in DECOR.Source: B. Source Code Metrics · p. 5
  • The remaining metrics are NAD, NADC, NMD, and WMC, measuring attributes, associated data classes, methods, and method complexity.NAD, NADC, and NMD are also part of DECOR's detection process.Source: B. Source Code Metrics · p. 6

C. Data Extraction

  • RepositoryMiner walks commits in reverse from a chosen snapshot, updates changed or renamed components, computes class metrics, and outputs revision-specific CSV files when values change.It accepts a repository URL, snapshot SHA, and directories of interest, and supports 12 class- and method-related metrics.Source: C. Data Extraction · p. 6

D. Evaluation

Evaluation pools class-level predictions from the three test systems into a confusion matrix for each classifier and uses it to calculate performance metrics. The paper also provides its study data and CAME implementation for God Class in an online appendix.

D. Evaluation

  • Predictions for all instances of the three test systems are combined into one confusion matrix per classifier for overall evaluation.Source: D. Evaluation · p. 6
  • Precision and recall are computed from true and false positives and false negatives, and F-measure is their harmonic mean.Source: D. Evaluation · p. 6

E. Replication Package

V. ASSESSING THE IMPACT OF THE METRICS HISTORY LENGTH ON CAME’S PERFORMANCES

The experiments assess how metrics-history length affects CAME’s God Class detection on three test systems, with separate hyper-parameter tuning for each history length. F-measure and precision improve through 500 revisions, while performance declines beyond that length.

B. Hyper-parameters Tuning

C. Results

  • F-measure rises from 0.51±0.06 at Lh = 1 to 0.68±0.08 at Lh = 500, an improvement of 33% ± 6% across the three test systems.Source: C. Results · p. 7, Figure 2 · p. 7
  • At Lh = 500, precision improves 61% ± 11% over Lh = 1; recall rises through Lh = 100, then drops to its initial value at Lh = 1000.Source: C. Results · p. 7
  • Performance, especially precision, declines for Lh > 500; the authors cite tuning uncertainty and the two-layer convolution limit as possible explanations.Source: C. Results · p. 8
  • For Lh > 1, recall is at least its single-revision level, while precision increases with history length, reducing false positives.Source: C. Results · p. 8

VI. COMPARING CAME WITH OTHER APPROACHES

The comparison evaluates CAME against static machine-learning classifiers and three established detection approaches. CAME’s ensemble is evaluated on three test systems, while the other five systems are used for training and calibration.

A. Approach

  • CAME’s final prediction uses Bayesian averaging of ten CNNs trained with Lh = 500 on the three test systems.Source: A. Approach · p. 8
  • Decision Tree, MLP, and SVM classifiers use the same seven metrics as CAME, measured on the current revision, and their predictions are ensemble-averaged across ten calibrated models.Source: A. Approach · p. 8
  • The tool comparison includes DECOR’s rule cards, JDeodorant’s refactoring-opportunity detection, and HIST’s use of change history.Source: A. Approach · p. 8

B. Hyper-parameters Tuning

Across the evaluated comparisons, CAME outperformed static classifiers and existing God Class detection techniques. The paper also notes comparator-specific caveats that affect how some results should be interpreted.

B. Hyper-parameters Tuning

  • Hyperparameters were tuned with 100-iteration random search and five-fold cross-validation on instances from the five training systems, using CAME's procedure for MLP.Source: B. Hyper-parameters Tuning · p. 8

RQ3: How does CAME compare to existing detection techniques?

  • Against the best tool for each metric, CAME improves precision by 196%, recall by 51%, and F-measure by 166% for God Class detection.Overall, CAME reports 71% precision, 86% recall, and 77% F-measure; the authors suggest practitioners consider it for finding components to refactor.Source: Table V · p. 9, RQ3: How does CAME compare to existing… · p. 9

C. Results

  • Table V reports precision, recall, and F-measure by subject system and overall; “–” marks cases where no God Class occurrence was detected.Source: Table V · p. 9, C. Results · p. 9

RQ2: How does CAME compare to other static ML algorithms?

  • Overall F-measure improved from 56% to 77% (38%) versus MLP, and CAME significantly outperformed all three static classifiers.CAME matched MLP recall (86%) while Decision Tree and SVM reached 29% and 14%, respectively.Source: RQ2: How does CAME compare to other… · p. 9

RQ3: How does CAME compare to existing detection techniques?

  • HIST's comparison depends on a replication because its original implementation was unavailable, and implementation differences may have affected reported performance.JDeodorant suggests refactoring opportunities rather than detecting anti-patterns directly.Source: RQ3: How does CAME compare to existing… · p. 9

VII. THREATS TO VALIDITY

The authors flag possible limits in the metrics, study scope, and model choice. Their conclusion reports that CAME benefits from longer metrics histories and outperforms both other ML classifiers and existing detection tools.

VII. THREATS TO VALIDITY

  • Overlooked metrics may have improved performance, and differences may remain between the HIST implementations used in this study and the authors’ implementation.Source: §VII · p. 10
  • The sample size and evaluation of God Class alone may limit generalizability, although the systems span different sizes and domains.Source: §VII · p. 10
  • Classifier predictions may vary with initialization, and the CNN choice rests on intuition; other deep-learning models could perform better.The evaluation used ten independent models per classifier, treating first-study performances as a Gaussian distribution and using an ensemble in the second study.Source: §VII · p. 10

VIII. CONCLUSION AND FUTURE WORK

  • CAME’s performance increases with the length of the metrics history supplied to the model.The authors interpret this as evidence that the model leverages historical information to improve performance.Source: §VIII · p. 10
  • CAME significantly outperforms ML classifiers that do not rely on historical data.Source: §VIII · p. 10
  • CAME significantly outperforms existing detection tools for the evaluated anti-pattern detection task.Source: §VIII · p. 10
Loading 1910.07658v1…