PaperlayerView exact revision on arXiv

Source-linked AI summary

Code Smells and Refactoring: A Tertiary Systematic Review of Challenges and Observations

Guilherme Lacerda, Fabio Petrillo, Marcelo Pimenta, Yann Gael Gueheneuc

arXiv:2004.10777v1cs.SE

TL;DR

Software maintenance consumes substantial software costs, while refactoring decisions remain difficult to justify because benefits are vague and long-term. This tertiary review analyzes 40 secondary studies and finds shared quality-attribute relationships between code smells and refactoring, with refactoring affecting quality more than code smells.

  • Problem

    Maintenance accounts for 50% to 80% of software costs, while refactoring benefits are vague and long-term, making refactoring decisions difficult to justify.

  • Method

    The paper analyzes 40 secondary studies to identify discussed topics and synthesize related observations and challenges.

  • Results

    Code smells and refactoring relate to understandability, maintainability, testability, complexity, functionality, and reusability, with refactoring affecting quality more than code smells.

  • Takeaways & Limitations

    The study presents implications for practitioners, researchers, and instructors and proposes open questions for future studies on smell detection, refactoring, and supporting tools.

  • Takeaways & Limitations

    Subjective data extraction and differing classification schemes or study granularity may affect the conclusions.

Abstract

from arXiv · show

In this paper, we present a tertiary systematic literature review of previous surveys, secondary systematic literature reviews, and systematic mappings. We identify the main observations (what we know) and challenges (what we do not know) on code smells and refactoring. We show that code smells and refactoring have a strong relationship with quality attributes, i.e., with understandability, maintainability, testability, complexity, functionality, and reusability. We argue that code smells and refactoring could be considered as the two faces of a same coin. Besides, we identify how refactoring affects quality attributes, more than code smells. We also discuss the implications of this work for practitioners, researchers, and instructors. We identify 13 open issues that could guide future research work. Thus, we want to highlight the gap between code smells and refactoring in the current state of software-engineering research. We wish that this work could help the software-engineering research community in collaborating on future work on code smells and refactoring.

1. Introduction

Software maintenance is costly and difficult, while code smells and refactoring raise unresolved questions about detection, selection, application, and benefits. This tertiary review synthesizes prior secondary studies to connect these topics, their quality implications, tools, and open issues.

  • Motivation: 50% to 80% of software costs are related to maintenance activities, including repairing faults, adapting environments, and modifying functionalities.The introduction presents maintenance as essential but difficult, partly because helpful documentation is lacking.
  • Motivation: Code smells violate coding design principles, increase technical debt, impair maintenance and evolution, and lower software quality as systems become more complex.The paper focuses on source-code smells because they negatively affect maintainability.
  • Motivation: Refactoring improves software systems through behavior-preserving transformations and can remove code smells, but deciding what and when to refactor remains challenging.The paper identifies unresolved questions about detecting smells, selecting and applying refactorings, and measuring gains.
  • Method: The study performs a tertiary systematic literature review of surveys, systematic mappings, and secondary reviews to examine code smells, refactoring, their relationship, and related evidence.The review addresses five research questions covering investigated topics, tools, cited studies, and annual trends.
  • Method: 40 secondary studies are analyzed to summarize observations and challenges, with observations defined as literature consensus and challenges as topics still open for investigation.The analysis is organized around the most discussed topics and Figure 1 summarizes the research method.
  • Contributions: The review cross-references frequent smells with detection approaches, tools, suggested refactorings, and refactoring tools, while relating smells and refactoring to quality attributes.Reported contributions include quality analysis, implications for practitioners, researchers, and instructors, and 13 open issues.

2. Background

The background defines smells as software problems that can impair evolution and distinguishes code- and design-level smells. It also presents established smell catalogs, taxonomies, and the role of refactoring in addressing them.

  • Definitions: There is no rigorous definition or shared categorization and organization for smells.Different authors may hold distinct views; inheritance, for example, is described as both good OO practice and a maintenance problem.
  • Definitions: Smells describe internal software problems whose symptoms can impair software evolution at code or design level.Code smells occur at a lower level, while design smells occur at a higher level.
  • Code-smell catalogs: Fowler’s catalog is the most frequently mentioned source, describing smells together with refactoring sequences intended to mitigate them.Fowler initially described 22 smells, and later information introduced six additional smells.
  • Categorization: Smell taxonomies organize problems by relationships such as coupling, class scope, code changes, object-orientation abuse, and dispensability.Examples include Feature Envy and Inappropriate Intimacy as couplers, and Duplicated Code and Lazy Class as dispensables.
  • Refactoring: Refactoring is presented as behavior-preserving transformation that can remove code smells, but selecting when and which transformation to apply remains challenging.The review also notes that quality benefits may be diluted by refactoring’s high costs and lower priority than bug fixes or new features.

3. Study Design

The study uses a tertiary systematic literature review to synthesize research on code smells and refactoring. It searches and assesses secondary studies systematically, then organizes the evidence into topics, observations, and challenges.

  • Study process: The review follows five main steps: defining goals and questions, identifying papers, selecting studies, assessing quality, and extracting data.These steps follow Kitchenham et al.’s SLR guidelines.
  • Goals and research questions: The study examines how the research area evolved and structures its research questions using prior tertiary studies.The questions cover investigated topics, tools, research questions, citations, and annual trends.
  • Search strategy: The search covers eight digital libraries and uses PICOC to define the population, intervention, comparison, outcomes, and context.The comparison is not applicable because the study characterizes available secondary studies.
  • Selection criteria: The review selects English academic secondary studies on smells and refactoring, including systematic reviews, systematic mappings, surveys, and tool or method studies.The search spans 1992–2018, beginning with Opdyke’s doctoral thesis as the starting reference for refactoring research.
  • Execution: 12.63% was the average search accuracy across databases; Google Scholar reached 33.33%, while Springer reached 5.62%.The selected studies most often used IEEE Xplore (90%), ACM Digital Library (80%), and ScienceDirect (75%).
  • Analysis and scope: The analysis identifies refactoring and smell topics, with object-orientation as the topic covered by both areas.The researchers use card sorting to organize topics emerging from the analyzed studies and focus discussion on code-related topics.

4. Findings

The findings compile evidence from the selected secondary studies to characterize smell-detection approaches, supporting tools, refactoring techniques, and tool support. They summarize these results around frequently cited smells and related practices.

  • Findings: The review compiles results from 40 selected studies on code-smell detection, detection tools, refactoring techniques, and refactoring-tool support.Table 8 summarizes the ten most-cited smells and associated approaches, tools, suggested refactorings, and refactoring tools.
  • Findings: The findings are organized in subsections grouped by the research questions.This structure presents the reviewed evidence according to the study’s analytical agenda.

4.1. RQ1: What refactoring-related topics have been investigated in secondary studies?

Secondary studies most often investigated refactoring opportunities, application, tools, techniques, and quality effects. The literature emphasizes extraction techniques and multiple opportunity-identification approaches, while reporting limited technique coverage, inconsistent quality effects, and unresolved automation challenges.

  • 13 secondary studies addressed refactoring, with topics categorized across the reviewed literature.The review used these studies to summarize observations and challenges.
  • 10 studies mentioned techniques, especially extraction techniques such as Extract Method, Extract Variable, and Extract Class.
  • The same refactoring can address multiple smells, so developers should select techniques according to context.Extract Class, for example, was associated with Duplicated Code/Clones, Large Class, God Class, Divergent Change, and Data Clumps.
  • Refactoring opportunities, application, and tools support were the most studied topics.
  • Quality-metrics, precondition, clustering, graph, slicing, dynamic-analysis, and search-based approaches were used to identify opportunities, but comparing approaches remains open.Code slicing is practical for small snippets but has scalability issues; no fully automatic process covers all refactoring tasks.
  • Quality effects can conflict across attributes, and unnecessary refactoring may degrade software quality rather than improve it.

4.2. RQ2: What smells-related topics have been investigated in secondary studies?

Secondary studies mainly examined design and code smells, their co-occurrence, terminology, detection approaches, and effects on maintenance and design. The literature highlights frequently studied smells but also inconsistent detection results and underexplored smell categories.

  • 14 studies discussed design smells, with Blob the most mentioned design smell.Blob was often linked with Spaghetti Code, Swiss Army Knife, Lava Flow, Functional Decomposition, and Poltergeist.
  • Design and code smells were studied together through reported relationships such as Blob–Large Class, God Class–God Method, and Data Class–Data Clumps.
  • 28 studies discussed code smells, and Duplicated Code/Clones was the most studied, appearing in 7 studies.
  • Different detectors for the same smell can produce inconsistent results, and several less-studied smells remain unexplained in the literature.The review specifically identifies Alternative Classes with Different Interfaces, Incomplete Class Library, Primitive Obsession, Inappropriate Intimacy, and Comments as receiving little attention.
  • God Class/Large Class, Long Method, Feature Envy, and Duplicated Code/Clones frequently co-occurred in selected studies.The review describes possible relationships among these smells, including Long Parameter List preceding Long Method and Long Method indicating God Class/Large Class.
  • Smell terminology lacks standardization: different names may describe one definition, while one name may have distinct definitions.The review proposes a unified catalog with alternative names to support precise terminology.
  • Detection research used human-perception, metrics-based, rule-based, probabilistic/search-based, and visualization-based approaches.Manual and visualization approaches retain human involvement, while metrics and rules target particular source-code elements or smells.

4.3. RQ3: Which tools have been mentioned for code-smell detection and refactoring support?

The reviewed studies identify widely used smell-detection and refactoring tools, but tool coverage, accuracy, scalability, and contextual validity remain uneven.

  • Platforms/Programming Languages: Java is the most commonly used platform and programming language for tool development and experiments, followed by C++ and C#.Among 35 studies mentioning platforms or programming languages, C++ and C# followed Java at 57.14% and 42.85%, respectively.
  • Smells Detection Tools: CCFinder and PMD were the most frequently cited smell-detection tools, each appearing in 47.36% of studies.inCode and DECOR/DETEX appeared in 42.10%, while inFusion appeared in 36.84%.
  • Smells Detection Tools: DECOR/DETEX detects more than 10 Fowler and design smells, whereas inCode detects several smells with visualization support.The reviewed tools differ substantially in detected smell types and documented detection strategies.
  • Tool Considerations: Tool validation is constrained because smell detectors use thresholds or ad-hoc rules, only about 30% report precision and recall, and standard benchmarks are unavailable.Different tools and unavailable implementations also make published results difficult to compare.
  • Tool Considerations: Most tools target a single language, commonly Java, and none detect all 22 Fowler code smells; typical coverage averages three to four smells.Only a few tools analyze very large projects, while many omit expert feedback and contextual factors such as domain and project status.
  • Refactoring Tools: JDeodorant was the most frequently studied refactoring tool, appearing in 71.42% of studies, followed by TrueRefactor at 21.42%.Eclipse Refactoring and IntelliJ IDEA Refactoring each appeared in 14.28%, while Wrangler appeared in 7.14%.
  • Refactoring Tools: Refactoring tools automate transformations and check preconditions and postconditions, but developers must still identify smells and select appropriate refactorings.TrueRefactor uses metrics, control-flow graphs, and a genetic algorithm to search for refactoring sequences.

4.4. RQ4: Which RQs have been studied on smells and refactoring? What are the highest cited secondary studies?

The 40 reviewed secondary studies predominantly ask descriptive and exploratory questions, while relationship, causality, design, and empirical-evidence questions remain uncommon. Citation patterns and research focuses show a broad but uneven evidence base.

  • Analysis of RQs: The 40 selected secondary studies contained 181 research questions, averaging 4.5 questions per study.Study S40 contained the most, with 13 questions addressing definitions, detection, tools, impact, trends, and researchers.
  • Analysis of RQs: Description-and-Classification questions were most common, appearing in 75% of studies, followed by Frequency Distribution, Existence, and Descriptive Process questions.The authors propose revising the RQ list to support better and more systematic questions in future secondary studies.
  • Analysis of RQs: Exploratory and Base-rates questions accounted for 152 RQs, or 83.97%, and appeared in at least one question in 34 studies.These categories were especially common in systematic mappings and surveys.
  • Analysis of RQs: Relationship and Causality questions were a minority, and no secondary study used Causality-Comparative Interaction or Design questions.The authors identify comparative efficacy, efficiency, and phenomenon-evaluation questions as open opportunities.
  • Analysis of RQs: Only 9 RQs, or 4.97%, explicitly addressed empirical or evidence-based software engineering.The authors suggest more secondary studies should cross-analyze the growing body of empirical primary studies.
  • Analysis of RQs: The number of RQs did not determine study comprehensiveness; comprehensiveness was related to the scope of the RQs themselves.For example, S15 had five RQs but covered six focuses.
  • Research Topics and Trends: The reviewed topics prominently included code clones and technical debt, with technical-debt studies increasingly addressing policies, tools, and mitigation techniques.The authors also identify smell detection, automation, quality and refactoring, and specialized topics such as testing and model refactoring.
  • Ranking of Cited Secondary Studies: The 40 studies received 2568 citations, averaging 64.2 per study with a median of 9, while citation counts varied sharply across studies.S10 had 1301 citations and S30 had 272; the authors also note that recent studies had fewer citations.

4.5. RQ5: What are the annual trends of types, quality, and the number of primary studies reviewed by the secondary studies?

Secondary studies on code smells and refactoring increased after 2013, with SLRs predominating and empirical or case studies commonly used for validation. The evidence base relies heavily on FLOSS projects, while expert validation, industrial datasets, reproducible protocols, and recognized benchmarks remain limited.

  • Publication trends: After 2013, secondary-study publications increased, and 75% appeared in journals versus 25% in conferences.The first secondary study appeared in 2004, with only two more publications through 2012.
  • Study types: 65% of secondary studies were SLRs, while systematic mappings and surveys each represented 17.5%.Only one study used multivocal literature mapping.
  • Validation approaches: Around 58% of secondary studies relied on empirical and case studies, but expert validation was lacking.These approaches were preferred for validating tools and prototypes.
  • Referenced projects: 57% of mentioned projects were FLOSS, with Apache, Eclipse, jHotDraw, ArgoUML, and GanttProject among the most cited.Frequently cited projects were generally long-lived, large, structured into sub-projects, and recognized in the open-source community.
  • Study quality: More than 80% of selected studies scored between 3 and 4, although some studies lacked explicit search strings or inclusion and exclusion criteria.The absence of explicit protocols made reproduction difficult.
  • Challenges: The review identifies a lack of recognized benchmarks and calls for more industrial experiments and expert participation.PROMISES12 lacked updated or smell- and refactoring-specific datasets.

5. The relationship between Code Smells and Refactoring

Code smells and refactoring are closely related to software quality: smells signal design problems, while refactoring applies behavior-preserving transformations that may address them. The reviewed evidence indicates that refactoring affects more quality attributes than code smells, but its effects remain incompletely studied and can involve trade-offs.

  • Relationship: Code smells are symptoms or design problems that may affect software evolution and maintenance, while refactorings can remove them.Refactorings may be primitive or composite sequences, and one smell can be associated with multiple refactoring options.
  • Relationship: Code smells and refactoring are closely related to software quality.The review links them through quality attributes and their effects on software evolution and maintenance.
  • Code-smell effects: Maintainability, understandability, and complexity are the quality attributes most affected by code smells.Evolvability, stability, performance, and testability were each mentioned for only one code smell.
  • Code-smell effects: God Class/Large Class, Long Method, and Feature Envy affect the widest sets of quality attributes.God Class/Large Class was reported to affect maintainability, complexity, evolvability, stability, performance, readability, reusability, and changeability.
  • Refactoring effects: Refactoring does not always improve every quality attribute, so developers should assess whether a proposed transformation achieves its intended quality goal.Some refactorings can affect attributes negatively and may introduce other code smells.
  • Evidence gaps: Fourteen refactoring options for the reviewed code smells lacked studies of their quality impact.The review concludes that effects on most measured external quality attributes require further research.
  • Refactoring effects: Refactoring affects understandability, functionality, reusability, and maintainability more than code smells after internal attributes are grouped with external attributes using QMOOD.Extract Method and Extract Class were reported as affecting the greatest number of quality attributes, with Extract Method affecting 10.

6. Implications

The review translates its findings into implications for practitioners, researchers, and instructors. It emphasizes experience-informed refactoring, monitoring and measurement, industry-connected research, and classroom practices that integrate code smells, refactoring, testing, and reviews.

  • Practitioners: Practitioners should use developers’ perceptions and experience when making refactoring decisions because refactoring need is subjective and empirical.A decision-making framework based on practitioners’ perceptions is cited as support for this view.
  • Practitioners: Version control, testing, and code reviews are encouraged because they can help identify refactoring targets, reduce complexity, and maintain quality.The review calls for further research connecting these practices with refactoring.
  • Practitioners: Measuring and monitoring refactoring impacts is needed because insufficient impact measures can prevent developers from applying refactoring.The review links prioritization of smells and assessment of refactoring to quality-attribute impacts.
  • Instructors: Teaching code smells and refactoring can help students acquire good programming practices and design principles.The review discusses curricula, exposure to change, and reasoning about when, what, and why to refactor.
  • Instructors: Coding Dojos are recommended for practicing test-driven development, refactoring, code review, pair programming, and tool use.The authors report using Coding Dojos in undergraduate and graduate courses.
  • Researchers: Researchers should standardize code-smell definitions, detection methods, and tools while studying automation, prioritization, inference, and refactoring recommendation together.The review presents these areas as connected research interests.
  • Researchers: Researchers should bring studies closer to industry and examine code smells, refactoring, software quality, and technical debt.The review identifies a gap between software-engineering research and industry in quality and design topics.

7. Open Issues

The review identifies open issues spanning definitions, detection, tools, refactoring effects, monitoring, context, education, datasets, and industry collaboration. Central gaps concern inconsistent terminology, limited validation, incomplete quality evidence, obsolete tools, and insufficiently representative research settings.

  • Code-smell detection: Researchers should standardize code-smell naming and organization because existing definitions can be complex and inconsistent.The review identifies this as an unresolved issue after more than 15 years of earlier proposed research trends.
  • Code-smell detection: The effectiveness of competing code-smell detection approaches remains unclear, and some smells have no detection approach.The review proposes studying combinations of approaches and developing methods for currently undetected smells.
  • Quality impact: The relationship between code smells and quality attributes remains incompletely established across reviewed studies.Some studies discussed quality impacts, while others did not make the connection explicit.
  • Support tools: Many code-smell detection tools are obsolete or limited in scalability, prioritization, visualization, multi-smell and multi-language support, precision, or recall.The review calls for more comprehensive tools.
  • Refactoring impact: Refactoring-quality research uses small numbers of refactorings and different quality models, limiting systematic comparison.The review proposes studying refactoring systematically and measuring effects on individual attributes.
  • Refactoring impact: Refactoring monitoring is an open research opportunity because applying refactoring can cause problems during software evolution.The proposed direction is to monitor refactoring effects over time.
  • Refactoring context: Researchers should investigate how architectural-refactoring contexts and their benefits relate to quality.The review specifically contrasts contexts such as methods, classes, and packages.
  • Research settings: Future studies should use reliable datasets, large academic and industrial projects, and analyses of inconsistencies between industrial and academic data.These directions are intended to support more reliable conclusions.

8. Threats to validity

The review identifies threats from search coverage, language and database selection, subjective classification, and the possibility that some topics or studies were missed.

  • Search strategy: The search string used synonyms and pilot searches, but some topics may not have been covered by secondary studies.The authors caution that uncovered topics do not necessarily mean the community has excluded them.
  • Database coverage: Searching eight electronic databases may have omitted relevant work indexed elsewhere, despite using snowballing to mitigate this threat.Snowballing followed references in selected papers to find potentially relevant studies.
  • Language scope: Restricting the review to English may exclude relevant studies in other languages, although the authors consider English sufficient for the main publication venues.
  • Classification: Subjective data extraction and categorization may yield different classifications and analyses in other reviews.The authors describe procedures intended to reduce bias, but acknowledge alternative schemas remain possible.
  • Reliability: The study is intended to be replicable because its steps and search string are described.

9. Conclusion and Future Work

The tertiary review finds that code-smell and refactoring research is fragmented, with shared quality-attribute relationships but substantial gaps in joint investigation, detection consistency, refactoring coverage, and tool support. It identifies open issues and calls for future studies connecting smells, refactoring, quality impact, datasets, and practice.

  • Conclusion: 40 secondary studies were systematically analyzed through a tertiary review addressing observations and challenges in code smells and refactoring.The review used five research questions to organize its synthesis.
  • Conclusion: 62.5% of secondary studies focused on smells, while only two explicitly explored smells and refactoring together.Duplicated Code/Clones and God Class/Large Class were the most mentioned smells.
  • Conclusion: Smell detection results are highly inconsistent because approaches vary and no standard detection thresholds exist.Metrics-based approaches and strategies or rules were the most cited detection approaches.
  • Conclusion: Extraction refactorings were most studied, with only about 27 of 72 refactorings examined; refactoring did not always improve code quality.
  • Conclusion: The review found 162 smell-detection tools and 24 refactoring tools, while identifying gaps in refactoring support for unexplored techniques, execution, developer support, and quality impact.CCFinder and JDeodorant were the most cited tools in their respective categories.
  • Future Work: The authors propose open questions for future research on smell detection, refactoring, and supporting tools.These questions are presented as a basis for future studies.
  • Future Work: Future work should produce reliable datasets, analyze large-scale academic and industrial projects, investigate data inconsistencies, and consider both academia and industry.
  • Conclusion: Code smells and refactoring relate to understandability, maintainability, testability, complexity, functionality, and reusability, with refactoring affecting quality more than code smells.
Loading 2004.10777v1…