Source-linked AI summary
Do Code Clones Matter?
Elmar Juergens, Florian Deissenboeck, Benjamin Hummel, Stefan Wagner
TL;DR
Code cloning’s consequences for program correctness had not been thoroughly investigated. The paper conducts a large-scale case study using a novel inconsistent-clone detector, finding frequent inconsistencies and faults associated with them. It also provides scalable open-source tool support for further investigation and clone tracking.
Problem
Prior research provided little thorough evidence about how code cloning affects program correctness and how harmful cloning really is.
Method
The authors analyzed commercial and open-source systems using a novel inconsistent-clone detection algorithm and manually inspected about 900 clone groups.
Results
Nearly every second unintentionally inconsistent change to a clone leads to a fault, and the study found strong evidence that inconsistent clones constitute a major source of faults.
Takeaways & Limitations
Reducing cloning and using tools that prevent unintentionally inconsistent clone changes are proposed approaches for reducing related faults.
Takeaways & Limitations
The conclusions are limited to cloning’s consequences for program correctness; the impact on software maintenance effort remains future work.
Abstract
from arXiv · showhide
Code cloning is not only assumed to inflate maintenance costs but also considered defect-prone as inconsistent changes to code duplicates can lead to unexpected behavior. Consequently, the identification of duplicated code, clone detection, has been a very active area of research in recent years. Up to now, however, no substantial investigation of the consequences of code cloning on program correctness has been carried out. To remedy this shortcoming, this paper presents the results of a large-scale case study that was undertaken to find out if inconsistent changes to cloned code can indicate faults. For the analyzed commercial and open source systems we not only found that inconsistent changes to clones are very frequent but also identified a significant number of faults induced by such changes. The clone detection tool used in the case study implements a novel algorithm for the detection of inconsistent clones. It is available as open source to enable other researchers to use it as basis for further investigations.
1. Clones & correctness
This paper investigates whether inconsistent changes to cloned code affect program correctness, addressing limited prior evidence about cloning’s harmfulness. A large-scale case study and novel detector found frequent inconsistencies and faults associated with them.
- Cloned code can increase maintenance costs, while inconsistent changes can create faults and incorrect program behavior.
- An inconsistent bug fix can leave a fault active when only one clone instance receives the correction.
- The study examined whether clones change inconsistently, whether inconsistencies are intentional, and whether unintentional inconsistencies can represent faults.
- The case study covered three commercial C# systems, one commercial Cobol system, and one open-source Java system.
- The authors developed a novel inconsistent-clone detection algorithm and manually inspected about 900 clone groups to address false positives.
2. Terms and definitions
The paper defines clones syntactically as repeated code substrings, with inconsistent clones allowing bounded differences. It distinguishes faults, failures, and defects by their roles in incorrect software behavior.
- An exact clone is a normalized code substring appearing at least twice, and an exact clone group contains at least two such clones at different positions.
- An inconsistent clone is a code substring with another non-overlapping substring whose edit distance is below a chosen threshold.
- An inconsistent clone group contains at least one pair of inconsistent clones within its connected clone graph.
- A failure is incorrect user-visible output, a fault is its potential cause inside code, and defects include both faults and failures.
3. Related work
Prior research used diverse clone representations and reported mixed evidence about cloning’s effects on correctness and maintainability. This paper addresses limitations through manual inspection, broader systems, creation-independent analysis, and an open-source scalable tool.
- Earlier studies reported inconsistent clone evolution, higher revision counts for cloned files, and bugs discovered through inconsistent clone changes.
- Other findings questioned cloning’s harmfulness, reporting limited later consistency and no statistically verified relationships with change coupling or changeability.
- The overall effect of cloning on maintainability and correctness remained unclear because prior findings had transferability limitations.
- This study manually rated actual inconsistent clones, analyzed open-source and industrial systems, and considered inconsistencies independently of how they were created.
- Existing approaches faced limitations involving legacy-language grammars, edit-distance control, recall, industrial scalability, or unavailable implementations.
- The presented approach supports modern and legacy languages, controls similarity through statement edit distance, scales to industrial projects, and is open source.
4. Detecting inconsistent clones
The detector preprocesses source into normalized statements and searches for approximate matches using edit distance over a suffix tree. Filtering and practical limits make the pipeline usable on large systems, including Eclipse.
- Preprocessing: The detector tokenizes code, removes comments and generated code, normalizes statements, and feeds the resulting sequence into the detection pipeline.
- Detection algorithm: The algorithm searches for common substrings whose edit distance is bounded by a chosen threshold.
- Detection algorithm: The detector traverses a suffix tree recursively, extending candidate substrings while consuming an allowed edit-distance budget.
- Implementation: Dynamic programming computes longest edit-distance matches but requires quadratic time and space in its straightforward form.
- Implementation: The search examines at most the first 1000 statements of an edge, reporting longer clones in chunks and excluding self-matches.
- Filtering and constraints: The search requires initial statements to match exactly, preventing arbitrary extension of exact clones into inconsistent clones.
- Filtering and constraints: Filtering removes overlapping or contained clone groups and applies absolute and relative inconsistency limits.
- Performance: The pipeline processed Eclipse’s 5.6 MLOC in about 3 hours on a single-threaded 2.4 GHz Intel Core 2 Duo system.
5. Study description
The study examines inconsistent cloning across five production systems and uses clone-group sets to structure three research questions about inconsistency, intentionality, and faults. It combines automated detection with manual evaluation and compares set sizes to characterize these phenomena.
- 5.1. Study objects: Five production projects—three C#, one Java, and one Cobol system—were analyzed across different teams, languages, domains, ages, and sizes.The systems came from two companies and one open-source project; commercial systems were anonymized, while Sysiphus enabled external replication.
- 5.2. Research questions: RQ 1 asks whether inconsistent clones exist in real-world systems and constitute a significant part of all clone groups.The study measures the size of IC relative to C, where C is the set of all clone groups and IC is the set of inconsistent clone groups.
- 5.2. Research questions: RQ 2 asks whether inconsistent clones were created unintentionally, distinguishing unintentional divergence from changes made to satisfy different requirements.Developers rated identified inconsistent clones as intentional or unintentional, yielding the ratio |UIC|/|IC|.
- 5.2. Research questions: RQ 3 asks whether inconsistent clones indicate faults, focusing on the fault-indicating subset F of unintentionally inconsistent clone groups UIC.The study uses |F|/|IC| as a lower bound because faults in intentionally inconsistent clones are excluded.
- 5.4. Procedure: Fault density in inconsistencies is compared with average fault-density values, using faults divided by the logical lines of code in inconsistencies.The study does not test the hypothesis statistically because the number of data points is low; the comparison uses a reported typical range of 0.1–50 faults per kLOC.
- 5.4. Procedure: The study detects inconsistent clone candidates automatically, then manually rates candidates to remove false positives and classify inconsistent groups.Detection was run on each system; configurations used a 10-statement minimum for C# and Java and 20 statements for Cobol, with corresponding edit-distance thresholds of 5 and 10.
6. Results
Inconsistent changes are common in clones and can indicate faults, including faults with varied practical effects. The study also found lower precision for inconsistent clone groups in most systems, with Cobol system D as an exception.
- Detection results: Inconsistent clone groups generally had lower precision than exact clone groups, except in Cobol system D.Greater allowed deviation in inconsistent groups explains the expected precision difference; conservative parameters and unrelated database-access code affected system-specific precision.
- Inconsistent changes: 52% of clones contain inconsistencies, and 28% of those inconsistencies were introduced unintentionally.The authors describe intentional cloning and subsequent changes as a frequent development and maintenance pattern.
- Fault indicators: 3–23% of inconsistencies presented a fault, rising to 18% when Cobol system D was excluded.The reported fault densities ranged from 3.4–91.4 faults per kLOC, with system D again an outlier.
- Fault indicators: About every second to third unintentional change to a clone led to a fault across the projects after accounting for clone-awareness differences.The ratio of unintentional changes leading to faults was in the same range for all projects, although project D had lower unintentional-change and fault rates.
- Fault consequences: The 107 inconsistent clones included 17 category (1), 44 category (2), and 46 category (3) faults.Examples included incorrect messages, inconsistent defaults, editing or validation behavior, memory leaks, performance issues, and exception-handling differences.
7. Threats to validity
The study’s validity is constrained by imperfect fault-density comparisons, subjective assessment, tool configuration, preprocessing, incomplete histories, and a non-random, small sample of systems. The authors describe mitigation steps, but these boundaries limit interpretation and representativeness.
- Construct validity: Development repositories were not analyzed, partly because industrial development histories were incomplete.This prevented determining whether inconsistencies arose from incomplete changes rather than random similarities, while preserving visibility into copy-and-modify cases from a single commit.
- Construct validity: Average fault-probability comparisons cannot establish reliably whether inconsistencies are more fault-prone than random code.Actual system fault densities were unavailable because defect databases were incomplete, and checking random code lines was impractical.
- Internal validity: Developers’ judgments of intentionality and faultiness may be incorrect, while inconclusive candidates were classified as intentional and non-faulty.The authors required complete certainty and consulted knowledgeable colleagues to reduce false-positive fault classifications.
- Internal validity: Clone-detection configuration strongly influences detection results and varied across programming languages.Parameters were calibrated using a pre-study and experience, with care taken to produce sensible inconsistent-clone results.
- Internal validity: Preprocessing to remove false positives may have excluded clones that were actually faulty.The authors state that this would reduce the chances of answering the research questions positively.
- External validity: The systems were not randomly sampled, and only five systems—mostly written in C#—were analyzed.The sample included different organizations, technically different C# systems, a legacy Cobol system, and an open-source Java system.
8. Discussion
The study finds that clones can lead to faults and that inconsistencies are often attributable to developer mistakes rather than different requirements. It proposes prevention and detection practices, supported by an open-source, independently replicable analysis.
- Clones can lead to faults, and inconsistencies are often explained by developer mistakes rather than different requirements.
- The Sysiphus analysis is independently replicable because both the project and detection tools are open source.
- Reducing clone-related faults requires less cloning and tools that prevent or detect unintentionally inconsistent clone changes.
9. Conclusion
The paper concludes that inconsistent clones are a major source of faults and provides a scalable detection algorithm and tool support. It also identifies methodological and scope boundaries requiring further work, especially beyond correctness consequences.
- Inconsistent clones constitute a major source of faults during software development and maintenance unless their evolution is tracked carefully.
- A scalable algorithm and tool support are provided for finding inconsistent clones and enabling future experiments.
- Further work should refine detection tools, improve usability, compare algorithms, and classify defect types more quantitatively.
- The paper is a first step toward a usable and economically sensible methodology for reducing development and maintenance costs.
- The results are limited to clones’ consequences for program correctness, leaving their impact on software maintenance effort for future research.