PaperlayerView exact revision on arXiv

Source-linked AI summary

Bug Localization from Bug Reports: A Multi-Objective Approach

Waleed Ahmad, Mehtab Kiran Suddle, Maryam Bashir

arXiv:2608.27089v1cs.NEcs.SE

TL;DR

Bug localization is labor-intensive, and lexical matching is limited by the gap between natural-language bug reports and source code. The paper proposes a multi-objective search system using evolutionary algorithms to rank buggy classes while balancing similarity and recommendation size. SPEA-2 achieved the strongest reported performance, with correct buggy files ranked in the top 20 for nearly 94% of error reports, and the approach was evaluated across Java and Kotlin projects.

  • Problem

    Lexical matching is limited by the linguistic gap between natural-language bug reports and programming-language source code, making automated bug localization challenging.

  • Method

    The paper formulates class-level bug localization as multi-objective search using SPEA-2 and MOEA/D to maximize similarity while minimizing recommended files.

  • Results

    Nearly 94% of error reports had correct buggy files ranked within the top 20, and SPEA-2 achieved the highest precision and recall among evaluated models.

  • Takeaways & Limitations

    The framework identified buggy code across six Java datasets and a Kotlin-based Android project, supporting its evaluation across programming-language settings.

  • Takeaways & Limitations

    The study evaluated a small Kotlin-based Android dataset and plans further testing on other programming platforms and error types.

Abstract

from arXiv · show

Bug localization is a labor-intensive task, particularly in large software systems. When abnormal behavior occurs, developers must perform repetitive and time-consuming steps to identify faulty files. Previous studies have mainly focused on single-objective localization methods, many of which are limited to specific programming languages. In addition, relying solely on lexical similarity between source code and bug reports is often insufficient due to the natural language nature of bug descriptions. In this study, we propose a class-level automated multi-objective search-based system to identify and rank potentially buggy classes from bug reports. The main objective is to maximize similarity while minimizing the number of suggested faulty files. The evolutionary optimization algorithm SPEA-2 was applied to six open-source Java projects comprising more than 22,000 bug reports. The proposed approach was evaluated against two widely used algorithms, NSGA-II and MOEA/D. Results indicate that SPEA-2 achieved higher precision and recall than both multi-objective and single-objective baseline methods. The proposed recommender system successfully identified buggy classes or files for 88.5\% of bug reports within the top 10 recommendations and 94\% within the top 20. The effectiveness of the model was further validated on an industrial Android project written in Kotlin, demonstrating its adaptability across programming languages.

1. Introduction

Bug localization is difficult because software systems and bug reports are complex, while lexical matching between natural-language reports and source code is often insufficient. This study formulates localization as a multi-objective search problem that balances relevance with a small recommendation set.

  • Software complexity increases the likelihood of bugs that can cause operational failures.
  • Natural-language bug reports and programming-language source code create a linguistic gap that limits purely lexical matching.
  • The model ranks classes or files by balancing lexical and history-based similarity against the number of recommendations.
  • The study proposes SPEA-2 and MOEA/D-based search approaches with novel tokenization for bug localization.The evaluation covers six open-source Java projects and an industrial Kotlin-based Android project with 100 bug reports.
  • The framework compares multi-objective models with lexical-similarity, history-based-similarity, and NSGA-II approaches across Java and Android projects.

2. Literature Review

Prior bug-localization research spans information retrieval, deep learning, large language models, and computational search-based methods. The literature reports progress but continues to identify lexical mismatches, false positives, and difficulty balancing relevance with result size.

  • Bug-localization research includes information retrieval, deep learning, large language models, and computational search-based approaches.
  • Classical information-retrieval methods use LDA, LSI, and VSM to model similarities between bug reports and source artifacts.
  • Representative systems incorporate structural retrieval, topic models, version history, transfer learning, multimodal features, and changeset matching.
  • Large-language-model research examines automated localization and repair, including query reformulation and result reranking.
  • IR-based techniques remain susceptible to false positives, while newer work explores classification, evolutionary optimization, and hybrid methods.
  • The literature identifies balancing relevance and result size as an area requiring further improvement.

3. Multi-Objective Problem Formulation for Bug Localization

The proposed formulation searches combinations of source classes for recommendations that maximize relevance while minimizing list size. It uses evolutionary optimization, with SPEA-2 preserving non-dominated solutions and incorporating opposition-based learning.

  • 3.1. Model Overview: The approach explores combinations of project classes because multiple classes may contribute to a reported error.
  • 3.1. Model Overview: Metaheuristic techniques address the large search space through contradictory objectives: maximizing correction and minimizing recommendation size.
  • 3.1. Model Overview: The correction function combines lexical similarity between reports and files with a history-based score.
  • 3.1. Model Overview: Four inputs—API specification, previous bug reports, the current report, and modification history—produce an optimized ranked list of files or classes.
  • 3.2. Strength Pareto Evolutionary Algorithm (SPEA-2) using opposition-based learning: SPEA-2 ranks solution dominance and preserves a Pareto front of non-dominated solutions through crossover and mutation.
  • 3.2. Strength Pareto Evolutionary Algorithm (SPEA-2) using opposition-based learning: SPEA-2 uses an external archive, strength-based fitness, environmental selection, truncation, mating, and variation across generations.
  • 3.2. Strength Pareto Evolutionary Algorithm (SPEA-2) using opposition-based learning: Opposition-based learning generates an opposite population after recombination and mutation before the next fitness-calculation cycle.

3.3. Multi Objective Evolutionary Algorithms using Decomposition Approach (MOEA/D)

MOEA/D decomposes a multi-objective problem into scalar sub-problems that are solved simultaneously. In this model, MOEA/D uses opposition-based learning as part of the proposed search recommendation system.

  • The decomposition approach converts a multi-objective problem into several sub-problems using aggregation functions.
  • The approach can use penalty-based boundary intersection, Tchebycheff, or weighted-sum decomposition techniques.
  • A combined set of Pareto-optimal sub-problem solutions can provide an approximation of the overall Pareto front.
  • Figure 4 provides an overview of the proposed multi-objective search recommendation system for bug reports.
  • The proposed model implements MOEA/D with opposition-based learning.
  • MOEA/D optimizes N scalar problems simultaneously rather than directly solving one multi-objective problem.
  • MOEA/D is described as having lower computational complexity than NSGA-II while maintaining solution quality.

3.4. Non-dominated Sorting Genetic Algorithm (NSGA-II)

NSGA-II formulates bug localization as a multi-objective search for Pareto-optimal recommendations, representing each candidate as a vector of potentially buggy classes. The study adapts tokenization to emphasize essential class elements rather than splitting the entire codebase by camel case.

  • NSGA-II: NSGA-II identifies a Pareto set of balanced solutions across conflicting objectives.Each solution represents a compromise without sacrificing either objective.
  • NSGA-II: Each candidate solution is a vector whose dimensions correspond to potentially buggy classes recommended for a bug report.
  • Tokenization approach: The proposed tokenization focuses on comments, class names, function names, and variable names instead of applying camel-case splitting across the entire codebase.The approach targets classes more likely to be responsible for the reported bug.

3.6. Fitness Function (Objectives)

The fitness function balances a correction objective that increases lexical and history-based similarity with a minimization objective that reduces the number of recommended classes. These objectives conflict because retrieving more similar classes can increase the candidate set.

  • Objectives: The fitness function contains correction and minimization sub-functions as two objectives.
  • Correction objective: The correction objective combines lexical-based similarity and history-based similarity.Lexical similarity compares bug reports with source-code and API-documentation representations, while history-based similarity uses prior changes and bug reports.
  • Lexical similarity: Cosine similarity compares vector representations of bug reports and source code after tokenization, stop-word removal, and stemming.Source-code identifiers are tokenized with a camel-case splitter, and vector weights use information-retrieval strategies such as term frequency–inverse term frequency.
  • Lexical similarity: Detailed comments or explicit code-element names in bug reports can increase cosine similarity between reports and source code.
  • Minimization objective: The minimization objective reduces the number of suggested buggy classes, conflicting with similarity maximization when higher similarity retrieves more classes.

3.7. Opposition-based Learning (OBL)

Opposition-based learning supplements evolutionary optimization by generating opposite solutions during initialization and evolution, supporting faster convergence. The system evaluates contradictory objectives: improving correction while minimizing recommendation-set size.

  • Optimization process: Multi-objective optimization evolves an initial population across generations until stopping criteria are satisfied.
  • Opposition-based learning: Opposition-based learning generates opposite solutions during initialization and evolution rather than relying only on random values.The technique is described as accelerating convergence.
  • History-based scoring: A solution consists of a set of recommended classes, and history-based scoring considers earlier bug fixes, modification dates, and consistency with prior bug reports.
  • Contradictory objectives: The correction objective maximizes lexical and history-based similarity, while the minimization objective reduces the number of recommended classes.
  • Opposition-based learning: Opposite-based selection compares candidate and opposite solutions using an evaluation function and continues with the one favored by the optimality criterion.

3.8. Dataset Description

The evaluation uses six open-source projects with more than 22,000 bug reports and an additional industrial Android dataset containing 100 reports. The industrial project combines Kotlin, Java, and XML files, requiring extra parsing effort.

  • Open-source datasets: The primary experiments cover six open-source software projects comprising more than 22,000 bug reports.
  • Industrial validation: Additional validation uses a non-open-source industrial Android dataset containing 100 bug reports.A database linking buggy classes to corresponding bug reports was created for validation.
  • Industrial validation: The Android project contains Kotlin, Java, and XML files, unlike the entirely Java-based open-source projects.Its heterogeneous structure required additional parsing effort.

4. Evaluation Techniques

The evaluation uses structured research questions, repeated experiments, multiple datasets, and precision, recall, and accuracy measures to assess bug-class recommendation. It also compares tokenization variants, mono-objective baselines, and an industrial Android project.

  • Experimental design: Experiments used open-source Java projects and an Android project, with each experiment replicated 30 times and compared against published heuristic and non-heuristic approaches.The evaluation was designed to assess robustness across datasets and baseline approaches.
  • Research questions: The evaluation addresses efficacy, improved tokenization, comparison with mono-objective heuristics, and cross-domain validation.These four research questions structure the assessment of the proposed system.
  • Evaluation metrics: Precision@k measures accurately suggested classes among the top k relative to the lowest number of classes examined in the ranked list.The definition uses true-positive and false-positive results in the accompanying calculation.
  • Evaluation metrics: Recall@k measures correct suggested classes among the top k relative to the total expected classes containing an error.Recall uses true positives and false negatives in its calculation.
  • Comparison procedures: The improved NSGA-II tokenization was compared with full-code tokenization, while mono-objective comparisons used a Knee-point solution as the multi-objective representative.The improved variant tokenizes only relevant components, and the Knee-point selection enables comparison with a single solution.
  • Cross-domain validation: The industrial Android validation used a database pairing bug reports with buggy classes across XML, Kotlin, and Java files.The model was executed on this project to validate accuracy beyond the open-source Java datasets.

5. Results and Discussion

Across six Java projects, SPEA-2 generally achieved the strongest localization performance, while improved tokenization enhanced NSGA-II and the multi-objective models outperformed mono-objective baselines. On the industrial Kotlin Android project, SPEA-2 also led across metrics, supporting cross-language applicability.

  • Results for RQ1: 86%, 82%, 76%, and 71% were SPEA-2’s average precision@k values, exceeding MOEA/D and NSGA-II across the evaluated k values.MOEA/D achieved 75%, 72%, 69%, and 64%, while NSGA-II achieved 83%, 80%, 71%, and 68%.
  • Results for RQ2: 90% average recall at k=20 showed that most actual buggy classes appeared within the top 20 rankings for improved NSGA-II.At k=5, the average recall was 71%.
  • Results for RQ2: 95% accuracy at k=20 was achieved by NSGA-II with improved tokenization, which processes only critical source-code sections.Standard NSGA-II tokenizes the entire codebase; the improved version also reduced computational time and storage requirements.
  • Results for RQ3: 91.3% recall at k=20 for SPEA-2 exceeded the highest recall of lexical similarity and history-based similarity, 71% and 68%.Average precision was 58.5% for lexical similarity and 56.5% for history-based similarity.
  • Results for RQ3: Combining lexical and history-based objectives significantly improved performance across all six open-source datasets.Buggy classes missed by one mono-objective algorithm were sometimes identified by the other.
  • Results for RQ4: 78.2% recall and 88.5% accuracy at k=20 were achieved by SPEA-2 on the industrial Kotlin Android project.At k=20, NSGA-II achieved 84% accuracy and MOEA/D achieved 75.3%.
  • Results for RQ4: Shorter running times than lexical and history-based similarity methods were reported for the proposed models.The comparison used milliseconds as the efficiency measure.

6. Conclusion and Future Work

The paper concludes that its automated class-level multi-objective search system effectively ranks buggy code elements by balancing correctness and recommendation-list size. SPEA-2 achieved the strongest overall results, while future work targets preprocessing, additional platforms, and error-type complexity.

  • Conclusion: The proposed automated class-level approach ranks relevant buggy code elements from bug reports using multi-objective search.Its objectives balance maximizing correctness with minimizing the number of recommended buggy files.
  • Conclusion: The correctness function combines bug-report, source-code, and API-documentation similarity with bug-fixing and code-change history.Evaluation covered six Java datasets and a Kotlin Android dataset containing more than 22,000 bug reports collectively.
  • Conclusion: Nearly 94% of error reports had correct buggy files ranked within the top 20 suggested classes.SPEA-2 achieved the highest precision and recall among the evaluated models.
  • Future Work: Future work will improve preprocessing, extend evaluation to Angular-JS and iOS, and investigate error-type complexity.The stated goal is to identify a larger proportion of buggy files among top-ranked classes, including within the top five recommendations on additional platforms.
Loading 2608.27089v1…