PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Effective Bug Triage with Towards Effective Bug Triage with Software Data Reduction Techniques

Jifeng Xuan, He Jiang, Yan Hu, Zhilei Ren, Weiqin Zou, Zhongxuan Luo, Xindong Wu

arXiv:1704.04761v1cs.SE

TL;DR

Bug triage is costly because repositories contain large, noisy, and redundant data. The paper combines instance and feature selection and predicts their order from historical datasets; experiments on Eclipse and Mozilla show reduced data can improve triage accuracy. The authors note that noise and redundancy removal remains incomplete and order-prediction performance is not yet ideal.

  • Problem

    Bug triage must handle large, noisy, and redundant repositories, motivating reduction that lowers data scale and improves quality.

  • Method

    The paper combines instance selection and feature selection, then trains a predictive model on historical datasets to select their application order.

  • Results

    Data reduction reduces reports and words while improving bug-triage accuracy in experiments on Eclipse and Mozilla.

  • Takeaways & Limitations

    The approach forms reduced and higher-quality bug data for software development and maintenance.

  • Takeaways & Limitations

    The method does not remove all noise and redundancy, and its reduction-order prediction is not an ideal resolution.

Abstract

from arXiv · show

Software companies spend over 45 percent of cost in dealing with software bugs. An inevitable step of fixing bugs is bug triage, which aims to correctly assign a developer to a new bug. To decrease the time cost in manual work, text classification techniques are applied to conduct automatic bug triage. In this paper, we address the problem of data reduction for bug triage, i.e., how to reduce the scale and improve the quality of bug data. We combine instance selection with feature selection to simultaneously reduce data scale on the bug dimension and the word dimension. To determine the order of applying instance selection and feature selection, we extract attributes from historical bug data sets and build a predictive model for a new bug data set. We empirically investigate the performance of data reduction on totally 600,000 bug reports of two large open source projects, namely Eclipse and Mozilla. The results show that our data reduction can effectively reduce the data scale and improve the accuracy of bug triage. Our work provides an approach to leveraging techniques on data processing to form reduced and high-quality bug data in software development and maintenance.

——————————  —————————— 1 INTRODUCTION

Bug triage is costly because repositories contain large, low-quality bug data, motivating reduction methods that preserve or improve classification accuracy. The paper combines instance and feature selection, predicts their application order, and reports improved triage accuracy on Eclipse and Mozilla data.

  • Motivation: Bug repositories challenge software development through large scale, noise, and redundancy.Eclipse received 333,371 reports from 2001 to 2010, while noisy bugs may mislead developers and redundant bugs waste handling time.
  • Motivation: Manual bug triage is time-consuming and error-prone, motivating automated text-classification approaches.In Eclipse, 44 percent of bugs were assigned incorrectly and first triaging took 19.3 days on average.
  • Approach: The paper combines instance selection and feature selection to reduce bug reports and words while improving bug-triage accuracy.The reduced data are evaluated by dataset scale and triage accuracy across four algorithms of each type.
  • Approach: The proposed predictive model uses attributes from historical bug datasets to determine the order of the two reduction techniques.A binary classifier predicts the order for a new bug dataset.
  • Results: When 50% of bug reports and 70% of words are removed, Naive Bayes accuracy improves by 2% to 12% on Eclipse and by 1% to 6% on Mozilla.The experiments use bug reports from the two large open-source projects.
  • Results: The predictive model achieves 71.8% accuracy for predicting the reduction order.Top-node analysis finds that no individual attribute determines the order and that each attribute helps prediction.

2. We propose a combination approach to addressing

The paper identifies the order of applying instance and feature selection as an underexplored issue and extends prior work with prediction and broader experiments.

  • Contribution: The paper introduces prediction of the order for applying instance selection and feature selection.It frames this order as a binary classification problem using attributes extracted from bug datasets.
  • Novelty: The order of applying instance and feature selection had not been investigated in related domains.The paper presents this as a novel research direction.
  • Extension: This extension adds new bug-data attributes, order prediction, and experiments covering four algorithms of each selection type and their combinations.It extends the authors’ previous work.

2 BACKGROUND AND MOTIVATION

Bug repositories support bug handling, but manual triage and growing noisy or redundant data make effective use difficult. The paper illustrates these issues through bug-report histories and examples, then motivates combined data reduction and order prediction.

  • 2.1 Background: Bug reports record textual bug information and are assigned to developers who attempt to fix the reported problem.Reports include summaries, descriptions, product, platform, importance, and assignment history.
  • 2.1 Background: Historical bug handling may require examining similar reports and previously fixed bugs, while reassignment can prolong resolution.Bug 284541 was assigned to three developers over more than seven months, although fixing it took three days.
  • 2.1 Background: Text classification uses a bug report’s summary and description as text and the fixing developer as its label.These techniques predict a developer for a new bug and support human triagers.
  • 2.2 Motivation: Common words can be non-informative, unclear reports can be noisy, and duplicate reports can create redundancy.The examples motivate removing words and selecting representative bug reports.
  • 2.2 Motivation: The proposed data-reduction phase combines instance and feature selection before classifier training and predicts their application order.Figure 2 frames the process as existing triage augmented with reduction and binary order prediction.

3 DATA REDUCTION FOR BUG TRIAGE

The paper combines instance selection and feature selection to reduce both bug reports and words before bug triage, targeting lower data scale and higher accuracy. It also examines how the reduction order affects outcomes.

  • 3 DATA REDUCTION FOR BUG TRIAGE: Data reduction combines instance selection and feature selection to reduce the bug and word dimensions of bug data.The reduced data set replaces the original data set for bug triage.
  • 3 DATA REDUCTION FOR BUG TRIAGE: Instance selection removes relevant subsets of bug reports, while feature selection removes relevant subsets of words.The paper evaluates their combination rather than either technique alone.
  • 3 DATA REDUCTION FOR BUG TRIAGE: The two reduction techniques can be applied in either order, and the order is treated as a prediction problem.The paper denotes the two sequential orders and compares them for bug-data reduction.
  • 3 DATA REDUCTION FOR BUG TRIAGE: The reduction has two goals: reducing data scale to save developer labor and improving bug-triage accuracy by removing noisy or duplicate information.The paper links bug-dimension reduction to uninformative reports and word-dimension reduction to noisy or duplicate words.
  • 3 DATA REDUCTION FOR BUG TRIAGE: Feature selection improves bug-triage accuracy and can recover accuracy lost when instance selection removes bug reports.The paper also reports that instance selection may decrease accuracy.

4 PREDICTION FOR REDUCTION ORDERS

The paper predicts which reduction order to use for a new bug data set instead of manually checking both orders. It represents data sets with attributes and applies binary classification to predict the better order.

  • 4 PREDICTION FOR REDUCTION ORDERS: Prediction for reduction orders chooses between applying instance selection first and applying feature selection first.The order can affect bug-triage results, creating the need to choose between the two alternatives.
  • 4 PREDICTION FOR REDUCTION ORDERS: A bug data set is mapped to a classifier instance, while its associated reduction order is mapped to a class label.This converts reduction-order prediction into a binary classification problem.
  • 4 PREDICTION FOR REDUCTION ORDERS: A classifier can be trained once and then used to predict reduction orders for many new bug data sets.This avoids checking both reduction orders separately for each new data set.
  • 4 PREDICTION FOR REDUCTION ORDERS: The method extracts 18 attributes from each bug data set before new bugs are triaged.The attributes describe characteristics of the bug data set and are normalized in the data-preparation setup.
  • 4 PREDICTION FOR REDUCTION ORDERS: The 18 attributes comprise 10 bug-report attributes and 8 developer attributes.The paper labels these categories B1 to B10 and D1 to D8.

5.1 Data Preparation

The experiments prepare bug-triage data from Eclipse and Mozilla repositories, filtering reports and representing report text for classification. The resulting data sets are formed from continuous bug units.

  • 5.1 Data Preparation: The evaluation uses bug repositories from the Eclipse and Mozilla open source projects.Eclipse is described as a software development environment, while Mozilla is described as an Internet application suite.
  • 5.1 Data Preparation: The study collects continuous ranges of bug reports, obtaining 298,785 reports for Eclipse and 281,180 for Mozilla after removals.The target collection was 300,000 reports for each project.
  • 5.1 Data Preparation: The preparation retains fixed and duplicate bug reports and removes unfixed reports such as new or will-not-fix reports.Inactive developers are also excluded because they provide insufficient information for predicting correct developers.
  • 5.1 Data Preparation: Bug summaries and descriptions are converted into vector-space representations using tokenization and stop-word removal.These fields are selected as representative bug content and inputs to the classifiers.

5.2 Experiments on Bug Data Reduction

Experiments across Eclipse and Mozilla show that combining instance selection with feature selection can reduce both bug reports and words while improving bug-triage accuracy. Results also show that algorithm order matters, and that reductions are especially effective when built on Naive Bayes.

  • Rates of selected bug reports and words: 50% bug reports and 30% words are selected in subsequent experiments after ICF and CH showed favorable reduction rates.ICF performed slightly better than LVQ, while CH outperformed IG; 50% or 70% of bug reports and 30% or 50% of words were effective settings.
  • Individual reduction techniques: Feature selection generally increased accuracy, whereas instance selection could decrease it, especially as the number of developers increased.The data sets contained over 200 classes, and most loss from origin to ICF increased with the number of developers.
  • Combined data reduction: Combining instance and feature selection improved accuracy while reducing both bug reports and words, with feature selection supplementing instance-selection accuracy loss.The experiments evaluated combinations with SVM, KNN, and Naive Bayes across multiple data sets.
  • Combined data reduction: Data reduction improved Naive Bayes accuracy by 2% to 12% on Eclipse and by 1% to 6% on Mozilla.For list size 5, Naive Bayes-based reduction also produced 13% to 38% better results than SVM-based reduction and 21% to 28% better results than KNN-based reduction.
  • Time cost: Data reduction required 298 to 1,558 seconds, compared with 23 to 57 days for manual triage per bug report.Original Naive Bayes required 88 to 139 seconds, so reduction added computation but remained much faster than manual triage.
  • Reduction order: The order of applying instance and feature selection changed accuracy, retained words, removed duplicates, and produced different reduced data sets.For example, one order removed 198 duplicate bug reports and the other removed 262; the reduced sets also retained different words and blank bug reports.

5.3 Experiments on Prediction for Reduction Orders

The paper predicts whether instance selection or feature selection should be applied first by training classifiers on historical bug-data attributes. Experiments on Eclipse and Mozilla show that classifier performance varies by project and class imbalance, while multiple attributes contribute to order prediction.

  • Prediction setup: The authors map each bug data set to an instance and its reduction order to a label, then train classifiers to predict the appropriate order for new data sets.The data sets are formed from continuous bug units, with 300 Eclipse data sets and 399 Mozilla data sets.
  • Evaluation: AdaBoost with C4.5 decision trees is evaluated using resampling and reweighting, alongside C4.5, with precision, recall, F1-measure, and accuracy as criteria.Ten-fold cross-validation is used, and AdaBoost is selected because the order classes are imbalanced.
  • Results: AdaBoost C4.5 reweighting provides the best results among the three classifiers on Eclipse, while AdaBoost with resampling provides the best accuracy and F1-measure on Mozilla.On Eclipse, C4.5 obtains the best accuracy, whereas reweighting obtains the best F1-measure.
  • Results: The Eclipse and Mozilla data sets are combined into 699 data sets to increase the amount of training data for predicting reduction orders.The combined collection contains 202 data sets with one order and 497 with the other.
  • Attribute analysis: Top-node analysis identifies bug-report length and words per fixer as recurring decisive attributes, but no single attribute determines reduction-order predictions.Bug-report length appears at all three analyzed tree levels, while attributes collectively contribute to prediction.

6 DISCUSSION

The discussion characterizes data reduction as useful but incomplete: it reduces bug-data scale and can improve triage quality, yet real-world noise is difficult to identify and prediction remains imperfect. The approach assists human triagers rather than replacing them.

  • Data-reduction limitations: Instance selection and feature selection reduce bug-data noise and redundancy, but fewer than 50% of duplicate bug reports can be removed in the reported setting.The authors attribute this limitation to inadequate noise labels and useful information remaining in noisy or redundant natural-language data.
  • Triage complexity: Bug-triage accuracy remains below 61% despite recommendation lists, reflecting natural-language complexity, many potential developers, and incomplete project knowledge.The paper states that human triagers may also assign developers incorrectly.
  • Prediction model: The predictive model uses statistical attributes such as word counts and report length, without extracting representative words from bug data sets.The authors plan to investigate more detailed attributes in future work.
  • Prediction limitations: F1-measure and accuracy for reduction-order prediction are not large enough for binary classifiers, so the method is presented as a step toward automatic prediction rather than an ideal solution.The model can be trained once and used for new bug data sets at lower cost than trying both reduction orders.
  • Assumption: The method assumes that the time range of bugs within a data set can be ignored relative to the time of bug triage.The authors argue that this assumption permits attribute extraction to be applied to real-world repositories.

7 RELATED WORK

Prior work studies bug-data characteristics, duplicate reports, bug triage, and software-defect quality, whereas this paper combines data reduction with bug-triage preprocessing. It also distinguishes bug triage from broader expert-finding and ticket-routing problems.

  • Bug-data research: Earlier studies characterize bug-data quality, detect duplicate reports, and analyze bug-report or developer relationships.These approaches include questionnaires, natural-language processing, retrieval optimization, and network analysis.
  • Contribution relative to prior work: This paper differs from prior bug-data-quality work by providing preprocessing that both reduces data scale and improves data quality for bug triage.The paper contrasts its approach with studies of data characteristics and duplicate-report detection.
  • Bug triage: Existing bug-triage research applies text classification, tossing graphs, semi-supervised learning, and collaborative filtering to reduce manual effort or reassignment.Bug triage predicts an appropriate developer from the textual content of a bug report.
  • Related task distinctions: Unlike expert finding and ticket routing, bug triage assigns developers to bug-report documents through content-based classification rather than keyword- or sequence-based methods.The distinction is based on the task scope and representation of bug reports.
  • Defect prediction: Compared with defect prediction, which is binary classification over individual software artifacts, this work addresses multiple-class bug triage and predicts a value for a set of software artifacts.The authors frame the work as an extension of software metrics.

8 CONCLUSIONS

The paper combines instance selection, feature selection, and predictive order selection to reduce bug-data scale and improve quality for bug triage. It evaluates the approach on Eclipse and Mozilla and identifies improving reduction results and understanding order attributes as future directions.

  • Conclusions: The approach combines feature selection with instance selection to reduce bug-data scale and improve data quality, using historical data to predict the reduction order for new data sets.The empirical investigation covers bug repositories from Eclipse and Mozilla.
  • Future work: The authors identify improving data-reduction results and investigating relationships between bug-data attributes and reduction orders as future work.The stated goal is to prepare higher-quality bug data for domain-specific software tasks.
Loading 1704.04761v1…