Source-linked AI summary
Developer Prioritization in Bug Repositories
Jifeng Xuan, He Jiang, Zhilei Ren, Weiqin Zou
TL;DR
The paper addresses the previously unstudied problem of ranking developer contributions in bug repositories. It models developer prioritization with a social-network technique, analyzes its characteristics, and applies it to three predictive tasks. Results from Eclipse and Mozilla indicate that prioritization assists software tasks, especially bug triage.
Problem
No prior work had considered developer priorities in bug repositories, although developer contributions affect bug-repository tasks.
Method
The paper assigns developer priorities from a socio-technical, comment-based network and applies them to bug triage, severity identification, and reopened bug prediction.
Results
Developer prioritization improves bug-repository prediction tasks, with bug-triage accuracy improving by up to 13%.
Takeaways & Limitations
Developer prioritization provides developer-priority information that can assist software tasks, especially bug triage.
Abstract
from arXiv · showhide
Developers build all the software artifacts in development. Existing work has studied the social behavior in software repositories. In one of the most important software repositories, a bug repository, developers create and update bug reports to support software development and maintenance. However, no prior work has considered the priorities of developers in bug repositories. In this paper, we address the problem of the developer prioritization, which aims to rank the contributions of developers. We mainly explore two aspects, namely modeling the developer prioritization in a bug repository and assisting predictive tasks with our model. First, we model how to assign the priorities of developers based on a social network technique. Three problems are investigated, including the developer rankings in products, the evolution over time, and the tolerance of noisy comments. Second, we consider leveraging the developer prioritization to improve three predicted tasks in bug repositories, i.e., bug triage, severity identification, and reopened bug prediction. We empirically investigate the performance of our model and its applications in bug repositories of Eclipse and Mozilla. The results indicate that the developer prioritization can provide the knowledge of developer priorities to assist software tasks, especially the task of bug triage.
I. INTRODUCTION
The paper addresses the previously unstudied problem of ranking developer contributions in bug repositories and evaluates developer prioritization as support for bug-related prediction tasks. It analyzes prioritization across products, over time, and under noisy comments, using Eclipse and Mozilla repositories.
- Bug repositories support software development by storing and updating information about project problems and suggestions.
- Prior work studied social networks in several software repositories, but not developer priorities and their applications in bug repositories.
- The paper ranks participating developers with a socio-technical approach rather than dividing the developer network into communities.
- The analysis examines developer priorities in products, their evolution over time, and tolerance of noisy comments.
- The approach is applied to bug triage, severity identification, and reopened bug prediction.
- Combining developer prioritization improves bug-triage accuracy by up to 13%.
III. IDENTIFYING THE DEVELOPER PRIORITIZATION
The developer prioritization framework builds a weighted communication network from bug comments, computes developer scores, and ranks developers by those scores. It also supports product- or component-specific prioritization.
- The model can be adapted to a specified product or component by using comments associated with that topic.
- Developer prioritization assigns each developer a score and ranks all participating developers by descending score.
- The network uses comment counts as link weights, removes self-links, and adds a virtual developer with bidirectional links to connect the graph.
- The framework uses out-degrees to reflect the influences of both bug reporters and commenters.
- Developer scores are computed over time, followed by final-score calculation and descending-score ranking.
- The resulting scores can distinguish developers with equal out-degrees and identify a highest-priority developer in the illustrative example.
B. Research Questions
The paper organizes its research questions into characteristic analysis and applications. It studies product-level differences, temporal evolution, noise tolerance, and whether prioritization assists bug-repository tasks.
- Characteristic analysis: The first three research questions concern differences between product and project prioritization, temporal evolution, and tolerance to noisy comments.
- The study analyzes characteristics in Section IV and applications in Section V.
- Applications: The fourth research question asks whether developer prioritization can assist existing tasks in bug repositories.
IV. ANALYZING THE DEVELOPER PRIORITIZATION
The study examines developer prioritization using bug repositories from the Eclipse and Mozilla open-source projects.
- Experiments use bug repositories from two open-source projects: Eclipse and Mozilla.
A. Data Collection
The study analyzes developer prioritization in Eclipse and Mozilla using bug reports and comments, examining whole-project and product-level rankings. It compares prioritization with out-degree and fixed-bug counts and uses figures to inspect product differences.
- Data sources: The datasets contain bug reports collected before 2011 from the large open-source projects Eclipse and Mozilla.The study uses bugs 1–333371 in Eclipse and 1–662320 in Mozilla, with some reports removed or inaccessible.
- Data extraction: For each bug report, the study extracts identifiers, participants, textual fields, timestamps, comments, and fixer information.Heuristics are used to recognize correct fixers because fixer labels are not always reliable.
- Analysis design: Developer prioritization is examined across whole projects and products, including rankings for the top five whole-project developers in ten active products.The selected products are those contributed to by the most developers.
- Product-level findings: Prioritization differs between whole projects and products: some top whole-project developers contribute little or nothing to particular products.Among products in which they participate, top developers contribute substantially to most products.
C. Answer to RQ2, Evolution of Developer Prioritization
The paper studies how developer prioritization evolves across half-year periods and examines its robustness to noisy comments. It reports changing priorities over time and insensitivity to the tested noise-removal procedures.
- Evolution over time: Developer prioritization is generated for 18 half-year periods spanning nine continuous years from January 2002 through December 2010.The analysis uses bug reports from Eclipse and Mozilla and tracks leading developers over time.
- Evolution over time: The study defines leaders as developers ranked among the top 1000 in each period and compares new and old leaders.Figure 4 reports the ratios of new leaders, old leaders, and old developers becoming new leaders.
- Limitation: The authors identify project complexity and the half-year unit length as possible explanations for changing prioritization and call for further study of an appropriate unit time.The paper does not establish which explanation is correct.
- Evolution over time: Developer prioritization evolves over time, and new developers can join the projects and become highly prioritized.The ratios in Mozilla are more stable than those in Eclipse, while changes after 2005 are stable in both projects.
- Noise tolerance: The prioritization remains insensitive to the tested noisy-comment removals, with inconspicuous rank changes across datasets.The removed comments concern comments from inexperienced developers or comments written in very few words, using thresholds of 5, 10, 15 and 3, 6, 9.
V. LEVERAGING THE DEVELOPER PRIORITIZATION
The paper evaluates developer prioritization as an enhancement for bug triage, severity identification, and reopened bug prediction. For bug triage, priorities are combined with classifier outputs within incremental learning to reorder candidate developers.
- V. LEVERAGING THE DEVELOPER PRIORITIZATION: Developer prioritization is combined with task inputs or outputs: it adds features to predictive models or updates task results.For bug triage, predicted developers are reordered using priorities to discriminate among developers with similar classifier probabilities.
- A. Bug Triage: Incremental bug-triage evaluation sorts reports chronologically into 11 folds and performs 10 rounds, generating priorities from training data before combining them with classifier predictions.In round r, the first r folds train the classifier and generate priorities, while fold r+1 is tested.
- A. Bug Triage: Top-1 accuracy is higher with developer prioritization than with SVM alone in both Eclipse and Mozilla.The product-and-component combination has the highest accuracy among the seven approaches in each Figure 8 sub-figure.
- A. Bug Triage: Latest-fold developer priorities usually outperform priorities from accumulative folds, indicating that recent developer changes are more helpful for triage.The latest fold uses only the most recent training fold, whereas accumulative priorities use all preceding folds.
- A. Bug Triage: SVM plus developer prioritization improves average accuracy by around 10%, whereas NB plus developer prioritization improves it by about 2%.For SVM, some rounds exceed 15% improvement, including top-3 results in Eclipse Round 10 and Mozilla Round 9.
B. Severity Identification
The severity-identification experiment adds developer-priority features to bug-title vectors and evaluates precision, recall, and F-measure. Mixed vectors improve all three metrics for Mozilla, while Eclipse differences are not significant.
- B. Severity Identification: Reporter priority scores based on products and components are added as two numeric features to the original bug-title vectors.The resulting mixed vectors contain two additional developer-prioritization dimensions.
- B. Severity Identification: Severity identification is evaluated on severe and non-severe bugs from Eclipse and Mozilla using precision, recall, and F-measure.The datasets contain selected bugs from three components in each project.
- B. Severity Identification: Mixed vectors obtain better precision, recall, and F-measure for Mozilla, while Eclipse results differ insignificantly.The paper suggests that Eclipse’s larger number of original numeric features may obscure the effects of the two added dimensions.
C. Reopened Bug Prediction
The reopened-bug experiment adds reporter and fixer priority scores to an existing AdaBoost factor model. Priority-based factors slightly improve performance, but the improvement is not significant.
- C. Reopened Bug Prediction: The experiment evaluates whether developer prioritization supplies factors analogous to existing people-related predictors for reopened bugs.It uses 10-fold cross-validation on 1,530 Eclipse bugs, including 246 reopened and 1,284 non-reopened bugs.
- C. Reopened Bug Prediction: Reporter and fixer priority scores are added to the people dimension alongside developer names and experience factors.The six factors form name, experience, and priority groups evaluated with AdaBoost.
- C. Reopened Bug Prediction: The combination of name, experience, and priority factors provides the highest performance, while replacing name and experience with priority improves performance.The paper characterizes priority-based factors as similar features that can slightly improve existing results.
- C. Reopened Bug Prediction: The priority-score improvement is not significant, potentially because the training set is small or existing results are difficult to improve.These are the paper’s two proposed reasons for the limited improvement.
- C. Reopened Bug Prediction: Across the three examined tasks, developer prioritization helps improve results, especially for bug triage.The paper’s RQ4 conclusion covers bug triage, severity identification, and reopened bug prediction.
VI. THREATS TO VALIDITY
The paper identifies limitations in validating and constructing developer prioritization, including uncertain correspondence to real collaboration, imperfect product/component labels, sparse version metadata, and heuristic noise detection.
- The rankings are difficult to validate because no real developer ranking exists; a case study comparing them with development collaboration is needed.
- Product- and component-based prioritization depends on self-reported bug-report terms that may mismatch the actual products or components.The paper calls for a technique to identify correct product and component labels.
- Version-based evolution is not investigated because version information is difficult to collect: only 7.45% of bugs 200001-300000 include it.Those 7,450 version-identified bugs belong to 38 products.
- The noise-detection heuristic may discard meaningful comments, including concise comments such as “fixed” or “verified.”The paper states that manual labeling would better identify noisy comments.
C. Assisting Software Tasks
The paper positions developer prioritization as a social-network-based way to enrich bug-repository tasks, extending prior work on communities, failures, and developer–bug interactions.
- Earlier software-engineering studies used social-network measures or communities to analyze collaboration and predict software failures.Examples span email archives, build failures, change logs, binary repositories, and developer-module networks.
- Unlike prior community discovery, the approach analyzes developer rankings and applies them to three existing bug-repository tasks.
- Bug-repository applications include bug triage, severity identification, and reopened-bug prediction, alongside other quality-improvement tasks studied previously.
- The paper differs from work that treats developers and bugs separately by integrating developer prioritization with bug-repository analysis.Prior work transferred developer–bug interactions to requirements engineering, whereas this paper uses prioritization for software-task prediction.
- Developer prioritization ranks all participating bug-repository developers and is used to support predictive tasks.
C. Predicting Tasks in Bug Repositories
The paper evaluates whether developer prioritization improves bug triage, severity identification, and reopened-bug prediction using empirical studies of Eclipse and Mozilla.
- Bug triage automatically assigns new bugs to developers, while prior approaches use text categorization, recommendation lists, classifiers, or assignment history.
- Severity identification detects bug severity to guide resource allocation and bug-fixing planning.Prior work studied robotic, security, and critical severity categories.
- Reopened-bug prediction detects whether a bug was fixed correctly using factors from bug and source-code repositories.
- The paper empirically evaluates whether developer prioritization improves the three tasks and studies results on over 900000 Eclipse and Mozilla bug reports.
- The paper’s future work proposes task-based prioritization, such as weighting bug fixers more heavily for bug triage.