Source-linked AI summary
Taxonomy of Real Faults in Deep Learning Systems
Nargiz Humbatova, Gunel Jahangirova, Gabriele Bavota, Vincenzo Riccio, Andrea Stocco, Paolo Tonella
TL;DR
Faults in deep learning systems are especially important in safety-critical applications because they can arise beyond conventional source-code bugs, including during training and model construction. This paper builds and validates a faceted taxonomy from real-world artefacts and developer input, producing 92 unique fault types and identifying substantial gaps in existing mutation operators.
Problem
Deep learning faults matter in safety-critical systems, but they are more complex than traditional software faults because expected behavior can deviate through training and model structure beyond source-code bugs.
Method
The authors construct a bottom-up faceted taxonomy through manual analysis of 1,059 GitHub and Stack Overflow artefacts and interviews with 20 developers, then validate it with a survey of 21 different developers.
Results
The taxonomy contains 5 main categories, 375 instances, and 92 unique fault types; the survey confirmed the relevance and completeness of the identified categories, while existing mutation operators correspond to only 6 of 92 leaf faults.
Takeaways & Limitations
The taxonomy can guide DL testing and support the design of mutation operators that better represent real faults.
Takeaways & Limitations
Generalisation is constrained by the three considered frameworks, the artefact dataset, and the conducted interviews.
Abstract
from arXiv · showhide
The growing application of deep neural networks in safety-critical domains makes the analysis of faults that occur in such systems of enormous importance. In this paper we introduce a large taxonomy of faults in deep learning (DL) systems. We have manually analysed 1059 artefacts gathered from GitHub commits and issues of projects that use the most popular DL frameworks (TensorFlow, Keras and PyTorch) and from related Stack Overflow posts. Structured interviews with 20 researchers and practitioners describing the problems they have encountered in their experience have enriched our taxonomy with a variety of additional faults that did not emerge from the other two sources. Our final taxonomy was validated with a survey involving an additional set of 21 developers, confirming that almost all fault categories (13/15) were experienced by at least 50% of the survey participants.
1 INTRODUCTION
The paper frames faults in deep learning systems as a crucial topic because safety-critical applications depend increasingly on networks whose behavior can fail beyond source-code bugs. It builds and validates a faceted taxonomy of real DL faults using artefacts and developer experience.
- Motivation: DL systems can deviate from expected behavior because of training faults even when the network-building code is bug free.Examples include misconfigured learning parameters and unbalanced or non-representative training data.
- Goal: The paper builds a taxonomy of real DL faults to help developers avoid pitfalls, guide testers’ scenarios, and support realistic fault seeding.The authors identify resemblance to real faults as important for artificially injected faults.
- Approach: Faceted classification was used because DL is an evolving domain whose fault traits can be extended and combined.Categories and subcategories were created bottom up from multiple information sources.
- Approach: The initial taxonomy combined manual analysis of 477 Stack Overflow discussions, 271 GitHub issues and pull requests, and 311 commits with interviews of 20 researchers and practitioners.Manual analysis sought to identify the root causes behind reported problems, while interviews enriched the preliminary taxonomy.
- Validation: 13/15 fault categories were experienced by at least 50% of 21 survey participants, and no category remained non-validated.The least frequent category was confirmed by 24% of participants.
- Contribution: Interviews added faults absent from the other analysed artefacts, including 2 inner nodes and 27 leaf nodes that would otherwise have been missed.Several other nodes were highly represented in interviews but appeared rarely in the other sources.
2 RELATED WORK
Prior studies classified bugs in machine-learning frameworks and DL applications, whereas this study focuses on faults experienced while building DL systems using specific frameworks.
- Earlier ML studies: Thung et al. classified 500 bug reports and fixes from three open-source ML projects using existing categories and descriptive statistics.Their questions covered bug frequency, severity, and resolution effort.
- Earlier ML studies: Sun et al. manually analysed 329 closed bug reports from Scikit-learn, Caffe, and Paddle, organising them into seven categories.They assessed fault categories, fix patterns, and effort over time.
- Positioning: These studies analysed bugs in the frameworks themselves, while this paper examines faults experienced when building DL systems that use a framework.The distinction concerns the level at which faults are studied.
- DL application studies: Zhang et al. classified 175 TensorFlow-related bugs from Stack Overflow and GitHub into seven root-cause kinds and four symptom types.Their analysis also considered detection and localisation strategies, causes, and behavioral consequences.
- Positioning: This study covers TensorFlow, Keras, and PyTorch rather than only TensorFlow.The authors describe these frameworks as the most popular and representative of the field’s current situation.
- DL application studies: Islam et al. characterised bugs across five DL frameworks, examining their frequency, causes, impacts, patterns, and evolution.The listed frameworks were Theano, Caffe, Keras, TensorFlow, and PyTorch.
3.1 Manual Analysis of Software Artefacts
The manual artefact analysis mined GitHub and Stack Overflow for framework-usage problems, filtered and coded the data, and iteratively built the taxonomy until inner-category saturation.
- Data sources: The initial taxonomy drew on commits, issues, pull requests, and Stack Overflow discussions involving TensorFlow, Keras, and PyTorch.These were the three most popular DL frameworks considered in the study.
- Mining GitHub: The GitHub search generated 6,000 requests across size ranges and selected repositories using project activity and popularity criteria.The search used 2,000 requests per framework and excluded personal, inactive, trivial-history, and unpopular repositories.
- Mining GitHub: 151 TensorFlow, 237 Keras, and 326 PyTorch projects were initially selected before manual removal of tutorials, code collections, and false positives.The process retained projects representing real software systems used by developers in practice.
- Mining GitHub: 1,981 commits and 1,392 issues or pull requests remained after filtering artefacts for DL-relevant fault terms.The initial inspection found 97 false positives in a random sample of 100 artefacts.
- Mining Stack Overflow: Stack Overflow retrieval produced 9,935 TensorFlow, 3,116 Keras, and 1,007 PyTorch posts, from which the 1,000 most-viewed posts per framework were selected.Posts without accepted answers and general how-to, installation, or build questions were excluded.
- Open coding: All authors manually open-coded the collected data, assigning descriptive fault labels and using a web application to support classification and conflict resolution.Artefacts were marked as false positives or generic when they did not concern DL-system faults.
- Taxonomy construction: The taxonomy became hierarchical after the first three rounds, and labelling stopped when a new round produced no new inner categories.The process created 21 leaf categories from 35 DL-relevant artefacts in the first three rounds.
- Labelling process: 1,059 documents were labelled overall, with 111 requiring conflict resolution in open discussion meetings.The labelling process involved six rounds, each followed by an author meeting.
3.2 Developer Interviews
Developer interviews complemented artefact mining by targeting faults beyond mostly code-level reports, using balanced researcher–practitioner recruitment and semi-structured, open-ended questioning. Interviews were transcribed, independently coded, reconciled, and used to enrich the taxonomy.
- Rationale: Interviews addressed faults that Stack Overflow and GitHub sources might miss because those platforms mostly report code-level problems.DL decision logic also depends on the training phase and model structure.
- Participants: The study interviewed 20 developers, equally divided between researchers and practitioners, with varied expertise and backgrounds.Researchers included PhD students, post-docs, and professors; practitioners worked in industry or freelancing.
- Participants: Interviewees covered diverse domains and used TensorFlow 12 times, Keras 11 times, and PyTorch 8 times.Python was the main programming language, with additional mentions of Matlab, R, Java, Scala, C++, and C#.
- Interview design: Semi-structured interviews combined open-ended questions for unexpected information with specific questions to keep discussions within scope.The opening question asked broadly about problems and bugs encountered while developing ML/DL systems.
- Interview design: Two authors conducted each interview with separate leading and follow-up roles to support additional questioning.The second interviewer asked at least two additional questions in every interview.
- Procedure: Interviews lasted 26–52 minutes, averaging 37 minutes, and automated transcripts were checked and manually corrected.All but one interview were conducted remotely using Skype.
- Analysis: Each interview was open-coded by two evaluators under a moderator who resolved inconsistent labels.Across 20 interviews, evaluators tagged 297 text fragments and produced only six direct tagging conflicts.
- Analysis: After removing 19 non-DL issue tags, the authors retained 226 tags for taxonomy construction.The final meeting reviewed tags judged to concern general issues rather than DL-system problems.
3.3 Taxonomy Construction and Validation
The taxonomy was constructed bottom-up from manually analysed artefacts and iteratively refined, then validated through a separate survey of practitioners and researchers.
- Taxonomy Construction: The authors grouped similar tags into categories, created parent categories with “is a” relationships, and collaboratively refined each taxonomy version.The final review covered all categories, subcategories, and leaves.
- Validation: The validation used a new practitioner/researcher set distinct from the interview participants.This was intended to assess whether the taxonomy was comprehensive and representative of real DL faults.
- Validation: Candidates were recruited through personal contacts, Stack Overflow, and Upwork using the interview process’s selection strategy and criteria.The personal-contact group included 23 contacted individuals, 13 of whom completed the survey.
- Validation: The survey partitioned the taxonomy into inner categories or descendants to keep its presentation readable.Each partition included a category name, textual description, and examples of leaf tags.
- Validation: Participants reported whether they had encountered each fault and, when applicable, rated its severity and the effort needed to identify and fix it.The survey therefore assessed occurrence as well as developers’ perceived severity and effort.
- Validation: The survey also solicited additional DL problems not represented in the taxonomy to identify possible omissions.This provided a further check of coverage against developers’ experience.
4 RESULTS
The results present a five-category taxonomy spanning training, models, layers, tensors and inputs, and related fault sources, with complementary evidence from artefacts and interviews. Validation found broad developer experience with the categories, while detailed examples show faults across training data, model structure, layers, tensor operations, and inputs.
- The Final Taxonomy: The taxonomy contains five top-level categories, with three divided into inner subcategories; category labels report artefact assignments plus interview tag occurrences.The complete taxonomy is shown in Figure 1.
- The Final Taxonomy: Model faults concern overall structure and properties, while layer faults concern specific layers, their properties, or activation functions.Examples include unsuitable model types, unstable initialisation, excessive neuron counts, and unsuitable layer or activation choices.
- The Final Taxonomy: Tensor and input faults involve incompatible shapes, types, or formats, including incorrect tensor dimensions, transposition, datatypes, and channel ordering.Some wrong-input faults can run silently despite using an incorrect data format.
- The Final Taxonomy: Training is the largest category and covers data quality and preprocessing, hyperparameter tuning, loss or optimisation functions, and testing or validation.Its scope spans multiple stages of the training process.
- The Final Taxonomy: Incorrect learning rates, batch sizes, or epoch counts can affect training time and overall model performance without necessarily causing crashes or errors.The taxonomy also includes loss-function selection and implementation faults that may misrepresent optimisation goals or produce faulty behaviour.
- The Final Taxonomy: Training-data preprocessing faults are classified as missing or wrong preprocessing, including missing normalisation or scaling and incorrect pixel encoding.The appropriate preprocessing depends heavily on the application area, contributing to many leaf tags.
- Evidence Sources: The most frequent artefact-derived tags were wrong tensor shape (21+5), wrong input shape for a layer (16+2), and missing preprocessing (11+22).The interview top five instead included suboptimal network structure (1+15), wrong preprocessing (2+15), and not enough training data (0+14).
- Evidence Sources: Artefacts and interviews were complementary: their top-five lists intersected on only missing preprocessing, and omitting either source would produce an incomplete taxonomy.Interview tags contributed nothing to the API category, whereas artefacts captured API-related problems.
5 DISCUSSION
The discussion compares the taxonomy with prior fault classifications and mutation operators, showing that it captures many previously unrepresented real faults. It also reports interview-based problems and explains how the taxonomy addresses limitations of Stack Overflow evidence.
- The taxonomy contains 24 inner nodes and 92 leaf nodes, with 21 inner nodes representing new fault categories relative to Zhang et al.’s taxonomy.
- 79 leaf categories were discovered uniquely in this study after conservatively accounting for partially mapped categories from Zhang et al.
- Pre-training mutation operators correspond to taxonomy categories except for “Data Shuffle,” whereas post-training parameter mutations have no corresponding real fault.
- Only 6 of the taxonomy’s 92 unique real faults have corresponding mutation operators, so existing operators cover only a small portion of observed fault variety.
- The study excluded examples, toy programs, and tutorials from Stack Overflow and GitHub analyses, then complemented those sources with developer interviews.
- Interviewees frequently reported version incompatibilities, poor documentation, limited framework support, and insufficient tooling for common development tasks.
6 THREATS TO VALIDITY
The authors identify threats to internal and external validity and describe safeguards including multiple evaluators, generic interview questions, survey validation, broad artefact sampling, and diverse interview participants.
- Potential internal threats include biased labelling, interview-question influence from the initial taxonomy, and author-dependent choices in structuring categories.
- Each artefact and interview was labelled by at least two evaluators, interview questions were kept generic, and the taxonomy structure was validated through a survey.
- External validity is primarily limited by generalisation beyond the three frameworks, analysed artefacts, and interviewed developers.
- The authors sought broader coverage by sampling until inner-category saturation and recruiting developers with varied expertise, backgrounds, and application domains.
7 CONCLUSION
The paper constructs and validates a taxonomy of real deep-learning faults from artefact analysis and developer interviews. The taxonomy comprises five main categories and 92 unique fault types, with future use proposed for testing guidance and mutation-operator design.
- The taxonomy is based on manual analysis of 1,059 GitHub and Stack Overflow artefacts plus interviews with 20 developers.
- The taxonomy contains 5 main categories and 92 unique fault types represented by 375 instances.
- A survey with a different set of 21 developers confirmed the relevance and completeness of the identified categories.
- The authors plan to use the taxonomy to guide deep-learning testing and define novel mutation operators.