PaperlayerView exact revision on arXiv

Source-linked AI summary

Machine Learning Testing: Survey, Landscapes and Horizons

Jie M. Zhang, Mark Harman, Lei Ma, Yang Liu

arXiv:1906.10742v2cs.LGcs.AIcs.SEstat.ML

TL;DR

Machine learning systems require testing despite difficult trustworthiness questions, including unknown correct answers and evolving, data-driven behavior. The paper surveys 144 ML testing papers, organizes them across testing properties, components, and workflows, and analyzes datasets, trends, distributions, challenges, and future directions. Its analysis finds substantial concentration on supervised-learning testing and correctness or robustness, while also documenting concrete testing findings and scope boundaries.

  • Problem

    ML testing must address trustworthiness concerns in statistical, data-driven systems, including the Oracle Problem of judging bugs when correct answers are unknown.

  • Method

    The paper conducts a comprehensive survey using structured searches across Google Scholar, DBLP, and arXiv, organizing the literature by properties, components, workflows, and related dimensions.

  • Results

    The survey covers 144 papers and finds that around 120 address supervised learning, three address unsupervised learning, one addresses reinforcement learning, and 93 focus on correctness and robustness.

  • Takeaways & Limitations

    The survey provides an overview of ML testing research, datasets, tools, trends, open problems, challenges, and promising research directions.

  • Takeaways & Limitations

    The paper does not fully cover the separate literature on adversarial-example generation, focusing instead on selected promising intersections with software testing and machine learning.

Abstract

from arXiv · show

This paper provides a comprehensive survey of Machine Learning Testing (ML testing) research. It covers 144 papers on testing properties (e.g., correctness, robustness, and fairness), testing components (e.g., the data, learning program, and framework), testing workflow (e.g., test generation and test evaluation), and application scenarios (e.g., autonomous driving, machine translation). The paper also analyses trends concerning datasets, research trends, and research focus, concluding with research challenges and promising research directions in ML testing.

1 INTRODUCTION

ML testing addresses difficult trustworthiness questions for statistical, data-driven systems whose behavior can evolve and whose correct outputs may be unknown. This survey organizes 144 papers across testing concepts, properties, components, workflows, distributions, trends, datasets, challenges, and future directions.

  • Motivation: Machine learning systems raise testing challenges because their statistical, data-driven behavior can evolve with new data and exhibit emergent whole-system properties.These characteristics complicate unit isolation, fault localization, and traditional testing assumptions.
  • Motivation: The Oracle Problem is central: testers must judge whether a bug exists even when the correct answer is unknown.Much ML testing research seeks techniques that address this problem, often by adapting traditional software testing approaches.
  • Research emergence: 85% of ML testing papers appeared since 2016, indicating a sharp rise in research activity between 2007 and June 2019.The publication count is reported cumulatively over that period.
  • Survey scope: The survey organizes prior work by testing properties, ML components, and testing workflow while covering different ML approaches through correlated sections.The organization is intended to keep each aspect complete and support readers from software engineering and machine learning.
  • Contributions: The paper surveys 144 ML testing papers and analyzes research distribution, datasets, and trends across the literature.Its contributions also include defining ML testing and identifying open problems and promising research directions.

2 PRELIMINARIES OF MACHINE LEARNING

The preliminaries define machine learning as data-based decision or prediction and distinguish its learning types, tasks, algorithms, and core system elements. Supervised, unsupervised, and reinforcement learning map to different task families and use different training signals.

  • Machine learning foundations: Machine learning makes decisions or predictions from data and may use classic algorithms or deep neural networks.The paper includes both classic machine learning and deep learning, including CNN and RNN structures.
  • System elements: A machine learning system includes datasets, learning programs, frameworks, and the trained model that encodes decision or prediction logic.Training, validation, and test data serve distinct roles in model development and behavior validation.
  • Learning categories: Supervised learning uses labelled targets, unsupervised learning learns patterns without labels, and reinforcement learning interacts with an environment to pursue rewards.Reinforcement learning is typically modelled as a Markov decision process.
  • Learning tasks: Classification assigns categories and regression predicts values; both belong to supervised learning.Examples include image classification, handwriting recognition, and temperature, age, or income prediction.
  • Learning tasks: Clustering partitions instances into homogeneous regions, while dimension reduction reduces training complexity; both belong to unsupervised learning.The paper gives pattern recognition, segmentation, dataset representation, and preprocessing as examples.
  • Learning tasks: Control is a reinforcement-learning task that selects actions to maximize rewards, such as in game playing.The task is associated with reinforcement learning rather than supervised or unsupervised learning.

3 MACHINE LEARNING TESTING

ML testing is defined as activity designed to reveal bugs across ML properties, components, and testing activities. Its workflow combines offline and online testing, while accounting for data, learning-program, and framework interactions and the lack of consensus on fairness definitions.

  • Definition: ML testing refers to any activity designed to reveal machine learning bugs.The survey frames bugs as discrepancies between existing and required ML-system conditions.
  • Definition: ML testing examines required properties, potentially faulty components, and activities such as input generation, oracle identification, adequacy evaluation, and bug triage.Properties include correctness, robustness, and privacy; components include data, learning programs, and frameworks.
  • Testing workflow: Offline testing evaluates models before deployment, while online testing analyzes deployed behavior and user responses to detect problems missed by historical test data.Online testing includes runtime monitoring, A/B testing, and multi-armed bandits.
  • Testing workflow: The offline workflow proceeds from requirement analysis to test-input and oracle preparation, execution, bug reporting, repair, and regression testing before deployment.Regression testing checks that repairs solve reported problems without introducing new ones.
  • Testing components: ML testing must examine data, learning programs, and frameworks because closely bonded components can propagate errors and data defects can amplify over time.Model development involves collecting and preprocessing data, implementing learning architectures, and using framework algorithms and libraries.
  • Testing properties: Fairness remains unsettled because the literature proposes many definitions without firm consensus, making formulation an initial step in addressing fairness problems.The survey therefore discusses how different fairness types are formulated and measured.

4 PAPER COLLECTION AND REVIEW SCHEMA

The survey defines a broad collection scope for ML testing and reviews 144 papers using multiple search, snowballing, and author-feedback steps. It organises the literature by testing workflow, properties, components, application scenarios, and statistical trends.

  • Survey Scope: The survey covers software testing applied to machine learning, including ML properties, components, procedures, datasets, benchmarks, and measurement criteria.The scope excludes traditional ML performance validation and uses of ML for traditional software testing.
  • Paper Collection Methodology: Keyword searches across Google Scholar, DBLP, and arXiv used terms covering ML properties, components, and testing-related words such as “test” and “bug.”The search was expanded through snowballing until closure, with author feedback used to identify additional papers and check descriptions.
  • Collection Results: 144 papers were collected: 109 from keyword search and snowballing, plus 35 added through author feedback.The collection covered six research areas through June 2019.
  • Collection Results: 38.2% of papers appeared in software engineering venues, 19.4% in artificial intelligence venues, and 22.9% remained unpublished through peer-reviewed venues on arXiv.Systems and networking venues accounted for 6.9% of the papers.
  • Review Schema: The review presents collected work through testing workflow, testing properties, ML components, and application scenarios, while separately analysing research categories, structures, properties, datasets, and tools.The four presentation angles can overlap because one paper may fit multiple aspects.
  • Review Schema: These organisational and statistical analyses are used to identify research focus, trends, challenges, opportunities, and directions in ML testing.The survey reports these results in its concluding section.

5 ML TESTING WORKFLOW

ML testing research is organized around test-input generation and evaluation across offline workflows and application domains. Techniques include domain-specific synthesis, fuzzing, symbolic execution, and fault modeling, with reported gains in coverage and defect discovery.

  • Test input generation: ML testing research organizes test-input generation by adopted techniques and distinguishes adversarial inputs from natural, domain-distributed inputs.Adversarial inputs target robustness or security flaws, whereas natural inputs are synthesized for practical application scenarios.
  • Domain-specific test input synthesis: DeepXplore combines neuron coverage, cross-model behavioral differences, and real-world similarity to generate deep-learning tests.Its generated inputs cover 34.4% more neurons than randomly picked inputs and 33.2% more than adversarial inputs.
  • Domain-specific test input synthesis: Domain-specific transformations exposed more than 1,000 erroneous behaviors in autonomous-driving CNNs and RNNs with low false-positive rates.DeepTest used nine realistic image transformations, including brightness, weather, geometric, blur, fog, and rain effects.
  • Domain-specific test input synthesis: Other application-specific generators mutate translation words, transform audio or biological-cell images, and create adversarial driving billboards.The translation approach achieved 99% precision in manually inspected pairs expected to produce consistent translations.
  • General-purpose test input generation: Fuzzing and search-based generation support coverage exploration, numerical-error discovery, adversarial-example generation, and undesirable-behavior detection.TensorFuzz uses nearest-neighbor hill climbing, while DLFuzz uses neuron coverage and minimally changes original inputs without requiring a reference model.
  • Fault-based generation: Bayesian fault-injection modeling generated effective safety tests and revealed situations where faults caused safety violations in two production-grade autonomous-vehicle systems.The evaluation used systems from NVIDIA and Baidu.

5.2 Test Oracle

Test oracles are central to ML testing because probabilistic systems make correctness difficult to judge directly. The survey covers metamorphic relations, cross-referencing, and model-evaluation approaches, with evidence of their use across models and application domains.

  • Oracle problem: The oracle problem is a key ML-testing challenge because test oracles determine whether observed behavior constitutes a bug.The survey identifies metamorphic relations, cross-referencing, and model-evaluation metrics as studied oracle types.
  • Metamorphic relations: Metamorphic testing checks whether controlled input transformations produce unchanged or otherwise expected predictive-output changes.These relations can operate at different granularities, from dataset-level transformations to finer data changes.
  • Metamorphic relations: Functional-level metamorphic properties were 170% more effective than application-level properties across nine machine-learning applications.Other studies found model- and classifier-specific relations, including relations that detected 71% of injected faults.
  • Domain-specific oracles: Autonomous-driving metamorphic oracles achieved 0.94 precision when detecting errors in unlabelled data through consistency of similar images.Related checks assess whether steering angles remain stable under weather-related image transformations.
  • Cross-referencing: Cross-referencing uses multiple implementations, versions, or related programs to identify inconsistent outputs as evidence of defects.Differential testing and N-version programming are the principal forms described in this section.
  • Cross-referencing: CRADLE detected 104 unique inconsistencies and 12 bugs across three deep-learning libraries, 11 datasets, and 30 pretrained models.Differential testing also found 16 faults in seven Naive Bayes implementations and 13 faults in 19 k-nearest-neighbor implementations.

5.3 Test Adequacy

Test adequacy evaluates whether ML test suites can reveal faults, but traditional code coverage is weakly aligned with learned decision logic. Research therefore explores structural, mutation-based, surprise-based, and rule-based criteria.

  • Adequacy evaluation: Test adequacy measures the fault-revealing ability of existing tests and can also guide test generation.Traditional criteria include code coverage and mutation testing, both adopted in ML testing.
  • Coverage criteria: Traditional code coverage is seldom demanding for ML testing because a single random input can achieve 100% code coverage while learned behavior remains insufficiently exercised.ML testing instead proposes coverage criteria designed for model behavior.
  • Structural coverage: Neuron coverage measures the ratio of unique activated neurons to total neurons, with activation determined by a user-specified threshold.Later criteria refine this idea using training-data activation profiles and boundary-oriented behavior.
  • Coverage limitations: Initial experiments found no strong correlation between structural coverage and misclassified natural inputs, and the relation to decision logic remains unclear.The concern follows from fundamental differences between neural networks and human-written programs.
  • Mutation testing: Mutation-based criteria perturb DNN decision boundaries, and mutation scores count test instances whose results change relative to the total.Such criteria are described as more directly relevant to decision boundaries than structural coverage.
  • Alternative adequacy criteria: Surprise adequacy measures discretized input-surprise ranges relative to training data, favoring tests that are sufficiently but not overly surprising.The survey also reports rule-based checklists covering models, infrastructure, data, and system behavior over time.

5.4 Test Prioritisation and Reduction

ML test prioritization and reduction address large input spaces, labeling costs, and computational expense by selecting tests likely to expose unacceptable or adversarial behavior. Reported approaches prioritize uncertainty, noise sensitivity, or model-informed diversity.

  • Test prioritisation: Large ML input spaces and the need to label every test instance make test generation costly.Cross entropy, surprisal, and Bayesian uncertainty were evaluated as indicators of inputs exposing unacceptable behavior and supporting retraining.
  • Test prioritisation: Noise-sensitive ranking prioritizes test instances more likely to yield effective adversarial examples, reducing the search burden.The approach ranks inputs according to their sensitivity to noise.
  • Test reduction: Neuron-guided sampling reduced the test set to half its original size while achieving a similar level of performance to random sampling.The evaluation used pretrained models with MNIST, Udacity Challenge, and ImageNet datasets.
  • Test selection: Model-confidence metrics prioritize uncertain inputs because they are considered more informative for testing and retraining.The evaluation reported 80% more gain than random selection.

5.5 Bug Report Analysis

Bug-report analyses examine the symptoms, causes, and prevalence of ML-system failures across general-purpose frameworks and autonomous driving systems.

  • 22.6% of studied ML bugs were due to incorrect implementation.
  • 64% of autonomous-driving disengagements were attributed primarily to ML-system and decision-control issues.The analysis covered reports from 12 manufacturers and 1,116,605 cumulative California driving miles.

5.6 Debug and Repair

Debugging and repair approaches use generated data, program transformations, interactive inspection, human guidance, and synthesis to improve or fix ML systems.

  • Generated test inputs can both expose ML bugs and improve correctness through retraining.DeepXplore improved classification accuracy by up to 3%, while DeepTest improved model accuracy by 46%.
  • Fault-focused resampling targets training data influencing neurons identified as responsible for misclassification.
  • tfdbg supports debugging by exposing runtime-graph structure, inspecting or modifying nodes, and stepping through training iterations.
  • Human-in-the-loop repair compares simulated component improvements to guide designers toward better system configurations.
  • Distribution-guided inductive synthesis repairs decision-making programs by constructing semantically similar programs with correct predictive outputs.The approach uses sampled instances and predicted outputs to drive SMT-encoded program synthesis.

5.7 General Testing Framework and Tools

General testing frameworks and tools support test-input generation, validation, visualization, and regulation of undesired ML behaviours across multiple algorithm types.

  • CNN testing frameworks combine image generation, sampling methods, and visualization tools.
  • A framework for regulating ML algorithms can learn from potentially biased data while guaranteeing, with high probability, no bias on unseen data.The framework applies to regression, classification, and reinforcement algorithms, with user-specified bias definitions.

6 ML PROPERTIES TO BE TESTED

The survey organizes ML testing around correctness, overfitting, robustness, security, efficiency, and fairness, while showing that testing outcomes depend on models, data, metrics, and deployment context.

  • ML properties describe conditions of concern after training, but poor property performance may originate in any ML component.
  • Functional properties include correctness and overfitting, while non-functional properties include robustness, security, efficiency, and fairness.
  • Correctness testing commonly isolates test data through validation methods such as cross-validation and bootstrap to assess performance on new cases.
  • Accuracy, precision, recall, and AUC have distinct limitations, so performance metrics must be chosen carefully.Accuracy does not distinguish false positives from false negatives, while precision and recall can mislead on imbalanced data.
  • A mirror program can serve as a correctness oracle by reproducing behaviours expected on test data.
  • 40 of 175 TensorFlow bugs concerned poor correctness.
  • Overfitting occurs when overly complex models fit training noise, especially when training data is insufficient.
  • Cross-validation may miss overfitting when test data is unrepresentative of potential unseen data, and acceptable overfitting levels may be unclear.

7 ML TESTING COMPONENTS

ML testing examines the data, learning program, and framework because bugs in these components can affect model behaviour. The surveyed work covers data validation and cleaning, learning-program defects, and framework correctness, robustness, oracle, and efficiency issues.

  • Data Testing: Data testing targets bugs because model performance depends on data and feedback loops can amplify small defects.Reported approaches include linting, constraint validation, skew measurement, iterative cleaning, and performance-based detection.
  • Data Testing: MODE improved test effectiveness from 75% to 93% by resampling data to assess whether faulty neurons influence classification errors.
  • Learning Program Testing: 23.9% of 159 TensorFlow-project bugs arose from learning-program problems, with TensorFlow API changes identified as the most common bug source.
  • Framework Testing: Framework studies found different runtime training behaviours despite similar prediction accuracies, while approximately 10% of framework bugs concerned low efficiency.
  • Framework Testing: Approximately 22.6% of bugs were attributed to incorrect algorithm implementations, and differential testing found faults across multiple implementations and libraries.
  • Framework Testing: Between 5% and 24% of examined oracles were approximated, and 5%-27% used outputs from other libraries or frameworks.

8 APPLICATION SCENARIOS

ML testing methods are applied to autonomous driving, machine translation, and natural language inference. These approaches generate tests, detect safety or consistency violations, and sometimes repair identified defects.

  • Autonomous Driving: 64% of 5,328 autonomous-vehicle disengagements were caused by machine-learning-system bugs, while 42 disengagements led to accidents.Image-classification behaviours accounted for 44% of all reported disengagements.
  • Autonomous Driving: Autonomous-driving testing uses fuzzing, metamorphic testing, fault injection, search-based generation, adversarial examples, and risk-based frameworks.These methods target perception, safety, reliability, and critical traffic scenarios across components and system levels.
  • Machine Translation: Machine-translation testing uses metamorphic relations to check whether input changes preserve translation structure and consistency.One evaluation found Google Translate stronger on long sentences, while Microsoft Translator was stronger on short, simple sentences.
  • Machine Translation: TransRepair fixed 28% of Google Translate bugs and 19% of Transformer bugs on average using mutation and metamorphic testing.The approach supports automatic test-input generation, oracle generation, and translation repair.
  • Natural Language Inference: NLI testing with rule-based sentence mutants found that seven state-of-the-art models missed simple semantic differences when word-level information stayed unchanged.

9 ANALYSIS OF LITERATURE REVIEW

The literature review finds rapid growth but uneven coverage across learning categories, testing properties, tasks, and datasets. The 144-paper collection is concentrated on supervised learning, deep-learning-specific methods, correctness, robustness, and small evaluation sets.

  • Research Trends: 56 of 144 papers (38.9%) target deep learning alone, while 88 address general machine learning.
  • Research Trends: Before 2017, research mostly focused on general machine learning; after 2018, both general and deep-learning-specific testing increased notably.
  • Learning Categories: Among 144 papers, around 120 studied supervised-learning testing, three studied unsupervised learning, and one studied reinforcement learning.
  • Learning Categories: Reinforcement learning had higher search popularity than supervised learning, yet the survey found no direct reinforcement-learning testing work.
  • Testing Properties: 38.7% of papers tested correctness, 26.8% focused on robustness and security, and 12.0% focused on fairness.
  • Datasets: Most evaluated papers used only one or two datasets, while one data-cleaning study used 600 datasets.The survey suggests that model training and testing costs may contribute to the limited number of datasets used.

10 CHALLENGES AND OPPORTUNITIES

ML testing remains at an early stage, with challenges in generating realistic inputs, evaluating test adequacy, constructing reliable oracles, reducing costs, and testing diverse deployments. The survey highlights broader opportunities across learning categories, tasks, properties, definitions, and benchmarks.

  • Challenges: Test-input generation remains difficult because ML models have large behaviour spaces and generated adversarial inputs may be unnatural.Driving-scene generators can produce images that humans may not recognise.
  • Challenges: The relationship between test-assessment metrics and fault-revealing ability remains unclear, limiting conclusions about test sufficiency.
  • Challenges: Metamorphic relations provide pseudo oracles but often require human ingenuity, leaving automatic construction of reliable ML test oracles unresolved.
  • Challenges: Pseudo oracles may produce false positives, motivating more accurate oracle construction and methods to reduce incorrectly reported issues.
  • Challenges: Testing ML systems across diverse devices and deployment processes remains challenging under target-device resource limitations.
  • Opportunities: Research opportunities include unsupervised and reinforcement learning, speech recognition, natural language processing, agent play, transfer learning, efficiency, model relevance, and interpretability.
  • Opportunities: Fairness and interpretability lack consensus in their definitions and understanding, motivating clearer formalisation and empirical studies across contexts.
  • Opportunities: The survey calls for more ML-testing benchmarks, noting that few benchmarks currently exist.

11 CONCLUSION

The survey provides a comprehensive overview of ML testing research, covering testing properties, components, workflows, datasets, tools, trends, opportunities, and challenges.

  • The survey defines and analyzes the current research status of ML testing properties, components, and workflows.
  • It summarizes experimental datasets and available open-source ML testing tools and frameworks.
  • It analyzes research trends, directions, opportunities, and challenges in ML testing.
Loading 1906.10742v2…