Source-linked AI summary
Testing Deep Neural Networks
Youcheng Sun, Xiaowei Huang, Daniel Kroening, James Sharp, Matthew Hill, Rob Ashmore
TL;DR
DNN software needs testing for safety-critical deployment, but traditional coverage metrics do not directly fit DNNs. The paper proposes four MC/DC-inspired criteria for DNNs and finds them feasible while supporting adversarial-example discovery across experiments.
Problem
Safety-critical DNN software requires thorough testing, but traditional software coverage metrics cannot be applied directly to DNNs.
Method
The paper defines four white-box test criteria based on DNN features, decisions, and conditions, then uses them to guide test-case generation.
Results
The criteria achieve intensive yet computationally feasible testing, with over 90% SS coverage and a significant portion of adversarial examples on CIFAR-10.
Takeaways & Limitations
The coverage metrics provide evidence towards adversarial robustness and additional insight into dataset adequacy for safety cases.
Takeaways & Limitations
The ultimate oracle requiring agreement with perfect human perception is impractical, and some experimental parameter choices remain unspecified or experimental.
Abstract
from arXiv · showhide
Deep neural networks (DNNs) have a wide range of applications, and software employing them must be thoroughly tested, especially in safety-critical domains. However, traditional software test coverage metrics cannot be applied directly to DNNs. In this paper, inspired by the MC/DC coverage criterion, we propose a family of four novel test criteria that are tailored to structural features of DNNs and their semantics. We validate the criteria by demonstrating that the generated test inputs guided via our proposed coverage criteria are able to capture undesired behaviours in a DNN. Test cases are generated using a symbolic approach and a gradient-based heuristic search. By comparing them with existing methods, we show that our criteria achieve a balance between their ability to find bugs (proxied using adversarial examples) and the computational cost of test case generation. Our experiments are conducted on state-of-the-art DNNs obtained using popular open source datasets, including MNIST, CIFAR-10 and ImageNet.
1 Introduction
DNN deployment in safety-critical applications creates a need for testing methods that capture learned behavior and DNN-specific bugs. The paper proposes four MC/DC-inspired criteria designed to balance intensive testing with computational feasibility.
- Safety-critical DNN deployment raises requirements for testing, validation, and certification against relevant safety standards.
- Traditional white-box coverage cannot directly test DNNs because control flow does not represent learned knowledge, while adversarial examples create distinctive safety concerns.
- The proposed methodology introduces four MC/DC-inspired criteria tailored to DNN architecture, activation functions, and semantic relationships between layers.
- The criteria aim to avoid both weak coverage, such as easily achieved neuron coverage, and computationally intractable coverage requirements.
- Experiments on networks ranging from hundreds to millions of neurons evaluate bug finding, safety statistics, testing efficiency, and internal-structure analysis, treating adversarial examples as bugs.
2 Preliminaries: Deep Neural Networks
A DNN is modeled as connected layers of neurons whose weighted computations and activation functions transform inputs into output labels. The preliminaries define node variables, ReLU behavior, and the network’s input-output spaces.
- A DNN consists of layers, inter-layer connections, and one function for each non-input layer, with designated input, hidden, and output layers.
- Each non-input, non-output neuron has pre-activation and post-activation variables, while input and output nodes use variables reflecting their lack of activation functions.
- ReLU activation contributes to highly nonlinear network behavior by combining the neuron computation with activation operations.
- Pretrained weights and biases connect each neuron to the preceding layer and support both fully connected and convolutional functions.
- The network assigns each input the label corresponding to the output node with the largest value.
- For a fixed input, neuron values and the activation state of every ReLU operation are determined in the instantiated network.
3 Adequacy Criteria for Testing DNNs
The paper adapts MC/DC ideas to DNN-specific features and defines four coverage methods based on changes in adjacent-layer conditions and decisions. These methods quantify coverage over feature pairs and support test generation aimed at safety-relevant behaviors.
- DNN decisions and conditions: The framework treats each feature in a layer as a decision and features in the preceding layer as its conditions.Features are subsets of neurons and may overlap; singleton features correspond to individual neurons.
- Feature changes: DNN feature changes are characterized as either sign changes or value changes, with value functions specifying significant differences between feature vectors.Examples include absolute or relative activation changes, boundary constraints, norm-based distances, and structural similarity.
- Covering methods: The four coverage methods combine changes in condition features and decision features across adjacent layers to capture causal changes in DNN behavior.SS coverage, for example, requires a condition feature’s sign change without corresponding changes in other features and a sign change in the decision feature.
- Covering methods: SS coverage extends MC/DC by replacing Boolean condition changes with feature sign changes, while three additional criteria address DNN behavior not captured by this direct analogy.The added criteria use value changes for conditions, decisions, or both, according to developer-defined value functions.
- Test criteria: The test criterion Mf(N, T) measures the percentage of feature pairs covered by a test suite under covering method f.Instantiating f with the four methods yields MSS(N, T), MVSg(N, T), MSVg(N, T), and MVVg1,g2(N, T).
4 Comparison with Existing Structural Test Criteria
The proposed criteria are stronger than several neuron-based criteria but weaker and more practical than safety coverage. Their feature relations require more intensive testing while avoiding safety coverage’s computational intractability.
- Compared criteria: The paper compares its criteria with safety coverage, neuron coverage, neuron boundary coverage, multisection neuron coverage, and top neuron coverage.These criteria are formalized using the paper’s common notation for hidden nodes and test suites.
- Existing neuron criteria: Neuron coverage requires each neuron to be activated at least once, while boundary, multisection, and top-neuron criteria target additional activation-value regions or ranks.Multisection coverage requires every defined section of a neuron’s activation range to be reached by some test case.
- Relations between criteria: The paper proves that neuron coverage is weaker than SS coverage: MN ⪯ MSS.If a neuron is not covered, none of the corresponding feature pairs can be SS-covered.
- Relations between criteria: The proposed criteria require more test cases than criteria in [13], potentially enabling more intensive testing because they capture feature relations rather than individual activation statuses.The feature relation extends coverage beyond criteria based only on whether individual neurons activate.
- Practicality: SS coverage is weaker than safety coverage, whose hyper-rectangle scheme is computationally intractable for high-dimensional DNNs; the paper’s approach is presented as more practical.Safety coverage partitions inputs by ReLU activation patterns and generates one test case for each hyper-rectangle.
5 Automated Test Case Generation
The paper combines concolic LP solving with gradient search to generate test cases satisfying DNN coverage conditions. LP offers stronger guarantees, while gradient search addresses scalability to large networks but relies on favorable input pairs and search conditions.
- Approach: The authors apply two generation approaches to coverage criteria whose feature-relation conditions make existing testing methods and random generation unsuitable.The criteria require particular combinations of condition and decision features; random generation is described as prohibitively inefficient.
- LP-based generation: Concolic testing encodes a DNN instance and its test conditions as a linear programming model, then solves for new inputs.The model fixes a partial activation pattern from a concrete input and searches for another input satisfying the specified condition.
- LP-based generation: The LP constraints encode ReLU behavior and layer-to-layer activation values while preserving the reference input’s activation pattern.Variables u_k,i and v_k,i represent node values before and after ReLU, and C[x] combines the ReLU and activation constraints.
- Scalability: LP optimization can return a satisfying input pair when one exists, but its scalability depends on LP solvers and becomes difficult for networks with millions of neurons.This motivates the gradient-based heuristic developed for large-scale DNNs.
- Gradient-based generation: The gradient heuristic starts from inputs x1 and x2, updates x2 along the gradient, and returns the pair when the coverage condition is satisfied.The method adjusts the step size and can move x2 closer to or farther from x1 depending on a widened testing condition.
- Gradient-based generation: The heuristic succeeds under the assumption that a dataset input and a gradient-search input exist whose pair satisfies the specified test condition.The paper presents the criteria as a conjectured balance between guiding relevant test cases and computational cost, evaluated with both generation approaches.
6 Experiments
Experiments evaluate the proposed coverage methods on MNIST, CIFAR-10, and ImageNet-related settings, measuring adversarial-example discovery, safety statistics, efficiency, and layerwise behavior. The results show high coverage with substantial adversarial-example detection, practical LP-based generation, deeper-layer sensitivity to smaller perturbations, and comparable performance from top-weight simplification.
- Experimental scope: Experiments cover bug finding, DNN safety statistics, testing efficiency, and internal-structure analysis across multiple datasets and network settings.The study uses MNIST, CIFAR-10, and ImageNet, with experiments organized into four analysis classes.
- Bug finding: The test-generation algorithm achieves high coverage for all criteria while identifying a significant portion of adversarial examples.The MNIST results also show that neuron coverage alone can be high with a few non-adversarial test cases.
- DNN safety analysis: Coverage and adversarial-example percentages jointly provide statistics for assessing robustness, while distance curves indicate how close adversarial inputs are to originals.For the SS results, lower accumulated percentages near zero indicate greater robustness because adversarial inputs are farther from correct originals.
- Layerwise behavior: Deeper layers can be harder to cover, whereas most adversarial examples are found when testing middle layers.Improving deeper-layer coverage may require a larger dataset when generating test pairs.
- SS coverage with top weights: Restricting SS coverage to the κ largest absolute weights reduces test-suite size while producing coverage and adversarial-example percentages comparable to full SS coverage.The simplification assumes higher-weight condition neurons are likely to exert greater influence on decisions.
- CIFAR-10: On CIFAR-10, SS coverage exceeds 90% with a significant portion of adversarial examples, and deeper-layer feature changes detect smaller input perturbations.The authors suggest this behavior can provide feedback for debugging or tuning network parameters.
- Neuron boundary coverage: Neuron-boundary coverage produces 22.7% adversarial examples with average L∞ distance 3.49 and standard deviation 3.88, with larger distances than SS coverage.The experiments characterize boundary activation as requiring bigger changes in the DNN.
7 Related Work
Related work spans heuristic adversarial-example generation, structural coverage criteria, DNN adaptations of traditional testing, and automated verification. Existing approaches trade off efficiency, coverage strength, guarantees, and scalability.
- Heuristic gradient-descent and evolutionary methods can find adversarial examples efficiently but provide no guarantee when they fail.
- Existing structural criteria include neuron coverage, activation-boundary criteria, and coverage of neuron subsets or activation levels.
- Safety coverage exhaustively partitions activation patterns, but generating its test suite can be very expensive.
- Concolic testing, symbolic execution, fuzzing, mutation testing, metamorphic testing, and combinatorial methods have been extended to DNNs.
- DNN testing has also been studied in TensorFlow programs, Android malware detection, and autonomous-driving applications.
- Automated verification uses SMT, MILP, SAT, or hybrid-system formulations, typically handling only small networks without approximation.
8 Conclusions
The paper proposes novel DNN test criteria and reports promising evidence of their feasibility and effectiveness. The metrics support evidence about adversarial robustness and additional analysis of dataset adequacy for applications.
- The paper proposes a set of novel test criteria for DNNs and evaluates them across datasets and test-case generation methods.
- The metrics provide evidence towards adversarial robustness that is envisaged to contribute to safety cases.
- The metrics are expected to give domain experts additional insight when assessing whether a dataset is adequate for an application.
Appendix
The appendix establishes formal relationships among the proposed coverage metrics and connects them to neuron and safety coverage. These relationships are proved by relating uncovered neurons, intervals, feature pairs, and activation patterns.
- Lemma 2 states that neuron coverage is subsumed by SS coverage.
- The non-trivial-interval assumption is satisfied by the studied networks in practice, while an always-zero node can be treated as redundant.
- Theorem 1 states that safety coverage subsumes SS coverage because safety coverage exhaustively enumerates activation patterns.