Source-linked AI summary
DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang
TL;DR
DL systems are widely deployed despite vulnerabilities, while accuracy-based testing provides incomplete evidence of testing adequacy and generality. DeepGauge introduces multi-granularity testing criteria that examine DNN behavior and test-data coverage; experiments across datasets, models, and adversarial attacks show increased coverage for adversarial tests and indicate practical usefulness for evaluating DNN testing adequacy.
Problem
Accuracy-based testing does not fully gauge DL testing adequacy because it overlooks internal neuron behavior, depends on test-data representativeness, and may miss unseen attacks.
Method
DeepGauge defines multi-level and multi-granularity coverage criteria to measure DNN testing quality through neuron activities, network behavior, and covered test regions.
Results
Experiments on two datasets, five DNNs, and four adversarial testing techniques found obvious coverage increases for adversarially generated tests, supporting DeepGauge as a testing-adequacy indicator.
Takeaways & Limitations
The criteria can provide insights into DNN testing and support efforts toward understanding and building more generic and robust DL systems.
Takeaways & Limitations
Some results may not generalize to other datasets and DNN models, and configurable criterion hyperparameters remain a threat to validity.
Abstract
from arXiv · showhide
Deep learning (DL) defines a new data-driven programming paradigm that constructs the internal system logic of a crafted neuron network through a set of training data. We have seen wide adoption of DL in many safety-critical scenarios. However, a plethora of studies have shown that the state-of-the-art DL systems suffer from various vulnerabilities which can lead to severe consequences when applied to real-world applications. Currently, the testing adequacy of a DL system is usually measured by the accuracy of test data. Considering the limitation of accessible high quality test data, good accuracy performance on test data can hardly provide confidence to the testing adequacy and generality of DL systems. Unlike traditional software systems that have clear and controllable logic and functionality, the lack of interpretability in a DL system makes system analysis and defect detection difficult, which could potentially hinder its real-world deployment. In this paper, we propose DeepGauge, a set of multi-granularity testing criteria for DL systems, which aims at rendering a multi-faceted portrayal of the testbed. The in-depth evaluation of our proposed testing criteria is demonstrated on two well-known datasets, five DL systems, and with four state-of-the-art adversarial attack techniques against DL. The potential usefulness of DeepGauge sheds light on the construction of more generic and robust DL systems.
1 INTRODUCTION
DeepGauge addresses the lack of systematic testing-adequacy measures for deep neural networks by introducing multi-granularity coverage criteria. The evaluation indicates that these criteria expose differences between ordinary and adversarial test data and may help assess defect-detection ability.
- Motivation: DL systems are widely used in applications including speech processing, medical diagnostics, image processing, robotics, and autonomous driving.Autonomous vehicles have exhibited failures in unexpected or corner-case driving conditions, with reported accidents.
- Problem: Accuracy-based testing is incomplete because it ignores internal neuron activities, depends heavily on test-data representativeness, and may miss unseen adversarial attacks.High or low output performance alone does not establish that a model is generic or adequately tested.
- Approach: DeepGauge proposes scalable criteria that monitor neuron activities and intrinsic network connectivity at multiple granularity levels.The intended result is a multi-faceted portrayal of the DL system and testing quality.
- Coverage perspective: The criteria measure coverage of major-function and corner-case regions, where defects may occur, while revealing that existing test data favor major-function regions.This provides information about both DNN behavior and test-data quality from different levels and angles.
- Evaluation: Experiments with FGSM, BIM, JSMA, and CW show that adversarial examples increase the proposed coverage and distinguish correctly recognized original data from misrecognized adversarial data.The authors interpret higher coverage as potentially indicating a higher chance of detecting DNN defects.
- Evaluation: The study reports that the criteria behave differently depending on network complexity and the dataset under analysis.Together, the criteria are intended to provide insights into testing DNNs.
2 PRELIMINARIES
The paper frames DNN testing as a distinct software-testing problem because DNN behavior is learned from data and encoded through network computations rather than explicit developer-written control logic. It motivates coverage criteria that inspect runtime behavior and internal states, while noting that traditional criteria do not directly transfer.
- Traditional software testing: Traditional software testing compares actual runtime outputs with expected outputs and uses coverage criteria to quantify how thoroughly software is tested.Coverage criteria are practical measures of test adequacy.
- Traditional software testing: Traditional coverage criteria include statement, branch, data-flow, and model-based coverage at code or model levels.These criteria analyze runtime behavior from different perspectives and testing granularities.
- DNN testing gap: DNNs cannot directly use these traditional criteria because their unique architecture differs from conventional programs.The paper explicitly identifies this lack of direct applicability.
- DNN architecture: A DNN consists of interconnected neurons arranged in input, output, and hidden layers, with weighted edges and activation functions transforming inputs into outputs.Its function is determined by the aggregated effects of neuron computations and connection weights.
- DNN architecture: DNNs are programmed through training data, selected features, and network structures, and may contain defects caused by incorrect or incomplete training data or training procedures.Such defects can produce wrong outputs, including incorrect steering angles.
- DNN testing gap: DeepGauge defines coverage criteria to evaluate test adequacy and gauge the internal states covered by DNN test inputs.The criteria are intended to increase confidence in testing results.
3 COVERAGE CRITERIA FOR TESTING DL SYSTEMS
DeepGauge defines scalable, multi-level coverage criteria for DNN testing because prediction accuracy alone does not systematically probe internal behaviors. The criteria cover neuron activation ranges, corner cases, and layer-level activation patterns.
- Motivation: DNN testing cannot directly reuse traditional software coverage criteria and primarily relies on prediction accuracy, leaving systematic defect detection lacking.Traditional criteria target human-designed control flow, whereas DNN behavior is characterized by learned weights and nonlinear activations.
- Design goals: DeepGauge aims to provide simple, scalable, general criteria that gauge testing adequacy through multiple portrayals of DNN behavior.The design monitors neuron activities and network behavior at multiple granularity levels without depending on a specific DNN structure or activation function.
- Neuron-level criteria: k-multisection neuron coverage partitions each neuron’s training-derived major function region [lown, highn] into k equal sections and measures how many sections test inputs cover.The major function region is defined from training-data analysis, and coverage is the ratio of covered sections to the total k sections for each neuron.
- Neuron-level criteria: Neuron boundary coverage and strong neuron activation coverage target upper- and lower-corner regions outside each neuron’s major function range.A corner case occurs when a neuron output falls below lown or above highn; such regions may contain DNN defects despite being atypical under training-like inputs.
- Layer-level criteria: Layer-level criteria characterize DNN behavior using top-k hyperactive neurons and their cross-layer combinations, called top-k neuron patterns.Top-k neuron coverage counts neurons that have appeared among the k most active on a layer, while patterns record the sequence of top-k sets across layers.
4 EXPERIMENTS
Experiments evaluate DeepGauge across adversarial attacks, datasets, models, and criterion configurations. Adversarial tests generally increase coverage across multiple neuron-level perspectives, revealing differences between original and adversarial inputs and potentially exposing defects.
- Evaluation Setup: Four adversarial techniques—FGSM, BIM, JSMA, and CW—generate the evaluation tests, with JSMA and CW perturbing fewer MNIST pixels than FGSM and BIM.The same technique can also generate different perturbations for different DNNs.
- MNIST and ImageNet: Adversarial tests generally increase DeepGauge coverage over the original MNIST test set across multisection, boundary, activation, top-k neuron, and pattern criteria.For LeNet-4, JSMA raises 10,000-multisection coverage from 39.7% to 52.3% and strong neuron activation coverage from 13.5% to 27.7%.
- MNIST and ImageNet: On ImageNet, coverage patterns resemble MNIST but vary with model complexity; ResNet-50 reaches 22.8% KMNC on original tests versus 13.5% for VGG-19.The results do not generally support equating greater network size with lower coverage.
- MNIST and ImageNet: Top-k criteria capture hyperactive neuron states and structural differences in inputs, although top-k coverage improvements are generally less pronounced than other criteria.For VGG-19, original ImageNet tests achieve 58.8% top-1 and 81.6% top-3 neuron coverage; adversarial data still trigger additional top-k neurons in many cases.
- Findings and Remarks: Original and adversarial datasets cover both major-function and corner-case regions, while combining adversarial data with originals generally boosts DeepGauge coverage.The authors associate increased coverage with potentially higher chances of triggering DNN states and detecting defects.
- Findings and Remarks: Coverage responses depend on criterion parameters and attack characteristics: larger k reduces top-k coverage gains but increases top-k pattern gains, while CW is harder to distinguish.CW perturbations concentrate on smaller-magnitude objects and may trigger fewer internal behavior changes.
- Comparison with DeepXplore’s Neuron Coverage: DeepGauge uses per-neuron bounds derived from training-data statistics rather than a shared activation threshold.The comparison with DNC motivates this choice because neuron output distributions can differ substantially.
5 RELATED WORK
DL testing research addresses diverse methods for finding defects, measuring internal behavior, and verifying safety, but suitable coverage criteria remain underdeveloped. DeepGauge proposes multi-granularity coverage to differentiate adversarial techniques and guide more effective automated testing.
- Black-box accuracy testing may miss corner-case behaviors that induce unexpected errors in machine-learning systems.
- Existing DL testing methods include differential testing, image transformations, combinatorial neuron-input coverage, and model mutation.
- DeepGauge is an early attempt to provide multi-granularity criteria that differentiate adversarial test generation and guide DL test-suite diversity analysis.
- Formal verification can provide safety and robustness guarantees, but scalability remains a concern for real-world-sized DL systems.
- Adversarial attacks can fool DL systems, yet their effects on DNN internal states remain insufficiently studied.
- DeepGauge measures adversarial attack techniques quantitatively and may support interpreting DNN behavior and designing more effective defenses.
6 CONCLUSION AND FUTURE WORK
DeepGauge evaluates DNN testing adequacy through coverage criteria across diverse datasets, networks, and adversarial testing techniques. The results support its usefulness as an indicator and motivate future criteria and automated test generation.
- Experiments on two well-known datasets, five DNNs, and four state-of-the-art adversarial testing techniques evaluated DeepGauge’s testing criteria.
- Adversarially generated tests produced obvious increases in coverage across the paper’s defined metrics.
- DeepGauge could be a useful indicator for evaluating DNN testing adequacy.
- Future work will explore alternative criteria, including combined hyperactive and hypoactive neurons, and criteria-guided automated test generation.
- The authors aim for the criteria to illuminate DNN nature and mechanisms and support understanding and construction of generic, robust DL systems.