Source-linked AI summary
Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
Xingjun Ma, Bo Li, Yisen Wang, Sarah M. Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E. Houle, James Bailey
TL;DR
DNN adversarial examples motivate the need to characterize the regions in which they lie, because their small perturbations can induce high-confidence errors. The paper uses LID to characterize adversarial-region dimensionality and finds that LID-based features distinguish examples across five attacks and three datasets, while noting scope limitations in attack-strength tuning and estimation settings.
Problem
Small, often imperceptible perturbations can cause DNNs to make high-confidence errors, motivating characterization of the adversarial regions surrounding such examples.
Method
The paper characterizes adversarial regions using LID, which estimates local dimensional properties from neighbor-distance distributions and models adversarial perturbation in relation to subspace dimensionality.
Results
LID estimates are significantly higher for adversarial than normal examples, become more separated in deeper layers, and support detection that generally outperforms several existing measures across five attacks and three datasets.
Takeaways & Limitations
The results identify LID as a promising measure for characterizing adversarial examples and delivering state-of-the-art discrimination performance.
Takeaways & Limitations
The study uses attack examples generated with default parameters rather than tuning for the strongest attacks, and LID estimation uses a default minibatch size of 100 despite small-sample instability.
Abstract
from arXiv · showhide
Deep Neural Networks (DNNs) have recently been shown to be vulnerable against adversarial examples, which are carefully crafted instances that can mislead DNNs to make errors during prediction. To better understand such attacks, a characterization is needed of the properties of regions (the so-called 'adversarial subspaces') in which adversarial examples lie. We tackle this challenge by characterizing the dimensional properties of adversarial regions, via the use of Local Intrinsic Dimensionality (LID). LID assesses the space-filling capability of the region surrounding a reference example, based on the distance distribution of the example to its neighbors. We first provide explanations about how adversarial perturbation can affect the LID characteristic of adversarial regions, and then show empirically that LID characteristics can facilitate the distinction of adversarial examples generated using state-of-the-art attacks. As a proof-of-concept, we show that a potential application of LID is to distinguish adversarial examples, and the preliminary results show that it can outperform several state-of-the-art detection measures by large margins for five attack strategies considered in this paper across three benchmark datasets. Our analysis of the LID characteristic for adversarial regions not only motivates new directions of effective adversarial defense, but also opens up more challenges for developing new attacks to better understand the vulnerabilities of DNNs.
1 INTRODUCTION
The paper frames adversarial examples as small, often imperceptible perturbations that occupy regions whose spatial and dimensional properties are not reliably captured by existing density measures. It proposes Local Intrinsic Dimensionality (LID) to characterize these regions and reports strong preliminary detection results across attacks and datasets.
- Adversarial examples are small, often imperceptible perturbations that can cause DNNs to make high-confidence incorrect predictions.
- Adversarial regions are connected areas around adversarial examples in which points similarly subvert the classifier, and they can be studied in input or activation spaces.
- Existing density measures can fail because adversarial and normal examples may have identical neighbor counts, k-mean distance, and kernel density.In the cited example, both have 50 neighbors, KM=0.19, and KD=0.92.
- LID is introduced as an expansion-based measure for characterizing the intrinsic dimensionality of adversarial regions.The paper motivates LID as an alternative to density measures for capturing local spatial structure.
- The estimated LID of adversarial examples is significantly higher than that of normal examples, with the difference becoming more pronounced in deeper DNN layers.
- A LID-feature baseline generally outperforms several existing detection measures across five attacks and three benchmark datasets.The paper describes these as preliminary proof-of-concept results.
2 RELATED WORK
The related work surveys adversarial attack and defense methods, including gradient-based attacks and defenses that can be evaded by optimization-based attacks. It also notes that adversarial examples are generated under default parameters for this proof-of-concept study.
- The paper reviews prior work on adversarial attacks and defenses.
- Adversarial Attack: FGM perturbs inputs along the gradient direction, while BIM applies this perturbation iteratively.
- Adversarial Defense: Existing defenses include adversarial training, distillation, gradient masking, and feature squeezing, but optimization-based attacks can evade them wholly or partially.
- The study uses state-of-the-art attack methods with default parameters rather than tuning them to seek the strongest attacks under different conditions.This scope is intended to provide a proof of concept for LID's potential application.
3 LOCAL INTRINSIC DIMENSIONALITY
Local Intrinsic Dimensionality (LID) measures how rapidly probability mass, treated as a proxy for volume, expands around a reference sample. It is estimated locally from nearest-neighbor distances and approximates the dimensionality of the surrounding data structure.
- In Euclidean space, an m-dimensional ball’s volume grows proportionally to r^m, allowing dimensionality to be inferred from expansion with distance.
- LID transfers expansion-dimensionality ideas from volume growth to continuous distance distributions around individual samples.The underlying distance measure need not be Euclidean.
- LID describes the relative rate at which a cumulative distance function increases as radius grows from zero.
- Under ideal uniform sampling on a submanifold, LID equals its dimension; otherwise, it provides a rough, potentially noninteger approximation.
- The practical LID estimate uses the k nearest-neighbor distances, with the largest neighbor distance defining the neighborhood radius.
4 CHARACTERIZING ADVERSARIAL REGIONS
The paper characterizes adversarial regions by estimating local dimensionality around normal, adversarial, and noisy examples. It motivates minibatch-based LID detection because adversarial neighborhoods can span multiple manifolds and therefore exhibit higher local dimensionality.
- LID of Adversarial Subspaces: Adversarial perturbations can move an example just outside its original low-dimensional manifold while remaining close to it in a high-dimensional representation space.
- LID of Adversarial Subspaces: Because adversarial neighborhoods can include points from multiple manifolds, they may span substantially higher intrinsic dimensionality than individual manifolds.
- Efficiency through Minibatch Sampling: Minibatch sampling reduces the cost of neighborhood computation when the full dataset is too large or high-dimensional for efficient indexing.Sufficiently large minibatches preserve local neighborhoods, whereas smaller minibatches increase estimate variance.
- Efficiency through Minibatch Sampling: 100-example minibatches and neighborhoods of 20 examples were sufficient to discriminate adversarial from non-adversarial examples in the reported experiments.
- Characterizing Adversarial Regions: LID estimates are used to train a detector that distinguishes adversarial, normal, and noisy examples using features from DNN transformation layers.Each example’s LID is estimated from neighbors in a normal minibatch using activation-space distances.
5 EVALUATING LID-BASED CHARACTERIZATION OF ADVERSARIAL EXAMPLES
The evaluation tests LID-based detection across five attack strategies and three datasets, comparing it with KD and BU while examining layer depth, parameter stability, cross-attack transfer, and adaptive attacks.
- Experimental setup: LID-based detection is evaluated against FGM, BIM-a, BIM-b, JSMA, and Opt across three benchmark image datasets, with KD and BU as comparison measures.The detectors are trained and tested using normal, noisy, and adversarial examples.
- LID properties: At CIFAR-10’s softmax layer, Opt adversarial examples have substantially higher LID scores than normal or noisy examples.The result supports the expectation that adversarial regions have higher intrinsic dimensionality than normal data regions.
- LID properties: Detection becomes distinguishable at dense layers L18−24 and significantly different at the softmax layer L25, indicating stronger discrimination in deeper representations.The CIFAR-10 network has 26 transformation layers, including the final softmax layer L25.
- Parameter stability: LID is more stable than KD as its parameter varies, while KD requires substantially different optimal bandwidths across datasets.For MNIST and SVHN, KD performs best with relatively high bandwidth choices.
- Comparative performance: 99.24%, 98.94%, and 97.60% are LID detector AUC scores for Opt on MNIST, CIFAR-10, and SVHN, respectively.These scores exceed the corresponding KD and BU detection results reported for the same attack and datasets.
- Generalizability: A detector trained on FGM examples accurately detects more complex BIM-a, BIM-b, JSMA, and Opt attacks, suggesting similar dimensional properties across adversarial regions.The transfer results are reported for CIFAR-10, where LID is more consistent than KD and BU.
- Adaptive attacks: The adaptive attack fails to find valid adversarial examples 100%, 95.7%, and 97.2% of the time on MNIST, CIFAR-10, and SVHN in Scenario 2.When trained on all transformation layers, the LID detector correctly detected the attacks 100% of the time.
6 DISCUSSION AND CONCLUSION
The paper presents LID as a promising way to characterize adversarial regions and detect adversarial examples, while identifying unresolved questions about DNN transformations and estimation quality.
- LID characterizes adversarial-region dimensional properties and provides features for adversarial-example detection.
- Understanding how DNN transformations affect LID characteristics remains a challenging open question.
- The effect of LID estimation quality on adversarial-detection performance remains an open issue for empirical investigation.
- Increasing minibatch size from 100 to 1000 improved performance, suggesting that better estimators or sampling strategies may be beneficial.
A.1 STATISTICS OF ADVERSARIAL ATTACK STRATEGIES
Table 4 reports L2 mean perturbation and model accuracy on adversarial examples.
- Table 4 reports L2 mean perturbation and model accuracy (%) on adversarial examples.
A.2 LID CHARACTERISTICS OF ADVERSARIAL EXAMPLES
Figure 4 compares normalized LID scores for normal, noisy, and Opt adversarial examples on MNIST and SVHN. Opt adversarial examples have higher LID scores than normal or noisy examples in both plots.
- Opt adversarial examples have significantly higher LID scores than normal or noisy examples on both MNIST and SVHN.
- The MNIST plot reports LID at the pre-softmax layer, whereas the SVHN plot reports LID at the softmax layer.
- Normal and noisy-example curves appear superimposed in the SVHN plot because their LID values are similar.
A.3 EFFECT OF LARGER MINIBATCH SIZES IN LID ESTIMATION
Figure 5 evaluates how neighborhood size and minibatch size affect LID-based detection of Opt attacks across MNIST, CIFAR-10, and SVHN. The larger minibatch produces a higher peak detection AUC.
- The peak detection AUC is higher with a minibatch size of 1000 than with the default size of 100.
- Figure 5 evaluates Opt-attack detection across MNIST, CIFAR-10, and SVHN using neighborhood sizes from 10% to 90% of the batch size.