Source-linked AI summary
Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples
Guanhong Tao, Shiqing Ma, Yingqi Liu, Xiangyu Zhang
TL;DR
Adversarial attacks challenge DNN defenses because existing methods may require attack knowledge or disrupt normal features, while adversarial decisions are difficult to explain through human-perceptible attributes. AmI detects such samples in face recognition by extracting attribute-linked neurons, strengthening those neurons, weakening others, and comparing transformed and original predictions. It reports 94% detection accuracy across seven attacks with 9.91% false positives, versus feature squeezing's 55% accuracy and 23.32% false positives.
Problem
Existing defenses may require prior knowledge of attacks or degrade benign classification and detection by corrupting features used for normal DNN functionality.
Method
AmI extracts attribute witnesses through bi-directional attribute-neuron correspondence, strengthens witness neurons, weakens others, and compares attribute-steered with original model predictions.
Results
94% detection accuracy across seven attack types with 9.91% false positives, compared with feature squeezing's 55% accuracy and 23.32% false positives.
Takeaways & Limitations
AmI demonstrates that interpretability-based attribute reasoning can support adversarial-sample detection for face recognition systems.
Abstract
from arXiv · showhide
Adversarial sample attacks perturb benign inputs to induce DNN misbehaviors. Recent research has demonstrated the widespread presence and the devastating consequences of such attacks. Existing defense techniques either assume prior knowledge of specific attacks or may not work well on complex models due to their underlying assumptions. We argue that adversarial sample attacks are deeply entangled with interpretability of DNN models: while classification results on benign inputs can be reasoned based on the human perceptible features/attributes, results on adversarial samples can hardly be explained. Therefore, we propose a novel adversarial sample detection technique for face recognition models, based on interpretability. It features a novel bi-directional correspondence inference between attributes and internal neurons to identify neurons critical for individual attributes. The activation values of critical neurons are enhanced to amplify the reasoning part of the computation and the values of other neurons are weakened to suppress the uninterpretable part. The classification results after such transformation are compared with those of the original model to detect adversaries. Results show that our technique can achieve 94% detection accuracy for 7 different kinds of attacks with 9.91% false positives on benign inputs. In contrast, a state-of-the-art feature squeezing technique can only achieve 55% accuracy with 23.3% false positives.
1 Introduction
Adversarial samples exploit weaknesses in DNN behavior while remaining difficult to explain through human-perceptible attributes. AmI detects them by linking face attributes to internal neurons and comparing original and attribute-steered predictions.
- Slight, often imperceptible perturbations can cause DNN misbehavior, with consequences including unsafe driving decisions and unauthorized face-ID access.
- Existing defenses often require prior knowledge of attacks, while feature squeezing can damage features used by normal classification and detection.
- Adversarial classifications are difficult to attribute to human-perceptible features, unlike benign classifications based on attributes such as eye color and nose shape.
- AmI identifies attribute witnesses through bi-directional correspondence inference between face attributes and internal neurons, then strengthens witnesses and weakens other neurons.
- 94% true positive rate across seven attack types and 9.91% false positives outperform feature squeezing's 55% accuracy and 23.32% false positives.
2 Background and Related Work
Prior work studies adversarial attacks, DNN interpretation, and defenses, but AmI combines attribute-level reasoning with execution-time adversary detection. Its workflow extracts attribute witnesses and uses them to construct an attribute-steered model.
- Model Interpretation: Feature visualization and object-neuron association primarily reason from pixel perturbations to neuron changes, providing one-way rather than bi-directional correspondence.
- Adversarial Samples: Adversarial attacks include targeted and untargeted forms; targeted attacks seek a specific classification, whereas untargeted attacks cause misclassification without a target.
- Adversarial Samples: Targeted attacks include patches and pervasive perturbations, while FGSM and BIM generate untargeted samples through one-step and iterative gradient-based changes.
- Defense Techniques: Defense techniques comprise model hardening and adversary detection; AmI belongs to the detection category and operates during execution.
- AmI Workflow: AmI identifies face-attribute shapes, obtains human annotations, extracts attribute witnesses, builds an attribute-steered model, and compares its outputs with the original model.
- AmI Workflow: Attribute witness extraction intersects neurons changed by attribute substitution with neurons preserved by attribute preservation.
3 Approach
AmI identifies neurons corresponding to human-perceptible face attributes through bi-directional correspondence inference, then steers activations toward those witnesses to distinguish interpretable benign predictions from adversarial ones.
- Attribute witness extraction: AmI identifies attribute witnesses by analyzing how a face-recognition system responds to human-perceptible attributes such as eyes, nose, and mouth.The workflow uses annotated training images and extracts corresponding neurons for each attribute.
- Attribute witness extraction: Bi-directional reasoning combines attribute substitution, which finds neurons changing when an attribute changes, with attribute preservation, which retains the attribute while testing neuron stability.Substitution provides forward correspondence, while preservation captures the complementary backward relation.
- Attribute witness extraction: Witnesses are obtained by intersecting neurons selected through attribute substitution and attribute preservation, while witness sets may overlap across attributes.The intersection removes neurons that respond to changes without being specifically tied to the attribute.
- Attribute witness extraction: Attribute substitution replaces an attribute with counterparts from other people rather than eliminating it, enabling analysis of how different attribute instances affect internal neurons.The paper reports that eliminating an attribute with a simple image transformation does not work well.
- Attribute-steered model: The attribute-steered model strengthens witness neurons and weakens non-witness neurons without additional training, emphasizing attribute-based reasoning and suppressing other activations.The transformed model is expected to preserve benign predictions but produce inconsistencies for adversarial inputs that are difficult to explain through attributes.
- Attribute-steered model: Neuron weakening uses witness-neuron statistics to reduce sufficiently large non-witness activations, while neuron strengthening enlarges all witness activations during layer-by-layer inference.The weakening magnitude is controlled by α, and strengthening uses tunable factors ϵ and β.
4 Experiments
Experiments evaluate AmI on VGG-Face using attribute witnesses, compare it with feature squeezing and component variants, and test robustness across datasets and witness subsets.
- 4 Experiments: AmI uses VGG-Face and evaluates detection on seven adversarial attacks, comparing results with feature squeezing.Witnesses are extracted from 10 benign VGG Face images, with testing performed on disjoint datasets.
- 4.1 Evaluation of Extracted Attribute Witnesses: Attribute witnesses consistently exceed 93% accuracy for three attributes, compared with below 86% for VGG-Face’s face descriptor.Accuracy decreases on LFW but remains higher for attribute witnesses than for the face descriptor.
- 4.2 Detection of Adversarial Samples: AmI produces lower false positive rates than feature squeezing: 9.91% versus 23.32% on VF and 8.97% versus 28.43% on CelebA.Both methods are evaluated on benign inputs from the original and external datasets.
- 4.2 Detection of Adversarial Samples: Attribute substitution or preservation alone yields high false positive rates of 20.41% and 30.61%, while combining weakening and strengthening improves detection with a small false-positive trade-off.The variants extract too many witnesses or omit one of the neuron transformation operations.
- 4.2 Detection of Adversarial Samples: Excluding witnesses for an individual attribute degrades detection accuracy by less than 5% in most cases, indicating robustness to witness selection.The tested exclusions cover the left eye, right eye, nose, and mouth.
5 Conclusion
The paper concludes that AmI detects adversarial samples in face recognition by leveraging DNN interpretability and combining attribute-neuron reasoning with neuron strengthening and weakening.
- 5 Conclusion: AmI uses bi-directional correspondence inference, attribute-level mutation, and neuron strengthening and weakening to detect adversarial samples in face recognition.The method leverages interpretability of DNNs and is reported as effective in the targeted context.