Source-linked AI summary
Adversarial Perturbations Against Deep Neural Networks for Malware Classification
Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, Patrick McDaniel
TL;DR
The paper asks whether adversarial crafting developed for image classifiers also threatens malware detection, where inputs are discrete and functionality must be preserved. It adapts derivative-based attacks and evaluates them with DREBIN-trained neural classifiers, finding high attack effectiveness and mixed defense performance.
Problem
Whether adversarial crafting transfers effectively to security-critical malware detection despite discrete inputs and the requirement to preserve functional behavior.
Method
The study adapts forward-derivative crafting to binary static features, adding functionality-preserving features to DREBIN-trained malware classifiers.
Results
60% to 80% of malicious samples were misclassified, while feature reduction weakened defenses, distillation helped modestly, and adversarial retraining consistently reduced misclassification rates.
Takeaways & Limitations
Adversarial crafting is a real threat to neural-network malware classifiers, and adversarial retraining is the most consistently beneficial evaluated defense.
Takeaways & Limitations
The evaluation uses static features; dynamically gathered features are left for future work because they are more difficult to perturb.
Abstract
from arXiv · showhide
Deep neural networks, like many other machine learning models, have recently been shown to lack robustness against adversarially crafted inputs. These inputs are derived from regular inputs by minor yet carefully selected perturbations that deceive machine learning models into desired misclassifications. Existing work in this emerging field was largely specific to the domain of image classification, since the high-entropy of images can be conveniently manipulated without changing the images' overall visual appearance. Yet, it remains unclear how such attacks translate to more security-sensitive applications such as malware detection - which may pose significant challenges in sample generation and arguably grave consequences for failure. In this paper, we show how to construct highly-effective adversarial sample crafting attacks for neural networks used as malware classifiers. The application domain of malware classification introduces additional constraints in the adversarial sample crafting problem when compared to the computer vision domain: (i) continuous, differentiable input domains are replaced by discrete, often binary inputs; and (ii) the loose condition of leaving visual appearance unchanged is replaced by requiring equivalent functional behavior. We demonstrate the feasibility of these attacks on many different instances of malware classifiers that we trained using the DREBIN Android malware data set. We furthermore evaluate to which extent potential defensive mechanisms against adversarial crafting can be leveraged to the setting of malware classification. While feature reduction did not prove to have a positive impact, distillation and re-training on adversarially crafted samples show promising results.
1. INTRODUCTION
The paper investigates whether adversarial crafting transfers from image classification to malware detection, where inputs are discrete and modifications must preserve functionality. It demonstrates effective attacks and evaluates defenses on DREBIN-trained neural classifiers.
- Neural networks’ success on high-dimensional tasks motivates their application to security-critical malware classification.
- Unlike image attacks, malware attacks must handle low-entropy, binary inputs while preserving the application’s functional behavior.
- The study adapts forward-derivative crafting to malware classifiers trained on the DREBIN Android dataset.
- 97% accuracy, 7.6% false negatives, and 2% false positives were achieved by the trained malware classifier.
- Feature reduction failed to protect classifiers, distillation yielded small improvements, and adversarial retraining generally improved resistance with parameter sensitivity.
2. BACKGROUND
The background introduces neural-network architectures, adversarial samples, attack objectives, and defenses. It frames adversarial crafting as approximate optimization guided by model derivatives and saliency information.
- Deep neural networks compose multiple hidden layers that hierarchically extract representations before producing predictions.
- Adversarial samples use carefully selected perturbations to force targeted misclassification, exploiting training imperfections and model component linearity.
- For a binary classifier, crafting seeks the smallest perturbation that changes the original classification.
- Because deep-network optimization is difficult, approximate methods use gradients, saliency maps, or forward derivatives to select influential input components.
- Derivative-based attacks generally require full model knowledge, while substitute-model querying enables a black-box variant.
- Distillation and adversarial training are proposed defenses intended to improve robustness against crafted inputs.
3. METHODOLOGY
The methodology represents Android applications with sparse binary static features, trains feed-forward neural classifiers, and adapts Jacobian-guided crafting to discrete, functionality-preserving modifications. The attack iteratively adds selected features until misclassification or a change limit is reached.
- Application representation: Applications are represented as high-dimensional binary indicator vectors derived from statically analyzed features.
- Application representation: The study restricts attention to static features because dynamically gathered features are harder to perturb.
- Classifier training: Regular feed-forward networks are used because the binary feature vectors lack the structural properties exploited by convolutional or recurrent architectures.
- Classifier training: Classifiers vary from one to four hidden layers and from 10 to 300 neurons per layer.
- Classifier training: The networks use rectified hidden-unit activations, a softmax output, gradient descent, and batch-based training.
- Adversarial crafting: The attack targets malicious-to-benign misclassification by selecting absent features with the largest positive gradient toward the benign class.
- Adversarial crafting: Each iteration adds one binary feature, recomputes the gradient, and stops after successful misclassification or the maximum allowed changes.
4. EXPERIMENTAL EVALUATION
The evaluation tests neural-network malware classification and adversarial crafting on DREBIN, then compares attack resistance across architectures and training settings. The classifiers achieve strong conventional performance, but adversarial modifications misclassify many malware samples despite strict constraints.
- 4.1 Dataset: DREBIN contains 129,013 Android applications, including 5,560 malicious samples, represented by 545,333 binary static-analysis features.The features span eight classes, including permissions, API calls, application components, and intents.
- 4.1 Dataset: The baseline network uses two hidden layers of 200 neurons, while experiments vary layer width, depth, and malware ratio in training batches.Performance is measured using accuracy, false negative rate, and false positive rate.
- 4.2 Malware Detection: Around 98% overall accuracy, about 7% false negatives, and 3.3% false positives provide the baseline architecture’s best reported trade-off.The authors note that better hyperparameter search could further improve performance.
- 4.3 Adversarial Malware Crafting: 50% to 84% of previously correctly classified malware samples are misclassified after adversarial crafting across the evaluated network variants.The attack allows at most 20 modifications, and higher malware ratios generally correlate with lower misclassification rates.
- 4.4 Discussion: About 60% to 80% of malicious applications are misclassified despite restricted modifications, confirming transfer of adversarial-crafting findings from computer vision to malware detection.The reported range depends on architecture and excludes corner cases; it is below the cited 97% image-digit result.
5. DEFENSES
The defenses section evaluates mechanisms intended to reduce neural networks’ sensitivity to adversarial input manipulation. It considers feature reduction first, followed by distillation and adversarial-sample retraining.
- 5. DEFENSES: Defense effectiveness is measured by the reduction in misclassification rate relative to regular networks.The misclassification rate is the percentage of malware samples misclassified after adversarial crafting.
- 5. DEFENSES: Feature reduction is evaluated as a way to reduce input sensitivity and limit the adversary’s available feature choices.The study includes simple and more involved feature-reduction strategies.
- 5. DEFENSES: The study also evaluates distillation and retraining on adversarial samples as defenses proposed in prior literature.
5.1 Simple Feature Reduction
Simple feature reduction does not harden the malware classifiers against adversarial crafting. Across restrictions, attack success generally increases rather than decreases relative to regular networks.
- 5.1 Simple Feature Reduction: Manifestonly trains on 233,727 manifest-derived features, focusing the classifier on features that the attack can safely modify.
- 5.1 Simple Feature Reduction: Onlysmall removes features from the r largest feature classes, targeting classes dominated by mostly unique URLs and similar features.
- 5.1 Simple Feature Reduction: Onlyfreq retains features appearing in at least r applications, leaving 177,438, 95,871, 60,052, or 44,942 features for r ∈{1, 2, 3, 4}.The restriction aims to filter out features that do not consistently occur across representatives of a target class.
- 5.1 Simple Feature Reduction: Feature-restricted networks are evaluated with two hidden layers of 200 neurons using the same adversarial crafting procedure and varying malware ratios.Figure 2 compares their misclassification-rate differences with regular networks as baseline.
- 5.1 Simple Feature Reduction: Up to 99% misclassification occurs with manifestonly, compared with about 62% for the regular network, while most restrictions also increase attack success.The authors conclude that simple feature reduction is not suitable for hardening malware-detection networks.
5.2 Feature Reduction via Mutual Information
Feature reduction via mutual information generally did not improve resistance to adversarial crafting, although selecting highly informative features produced some less vulnerable cases.
- Feature selection: Mutual information ranks features by their dependence with target classes, and classifiers are trained using the top n or bottom n ranked features.The reversed-ranking approach uses lower-mutual-information features and yields larger training sets because many features have low mutual information.
- Experimental setup: The evaluated networks use two hidden layers with 200 neurons each, a 50% malware ratio, and feature counts ranging from n = 50 to n = 2000.Adversarial crafting is applied to determine susceptibility across these networks.
- Evaluation: For n ≤1600, false negative rates remain about 7%, while crafting succeeds in at least 70% of cases with average distortion 12.5.Networks with very small n were excluded because their classification performance was poor.
- Reversed ranking: Using lower-ranked features produces misclassification rates above 90% for very low n and between 70.1% and 87.6% for reported larger feature sets.The average misclassification rate is 82%, substantially worse than the original model.
- Discussion: Feature reduction can make perturbations easier because each retained feature has greater influence on the classifier output.The authors nevertheless report some cases where selecting the most informative features reduced vulnerability and leave finer analysis for future work.
5.3 Distillation
Distillation trains a second network from the original network’s soft probability outputs and reduces adversarial misclassification, but also worsens classifier performance and remains less effective than in computer vision.
- Method: Distillation transfers the original classifier’s probability distributions over target classes as training labels for a second network.These soft labels contain more information than selecting only the most likely class.
- Temperature: High temperature makes output probabilities more uniform, and the resulting soft labels are used to train the distilled network.T = 1 recovers regular softmax normalization.
- Method: The defensive procedure constructs a new network with the same architecture, labels data with high-temperature outputs, and trains the new network on those labels.Both network construction and training use the same high temperature T.
- Evaluation: Distillation reduces adversarial misclassification, in some cases to 38.5%, but generally increases false negative and false positive rates.For malware ratio 0.5, the false negative rate rises from 4 to 6.4; accuracy remains between 93% and 95%.
- Discussion: Distillation strengthens resistance, but misclassification rates remain around 40% and the resulting classifier is less effective overall.Its effect is weaker than reported image-classification results, where rates were around 5%.
5.4 Re-Training
Adversarial retraining adds crafted malware samples to further train the classifier and generally improves resistance, but its benefit depends strongly on the number of added samples and training parameters.
- Method: Adversarial retraining starts with an ordinary classifier, crafts samples using the forward-gradient method, and trains further with those samples as additional malicious data.The crafted samples are mixed with benign samples according to each network’s malware ratio.
- Method: Retraining is intended to improve generalization, which generally makes classifiers less sensitive to small perturbations.The evaluation uses networks with two 200-neuron layers and adds 20, 100, or 250 crafted malware samples.
- Evaluation: For malware ratios 0.3 and 0.4, 100 added samples reduce misclassification from 73% to 67% in one reported case and increase required distortion.Using too many samples can reverse the improvement.
- Evaluation: For malware ratio 0.5, misclassification falls from 69% to 68% only after adding 250 adversarial samples.Fewer added samples leave performance close to the original network.
- Discussion: Overall, simple adversarial retraining improves resistance, but the required sample count depends heavily on the original training parameters.The authors suggest iterative retraining could further improve resistance, while excessive sample counts may degrade it.
5.5 Summary of Results
Feature reduction usually weakens resistance to adversarial crafting, whereas distillation and adversarial retraining provide positive but qualified defenses.
- Feature reduction: Feature reduction usually makes networks weaker because fewer, more influential features make adversarial samples easier to craft.The authors do not recommend feature reduction as a defensive mechanism without more involved methods.
- Distillation: Distillation has a positive defensive effect, but performs less well in malware classification than in computer vision.The reasons for this difference remain open for future investigation.
- Re-training: Simple adversarial retraining consistently reduces misclassification across different networks, although the number of crafted samples strongly affects the reduction.Iterative retraining may provide additional improvement.
6. RELATED WORK
Prior work characterizes adversarial samples as test-time attacks that can generalize across learning algorithms. Research also links their existence to neural-network training and component linearity, while proposing defensive alternatives with trade-offs.
- Adversarial samples are test-time attacks, distinct from causative attacks that influence training data.
- Previous studies found that adversarial samples can target different algorithms and often transfer between machine-learning techniques.
- Deep contractive networks were proposed to address adversarial samples, but they are harder to optimize and have reduced capacity.
- A linear explanation attributes adversarial samples to the linearity of neural-network components rather than overall model nonlinearity.
- Adversarial samples also generalize to linear models such as logistic regression, whereas neural networks can be hardened against them.
7. CONCLUSION AND FUTURE WORK
The evaluation shows that adversarial crafting threatens malware classifiers despite their strong classification performance. Among examined defenses, feature reduction weakens robustness, distillation helps only modestly, and adversarial retraining consistently reduces misclassification across architectures; broader defensive and domain studies remain future work.
- Conclusion: Misclassification rates reached up to 80% against neural-network malware classifiers performing on par with state-of-the-art classifiers.
- Conclusion: Feature reduction generally made neural networks weaker against adversarial crafting.
- Conclusion: Distillation improved misclassification rates, but less strongly than reported in computer-vision settings.
- Conclusion: Adversarial retraining consistently reduced misclassification rates across architectures.
- Future Work: Future work should examine potentially helpful defenses, including adversarial loss functions, more carefully in security-relevant domains.
- Future Work: The applicability of adversarial crafting attacks to additional domains also remains to be studied.