Adversary Resistant Deep Neural Networks with an Application to Malware Detection

Qinglong Wang; Wenbo Guo; Kaixuan Zhang; Alexander G. Ororbia; Xinyu Xing; C. Lee Giles; Xue Liu

Source-linked AI summary

Adversary Resistant Deep Neural Networks with an Application to Malware Detection

Qinglong Wang, Wenbo Guo, Kaixuan Zhang, Alexander G. Ororbia, Xinyu Xing, C. Lee Giles, Xue Liu

arXiv:1610.01239v4Oct 5, 2016cs.LG

TL;DR

DNN malware detectors are vulnerable to adversarial samples, while prior defenses have limited theoretical support. The paper introduces random feature nullification during training and testing to obstruct impactful adversarial samples. Experiments report stronger resistance with negligible accuracy sacrifice across malware and image-recognition datasets.

Problem
DNN malware detectors can be bypassed by adversarial samples, while existing defenses rely on restrictive assumptions and lack theoretical robustness guarantees.
Method
Random Feature Nullification randomly nullifies features during DNN training and testing, making model processing nondeterministic and reducing attackers’ ability to exploit critical features.
Results
The method significantly improves resistance to adversarial samples with negligible accuracy sacrifice on malware data and demonstrates cross-data generality in image recognition experiments.
Takeaways & Limitations
The proposed technique provides a theoretically guaranteed, minimally modifying approach for improving DNN resistance across malware detection and other deep-learning applications.
Takeaways & Limitations
Data-augmentation defenses cannot exhaustively cover the potentially infinite adversarial space, motivating the paper’s model-complexity-based alternative.

Abstract

from arXiv · show

Beyond its highly publicized victories in Go, there have been numerous successful applications of deep learning in information retrieval, computer vision and speech recognition. In cybersecurity, an increasing number of companies have become excited about the potential of deep learning, and have started to use it for various security incidents, the most popular being malware detection. These companies assert that deep learning (DL) could help turn the tide in the battle against malware infections. However, deep neural networks (DNNs) are vulnerable to adversarial samples, a flaw that plagues most if not all statistical learning models. Recent research has demonstrated that those with malicious intent can easily circumvent deep learning-powered malware detection by exploiting this flaw. In order to address this problem, previous work has developed various defense mechanisms that either augmenting training data or enhance model's complexity. However, after a thorough analysis of the fundamental flaw in DNNs, we discover that the effectiveness of current defenses is limited and, more importantly, cannot provide theoretical guarantees as to their robustness against adversarial sampled-based attacks. As such, we propose a new adversary resistant technique that obstructs attackers from constructing impactful adversarial samples by randomly nullifying features within samples. In this work, we evaluate our proposed technique against a real world dataset with 14,679 malware variants and 17,399 benign programs. We theoretically validate the robustness of our technique, and empirically show that our technique significantly boosts DNN robustness to adversarial samples while maintaining high accuracy in classification. To demonstrate the general applicability of our proposed method, we also conduct experiments using the MNIST and CIFAR-10 datasets, generally used in image recognition research.

1 INTRODUCTION

Deep learning offers promising malware-detection capabilities but remains vulnerable to adversarial manipulation, while prior defenses lack reliable theoretical guarantees. The paper proposes random feature nullification to make DNN processing nondeterministic and reduce the effectiveness of adversarial samples across applications.

One million malware variants reportedly reach the Internet daily undetected by many common cybersecurity technologies.
DNN-based malware detectors can recognize complex patterns and potentially detect heavily mutated malware, but adversaries can exploit feature importance to craft misclassifying samples.
Malware authors have already bypassed DNN detectors by exploiting feature amplitude inequilibrium, demonstrating that adversarial vulnerability is operationally relevant.
Existing defenses rely on assumptions that often fail and provide empirical rather than theoretical robustness guarantees, a serious concern for malware detection.
Random feature nullification during training and testing makes the model nondeterministic, reducing attackers’ ability to identify critical features and weakening constructed adversarial samples.
The approach aims to preserve classification performance, require minimal architectural changes, provide theoretical resistance guarantees, and generalize beyond malware detection.

2 BACKGROUND

Adversarial samples exploit sensitive directions in a DNN’s high-dimensional input space, often causing incorrect predictions after small perturbations. The background defines their generation through gradient-based perturbations and explains why such manipulations must remain small.

Adversarial samples are perturbations that can make a DNN assign an input to an incorrect, sometimes unreasonable, class.
DNNs trained for the same task can share adversarial vulnerabilities, allowing samples generated for one model to fool another model.
Adversarial perturbations follow input-gradient directions where small changes strongly affect the cost function and model prediction.
The fast gradient sign method computes δX by multiplying the sign of the loss gradient with coefficient ϕ, which controls perturbation scale.
Perturbations must remain small because larger changes visibly distort real samples and make manipulation easier to detect.

3 RELATED WORK

The paper reviews data augmentation and model-complexity defenses against adversarial samples, emphasizing their coverage limits and vulnerability to approximation-based attacks. It then motivates a random feature-nullification DNN designed to resist adversarial samples generated from an attacker’s best approximation.

Adversarial defenses primarily augment training data or increase DNN model complexity.
3.1 Data Augmentation: Data augmentation expands training sets with potential blind spots, including through adversarial training.Data augmentation artificially expands datasets; adversarial training incorporates generated adversarial samples into learning.
3.1 Data Augmentation: An effectively infinite adversarial space prevents data augmentation from exhaustively covering all possible blind spots.
3.1 Data Augmentation: Adversarial training can defend against efficiently generated attacks but remains vulnerable because attackers can also generate adversarial samples efficiently.
3.2 Enhancing Model Complexity: Defensive distillation reduces sensitivity to adversarial samples but can be approximated by training other DNNs with similar functionality and performance.
3.2 Enhancing Model Complexity: The proposed adversary-resistant DNN targets both discovery of blind spots and adversarial samples generated from the model’s best approximation.

4 RANDOM FEATURE NULLIFICATION

Random feature nullification inserts stochastic masking between inputs and the DNN during training and testing, disrupting attackers’ ability to compute effective adversarial perturbations. The approach is analyzed theoretically as a lightweight alternative to regularization-based defenses.

Model description: The method adds a stochastic input layer that randomly masks features before they enter a standard DNN during training and testing.The masked input is formed by element-wise multiplication with a binary mask; training fixes each sampled mask during forward and backward propagation.
Model description: Each sample receives a binary mask with randomly distributed zeros, while the nullification rate is sampled from a Gaussian distribution during training.The number of masked features is determined by the sampled rate and the feature dimension.
Model description: At testing time, the method uses the Gaussian distribution’s expected nullification rate to obtain stable results after model parameters are fixed.The resulting mask follows the same construction procedure used during training, with p set to µ_p.
Resistance analysis: Randomness obstructs adversaries from computing the input derivative needed to construct an effective adversarial perturbation.Although the loss derivative with respect to the masked input can be backpropagated, the random mask in the input derivative prevents exact computation.
Resistance analysis: Even when an adversary approximates the mask and crafts a perturbation, the perturbation is further distorted when the synthesized sample passes through the model’s independently generated nullification layer.The effective perturbation becomes the interaction of the adversary’s estimated mask and the model’s random mask, rather than the original strongest perturbation.
Comparison with existing defenses: Unlike regularization-based defenses, the proposed approach is presented as a model-complexity method that adds few parameters and theoretically guarantees resistance to adversarial samples.The paper contrasts this with prior methods that impose limited penalties and may remain vulnerable when attackers adjust perturbation scale.

5 EVALUATION

The evaluation compares random feature nullification (RFN) with adversarial training and dropout across malware and image datasets, measuring classification accuracy and resistance to adversarial samples. RFN substantially improves resistance while generally retaining high accuracy, with trade-offs depending on nullification rate and dataset.

Experimental design: The experiments compare RFN, adversarial training, dropout, and their combinations using classification accuracy and resistance to adversarial samples.Adversarial samples are generated from models trained with the corresponding defense techniques across malware, MNIST, and CIFAR-10.
Adversarial sample construction: Malware adversarial perturbations use l0 distance because malware features are discrete and finite-valued.The samples represent events such as file-system accesses and system calls, making continuous image-style perturbations inappropriate.
Malware classification results: 93.66% average accuracy was achieved across malware nullification rates, while the highest accuracy was 95.22% at a 10% nullification rate.Accuracy generally decreases as more features are nullified, although it remains surprisingly high at a 90% rate.
Malware classification results: 90% nullification produced maximum malware resistance, revealing a trade-off between classification accuracy and resistance to adversarial samples.Higher nullification rates discard more carefully manipulated features, whereas standard dropout and adversarial training offered poor general resistance to specifically crafted samples.
Malware classification results: RFN improved malware resistance by more than 100% relative to a standard DNN, and combining RFN with adversarial training further improved resistance with near-negligible accuracy degradation.The hybrid method outperformed standalone RFN, while both RFN approaches exceeded adversarial training and dropout in malware resistance.
Image recognition results: 50% nullification maximized image-sample resistance while maintaining roughly similar classification performance across nullification rates.This rate was adopted for subsequent MNIST and CIFAR-10 experiments, and RFN was evaluated with both fully connected DNNs and CNNs.
Image recognition results: 91.28 resistance on MNIST and 74.12% on CIFAR-10 were achieved by combining RFN with adversarial training.Standard-DNN adversarial samples reduced accuracy to as low as 0.01% on MNIST and 10.68% on CIFAR-10; RFN-based defenses yielded the strongest resistance.

6 CONCLUSION

The paper presents Random Feature Nullification as a simple way to construct DNNs robust to adversarial samples. Experiments show improved malware-model resistance with negligible accuracy sacrifice, alongside cross-data generality in image recognition.

Random Feature Nullification constructs DNN models robust to adversarial samples.The authors state that specifically designed adversarial samples cannot force the resulting DNN to misclassify inputs.
Specifically designed adversarial samples cannot force the proposed DNN to misclassify its inputs.
The method significantly improves malware-model resistance with only negligible accuracy sacrifice compared with other defense mechanisms.
Experiments in image recognition demonstrate cross-data generality, while broader application coverage remains future work.

7 APPENDIX

The appendix reports hyperparameters and model-structure descriptions for the experimental models. It covers MNIST, malware, and CIFAR-10 evaluations, including the use of CNNs for CIFAR-10.

The appendix provides the hyperparameters used for model training in the experimental setup.
MNIST model descriptions specify hidden-layer structure, hidden units, final Softmax non-linearity, and the unregularized standard-DNN baseline.
A dedicated table reports the hyperparameters of the malware models.
CIFAR-10 evaluation uses CNNs instead of standard DNNs, with convolutional and fully connected activation functions described across the appendix tables.