PaperlayerView exact revision on arXiv

Source-linked AI summary

Exploring Adversarial Examples in Malware Detection

Octavian Suciu, Scott E. Coull, Jeffrey Johns

arXiv:1810.08280v3cs.LGcs.CRstat.ML

TL;DR

The paper asks how robust CNN-based malware detectors are to adversarial examples when binary semantics constrain input changes and prior evaluations use limited datasets. It evaluates attacks across Mini, EMBER, and Full settings, finding that small-dataset effectiveness does not always generalize, while MalConv’s architecture enables append-based attacks. The study also examines attack trade-offs and single-step transferability.

  • Problem

    Binary semantics restrict arbitrary perturbations, while limited datasets and robust public models leave adversarial-attack generalization insufficiently evaluated.

  • Method

    The paper evaluates adversarial attacks against MalConv using Mini, EMBER, and Full datasets, including a production-scale dataset with 12.5 million training samples.

  • Results

    Attack effectiveness on small-dataset models does not always generalize: FGM Append reaches up to 33% SR on EMBER and 71% SR on Full, but fails on Mini.

  • Takeaways & Limitations

    MalConv’s lack of positional information makes it vulnerable to append-based attacks, which represent an alternative threat to runtime packing.

  • Takeaways & Limitations

    Append-based attacks cannot be applied to files exceeding MalConv’s 2MB maximum size, and only 128 of 4,195 sequences can be selected.

Abstract

from arXiv · show

The convolutional neural network (CNN) architecture is increasingly being applied to new domains, such as malware detection, where it is able to learn malicious behavior from raw bytes extracted from executables. These architectures reach impressive performance with no feature engineering effort involved, but their robustness against active attackers is yet to be understood. Such malware detectors could face a new attack vector in the form of adversarial interference with the classification model. Existing evasion attacks intended to cause misclassification on test-time instances, which have been extensively studied for image classifiers, are not applicable because of the input semantics that prevents arbitrary changes to the binaries. This paper explores the area of adversarial examples for malware detection. By training an existing model on a production-scale dataset, we show that some previous attacks are less effective than initially reported, while simultaneously highlighting architectural weaknesses that facilitate new attack strategies for malware classification. Finally, we explore how generalizable different attack strategies are, the trade-offs when aiming to increase their effectiveness, and the transferability of single-step attacks.

I. INTRODUCTION

The paper examines adversarial examples against CNN-based malware detectors, where binary semantics constrain existing image-oriented attacks. It evaluates attack generalization, architectural weaknesses, and transferability using production-scale training data.

  • Motivation: Binary structure prevents arbitrary input perturbations because changing bytes can break executable functionality.Adjacent bytes are structurally interdependent, constraining evasion attacks.
  • Motivation: Limited representative datasets and robust public models leave attack generalization across production-scale malware detectors unevaluated.Prior attacks used victim models trained on very small datasets and made assumptions about attacker strategies.
  • Contributions: Training on 12.5 million binaries exposes that attack effectiveness can be misestimated when small datasets are used.The paper also reports that single-step attacks are more effective against robust models trained on larger datasets.
  • Contributions: The study measures adversarial-attack generalization across datasets and highlights common properties and trade-offs between strategies.This directly addresses how attack behavior changes across training settings.
  • Contributions: The paper identifies an architectural weakness in a published CNN architecture that facilitates existing attack strategies.It also investigates transferability of single-step examples across models trained on different datasets.

II. BACKGROUND

CNNs use convolution, pooling, and fully connected layers to transform input features into nonlinear classifications. Their success in vision has supported adoption in other domains.

  • CNN Adoption: CNNs have achieved substantial success in vision tasks such as image classification.This success contributed to their adoption in other fields and domains.
  • CNN Architecture: CNNs combine convolutional layers, pooling layers, and fully connected layers in a staged classification architecture.Convolutions select features, pooling downsamples the feature space, and fully connected layers classify the resulting representation.

A. CNNs for Malware Classification.

MalConv applies CNNs directly to raw bytes from Portable Executable files, using fixed-length embedded byte sequences. Malware evasion must preserve executable syntax and semantics, motivating append-based attacks.

  • CNNs for Malware Classification: Raw-byte malware classification represents executable files as byte sequences that can encode higher-level instructions or functions.This work focuses on raw byte representations rather than disassembled functions.
  • MalConv: MalConv reads up to 2MB from a PE file, pads smaller files with a distinguished token, truncates larger files, and maps bytes to 8-dimensional embeddings.The embeddings are passed through a gated convolutional layer.
  • Attack Constraints: Raw-byte evasion attacks must maintain PE-file syntactic and semantic fidelity because arbitrary changes can disrupt malicious functionality.PE files contain structured headers and variable-length code and data sections.
  • Append-Based Attacks: Appending adversarial bytes outside PE-defined boundaries avoids direct functional changes, but original-sample features can offset their classification impact.This creates a trade-off between preserving functionality and influencing the detector.
  • Datasets: The study evaluates evasion attacks on Full, EMBER, and Mini datasets, including a production-quality corpus with 12.5 million training samples.The Full dataset contains 3.8 million testing samples and uses stratified sampling by malware family.

III. BASELINE PERFORMANCE

The paper establishes its experimental setup by training and validating MalConv models on Mini and Full datasets before presenting attack strategies and their trade-offs.

  • Baseline Performance: MalConv classifiers are trained on both Mini and Full datasets and evaluated on the entire Full testing set.The Mini model trains for 10 epochs, while Full training stops early because of a small validation loss.
  • Baseline Performance: The study then introduces the attack strategies used throughout the evaluation and discusses their trade-offs.This frames the subsequent experiments as comparisons among evasion strategies.

A. Append Attacks

Append attacks preserve PE-file semantics by adding bytes after the original binary, using random, gradient-guided, benign, or one-shot FGM strategies. Their effectiveness depends on how appended bytes influence the classifier’s temporal max-pooling behavior.

  • A. Append Attacks: Append-based attacks add adversarial bytes after the original file, avoiding changes to defined PE-file contents.
  • A. Append Attacks: Random Append samples appended bytes uniformly, providing a baseline for measuring gains from more complex strategies.
  • A. Append Attacks: Gradient Append updates appended bytes using input gradients over multiple iterations until evasion or the iteration limit.
  • A. Append Attacks: Benign Append reuses byte sequences from benign-file beginnings, targeting MalConv’s temporal max-pooling susceptibility.
  • A. Append Attacks: FGM Append performs a one-shot embedding-space update of appended bytes, reducing the convergence cost associated with iterative Gradient Append.

B. Limitations of Append Strategies

Append attacks are constrained by MalConv’s fixed input size and selective pooling: some files cannot accept appended bytes, while many original discriminative features remain influential.

  • B. Limitations of Append Strategies: 13% of candidate instances cannot be attacked because their files exceed MalConv’s 2MB maximum input size.
  • B. Limitations of Append Strategies: MalConv selects only 128 of up to 4,195 non-overlapping 500-byte sequences, limiting how appended bytes can affect classification.
  • B. Limitations of Append Strategies: The first 1,000 sequences account for 79% of classifier features, although only 55% of files are shorter than 0.5MB.

C. Slack Attacks

Slack attacks modify bytes in compiler-inserted regions that can be changed without breaking the PE file, then apply gradient-based updates in embedding space. The supplied passages describe their construction but provide no attack results.

  • C. Slack Attacks: Slack FGM identifies modifiable slack bytes and replaces their mapped values using a gradient-based embedding update.
  • C. Slack Attacks: Slack regions are extracted from gaps between neighboring PE sections where RawSize exceeds VirtualSize.
  • C. Slack Attacks: Updated embeddings for allowed byte indexes are mapped back to byte values using the L2 distance metric.

V. RESULTS

The evaluation compares attack generalization, robust-model vulnerability, slack versus append strategies, and single-step transferability across models trained on Mini, EMBER, and Full datasets. Success is measured by the percentage of adversarial samples that evade detection.

  • V. RESULTS: The evaluation uses Mini, EMBER, and Full datasets to test how attacks generalize to classifiers trained on larger datasets.
  • V. RESULTS: The study tests the vulnerability of a robust MalConv architecture to adversarial samples.
  • V. RESULTS: The comparison asks whether slack-based attacks outperform append-based attacks.
  • V. RESULTS: Single-step adversarial samples are evaluated for transferability across models.
  • V. RESULTS: Success Rate is the percentage of 400 randomly selected, correctly classified malware candidates whose adversarial samples successfully evade detection.

A. Append Attacks.

Append attacks show that results from small datasets do not reliably generalize, while MalConv’s architecture remains vulnerable when attackers can append enough bytes.

  • Random Append fails on Mini, EMBER, and Full regardless of appended-byte count, contradicting previously reported success rates of up to 15%.The result suggests MalConv is immune to random noise and does not learn input size as a feature.
  • FGM Append reaches 33% SR on EMBER and 71% on Full but fails on Mini, whereas Benign Append increases only on Mini.The Mini model’s severe overfitting and poor generalizability help explain this reversal across datasets.
  • MalConv’s 500-byte-stride convolutions and global max pooling let appended activations replace legitimate features, exposing a positional-information weakness.The model selects its best 128 activation locations irrespective of location, enabling sufficiently large append regions to dominate classification.
  • The Gradient Append implementation reproduces none of the reported successes, with repeated byte updates empirically showing oscillation across iterations.Different byte positions may converge at different rates, so unnecessary successive updates can cause divergence.
  • Slack FGM evaluation on EMBER and Full is presented separately in Figure 3.The figure caption identifies the EMBER and Full models as the evaluation targets.

B. Slack Attacks.

Slack FGM modifies existing slack bytes within an embedding-space constraint and can improve the modification-success trade-off by exploiting contextual bytes.

  • Slack FGM achieves up to 15% SR on EMBER when modifying 14% of slack bytes and 28% SR on Full when modifying 58%.These correspond to 291/2103 modified bytes on EMBER and 1117/1930 on Full on average.
  • Slack FGM varies epsilon to control which slack-byte updates remain within an L2 embedding-space ball.Values outside the ball leave the original byte unchanged, so epsilon controls the percentage of modified slack bytes.
  • On EMBER, Slack FGM modifies 291 bytes for a success rate requiring approximately 500 FGM Append modifications.On Full, Slack FGM reaches 27% SR with 1005 modified bytes, while FGM Append is around 20% at the same setting.
  • Single-step attack samples transfer poorly: at most 2/400 evade the target model in each EMBER–Full direction.The paper contrasts this with prior image-classification observations and leaves stronger iterative transferability analysis for future work.

VI. RELATED WORK

Prior work established evasion attacks against machine-learning malware detectors, while this paper examines append and slack strategies on production-scale MalConv models and studies single-step transferability.

  • Barreno et al. systematized machine-learning attack vectors and distinguished evasion as a test-time attack.
  • Many malware-detector attacks use additive techniques that add capabilities or features to evade detection.
  • This paper tests trade-offs between append and slack strategies and evaluates MalConv robustness when trained on production-scale datasets.
  • The paper studies single-step adversarial transferability on malware, an area not previously examined according to the authors.Prior work suggested single-step attacks could suit black-box attacks.
  • The conclusion reports that small-dataset attack effectiveness does not always generalize, while MalConv remains vulnerable to append attacks.The authors frame adversarial examples as an alternative to evasion techniques such as runtime packing.
Loading 1810.08280v3…