PaperlayerView exact revision on arXiv

Source-linked AI summary

Android HIV: A Study of Repackaging Malware for Evading Machine-Learning Detection

Xiao Chen, Chaoran Li, Derui Wang, Sheng Wen, Jun Zhang, Surya Nepal, Yang Xiang, Kui Ren

arXiv:1808.04218v4cs.CR

TL;DR

Android malware detectors based on machine learning are vulnerable to adversarial examples, while earlier attacks largely targeted syntactic features or modified only application manifests. This paper introduces an automated method that inserts optimized perturbations directly into APK Dalvik bytecode and evaluates it against MaMaDroid and Drebin, reducing reported detection rates from 96% and 97% to 0%.

  • Problem

    Android machine-learning malware detectors are vulnerable to adversarial examples, while existing attacks mainly target syntactic features and modify application manifests rather than semantic Dalvik-bytecode features.

  • Method

    The paper uses a DNN substitute model to calculate APK-compatible perturbations and an automated tool to insert them into malware source files, including smali code, before rebuilding the APK.

  • Results

    Detection rates decreased from 96% to 0% in MaMaDroid and from 97% to 0% in Drebin using a small number of inserted codes.

  • Takeaways & Limitations

    The results show that adversarial examples can evade Android malware detectors using semantic or syntactic features when perturbations are implemented directly in APK bytecode.

Abstract

from arXiv · show

Machine learning based solutions have been successfully employed for automatic detection of malware on Android. However, machine learning models lack robustness to adversarial examples, which are crafted by adding carefully chosen perturbations to the normal inputs. So far, the adversarial examples can only deceive detectors that rely on syntactic features (e.g., requested permissions, API calls, etc), and the perturbations can only be implemented by simply modifying application's manifest. While recent Android malware detectors rely more on semantic features from Dalvik bytecode rather than manifest, existing attacking/defending methods are no longer effective. In this paper, we introduce a new attacking method that generates adversarial examples of Android malware and evades being detected by the current models. To this end, we propose a method of applying optimal perturbations onto Android APK that can successfully deceive the machine learning detectors. We develop an automated tool to generate the adversarial examples without human intervention. In contrast to existing works, the adversarial examples crafted by our method can also deceive recent machine learning based detectors that rely on semantic features such as control-flow-graph. The perturbations can also be implemented directly onto APK's Dalvik bytecode rather than Android manifest to evade from recent detectors. We demonstrate our attack on two state-of-the-art Android malware detection schemes, MaMaDroid and Drebin. Our results show that the malware detection rates decreased from 96% to 0% in MaMaDroid, and from 97% to 0% in Drebin, with just a small number of codes to be inserted into the APK.

I. INTRODUCTION

Android malware detectors increasingly use machine learning, but adversarial examples expose weaknesses in models relying on syntactic or semantic APK features. This paper proposes automated APK-level perturbations targeting both detector types.

  • Motivation: Machine-learning Android malware detectors are vulnerable to adversarial examples crafted with carefully designed perturbations.These perturbations can force false predictions by altering inputs presented to the models.
  • Research gap: Existing attacks mainly target syntactic features and modify AndroidManifest.xml, limiting their effectiveness against detectors using semantic Dalvik-bytecode features.The paper identifies this limitation as a gap addressed by its attack method.
  • Contribution: The paper proposes an automated method for crafting adversarial examples against recent Android malware detectors, including Drebin and MaMaDroid.The method is designed to target detectors that collect syntactic or semantic features from Dalvik bytecode.
  • Contribution: The tool calculates perturbations, modifies source files, and rebuilds APKs, adding perturbations directly to classes.dex rather than only to the manifest.This implementation targets the APK components from which detector features are extracted.
  • Evaluation: 96% to 0% in MaMaDroid and 97% to 0% in Drebin, using 5,879 malware samples and a small distortion.The evaluation used the same datasets as the two target detectors.

III. TARGETED SYSTEMS AND ATTACK SCENARIOS

MaMaDroid extracts behavioral features from API-call control-flow graphs, abstracts calls at family or package level, and trains classifiers on Markov-chain transition probabilities. The paper evaluates attacks against both operating modes.

  • A. MaMaDroid: API calls are abstracted to family or package level, with family mode more lightweight and package mode more fine-grained.The system recognizes nine families and 338 documented packages, plus categories for developer-defined and obfuscated packages.
  • A. MaMaDroid: MaMaDroid extracts API-call sequences from each application's control-flow graph to capture behavioral models.Its features use API-call sequences rather than the frequency or presence of individual APIs.
  • A. MaMaDroid: MaMaDroid converts the abstracted call graph into a Markov chain whose transition probabilities form the feature vector for classifier training.The extraction pipeline proceeds from decompiled code to call graph, abstracted call graph, and Markov chain.
  • A. MaMaDroid: RF, KNN, and SVM classifiers are trained and tested on datasets collected over different time periods; RF reaches F-measures of 0.98 and 0.99 in family and package modes.The reported values correspond to family mode and package mode, respectively.

B. Drebin

Drebin is a lightweight on-device detector that linearly scans manifest and disassembled bytecode for string-based features. Its binary feature representation supports iterative feature-injection attacks.

  • B. Drebin: Drebin is an on-device lightweight Android malware detector using features extracted from both the manifest and disassembled dexcode.It scans the manifest and smali files linearly and represents permissions, activities, and API calls as strings.
  • B. Drebin: Drebin's extracted features form a binary multidimensional vector, with each dimension indicating feature presence or absence.The detector retrieves eight feature sets from application files.
  • B. Drebin: Drebin trains a linear SVM on 5,560 malware samples and 123,453 benign applications collected between August 2010 and October 2012.The dataset supports evaluation of the detector's classification performance.

C. Attack Scenarios

The attack considers different levels of adversary knowledge and adapts C&W and JSMA to optimize viable API-call injections against MaMaDroid.

  • Attack Scenarios: Four scenarios vary whether the adversary knows the feature set, training set, and target detector outputs.Black-box querying reveals only the predicted label, while training-set access provides a copy that cannot be modified.
  • Attack Scenarios: The attack assumes feature-set knowledge and may use either a substitute model or MaMaDroid as a pilot classifier.The pilot determines whether an example has been misclassified during optimization.
  • Refined C&W: C&W minimizes feature distortion while maximizing an adversarial loss that targets misclassification into the benign class.The perturbation is constrained to valid feature ranges and normalized within each family or package group.
  • Refined C&W: The attack iteratively updates features until misclassification while enforcing group-wise normalization after each update.ReLu clipping automatically enforces non-negative API-call changes; normalization maintains the group-sum constraint.
  • Refined C&W: The optimization changes probability perturbations into non-negative API-call increments so the attack can be implemented in smali code.The objective minimizes the L2 norm of API-call changes, thereby limiting total code injections.

2) Refined 𝐽𝑆𝑀𝐴: JSMA

The refined JSMA uses forward derivatives to identify API-call features whose perturbation most effectively promotes misclassification.

  • JSMA: JSMA calculates a Jacobian linking input features to classifier outputs and uses it to guide iterative perturbations.For MaMaDroid, the relevant relationship links API-call numbers to model outputs through the input probabilities.
  • JSMA: The Jacobian indexes input features and output classes, with API-call numbers corresponding to the input features.The ground-truth output is reduced while outputs for other classes are increased during crafting.
  • JSMA: The saliency map scores features according to their derivative-based suitability for changing the classifier decision.At each iteration, the attack selects the API call with the highest saliency value.
  • JSMA: The selected API-call count increases by a small amount until misclassification or the maximum allowed change is reached.The maximum change is controlled by γ, while θ represents the per-iteration increment.

B. APK Manipulation

The APK manipulation process automatically inserts non-functional Dalvik bytecode that changes detector features while preserving the original application’s functionality.

  • Design Goals: The modification is designed to preserve original functionality and require no additional human effort.Scripts automate the APK modification process.
  • Manipulation Strategies: The strategy adds caller-to-callee API calls to alter MaMaDroid’s control-flow-derived feature values.MaMaDroid represents transitions between API families or packages as probabilities, which can be changed by inserting calls.
  • Manipulation Strategies: Arbitrarily many calls can be inserted by repeatedly invoking a callee from a caller.The example demonstrates adding Android-to-Android calls through self-defined methods.
  • Automated Insertion: The script injects self-defined classes and invokes caller methods from the entry activity’s onCreate() method.The entry activity is located through the android.intent.action.MAIN manifest entry, and insertion is performed on smali code.
  • Limitations: The process preserves functionality but modifies MaMaDroid’s extracted control-flow graph and feature values.Simple manipulation can potentially be countered by filtering API calls outside the standard Android SDK.
  • Limitations: Sophisticated manipulation inserts system-provided non-functional calls to bypass the white-list filter.Calls such as Log.d() can alter MaMaDroid’s abstraction of self-defined or obfuscated code.

C. Experiment Settings

The experiments evaluate whether crafted malware evades MaMaDroid and whether the APK modifications can be applied easily, using original and surrogate datasets.

  • Evaluation Goals: The evaluation asks whether modified malware evades the target detector and whether modification can be applied easily to the original APK.MaMaDroid was rebuilt from the authors’ published source code for the experiments.
  • Datasets: The original MaMaDroid datasets contain 5,879 benign applications and 5,560 malware samples.The benign samples came from PlayDrone, while the malware samples were initially used in Drebin.
  • Datasets: The surrogate dataset contains 4,560 malware samples and 5,879 benign applications selected from VirusShare and APKPure.The selection equalized the number of training samples relative to the original and surrogate settings.
  • Evaluation Settings: The experiments distinguish settings using the original dataset from settings using a surrogate dataset.These settings support evaluation under differing adversary knowledge conditions.

2) Experiment Work Flow:

The workflow decompiles malware, optimizes feature perturbations, inserts corresponding code into smali files, rebuilds APKs, and evaluates evasion across classifiers, scenarios, and distortion.

  • Experiment Work Flow:: Malicious APKs are decompiled, feature vectors are constructed, perturbations are optimized, code blocks are inserted into smali files, and modified APKs are rebuilt.The manipulated APK is submitted to the MaMaDroid oracle to determine whether the attack succeeds.
  • Experiment Work Flow:: The experiments use RF, SVM, KNN, and DNN detectors, with DNN F-measures of 0.92 in family mode and 0.95 in package mode.The DNN has five hidden layers, while MaMaDroid is used as the evaluation benchmark.
  • Experiment Work Flow:: Evasion rate measures malware misclassified as benign, while distortion measures the number of API calls added to each malware sample.These metrics jointly capture attack effectiveness and perturbation size.
  • Experiment Results: Across four scenarios and four machine-learning algorithms, the attack increased RF family-mode evasion from 4% to 56%-99% and package-mode evasion from 3% to 58%-99%.The results cover two attack algorithms and report pre-attack evasion as a baseline.
  • Experiment Results: In scenario F, JSMA achieved evasion rates of 62%, 75%, 58%, and 91% for RF, SVM, 3-NN, and DNN, respectively.Scenario F gives the adversary knowledge of the feature set only.
  • Experiment Results: Attack effectiveness generally improved with greater target-system knowledge, but C&W benefited more from black-box access whereas JSMA benefited more from training-set access.Training-set possession had little impact on C&W evasion, while black-box access was less important for JSMA.

3) Evaluation results by operation modes:

The attack is more effective against MaMaDroid’s fine-grained package mode, but package-mode evasion requires substantially more inserted API calls than family mode. Across Drebin and MaMaDroid, attack effectiveness depends on algorithm, detector knowledge, and manipulation constraints.

  • MaMaDroid operation modes: 100% evasion was reached against MaMaDroid package mode with JSMA and RF in scenario FTB, compared with 89% in family mode.The package-mode result used Fig. 6(a), while the family-mode comparison used Fig. 4(a).
  • MaMaDroid operation modes: 17 API calls were added on average in family mode, versus 257 in package mode, to craft adversarial examples.The package mode is more fine-grained, whereas family mode is more lightweight.
  • Evaluation design: MaMaDroid evaluations compare JSMA and C&W across family and package modes using evasion rate and average distortion.The experiments cover four machine-learning algorithms and four real-world attacker-knowledge scenarios.
  • MaMaDroid operation modes: More than 15 times as many calls were required for successful evasion against fine-grained package-mode features than against abstracted family-mode features.Package mode slightly improves classification accuracy but is significantly more resistant to the attack.
  • Drebin attack: Drebin’s Jacobian-based attack changes one binary feature from 0 to 1 per iteration, selecting the most influential currently absent feature.The process stops when malware is misclassified or the maximum allowed change γ is reached.
  • Drebin attack: Drebin features can be added through dexcode by inserting code containing required strings without invoking or executing those calls.Drebin scans AndroidManifest.xml and smali files for feature-presence strings rather than checking whether calls execute.

C. Experiments & Evaluations

The experiments evaluate the proposed attack on Drebin and compare it with prior methods under matched attacker knowledge. The attack achieves high evasion with few added features in the strongest-knowledge Drebin scenario and outperforms the compared methods.

  • Experimental setup: The evaluation reports evasion rate and average distortion for the proposed Drebin attack using the dataset described in Section IV-C.Table III summarizes the average number of inserted features by feature set.
  • Drebin evaluation: 99% of Drebin malware samples were misclassified in scenario FTB after adding an average of 3.5 features per sample.FTB grants access to Drebin’s feature set, training set, and oracle output.
  • Drebin evaluation: 60% of Drebin adversarial malware examples evaded detection in scenario F, where the adversary had access only to the feature set.The results therefore vary with the attacker’s available knowledge.
  • Comparison with prior work: The proposed methods outperform Chen et al.’s poisoning attack and Grosse et al.’s white-box attack in evasion rate under the compared scenario.The comparison uses scenario FTB because the prior methods require access to the feature set, training set, and machine-learning model.
  • Detector selection: The study selects Drebin and MaMaDroid as representative detectors using syntactic and semantic features, respectively.The authors state that they could not demonstrate effectiveness on every machine-learning detector.
  • Scope of method: The framework is intended for detectors extracting features from either an application manifest or bytecode, with attack algorithms refined for target-feature constraints and interdependencies.MaMaDroid and Drebin serve as the demonstrated cases.

C. Artifacts in Our Attack

The paper examines artifacts introduced by APK manipulation and evaluates adversarial and ensemble defenses. Adversarial training improves F-measure, while ensemble defense is reported as effective when the attacker has limited knowledge.

  • APK artifacts: 17.9% of benign applications and 16.3% of malware samples contained more than 100 android.util.log() calls.For more than 50 calls, the percentages were 28.7% for benign applications and 40% for malware samples.
  • APK artifacts: The inserted no-op calls are comparable to call patterns already observed in benign and malicious applications in the experiment dataset.The authors report no specific indication that either class tends to contain more such calls than the other.
  • Adversarial training: Adversarial training adds crafted adversarial malware examples recursively to the training dataset to strengthen model robustness.The evaluation varies the percentage of adversarial malware samples added to training.
  • Adversarial training: Adding 1% adversarial examples increased benign and malicious F-measure from 69% to 80% and from 64% to 82.5%, respectively.With more adversarial examples, the respective F-measures increased further to 83% and 87%.
  • Ensemble defense: Ensemble learning trains multiple sub-classifiers on subsets of features or training samples and combines their decisions using a rule such as majority vote.The defense uses diversity across classifiers instead of one classifier trained on the full feature set and all samples.
  • Ensemble defense: The ensemble defense is effective against the C&W attack on MaMaDroid when the attacker has the least knowledge of the target system.The evaluation uses 10 classifiers, each trained with either 1/10 of the training samples or 1/10 of the features.

B. Android Malware Detection

The paper positions its attack against a gap in Android malware detection research: existing attacks were limited when detectors used semantic bytecode features. It presents direct APK bytecode manipulation and reports strong evasion against representative detectors, while leaving broader defense analysis for future work.

  • Detection landscape: Recent Android malware detectors increasingly rely on semantic features to model malware behavior.Examples include sensitive subgraph analysis, inter-component communication patterns, and malicious-modality detection.
  • Research gap: Existing attacking and defending methods were reported as ineffective when detectors relied more on semantic features.The paper contrasts these methods with detectors using features from APK classes.dex rather than only AndroidManifest.xml.
  • Proposed approach: The proposed method applies perturbations directly to the APK classes.dex file, producing adversarial examples intended to affect semantic-feature detectors.This extends beyond prior approaches that primarily modified manifest-based syntactic features.
  • Security motivation: Machine-learning detectors are vulnerable to adaptive adversaries because they were not originally designed to cope with input manipulation intended to mislead learning systems.The paper frames this vulnerability as a potential weak point in security systems.
  • Reported outcome: Android malware detection rates decreased from 96% to 0% in MaMaDroid and from 97% to 0% in Drebin after the proposed attack.MaMaDroid is characterized as semantic-feature based, while Drebin uses syntactic features and also collects some classes.dex features.
  • Scope boundary: The paper’s future work includes deeper analysis of defense mechanisms, attack modifications against those defenses, and comparisons of substitute-model architectures.Only a brief discussion of adversarial training and ensemble learning is presented in this paper.
Loading 1808.04218v4…