PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning for Android Malware Defenses: a Systematic Literature Review

Yue Liu, Chakkrit Tantithamthavorn, Li Li, Yepang Liu

arXiv:2103.05292v3cs.CRcs.LGcs.SE

TL;DR

Android malware defenses face pressure from evolving threats and limitations in manual and traditional machine-learning approaches, while prior reviews did not systematically cover the full range of deep-learning defense research. This paper systematically reviews 132 studies from 2014–2021 and finds that most focus on detection, while other forms of analysis are gaining attention.

  • Problem

    Earlier reviews did not systematically provide a comprehensive, current account of deep-learning Android malware defenses beyond detection, despite evolving malware and limits of manual and traditional machine-learning approaches.

    Source: §1 · p. 2

  • Method

    The authors systematically collect and synthesize 132 primary studies published between 2014 and 2021 across several themes in deep-learning Android malware defenses.

    Source: §1 · p. 3

  • Results

    Most reviewed studies focus on malware detection, while more detailed analysis of malicious apps is receiving increasing attention.

    Source: §6 · p. 28

  • Takeaways & Limitations

    The review identifies current issues and research trends to inform future work on deep-learning Android malware defenses.

    Source: §1 · p. 3

    Publication selection was conducted by one researcher, which the authors note may have caused studies to be missed.

    Source: §5 · p. 28

Abstract

from arXiv · show

Malicious applications (particularly those targeting the Android platform) pose a serious threat to developers and end-users. Numerous research efforts have been devoted to developing effective approaches to defend against Android malware. However, given the explosive growth of Android malware and the continuous advancement of malicious evasion technologies like obfuscation and reflection, Android malware defense approaches based on manual rules or traditional machine learning may not be effective. In recent years, a dominant research field called deep learning (DL), which provides a powerful feature abstraction ability, has demonstrated a compelling and promising performance in a variety of areas, like natural language processing and computer vision. To this end, employing deep learning techniques to thwart Android malware attacks has recently garnered considerable research attention. Yet, no systematic literature review focusing on deep learning approaches for Android Malware defenses exists. In this paper, we conducted a systematic literature review to search and analyze how deep learning approaches have been applied in the context of malware defenses in the Android environment. As a result, a total of 132 studies covering the period 2014-2021 were identified. Our investigation reveals that, while the majority of these sources mainly consider DL-based on Android malware detection, 53 primary studies (40.1 percent) design defense approaches based on other scenarios. This review also discusses research trends, research focuses, challenges, and future research directions in DL-based Android malware defenses.

1 INTRODUCTION

Android’s scale, evolving malware, and the limits of manual and traditional machine-learning defenses motivate a review of deep learning approaches. The paper addresses gaps in earlier reviews by systematically synthesizing research across multiple defense objectives and identifying trends and open issues.

1 INTRODUCTION

  • Android’s popularity, large app ecosystem, and malware that can bypass Google Play Protection make reliable defenses a pressing need.The introduction also notes risks from unofficial markets and malicious apps that can expose privacy or perform dangerous operations.Source: §1 · pp. 1–2
  • Manual analysis does not scale, while traditional machine-learning models struggle with complex representations and can become obsolete as apps and attack strategies evolve.The authors describe constructing robust and transparent defenses using traditional machine learning alone as non-trivial.Source: §1 · p. 2
  • Deep learning can learn representations automatically from raw data with little prior knowledge, prompting growing use in Android malware defenses and demand for an up-to-date survey.The paper notes that deep-learning tools were applied to Android malware defenses in 2014.Source: §1 · p. 2
  • Earlier reviews often focused on malware detection, omitted other defense objectives, lacked systematic methods, or predated recent research.The paper identifies unanswered questions about current approaches and future research needs.Source: §1 · p. 3
  • The authors systematically review 132 studies published from 2014 to 2021, synthesize research across several themes, and identify issues and potential trends.The synthesis covers research objectives, APK characterization, deep-learning techniques, deployment, and model evaluation.Source: §1 · p. 3

2 REVIEW METHODOLOGY

The authors follow a systematic review process to define research questions, search for studies, and select a final set for analysis. The selected literature spans 2014 through 2021, and its publication volume and venue mix indicate growing interest across several research domains.

2 REVIEW METHODOLOGY

  • The review follows five steps: planning a protocol, defining research questions, designing searches and exclusion criteria, extracting data, and synthesizing findings.The process is based on the methodology suggested by Kitchenham.Source: §2 · p. 3

2.1 Research Question

  • The research questions cover defense objectives, feature processing, deep-learning architectures, deployment, evaluation, and emerging trends.Source: §2.1 · p. 4

2.2 Search Strategy

  • The search uses five digital libraries, two citation engines, and one round of backward snowballing to find relevant studies.The libraries include IEEE, ACM Digital Library, Springer, Science Direct, and Wiley Online Library.Source: §2.2 · p. 4

2.3 Data Selection Process

  • After applying exclusion criteria and full-text quality appraisal, the authors selected 132 primary studies.Source: §2.3 · p. 4
  • Publications increased from 2014 through the review’s November 30, 2021 cutoff, and more than 35% of studies appeared in security venues.AI and software engineering/programming-languages venues each account for more than 10%; conference and journal shares are about 50% each.Source: §2.3 · pp. 4–5

3 RESULTS ANALYSIS

The results section answers the review questions through analyses of defense objectives, approaches, and trends. Figure 2 summarizes research objectives, while detailed information about individual studies is provided in supplementary material.

3 RESULTS ANALYSIS

  • The results are organized around research objectives, feature processing, architectures, deployment, evaluation, and research trends.Source: §3 · p. 5
  • Figure 2 summarizes primary studies by research objective, and its percentages can sum to more than 100% because some studies address multiple objectives.Source: Figure 2 · p. 6
  • A detailed table about each primary study is available in the online supplementary materials.Source: §3 · p. 6

3.1 Malware Defenses Objectives

The review classifies DL-based Android malware defenses into six objectives, with binary malware detection receiving the most attention. It also reports work on family attribution, fake apps, adversarial threats, malware evolution, and behavior analysis.

Malware Detection (Binary Classification)

  • 68% of reviewed sources address binary malware detection, the largest objective; Droid-Sec was the first DL-based Android malware detector.Droid-Sec’s results showed greater effectiveness than traditional machine-learning techniques such as SVM.Source: §3.1 · p. 6

Malware Family Attribution

  • 20 reviewed articles (15%) target family attribution; most classify existing families, while Qiu et al. address zero-day families.Source: §3.1 · p. 6

Repackaged/Fake App Detection

  • 5% of sources study repackaged/fake-app detection, including detectors proposed to locate counterfeit apps in application markets, [74].Source: §3.1 · p. 7

Adversarial Learning Attacks and Protections

  • 16 studies (12%) examine adversarial attacks and protections; 14 (87%) focus on evasion versus two on poisoning, and Chen et al. [25] report accuracy falling from over 90% to 0%.Source: §3.1 · p. 7

Malware Evolution Detection and Defense

  • Seven papers (5%), all published within the last three years, address malware evolution; Pendlebury et al. [119] report future-sample detection falling from almost 90% to below 30%.Retraining and active learning are used, but the cited models still rely on humans to decide when retraining should occur.Source: §3.1 · p. 7

Malicious Behavior Analysis

  • Six studies (5%) analyze malicious behavior, a difficult task amid diverse behaviors, code obfuscation, and dynamic payloads; some use recurrent networks to visualize potential risks,.Source: §3.1 · p. 7
  • Some studies compare risk permissions with app descriptions or icon widgets to help assess application security before installation or permission grants.Source: §3.1 · p. 8

Discussion

  • 53 primary studies address other objectives, but non-detection work remains relatively small and the six-category taxonomy does not exhaust the defense scope.Robustness, effectiveness, stability, and reliability remain open issues despite recent studies of practical scenarios.Source: §3.1 · p. 8

3.2 APK Characterization

APK characterization traces how studies extract features from apps, categorize them, and encode them for deep-learning models. The reviewed approaches span static, dynamic, and hybrid analysis, diverse feature types, and multiple encoding strategies.

3.2 APK Characterization

  • APK characterization follows a pipeline of program analysis, feature extraction, and feature encoding before the resulting representation is used by a deep-learning model.The review examines these mechanisms through program-analysis approaches, feature categories, and feature-encoding approaches.Source: §3.2 · pp. 8–9

3.2.1 Program analysis approaches.

  • 73% of studies use static analysis, 17% use dynamic analysis, and 10% use hybrid analysis combining both.Static analysis offers full code coverage at lower computational cost but is vulnerable to obfuscation; dynamic analysis observes runtime behavior but may have limited code coverage, while hybrid analysis is computationally intensive.Source: Figure 3 · p. 9, §3.2.1 · pp. 9–10

3.2.2 Feature categories.

  • API calls (55.3%) and permissions (51.5%) are the most frequently used feature types; 13 studies use program graphs and eight use app metadata.The review identifies 13 feature categories overall; 18 studies use Linux kernel system calls and 14 examine dynamic activity characteristics such as network access and memory dumps.Source: Figure 4 · p. 10, §3.2.2 · p. 10
  • Raw opcode sequences are the most frequent raw-code feature, used in 22 studies, while 13 studies feed raw classes.dex bytecode to deep neural models.Four studies convert disassembled code to Java source; one example maps bytecode hexadecimal values to RGB color values.Source: §3.2.2 · pp. 10–11

3.2.3 Feature encoding approaches.

  • Categorical encoding appears in 47% of sources (62 studies), and 55 of those studies use one-hot encoding to mark feature presence.Categorical encoding is simple but can produce high-dimensional vectors and represent distinct patterns in isolation.Source: Figure 5 · p. 11, §3.2.3 · p. 11
  • Text-based encoding is used by 26 studies (20%); approaches include BOW, TF-IDF, N-Gram, and pretrained Word2vec or GloVe embeddings.The review notes sparsity and high dimensionality issues for discrete approaches and high dimensionality for one-hot text encoding.Source: §3.2.3 · p. 11
  • Graph-based, image-based, and hybrid encoding are used by 15 studies (11%), 16 studies (12%), and 6% of studies, respectively.Graph approaches encode structures such as system-call or API-call graphs; image approaches commonly transform bytecode into grayscale or color images; hybrid approaches combine encodings, including similarity-based sequential features intended to alleviate obfuscation effects.Source: §3.2.3 · pp. 11–12
  • Feature lists are difficult to make robust and may miss malicious characteristics; malware evolution can also cause concept drift, while evolving obfuscation challenges static analysis.The review identifies effective practical feature lists and analysis of obfuscated apps as continuing research challenges.Source: §3.2.3 · p. 12

3.3 Deep Learning Techniques

The review finds supervised learning dominant, with MLPs, CNNs, and RNNs most common among varied architectures. Explainability is emerging, while data-labeling and computational demands remain important challenges.

3.3.1 Learning paradigms.

  • 81.8% of reviewed approaches use supervised learning, while only one study uses unsupervised learning alone and three use reinforcement learning.Twenty studies combine unsupervised and supervised scenarios, often using autoencoders to initialize weights before fine-tuning on labeled samples.Source: §3.3.1 · p. 13

3.3.2 Deep learning models.

  • MLPs appear in 45 studies (34.1%), CNNs in 41 (31.1%), and RNNs in 17 (12.9%), making them the most-used architectures.CNNs are used for opcode, bytecode, and API-call sequences; RNN variants such as LSTM and GRU are used for sequential properties.Source: §3.3.2 · p. 14
  • Hybrid models combine basic DNN blocks and occur in 11 studies (8.3%), while DRL appears in three studies, including work on evasion and protection strategies.One study uses a transformer-based model for sequential features in highly imbalanced malware data; two studies use heterogeneous graph representation learning, and one proposes an explainable attention-based detector.Source: §3.3.2 · p. 15
  • DL architectures shifted from early unsupervised AE and DBN models toward supervised MLP, CNN, and RNN models from 2017, with newer advanced approaches emerging later.The newer approaches include GAN, GNN, attention, transformer, DRL, and hybrid models.Source: §3.3.2 · pp. 14–15

3.3.3 Model explanation.

  • Black-box DL models provide insufficient transparency to explain why a sample is identified as malicious, hindering practical malware analysis.The review identifies reliable labeling as a challenge because antivirus labels can be inconsistent and complex behavior or evolved-malware labeling requires expertise.Source: §3.3.3 · pp. 15–16
  • Ten studies (7.5%) use interpretable DL techniques, and nine of those ten were published after 2019.Most use local explanations; model-agnostic tools are most frequent, while gradient-based and customized-attention approaches also appear.Source: §3.3.3 · p. 16
  • The review identifies computational efficiency as a growing need because Android files are large and complex, reverse-engineering feature processing is time-consuming, and DL frameworks require substantial resources.It also notes that training very deep networks on small datasets risks overfitting, while pre-trained models could ease repeated training from scratch.Source: §3.3.3 · p. 17
  • Future explanation work should cover richer features and source-code semantics, and develop evaluation methods that can assess and compare explanation quality and reliability.The review notes that current studies mainly use simple semantic APK features and that no effective evaluation system for explanations is available.Source: §3.3.3 · p. 17

3.4 Deployment of Analysis

Deployment is overwhelmingly off-device, while only a few studies place analysis on phones or distribute it across phones and servers. The review highlights privacy, communication, and real-time performance as constraints on practical deployment.

3.4 Deployment of Analysis

  • 93.2% of reviewed sources deploy their proposed tools off-device, generally training and evaluating on PCs or GPU servers.The review attributes this pattern to the substantial computational resources required for automated malware defenses at scale.Source: §3.4 · p. 18
  • Only two studies propose on-device approaches, which can analyze apps on the phone without sharing or uploading private data.MobiDroid and MobiTive migrate server-trained models to TensorFlow Lite for real-time mobile detection, reported as reliable and fast.Source: §3.4 · p. 18
  • Seven studies (5.3%) use distributed approaches, splitting on-device analysis from detailed, computationally expensive server analysis.The review identifies privacy risks when app data are uploaded, and says privacy protection for distributed DL-based approaches needs further study.Source: §3.4 · p. 18
  • The review describes model aging as inevitable for off-device approaches, which can leave them slow to respond to new malware and unable to protect users promptly.It identifies trade-offs involving communication overhead, detection performance, real-time demands, and privacy in distributed or on-device frameworks.Source: §3.4 · pp. 18–19

3.5 Performance Evaluation

The review finds substantial variation in datasets and evaluation practices, alongside frequent high reported accuracy and limited public tool availability. It cautions that outdated or imbalanced data can make evaluations less representative and reproducibility difficult.

3.5.1 Dataset.

  • 61 primary studies (46.2%) collected malware samples from Drebin, whose 2011–2014 data may be outdated and whose evaluations may not reflect detection capability on recent malware.The review reports increasing interest in AndroZoo and VirusShare for collecting recent malicious samples.Source: §3.5.1 · p. 20
  • The median evaluation set contains 9,945 goodware and 7,149 malware samples, and many studies use unrealistic class ratios.Nineteen studies use balanced 1:1 data; only seven with ratios of 5<a<=10 and six with a>10 appear to use more realistic goodware-to-malware ratios.Source: §3.5.1 · p. 20

3.5.2 Evaluation approaches.

  • 59 studies (44.6%) use k-fold cross-validation, 44 (33.3%) use holdout validation, and 14 (10.6%) use timestamp-based splits.Half compare against traditional ML approaches, and 33.3% compare against other DL approaches.Source: §3.5.2 · p. 20

3.5.3 Evaluation metrics.

  • 98% is the median reported accuracy/F1 value, and 25% of sources report accuracy/F1 values above 99%.The review notes that biased settings, including data imbalance and outdated data, have been identified as causes of overly optimistic results.Source: §3.5.3 · pp. 21–22

3.5.4 Availability.

  • Only 30 studies (22.7%) provide publicly available tools, of which 26 are open-sourced.Twenty-six of the 30 sources appeared from 2019 to 2021, indicating increasing interest in sharing research efforts.Source: §3.5.4 · p. 22
  • Outdated malware samples and imbalanced datasets constrain how well evaluations represent current threats and minority malware families.The review also reports limited source-code sharing and missing dataset details, making validation and reproduction more difficult.Source: §3.5.4 · pp. 22–23

3.6 Trend Analysis

The review finds malware detection remains the central focus, while adversarial attacks and malware evolution are emerging research topics. It identifies robustness, reliability, and practical deployment as continuing challenges.

3.6.1 Statistic analysis.

  • Malware detection remains the main focus among highly cited studies, while recent top-venue work also examines evaluation, imbalance, interpretability, model aging, and other specific issues.The review identifies adversarial attacks and protections and malware evolution as recurring recent topics connected to the practicability and effectiveness of defenses.Source: §3.6.1 · p. 24

3.6.2 Adversarial learning attacks and protections.

  • Evasion accounts for 87.5% of adversarial attack studies, and one cited Drebin evaluation found an evasion algorithm misclassified 63% of malware samples.Most reviewed attack studies examine white-box, black-box, or both settings; 75% of sources adopt defenses, commonly adversarial training or ensemble learning.Source: §3.6.2 · pp. 24–25

3.6.3 Malware evolution detection and defense.

  • Only seven studies address malware evolution, and the review says existing approaches either slow performance degradation or require periodic retraining and substantial labeling effort.Four studies use semantic similarity, while a time-aware metric and contrastive learning are also discussed as approaches to evolution and model updating.Source: §3.6.3 · p. 26

3.6 Trend Analysis

  • The review concludes that reliability, robustness, and practicability remain future challenges for DL-based Android malware defenses.It characterizes advanced DL applications in this domain as still preliminary and expects further development.Source: §3.6.1 · p. 24, §3.6.3 · p. 26

4 OPEN ISSUES AND FUTURE TRENDS

The review identifies open problems in moving Android malware defenses beyond binary detection toward richer analysis, practical deployment, and better handling of data and APK complexity. It also calls for broader learning approaches and further study of underexplored defense tasks.

Future directions

  • The review presents its findings as a basis for future research and practice discussions on Android security, while noting that relevant research is increasing.It frames mobile security as a major concern alongside phones’ role in daily life and users’ attention to private mobile security.Source: §4 · p. 26

Research focus

  • Other analyses—including malware attribution and behavior, variant analysis, triage, and infection treatment—remain scant compared with binary malware detection.The review argues that binary labels alone are insufficient for current issues and improving mobile security.Source: §4 · p. 26

Data imbalance

  • Only two reviewed studies address data imbalance, and both handle relatively simple scenarios, leaving broader imbalance challenges unresolved.The review notes imbalance between goodware and malware and between malware families, which prior work has linked to hindered performance.Source: §4 · pp. 26–27

Practicality and reliability

  • Effective practical use remains unresolved amid mobile-resource constraints, privacy protection, information updating, and neural networks’ black-box nature.The review calls for deployment solutions and a comprehensive framework addressing realistic challenges, as well as improved transparency.Source: §4 · p. 27

Learning approaches

  • Android security DL research is relatively narrow: supervised DNNs and simple binary tasks dominate, while reinforcement and online learning receive little attention.The review encourages more unsupervised learning, noting that labeling requires time and substantial expertise.Source: §4 · p. 27

APK embedding

  • APK embedding remains underexplored because apps contain complex, multimodal structures that current DL approaches struggle to model.Existing approaches either use a single feature type or multimodal architectures; imported CV and NLP embeddings may be too shallow for APK structures.Source: §4 · p. 27

5 THREATS TO VALIDITY

The authors cannot guarantee that the review captured every relevant study. They identify possible publication-collection and publication-selection bias as threats to validity.

Threats to validity

  • The review acknowledges that its methodology cannot guarantee coverage of all relevant studies.Source: §5 · p. 27

Search items and strategies

  • Search strategies may miss relevant studies because the rapidly developing DL field makes it difficult to identify every search item.The review used terms informed by AI experts, five electronic databases, two citation engines, and backward snowballing to broaden collection.Source: §5 · p. 28

Data selection bias

  • One researcher selected publications, which may have caused studies to be missed, although the other authors reviewed the scheme and monitored execution.The authors also used venue H5-index as a quality-appraisal criterion, while noting that H5 values can change over time.Source: §5 · p. 28

6 CONCLUSIONS

The review synthesizes 132 studies on deep learning for Android malware defenses. It finds detection remains the dominant focus, alongside patterns in features, architectures, learning tasks, analysis settings, and emerging topics.

Conclusions

  • The review collected 132 relevant studies and concludes that deep learning is a promising tool for defending against Android malicious applications.Source: §6 · p. 28
  • Most studies focus on malware detection, while more detailed analyses of malicious apps are receiving increasing attention.Source: §6 · p. 28
  • Static program analysis is widely used for feature collection, semantic features occur frequently, and MLPs and CNNs are the most widely used architectures.Source: §6 · p. 28
  • Most approaches use supervised classification, while distributed and on-device analysis gain attention and adversarial learning and malware evolution emerge as hot topics.Source: §6 · p. 28
Loading 2103.05292v3…