A New Android Malware Detection Approach Using Bayesian Classification

Suleiman Y. Yerima; Sakir Sezer; Gavin McWilliams; Igor Muttik

Source-linked AI summary

A New Android Malware Detection Approach Using Bayesian Classification

Suleiman Y. Yerima, Sakir Sezer, Gavin McWilliams, Igor Muttik

arXiv:1608.00848v1Aug 2, 2016cs.CRcs.LG

TL;DR

The paper addresses increasingly sophisticated Android malware that can evade signature-based scanners. It uses Bayesian classification trained on static code and app characteristics, achieving higher malware detection rates than reported signature-based scanners on the same sample set. Performance generally improves with more features, while 15-feature and 20-feature settings are close.

Problem
Polymorphism, obfuscation, encrypted payloads, and runtime payload execution make Android malware increasingly difficult for signature-based scanners to detect.
Method
The approach uses Bayesian classification with static-analysis features extracted from Android applications and selected for model training.
Results
TPR reached 0.906 for both 10f and 20f, compared with a reported best signature-based detection rate of 0.796 on the same malware sample set.
Takeaways & Limitations
The approach demonstrated capability to detect unknown malware and can prioritize suspicious applications for further scrutiny.

Abstract

from arXiv · show

Mobile malware has been growing in scale and complexity as smartphone usage continues to rise. Android has surpassed other mobile platforms as the most popular whilst also witnessing a dramatic increase in malware targeting the platform. A worrying trend that is emerging is the increasing sophistication of Android malware to evade detection by traditional signature-based scanners. As such, Android app marketplaces remain at risk of hosting malicious apps that could evade detection before being downloaded by unsuspecting users. Hence, in this paper we present an effective approach to alleviate this problem based on Bayesian classification models obtained from static code analysis. The models are built from a collection of code and app characteristics that provide indicators of potential malicious activities. The models are evaluated with real malware samples in the wild and results of experiments are presented to demonstrate the effectiveness of the proposed approach.

I. INTRODUCTION

Android malware is increasing in volume and sophistication, challenging signature-based detection. The paper proposes Bayesian classification from static app characteristics to identify suspicious and unknown malware.

Android malware grew sharply, with approximately 2000 samples across 80 families reported by November 2011.
Polymorphism, code obfuscation, encrypted payloads, and runtime payload execution make newer malware difficult for signature-based scanners to detect.
New threats took an average of 48 days for signature-based antivirus engines to detect, while malware could remain unnoticed for up to three months.
The proposed Bayesian approach uses malware-associated code and app characteristics to classify applications as suspicious or benign, including known and unknown families.
The approach complements signature scanning by verifying harmless apps and isolating suspicious samples for further scrutiny in large marketplaces.
The paper’s contributions include signature-free static-code detection, Bayesian model construction, and empirical evaluation across malware from 49 families.

III. ANDROID SYSTEM ARCHITECTURE

Android is a layered mobile software stack built on a modified Linux kernel, with applications distributed through official, alternative, or manual installation channels. Downloaded applications can threaten user security and privacy when malicious.

Android combines an operating system, middleware, application API, and key applications above a customized embedded Linux system.
Applications may be pre-installed, downloaded from Google Play or alternative marketplaces, or installed manually from .apk files.
Malicious applications distributed through these channels pose potential threats to users’ security and privacy.

A. Android application basics

Android applications are packaged as APKs containing code, manifests, resources, and assets. The paper reverse-engineers these packages and extracts static indicators such as API calls, commands, permissions, encryption, and embedded payloads for classification.

A. Android application basics: Android SDK tools compile Java code, data, and resources into installable .apk archives.
A. Android application basics: The manifest declares application components and requested permissions, including internet or contacts access.
A. Android application basics: An APK typically contains AndroidManifest.xml, classes.dex, and resource or asset directories.
B. Application Reverse Engineering: The analyzer reverse-engineers APK files and applies detectors to identify API calls, Linux commands, permissions, encryption, and secondary APK or JAR files.
B. Application Reverse Engineering: API detectors profile intended runtime activity using calls related to telephony, SMS, phone numbers, and package installation.
B. Application Reverse Engineering: Command detectors inspect code, resources, assets, and libraries for patterns that may indicate malicious scripts or hidden payloads.

IV. BAYESIAN CLASSIFICATION MODEL

The Bayesian classifier learns feature likelihoods from known malicious and benign applications, then evaluates new applications against the selected features. Bayesian classification is chosen for fast, low-overhead filtering with prior-knowledge integration.

Bayesian classification supports relatively fast classification with low computational overhead after training, making it suitable for filtering many applications.
The model can incorporate expert prior probabilities before training alongside probabilities learned from application data.
The learning stage extracts features from known malicious and benign applications, reduces the feature set, and calculates feature likelihood probabilities.
During classification, new applications are assessed against the selected feature set and stored likelihood probabilities.

B. Feature ranking and selection

The feature-reduction function represents each application with detector-derived characteristics, calculates their Mutual Information with the application class, and ranks features to select the most relevant ones for classification.

Each application is assigned a vector of detector-derived characteristics, with each characteristic represented as a random variable.
A detector-derived feature is binary, taking the value 1 when the corresponding property is discovered and 0 otherwise.
Mutual Information is calculated for each feature to measure its relationship with the suspicious or benign application class.
Features are ranked from largest to smallest Mutual Information, and the highest-ranked features are selected to maximize class-related information and classifier performance.

C. Bayesian classification

The Bayesian classifier estimates the probability that an application belongs to the benign or suspicious class given its feature vector, then assigns the more probable class. Misclassifying suspicious applications as benign is treated as the more critical error.

The classifier uses an application feature vector and estimated frequencies from the learning corpus to calculate class probabilities.
Applications are assigned to the benign class when P(C=benign|R=r) exceeds P(C=suspicious|R=r).
Applications not meeting the benign-probability condition are classified as suspicious.
The classifier distinguishes two errors: benign applications misclassified as suspicious and suspicious applications misclassified as benign.
Misclassifying a suspicious application as benign is considered more critical because the malicious app may reach an end-user device.

D. Evaluation measures

The experiments evaluate classifier predictions using accuracy, error rate, false-positive and false-negative rates, true-positive and true-negative rates, and precision. These measures are defined from counts of correct and misclassified benign and suspicious applications.

The evaluation uses accuracy and error rate as measures of predictive classifier accuracy.
The confusion-count definitions distinguish correctly classified benign and suspicious applications from benign and suspicious misclassifications.
The reported classification measures also include false positive rate, false negative rate, true positive rate, true negative rate, and precision.

V. METHODOLOGY AND EXPERIMENTS

The methodology analyzes balanced malware and benign app collections, extracts and ranks detector features, and examines how selected properties differ between classes. The results indicate that Bayesian classification combines these properties probabilistically to discriminate between malicious and legitimate applications.

The analyzer processed 2000 APKs: 1000 malware samples from 49 families and 1000 benign applications from official and third-party markets.
Table I lists the malware families used and the number of samples from each family.
The benign set covered varied categories including entertainment, tools, sports, health, finance, education, and games.
Of 58 defined feature attributes, 10 produced no matches and were discarded; the remaining 48 were ranked by Mutual Information, with the top 25 reported.
Table II reports the selected features and their frequencies in the 1000-sample benign and malware sets.
Some properties appeared only in malware, system-command references mainly occurred in malware, and Telephony manager calls occurred more often in malware than benign apps.
References to .apk and .jar files occurred in both classes but were more frequent in malware, while legitimate ad and mobile-payment frameworks also used secondary apps.
Bayesian classification probabilistically combines multiple properties, including class-specific and shared features, to improve discrimination between classes.

A. Bayesian Classifier training

Training uses 5-fold cross-validation on 2,000 balanced samples, with 1,600 samples for training and 400 for testing in each evaluation.

5-fold cross-validation evaluates the classifier using 1,600 training samples and 400 testing samples per fold.Each split contains 800 benign and 800 malware training samples, plus 200 benign and 200 malware testing samples.

VI. RESULTS AND DISCUSSIONS

The experiments compare ranked feature sets and training-sample sizes across accuracy, error, TNR, FPR, TPR, FNR, precision, and AUC. Results favor the top-ranked features, with strong malware detection and limited gains beyond 15 features, while some errors arise from zero feature vectors.

Top-ranked feature sets generally increase accuracy and decrease error rates as the number of features rises.Performance with 15f and 20f is very close, indicating only marginal improvement from using more features.
5fT achieves close to 85% average accuracy, compared with 65% for 5fL using the same number of differently ranked features.The comparison supports the effectiveness of the analyzer’s feature ranking.
Around 6% FPR is observed for the feature sets, while 15f and 20f show little difference in TNR and FPR.Lower FPR reduces the number of benign apps sent for further scrutiny.
10f, 15f, and 20f yield TPRs of 0.906, 0.904, and 0.906, with corresponding FNRs of 0.094, 0.096, and 0.094.These TPRs exceed the reported signature-based antivirus detection range of 0.202 to 0.796 on the same malware sample set.
Some false negatives result from zero feature vectors; adding features may reduce these occurrences without necessarily improving classification accuracy.Combining related features, such as .apk and .jar secondary-payload indicators, is proposed as one possible remedy.
Precision and AUC generally improve with more features, while 10f, 15f, and 20f remain close to one another.AUC measures area under the ROC curve, and values closer to 1 indicate better predictive power.
Increasing training samples noticeably affects TPR, has less impact on some other metrics, and improves AUC.The training-sample experiments use the 20-feature set and balanced benign-malware samples under 5-fold cross-validation.

VII. CONCLUSION

The paper proposes and evaluates Bayesian classification for Android malware detection using statically extracted and ranked application properties. Experiments on balanced malware and benign-app samples found that 15–20 features provide optimum performance, with significantly better detection rates than previously tested signature-based antivirus software.

The approach applies Bayesian classification to Android malware detection using properties extracted through reverse engineering and static analysis.An APK analyzer extracts 58 properties by detecting patterns and references to API calls, system commands, and other malware-associated indicators; a feature-selection function ranks them.
2,000 applications—1,000 samples from 49 Android malware families and 1,000 benign apps—were used for feature extraction and Bayesian-classifier training.The benign applications represented a variety of categories.
15 to 20 features were sufficient to provide optimum performance in the experiments.This range was determined from detected properties and their ranking by the feature-selection function.
The approach achieved significantly better detection rates than popular signature-based antivirus software previously tested on the same malware samples.The comparison used the largest publicly available malware collection at the time of writing.