PaperlayerView exact revision on arXiv

Source-linked AI summary

A Comparison of Static, Dynamic, and Hybrid Analysis for Malware Detection

Anusha Damodaran, Fabio Di Troia, Visaggio Aaron Corrado, Thomas H. Austin, Mark Stamp

arXiv:2203.09938v1cs.CRcs.LG

TL;DR

The paper examines whether hybrid malware analysis offers an inherent advantage over static or dynamic approaches while limiting confounding factors. Comparing these strategies, it finds that fully dynamic detection is generally most effective and that hybrid methods do not consistently outperform fully static or fully dynamic detection.

  • Problem

    The paper asks whether hybrid malware analysis has an inherent advantage over static and dynamic techniques, amid difficult cross-study comparisons and confounding factors.

  • Method

    The study compares static, dynamic, and hybrid analysis using a straightforward training-and-scoring framework.

  • Results

    Fully dynamic detection is generally most effective, while hybrid approaches do not consistently outperform fully static or fully dynamic detection.

  • Takeaways & Limitations

    Hybrid detection claims should be tested against comparable fully dynamic and fully static techniques across malware families.

  • Takeaways & Limitations

    The conclusions do not establish whether hybrid benefits generalize across broad malware samples or remain limited to narrower sample ranges.

Abstract

from arXiv · show

In this research, we compare malware detection techniques based on static, dynamic, and hybrid analysis. Specifically, we train Hidden Markov Models (HMMs ) on both static and dynamic feature sets and compare the resulting detection rates over a substantial number of malware families. We also consider hybrid cases, where dynamic analysis is used in the training phase, with static techniques used in the detection phase, and vice versa. In our experiments, a fully dynamic approach generally yields the best detection rates. We discuss the implications of this research for malware detection based on hybrid techniques.

1 Introduction

The paper compares static, dynamic, and hybrid malware detection using Hidden Markov Models over API-call and opcode sequences. It seeks to understand their relative advantages, particularly whether hybrid analysis offers an inherent benefit, rather than optimizing detection accuracy.

  • Problem Motivation: More than 317 million new malware samples were created in 2014, a 26% increase over 2013, underscoring the importance of malware detection research.These figures are attributed to Symantec.
  • Problem Motivation: Signature scanning is widely used and effective for many malware types, but it cannot reliably detect new malware or significant variants.Signature scanning relies on pattern matching.
  • Related Work: Prior research has applied static analysis, dynamic analysis, and hybrid approaches using features obtained without execution, execution behavior, or both.Static methods use features obtainable without executing software, while hybrid methods combine static and dynamic features.
  • Contribution and Method: The study compares static, dynamic, and hybrid malware detection techniques using a straightforward Hidden Markov Model training and scoring approach.The feature sets consist of API call sequences and opcode sequences.
  • Research Goal: The research examines whether hybrid analysis has an inherent advantage over static or dynamic techniques.The analysis emphasizes relative advantages and disadvantages rather than maximizing detection accuracy through combined scores or scoring techniques.

2 Background

This section introduces malware detection techniques, emphasizing Hidden Markov Models as the research basis, reviews related work, and explains ROC curves for quantifying experiments.

  • Hidden Markov Models are emphasized because they form the basis of the research presented in this paper.
  • The section provides a brief discussion of malware detection techniques and reviews relevant related work.
  • ROC curves are introduced as a convenient way to quantify the experiments conducted in the paper.

2.1 Malware Detection

Malware detection includes signature-, behavior-, and statistical-based approaches, with HMMs selected as the basis for the schemes studied because they have benchmarked metamorphic-malware classification in prior work,.

  • Signature-Based Detection: Signature-based detection scans for byte-sequence signatures of known malware but depends on frequent database updates and can be evaded by simple obfuscation.It is simple, relatively fast, and effective against most common malware, but misses threats absent from its signature repository.
  • Behavior-Based Detection: Behavior-based detection learns malware and benign-file action patterns during training, then classifies executables during testing or monitoring.The approach focuses on actions performed during execution and bases classification on patterns derived in the learning phase.
  • Statistical-Based Detection: Statistical detection derives evidence from program features, and this research uses HMMs because prior work applied them to metamorphic-malware classification and established a benchmark,.HMMs therefore provide the foundation for the malware-detection schemes evaluated in this research.

2.2 Hidden Markov Models

The paper models malware-family observation sequences with Hidden Markov Models, training models to maximize sequence likelihood and scoring malware and benign files for detection. An HMM represents unobserved Markov states through probability distributions over observed sequences and is parameterized by λ = (A, B, π).

  • HMM Definition: An HMM links observations to an unobserved Markov process through discrete probability distributions.The observations are related to hidden states that cannot be directly observed.
  • HMM Definition: The model is defined as λ = (A, B, π), comprising state-transition probabilities, observation probabilities, and the initial-state distribution.A, B, and π respectively specify transitions, observation probabilities, and the initial state distribution.
  • Application to Malware Detection: HMMs are trained on observation sequences from malware families, then score malware and benign-file sequences to measure each detection technique’s success.Training uses Problem 3, while detection uses Problem 1’s sequence-likelihood scores.

2.3 Related Work

Prior malware-detection research spans static, dynamic, and hybrid analysis, using diverse program representations and machine-learning methods. Static approaches avoid execution, dynamic approaches model runtime behavior, and hybrid approaches combine both to balance detection fidelity with efficiency.

  • Static Analysis: Static analysis detects malware without executing programs, using features such as opcode sequences and control-flow graphs.Prior work applies these representations with Hidden Markov Models, Profile Hidden Markov Models, Principal Component Analysis, Support Vector Machines, clustering, and graph-based methods,,,,,,, [15].
  • Dynamic Analysis: Dynamic analysis executes programs, often virtually, to capture runtime evidence including API and system calls, instruction traces, registry changes, and memory writes.Related methods model system-call dependencies, API-call spatio-temporal statistics, API-control-flow combinations, kernel traces, behavioral graphs, and dynamic instruction sequences [27],,,,,,,, [2].
  • Hybrid Analysis: Hybrid techniques combine static and dynamic analysis to use information from both forms of program examination.Mal-DNA uses a debugging-based behavior monitor to extract dynamic characteristics within a framework that uses both analyses.
  • Hybrid Analysis: HDM Analyser uses static and dynamic analysis during training but only static analysis during testing, targeting dynamic fidelity in training and static efficiency during scoring.The passage reports that HDM Analyser has better overall accuracy and time, but the supplied text is truncated before specifying the comparison details.

2.4 ROC Analysis

ROC analysis evaluates malware-score thresholds by relating false-positive and true-positive rates. Its AUC-ROC summarizes ranking quality, with 1.0 indicating ideal separation between malware and benign files.

  • 2.4 ROC Analysis: An AUC-ROC of 1.0 denotes ideal detection, while generally representing the probability that a randomly selected positive instance scores above a randomly selected negative instance.The ROC curve plots false-positive rate against true-positive rate as the threshold varies.
  • 2.4 ROC Analysis: In malware classification, ROC analysis treats malware scores as positive instances and benign-file scores as negative instances, assuming higher scores indicate better classification.Figure 2 illustrates these positive and negative scores with a scatterplot and corresponding ROC curve.
  • 2.4 ROC Analysis: The true-positive rate is sensitivity, the true-negative rate is specificity, and the false-positive rate equals 1 − specificity.
  • 2.4 ROC Analysis: Every ROC curve includes (0, 0) and (1, 1), with intermediate points determined by moving the threshold through the observed scores.
  • 2.4 ROC Analysis: At the illustrated threshold, sensitivity is 0.7 and the false-positive rate is 0.2, based on correctly classifying 7 of 10 positives and misclassifying 2 of 10 negatives.

2.5 PR Analysis

PR curves plot precision–recall pairs across classification thresholds and can be more informative than ROC curves when the nomatch set greatly exceeds the match set. In the illustrated example, the PR curve has an AUC-PR of about 0.69.

  • 2.5 PR Analysis: PR curves may be preferable to ROC curves when the nomatch set is large relative to the match set.Although the two curve types are connected, PR curves can be more informative in this class-imbalanced setting.
  • 2.5 PR Analysis: Recall is the fraction of match cases classified correctly, whereas precision is the fraction of positive classifications belonging to the match set.Recall corresponds to the true positive rate, while precision differs from the ROC false positive rate.
  • 2.5 PR Analysis: True negatives do not directly affect PR-curve computation because neither recall nor precision uses them.The relevant counts are true positives, false positives, and false negatives.
  • 2.5 PR Analysis: PR curves plot recall–precision pairs as the classification threshold varies across a scatterplot.The procedure is illustrated with the same data used for the ROC-curve example in Section 2.4.
  • 2.5 PR Analysis: About 0.69 AUC-PR is obtained for the illustrated scatterplot example.The complete PR curve is generated from the threshold-varying data shown in Figure 3.

3 Experiments

The experiments compare static, dynamic, and hybrid HMM detection using opcode and API-call sequences extracted from malware and benign Windows files. They evaluate four training/scoring combinations across seven malware families with five-fold cross-validation and ROC AUC as the success measure.

  • Analysis tools: IDA Pro supports static disassembly and dynamic tracing, while Buster Sandbox Analyzer records process behavior including registry, file-system, port, and API-call activity.Ether is also tested for dynamic analysis, but it produces no significant difference in API-call sequences from BSA for the datasets considered.
  • Datasets: The dataset comprises seven malware families plus a benign set of Windows System 32 files.Zbot execution could only be traced for about 5 to 10 minutes because it eventually vanished from the process list.
  • Feature extraction: Each program supplies four observation sequences by combining static or dynamic extraction of opcode sequences and API-call sequences.Static opcodes come from IDA Pro disassembly, dynamic opcodes from traced execution, static API calls from disassembly, and dynamic API calls from BSA logs collected during a fixed execution window.
  • Experimental design: Four cases compare static/static, dynamic/dynamic, dynamic/static, and static/dynamic training and scoring combinations.The dynamic/static case represents a hybrid approach intended to combine dynamic-training accuracy with efficient static scoring, whereas static/dynamic offers no clear advantage and is included for completeness in opcode experiments.
  • Experimental design: Five-fold cross-validation trains an HMM on four malware subsets and scores the held-out malware subset together with benign files.A separate experiment is conducted for each malware dataset and each static/dynamic combination.
  • Evaluation: ROC curves are generated from experiment score scatterplots, with area under the ROC curve used as the success measure.The metric is defined as the area under the ROC curve.

4 Results

Across API-call and opcode experiments, fully dynamic analysis generally produced the strongest malware-detection results, while hybrid dynamic/static analysis was not generically competitive. Simulated class imbalance further favored fully dynamic and fully static approaches when evaluated with AUC-PR.

  • Experimental design: Experiments used HMMs with five-fold cross-validation on API-call and opcode sequences, evaluating static, dynamic, and hybrid training/scoring combinations primarily with AUC.The results report AUC-ROC and AUC-PR, with corresponding tables and figures for the sequence types.
  • API call sequences: For API-call sequences, dynamic/dynamic performed best, static/static was usually similarly effective, and dynamic/static was worse than static/static except for Security Shield and Zbot.The dynamic/static approach was significantly worse than static/static for all families except Security Shield and Zbot.
  • Opcode sequences: For opcode sequences, results were weaker than for API calls, but dynamic/dynamic remained best while static/static and dynamic/static were roughly equivalent.In four cases, dynamic/static was marginally better than static/static, but in one case it was significantly worse.
  • Imbalance analysis: Under simulated benign-sample imbalance, AUC-ROC stayed unchanged, whereas AUC-PR decreased; fully dynamic superiority and fully static advantage over hybrids increased as imbalance grew.Dynamic/static lost competitiveness at large imbalance, even where it was initially competitive.
  • Overall results: Across API-call and opcode experiments, fully dynamic analysis generally achieved the best detection results, while hybrid approaches were not generically superior.Dynamic analysis is costly during scoring, whereas training can be performed offline; the authors recommend comparing hybrids against fully dynamic and fully static baselines.

5 Conclusion and Future Work

The study finds that fully dynamic API-call analysis is highly effective across malware families, while straightforward hybrid approaches do not consistently outperform fully dynamic or fully static detection. It calls for careful scrutiny of claimed hybrid benefits and further comparisons using additional features, scoring methods, and imbalance analyses.

  • Conclusion: The study compared Hidden Markov Models trained on static, dynamic, and hybrid API-call and opcode-sequence features to evaluate malware detection rates.The experiments covered static data, dynamic data, and hybrid approaches across the tested feature types.
  • Conclusion: Fully dynamic API-call detection was extremely effective across malware families, while fully static API-call detection was nearly as effective in most cases.Opcode sequences were effective in many cases but produced unimpressive results for some families, likely reflecting malware obfuscation techniques.
  • Conclusion: Straightforward hybrid detection was unlikely to outperform fully dynamic detection and did not consistently improve on fully static detection, although an impractical static/dynamic hybrid was superior in some cases by some measures.These findings are surprising given claims made for hybrid approaches.
  • Limitations: Claims of hybrid benefits should be scrutinized to determine whether gains arise from the hybrid model itself or factors such as score combinations, and whether they generalize across malware samples.The paper emphasizes distinguishing model-specific benefits from other contributing factors and assessing their breadth across samples.
  • Future Work: Future work should examine additional features, alternative scoring methods and optimal static/dynamic score combinations, including Support Vector Machines, alongside deeper analysis of imbalance issues.Suggested alternatives include graph-based, structural, machine-learning, and statistical scores.
Loading 2203.09938v1…