Source-linked AI summary
Dynamic Malware Analysis with Feature Engineering and Feature Learning
Zhaoqi Zhang, Panpan Qi, Wei Wang
TL;DR
Existing malware-detection methods often ignore API arguments or require complex processing for heterogeneous argument types. This paper introduces low-cost hashed API-call features and a gated-CNN/Bi-LSTM architecture, which outperforms all baselines on a large real dataset.
Problem
Existing methods often use API names while neglecting arguments, or struggle to extract features from heterogeneous argument types such as strings, integers, and addresses.
Method
The method applies feature hashing to API names, categories, and arguments, then uses multiple gated CNNs and a bidirectional LSTM to model API-call sequences.
Results
The approach outperforms all baselines on a large real dataset, while ablation studies support the feature-engineering and architecture choices.
Takeaways & Limitations
Hashing produces a homogeneous, low-cost representation, while the gated-CNN/Bi-LSTM design captures sequential correlations among API calls.
Takeaways & Limitations
The method does not handle virtual addresses, structs, and other argument types, which the authors leave for targeted future research.
Abstract
from arXiv · showhide
Dynamic malware analysis executes the program in an isolated environment and monitors its run-time behaviour (e.g. system API calls) for malware detection. This technique has been proven to be effective against various code obfuscation techniques and newly released ("zero-day") malware. However, existing works typically only consider the API name while ignoring the arguments, or require complex feature engineering operations and expert knowledge to process the arguments. In this paper, we propose a novel and low-cost feature extraction approach, and an effective deep neural network architecture for accurate and fast malware detection. Specifically, the feature representation approach utilizes a feature hashing trick to encode the API call arguments associated with the API name. The deep neural network architecture applies multiple Gated-CNNs (convolutional neural networks) to transform the extracted features of each API call. The outputs are further processed through bidirectional LSTM (long-short term memory networks) to learn the sequential correlation among API calls. Experiments show that our solution outperforms baselines significantly on a large real dataset. Valuable insights about feature engineering and architecture design are derived from the ablation study.
1 Introduction
The paper focuses on dynamic malware analysis because static signatures can be evaded, while API-call behavior captures runtime operations. It addresses the difficulty of representing heterogeneous API arguments by combining hashing-based features with a gated-CNN and bidirectional-LSTM architecture.
- Motivation: Dynamic analysis executes software in an isolated environment and collects runtime behavior, offering greater robustness than static analysis against obfuscation and zero-day malware.
- Research gap: API-call sequences capture operations such as network access and file manipulation, but their heterogeneous arguments are difficult to process with conventional feature engineering.
- Research gap: Existing deep-learning approaches can learn sequential features directly, yet most ignore API arguments because their heterogeneous types remain challenging to represent.
- Approach: The proposed representation hashes API names, categories, and arguments separately, concatenates the resulting features, and feeds them into the model.
- Approach: Multiple gated CNNs transform each API call’s high-dimensional features, while a bidirectional LSTM learns sequential correlations across API calls.
- Results: The solution outperforms all baselines with a large margin, and ablations identify feature engineering and architecture design as crucial for generalization.
2 Related Work
Related work represents API behavior with manually designed, statistical, textual, or learned features. Prior methods vary in how they encode arguments and sequences, often facing inconsistency, long vectors, or limited argument coverage.
- Feature engineering: Earlier MIST-style representations manually specify argument features for each API, creating inconsistent meanings across APIs that challenge machine-learning pattern discovery.
- Feature engineering: BBIS extends MIST but reports that only API category and name are effective, while CARL processes consecutively repeated API calls.
- Feature engineering: Statistical approaches extract frequencies, distributions, spatial statistics, and temporal n-gram relationships from API names, arguments, and sequences.
- Feature engineering: Other representations concatenate arguments with API names, use limited argument indicators, or combine manually selected argument features with sequence n-grams.
- Deep learning: Deep-learning approaches include a DBN over a 20,000-bit sandbox-report vector and an RNN-based feature-learning pipeline followed by classification.
- Deep learning: The reviewed approaches also include CNN-LSTM models that use stacked convolutional layers to simulate 3-gram features before sequence modeling.
3 System Framework
The system collects PE files, executes them in sandboxed virtual machines to obtain API-call traces, and extracts features for GPU-based malware classification. Its workflow comprises collection, behavior-information collection, and feature extraction with model training.
- System architecture: The architecture has three parts: PE-file collection, behavior-information collection, and feature extraction with model training.
- PE-file collection: PE files are collected and labeled using aggregated classifications from 12 antivirus engines before entering an execution queue.
- Behavior collection: Cuckoo2 runs PE files inside virtual machines, uses API hooks to record execution traces, and simulates user actions on 64-bit Windows 7 systems.
- Feature extraction and training: The system extracts features from execution logs in parallel and trains the deep-learning classifier on a GPU-equipped model server.
4.1 Feature Engineering
The feature-engineering approach preserves information from API names, categories, and heterogeneous arguments through separate hashed representations. It handles integer and structured string arguments while excluding some less-important argument types.
- Earlier approaches often ignored API arguments, losing discriminative distinctions such as whether a write targets a program-created file or a system file.
- The proposed representation separately hashes API names, categories, integer arguments, and string arguments before concatenating the resulting features.The design uses 8 API-name bins, 4 category bins, and 90 argument bins: 16 for integers and 74 for strings.
- Feature hashing maps strings or characters into fixed-length bins using a bin-index hash and a sign hash, then sums signed contributions within each bin.
- Integer arguments are hashed by argument name and value, with logarithmic normalization used to squash sparsely distributed numeric ranges.
- String arguments are processed by type, including paths, DLLs, registry keys, URLs, and IP addresses, with hierarchical substrings extracted for structured values.For a path such as C:\a\b\c, the method generates progressively longer substrings and hashes them individually.
- The method does not handle virtual addresses, structs, and other argument types considered relatively less important.
4.2 Model Architecture
The architecture transforms each API call’s engineered feature vector with gated CNNs, then models API-call sequence dependencies using a bidirectional LSTM and downstream pooling and classification layers.
- The model receives a normalized sequence of API features with dimension 102 per API call before applying several gated CNNs.The input shape is (N, d), where N is sequence length and d is 102.
- Each gated CNN combines two convolution outputs through element-wise multiplication with a sigmoid gate that controls transmitted information.
- Two gated CNNs use filter sizes 2 and 3, while all convolution layers use filter size 128 and stride 1.
- Concatenated gated-CNN outputs are batch-normalized and passed to a bidirectional LSTM to learn sequential patterns from both directions.Each LSTM has 100 units.
- Global max pooling extracts salient signals across the sequence instead of relying only on the Bi-LSTM’s final activation.
- The classifier reduces pooled features through a 64-unit ReLU dense layer, applies dropout at 0.5, and outputs a probability with a sigmoid layer.Training uses binary cross-entropy with Adam at a learning rate of 0.001.
5 Experiments
Experiments evaluate the proposed malware detector on archived PE-file data using ROC AUC, accuracy, recall at a 0.1% false-positive rate, and inference time. The model achieves the strongest overall baseline performance, while ablations show that argument features, Gated CNN configuration, batch normalization, and Bi-LSTM design affect results.
- 5.1 Dataset: The dataset uses PE files archived by date, with April and May data selected for experiments and API-call sequences collected through the system.Inconclusive antivirus-label cases are excluded from the dataset.
- 5.2 Model Evaluation: The evaluation compares the proposed model with three machine-learning and three deep-learning baselines using ROC AUC, accuracy, and recall at a 0.1% false-positive rate.Inference time includes feature processing and model prediction, and all three metrics receive 95% confidence intervals.
- 5.2 Model Evaluation: The proposed model achieves the best AUC, accuracy, and recall among all baselines on both cross-validation and test datasets.The test results are presented as evidence of performance on new and evolving malware.
- 5.2 Model Evaluation: Argument features increase test AUC by 3% for traditional machine learning and about 1% for deep learning, supporting their inclusion in the detector.Models using argument features generally outperform models that neglect arguments.
- 5.2 Model Evaluation: Models with argument features require slightly longer inference, but hundreds of milliseconds remain small relative to sandbox collection times of 3–5 minutes per sample.Training takes about 10 minutes per epoch and could be reduced through distributed training.
6 Conclusion
The paper combines hashed API-call features with multiple gated-CNNs and a Bi-LSTM for dynamic malware detection, outperforming all baselines in experiments.
- Hashing tricks encode API names, categories, and arguments into homogeneous, low-cost feature representations.
- Multiple gated-CNNs transform high-dimensional hash features for each API call before Bi-LSTM processing.
- The Bi-LSTM captures sequential correlations among API calls within each sequence.
- The approach outperforms all baselines, while ablation studies verify the architecture design decisions.