Source-linked AI summary
A Structured State Space Sequence Model for Multi-Class Classification of Malware
Emmanuela Andam, Rana Shaaban, Emanuel Grant, Naima Kaabouch
TL;DR
Malware detection and classification face rapidly growing variants, while established deep-learning approaches have limitations in long-range modeling or computational cost. This study evaluates an S4 model on binary detection and multiclass ransomware-family classification; it reports stronger family-level F1 than the compared baselines while retaining high detection accuracy.
Problem
Malware variants are growing rapidly, while CNNs, recurrent models, and Transformers face limitations in long-range modeling or computational cost.
Source: §I · pp. 1–2Method
The study evaluates an S4 state-space model for binary malware detection and multiclass family classification against deep-learning baselines.
Source: §I · pp. 2–3Results
89% macro F1-score and 88% weighted F1-score for S4 exceeded the CNN, LSTM, GRU, and Transformer baselines' 75%–76% range in family classification; S4 also reached 98.5% binary-detection accuracy.
Source: §IV · p. 8Takeaways & Limitations
The reported results indicate that structured state-space modeling improves family-level classification while maintaining strong malware-detection performance.
Source: §IV · p. 8The study uses a single dataset; the authors identify evaluation on additional benchmarks as future work.
Source: §IV · p. 8
Abstract
from arXiv · showhide
By 2030, Internet of Things (IoT) devices are projected to reach 40 billion, with fast-paced technological advancements in fields such as industry, healthcare, agriculture, automobiles, and building/home automation systems. This expansion has created a large attack surface for cybercrime, as the majority of these devices open the door for cybercriminals to exploit vulnerabilities, as they lack adequate built-in security. Cybercriminals launch malware attacks to compromise systems or steal sensitive data, and once a system is compromised, a ransom is typically demanded for its release. Current cybersecurity measures in place are being outpaced by the rapid growth of the IoT, which is accompanied by a subsequent growth in malware variants being created per day. Recognizing this pitfall, this research examines and proposes a novel approach to malware detection and classification to safeguard devices from further attacks and make IoT systems more robust and secure. The framework proposed utilizes a Structured State Space Sequence (S4) model, which discretizes sequences of malware samples in a sequence and captures long-range dependencies, essentially identifying the "cause" and "effect" hidden within malware execution flow. This study presents two novel contributions: the first empirical application of the S4 model for malware analysis, and a comprehensive comparison of its performance against other deep learning architectures, laying the stepping stone for future research in this new paradigm.
I. INTRODUCTION
The growth of connected devices and malware variants strains detection, while existing approaches face limits in long-range modeling or computational cost. The paper examines S4 for malware detection and family classification and benchmarks it against deep-learning baselines.
I. INTRODUCTION
- The introduction reports 39 billion projected Internet devices by 2030 and roughly 450,000 new malware variants per day, alongside rising vulnerabilities and challenges for timely detection.It also cites a 124% increase in IoT attacks within the year covered by the SonicWall report.Source: §I · p. 1
- Existing CNNs can struggle to model long-term dependencies, recurrent models can be computationally expensive and suffer gradient problems, and Transformers often incur high computation.Earlier signature-based detection may miss samples that do not match a known threat.Source: §I · pp. 1–2
- Malware classification research spans byte, opcode, behavioral, graph, metadata, and hybrid features, with prior work reporting varied task-specific results.The cited studies include byte-based F1-scores of 0.503 and 0.507 and opcode-model accuracy above 81% across twenty families.Source: §I · p. 2
- S4 is examined for binary malware detection and multiclass family classification, with shared state representations intended to capture long-range dependencies at linear computational complexity.The study presents its work as the first empirical application of S4 to sequential malware analysis.Source: §I · p. 2
- The study benchmarks S4 against CNN, LSTM, GRU, and Transformer models using accuracy and computational efficiency.The stated contributions include designing an S4 framework for binary detection and multiclass family classification.Source: §I · pp. 2–3
II. METHODOLOGY
The methodology section outlines the study's dataset and preprocessing, S4 architecture, training, evaluation, and hyperparameter tuning.
II. METHODOLOGY
- The methodology covers the dataset and preprocessing, proposed S4 architecture, training, model evaluation, and hyperparameter tuning.It points to Fig. 1 for the overall workflow.Source: §II · p. 3
A. Dataset and Preprocessing
The study uses static numerical features from a balanced malware dataset and transforms feature vectors into learned sequences for S4 processing.
A. Dataset and Preprocessing
- The Ransomware Dataset 2024 contains 21,752 samples, evenly divided between benign and malicious files, and spans twenty-six malware families.Its static numerical features describe executable structure; non-numeric identifiers and text labels are removed.Source: A. Dataset and Preprocessing · p. 3
- The 71 retained features are normalized using Z-scores, and family labels are integer-encoded for multiclass classification.The text says normalization prevents large-magnitude features from dominating learning and improves optimization stability.Source: A. Dataset and Preprocessing · p. 3
- Each normalized feature vector is projected into a learned sequence of length 32 with dimension 128 because S4 operates on sequential inputs.The sequence representation lets state-space layers capture interactions among feature subgroups rather than treating the vector as flat.Source: A. Dataset and Preprocessing · p. 3
B. Model Architecture
The architecture converts static feature vectors into latent sequences, processes them through stacked state-space blocks, and shares a pooled representation across malware detection and family classification heads.
B. Model Architecture
- A continuous-time linear state-space system is discretized over sequence positions, with a learned time step and forward Euler updates.The stated discretization uses Ad = I + ∆t A and Bd = ∆t B.Source: B. Model Architecture · p. 3
- The architecture transforms each fixed-length feature vector into a latent sequence, processes it with stacked state-space blocks, then pools the output for task-specific malware and family classifiers.The overview depicts sequence construction, residual-connected blocks, dual feed-forward classifiers, and discrete-time dynamics within a block.Source: B. Model Architecture · pp. 3–4, Figure 2 · p. 4
- The 71-feature input is projected and reshaped into a sequence of length 32 with 128 dimensions, without assigning temporal meaning to the features.The paper describes this structured representation as a way to apply repeated state updates over computational steps.Source: B. Model Architecture · pp. 3–4
- Three identical blocks each combine a discrete-time state-space layer with residual connections, layer normalization, and a positionwise feed-forward network.The feed-forward network expands the latent dimension to 2dmodel and projects it back; dropout follows the state-space layer and feed-forward network.Source: B. Model Architecture · p. 4
- Global average pooling yields a fixed-length representation shared by separate feed-forward classifiers for binary detection and malware-family prediction.Both classifiers output unnormalized logits and are trained using cross-entropy loss.Source: B. Model Architecture · pp. 4–5
C. Training and Hyperparameter Tuning
Models were trained on a stratified, fixed 60/20/20 train–validation–test split, with family classification prioritized in the joint objective. Hyperparameters were grid-searched using validation family accuracy before final retraining and held-out evaluation.
C. Training and Hyperparameter Tuning
- The stratified split allocated 60% of samples to training and 20% each to validation and testing, with random state 42 used for reproducibility.Training used AdamW for 50 epochs, batch size 256, gradient clipping at 1.0, dropout 0.1, and cosine annealing.Source: C. Training and Hyperparameter Tuning · p. 5
- The joint detection and family-classification objective prioritizes family classification, and validation family accuracy selects checkpoints.Validation and test losses are reported as unweighted sums of the two components.Source: C. Training and Hyperparameter Tuning · p. 5
- Grid search evaluated 32 ten-epoch trials per model, selected configurations by validation family accuracy, and retrained the best configuration for 50 epochs.S4 tuning covered architectural, optimization, and regularization parameters; baseline tuning used a restricted set of candidates.Source: C. Training and Hyperparameter Tuning · p. 5, Table II · p. 5
D. Evaluation Metrics
The evaluation examines binary detection, per-class performance, and dataset-wide averages to characterize performance and possible class bias.
D. Evaluation Metrics
- Evaluation considers whether models distinguish malware from benign samples and whether per-class performance reveals family-level bias.The metrics summary is provided in Table III.Source: D. Evaluation Metrics · p. 5, Table III · p. 6
- Table III summarizes the evaluation metrics used for malware classification.Source: Table III · p. 6
- Macro averages weight every malware class equally, whereas weighted scores scale results by sample counts and favor majority classes.The difference between these averages is used to indicate whether the model is biased.Source: D. Evaluation Metrics · p. 6
III. RESULTS AND DISCUSSION
The results section compares S4 with CNN, LSTM, GRU, and Transformer models on an independent unseen test set. Fig. 4 reports binary detection performance across four metrics.
III. RESULTS AND DISCUSSION
- The experiments compare S4 with CNN, LSTM, GRU, and Transformer models using an independent unseen test set.Evaluation was conducted on a Dell Precision 3260 workstation with GPU acceleration when available.Source: §III · p. 6
- Fig. 4 presents binary malware-detection performance across evaluated models.Source: Figure 4 · p. 6
- Fig. 4 reports accuracy, precision, recall, and F1-score for binary malware detection.Source: Figure 4 · p. 6
A. Training
Across 50 epochs, training and validation losses decline and stabilize, while detection accuracy rises early and family accuracy increases more gradually.
A. Training
- Training converges stably over 50 epochs, with losses stabilizing after an early sharp decline and family accuracy rising steadily before leveling off.Detection accuracy rises sharply within a few epochs and remains fairly consistent thereafter.Source: A. Training · p. 6
B. S4 Model Performance
S4 performs strongly in binary detection and outperforms the evaluated baselines in ransomware-family classification. Its reported metrics come from a single experimental run.
B. S4 Model Performance
- 98.5% accuracy for S4 compares with above 97% accuracy across all evaluated models for binary malware detection.S4’s precision, recall, and F1-score also each exceed 98%.Source: B. S4 Model Performance · p. 7
- S4 achieves an F1-score of at least 0.90 in 10 of the 12 selected ransomware families, including 0.98 for Phobos, 0.95 for Maze, and 0.94 for LockBit.Fig. 5 compares per-family F1-scores across model architectures.Source: B. S4 Model Performance · p. 7, Figure 5 · p. 7
- S4’s precision and recall remain balanced across families, while some baselines show sharp recall declines; the CNN on REvil records 0.85 precision and 0.68 recall.Figs. 6 and 7 present per-family precision and recall; the passage also reports that baseline margins reach 15 to 20 percentage points in challenging cases.Source: B. S4 Model Performance · p. 7, Figure 6 · p. 7, Figure 7 · p. 7
- 89% macro F1-score and 88% weighted F1-score for S4 exceed the Transformer and CNN scores of 75% and 74%, respectively, and the MLP’s 64%–67% range.The figure summarizes macro-averaged and weighted precision, recall, and F1-scores across evaluated models.Source: B. S4 Model Performance · p. 7, Figure 8 · p. 8
- All reported metrics reflect a single experimental run; future work will report means and standard deviations across multiple runs.The authors identify this as a next step for quantifying variance.Source: B. S4 Model Performance · p. 8
IV. CONCLUSION
The paper evaluates S4 for binary malware detection and multi-class ransomware-family classification, comparing it with CNN, LSTM, GRU, Transformer, and MLP baselines. S4 achieves higher reported family-level F1-scores while maintaining strong detection accuracy, but the study is limited to one dataset.
IV. CONCLUSION
- S4 reaches 89% macro F1-score and 88% weighted F1-score for family classification, versus 75%–76% for CNN, LSTM, GRU, and Transformer and below 67% for MLP.All evaluated models exceed 97% binary detection accuracy, with S4 at 98.5%.Source: §IV · p. 8
- On Phobos, S4 records 0.97 precision and 0.98 recall, a balance the baselines do not maintain across closely related variants.Source: §IV · p. 8
- The study uses a single dataset; future work will test generalizability on CIC-MalMem-2022 and BODMAS and examine dynamic features and concept drift.Source: §IV · p. 8