PaperlayerView exact revision on arXiv

Source-linked AI summary

A TTP by TTP Approach: Precise Malware Detection via Malicious TTP Recognition

Yashovardhan Sharma

arXiv:2609.06579v1cs.CRcs.AI

TL;DR

Malware detectors may ignore TTP knowledge and fail to identify which TTP was maliciously used. This paper supplies TTP features to a multi-label DNN, reporting the best overall performance and stronger average rare-TTP detection, with Macro F1-score improving by 37.55%.

  • Problem

    Malware detection methods may ignore available TTP knowledge and provide little or no explanation of which TTPs were maliciously used.

    Source: §1 · p. 1

  • Method

    The approach adds automatically generated TTP features to network-traffic samples and uses a multi-label DNN to classify sample maliciousness and maliciously used TTPs.

    Source: §1 · p. 1, §3 · p. 3

  • Results

    37.55%, 11.82%, and 46.05% average improvements in Macro F1-score, Macro Precision, and Macro Recall, respectively, accompany STTP2STTP’s best overall performance and significantly better rare-TTP detection.

    Source: §1 · p. 2

  • Takeaways & Limitations

    TTP-specific threshold tuning can adjust positive detections according to TTP frequency and risk in a given environment.

    Source: §6 · p. 13

    The dataset’s TTP labels are highly imbalanced, and several TTPs do not occur at least once on average in the train, validation, and test sets.

    Source: §4.1 · p. 6

Abstract

from arXiv · show

Machine learning methods, and especially neural networks, are now routinely used for malware detection in network traffic. Though very effective, systems based on such methods often (i) are purely data-driven, ignoring the substantial body of available knowledge about the tactics, techniques, and procedures (TTPs) possibly used, and, consequently (ii) are not precise, since they either cannot correlate malicious activity with TTP usage, or if they do, they are unable to explain which TTP has been maliciously used. In this paper we demonstrate that it is possible to precisely detect malware by (i) providing the neural network model with information about the TTPs used by any given sample, and (ii) teaching the neural network to detect not just the malicious activity as a whole, but which specific TTPs are maliciously used. We show that our approach consistently outperforms the three alternative models, which either do not exploit TTP information, or which are not taught to detect the malicious usage of TTPs, or both. Moreover, we show that our approach (i) is particularly beneficial in detecting malware that utilises rarely-used TTPs, a scenario which is particularly challenging for the other systems; (ii) allows for TTP by TTP tuning, further improving its ability to detect the malicious usage of TTPs; (iii) consistently outperforms other systems across a wide-range of scenarios, including when relying on limited training data or when subjected to adversarial attack.

1. Introduction

The paper proposes a multi-label DNN that combines network-traffic features with automatically extracted MITRE ATT&CK TTP features to detect malicious samples and identify which TTPs are maliciously used. This targets the limited TTP-level precision of data-driven malware detectors and is intended to help detect malware using uncommon TTPs.

Approach

  • The multi-label DNN uses network-traffic and automatically generated TTP features to classify sample maliciousness and identify the TTPs used maliciously.The approach analyzes each sample to extract its TTPs, then supplies the traffic and matched-TTP features to the DNN.Source: §1 · pp. 1–2

Evaluation

  • The evaluation uses more than 1.5 million malicious and benign samples from five publicly available datasets and assesses sample classification and TTP classification with micro- and macro-average metrics.The authors use micro-average metrics to assess overall sample classification on balanced datasets and macro-average metrics to assess TTP classification with unbalanced classes.Source: §1 · pp. 1–2

Results

  • STTP2STTP has the best overall performance; for rarely occurring malicious TTPs, its average Macro F1-score, Macro Precision, and Macro Recall improvements are 37.55%, 11.82%, and 46.05%, respectively.STTP2STTP, STTP2S, and S2STTP also consistently outperform the standard S2S model.Source: §1 · p. 2
  • Tuning decision thresholds separately for each TTP improves STTP2STTP’s average Macro F1-score, Macro Precision, and Macro Recall by 12.5%, 12.0%, and 47.6%, respectively.The system supports TTP-by-TTP tuning.Source: §1 · p. 2

Challenging scenarios

  • STTP2STTP detects malicious TTPs better than S2STTP with limited training samples and when adversaries camouflage activity by flooding malicious samples with benign traffic.With 25% of training data, STTP2STTP has a higher average F1-score than S2STTP with 100%; with 10%, it has higher average Recall than S2STTP with 100%.Source: §1 · p. 2

2. Background and Related Work

MITRE ATT&CK organizes real-world adversary behavior into tactics, techniques, and procedures; prior work has applied TTPs broadly, while automated network-malware detection studies focus on approaches with different tradeoffs.

Background

  • ATT&CK is a public, real-world-observation-based knowledge base distinguishing adversary goals (tactics), methods (techniques), and specific implementations (procedures).Techniques can include more specific sub-techniques.Source: §2 · p. 2
  • In the framework version described, Enterprise ATT&CK has 14 tactics, 201 techniques, and 424 sub-techniques; this study focuses on network-using techniques and their encompassing tactics.Source: §2 · p. 3

Related work

  • ATT&CK TTPs have supported diverse cybersecurity tasks, including malware trend tracking, CTI extraction, TTP-chain analysis, and malware detection.Source: §2 · p. 3
  • Prior TTP-based network-malware detectors either use separate per-TTP decision trees or add TTP features to a binary classifier.The decision-tree system, [30] sacrifices cross-TTP correlation and analyzes flows separately; TTPxML uses sample and TTP features to label the whole sample. The paper models TTPxML as the state-of-the-art comparison for its multi-label approach.Source: §2 · p. 3

3. System Architecture

The system converts network traffic into a Bag of Flows (BoF) augmented with automatically extracted TTP information, then uses a DNN to classify sample maliciousness and, in some configurations, identify maliciously used TTPs.

3.1. Constructing the Bag of Flows

  • Flows are annotated at flow level with 13 MITRE ATT&CK TTPs, converted to numeric vectors, and aggregated into BoFs using min-max-scaled normalized histograms; STTP2S and STTP2STTP also include normalized TTP statistics.For each TTP, those models include the proportion and total number of matching flows; total counts are normalized over all samples.Source: §3.1 · pp. 3–4

3.2. Multi-label DNN Classification

  • A four-hidden-layer, fully connected ReLU DNN has 14 outputs—13 TTP labels and Mal-Sample—and supports four input/output configurations.STTP2STTP uses TTP information as input and predicts TTPs and sample maliciousness; S2STTP, STTP2S, and S2S vary whether TTP information is input or output.Source: §3.2 · p. 4
  • Because datasets lack TTP-use labels and the supported TTP list is incomplete, detected TTPs supply labels while Mal-Sample captures malicious samples regardless of TTP matches.Mal-Sample also covers unseen TTPs and detection failures.Source: §3.2 · p. 5
  • When a TTP is predicted malicious but the sample benign, the system flips Mal-Sample to malicious; this rare inconsistency produced no F1-score, Precision, or Recall differences through the third decimal place.This flip enforces coherence in the one-level label hierarchy.Source: §3.2 · p. 5

4. Evaluation

Across five datasets, the evaluation compares four models and finds that combining TTP features with training to identify malicious TTP use performs best for whole-sample detection. TTP features particularly improve malicious-use detection for less frequent TTPs.

4.1. Datasets

  • The five derived datasets use 60/20/20 train/validation/test splits, balance malicious and benign samples, and have pairwise disjoint test sets.TTP occurrence is highly imbalanced; five TTP labels absent from the sets on average are excluded, leaving nine labels including Mal-Sample.Source: §4.1 · p. 6, Table 4 · p. 6

4.2. Metrics

  • The evaluation defines metrics from label-level confusion matrices and reports both micro and macro averages; macro metrics give equal weight to labels and better expose performance on imbalanced TTP labels.The datasets are highly unbalanced at the TTP level.Source: §4.2 · pp. 6–7, §4.1 · p. 6

4.2.1. Micro-average metrics.

  • Micro evaluation reports F1-score, accuracy, precision, recall, false-positive rate, and subset accuracy, the share of samples whose full label sets are classified correctly.F1 is the harmonic mean of precision and recall.Source: §4.2.1 · p. 7

4.2.2. Macro-average metrics.

  • Macro F1 and Macro Recall average their corresponding metrics across nine labels: eight TTPs observed on average and the Mal-Sample label.Unlike micro averages, macro averages give each label equal weight regardless of support.Source: §4.2.2 · p. 7, §4.2 · p. 7

4.3. Evaluating Malicious Usage of TTPs

  • Macro F1 rises from 0.478 to 0.658 (+37.55%), Macro Precision from 0.638 to 0.713 (+11.82%), and Macro Recall from 0.422 to 0.617 (+46.05%) with TTP features.TTP features help for every evaluated TTP except T1021, with larger gains for many less frequent TTPs; for T1105, F1 is 0.686 for STTP2STTP and 0 for S2STTP. Micro Accuracy and Precision are approximately 0.97 and 0.95, while Recall is approximately 0.74.Source: §4.3 · pp. 7–8, Table 5 · p. 7, Table 6 · p. 8

4.4. Evaluating Sample Detection

  • STTP2STTP performs best across all sample-detection metrics, followed by S2STTP; TTP-aware training and TTP features each improve performance, and their combination performs best overall.The authors attribute the benefit of TTP-aware training in part to the model’s ability to focus on malicious samples containing rarely used TTPs.Source: Table 7 · p. 8, §4.4 · p. 9

5. Experimental Analysis

The analysis finds that per-TTP threshold tuning improves macro metrics, while STTP2STTP generally outperforms S2STTP with less training data and under benign-flow injection. The experiments also identify low-support TTP labels as a limitation in data-scarce training and show that detection performance stabilizes at higher noise ratios.

5.1. Threshold Variation per TTP

  • Macro F1-score rises from 0.658 to 0.741 (12.5%), Macro Precision from 0.713 to 0.799 (12.0%), and Macro Recall from 0.617 to 0.911 (47.6%) with per-TTP threshold tuning.The best threshold can differ by label and metric; for T1105, F1-score improves from 0.686 at θ = 0.5 to 0.721 at θ = 0.4.Source: §5.1 · p. 10

5.2. Data Scarcity

  • STTP2STTP outperforms S2STTP in F1-score, Subset Accuracy, and Recall across training-data sizes, though the gap varies and S2STTP occasionally leads in Precision and False Positive Rate.With only 5% of training data, the three higher-support labels retain substantial expected support (T1124: 3594.63, T1571: 4304.44, Mal-Sample: 4654.65), while most other labels have support below 3; their micro-average scores are usually zero.Source: §5.2 · pp. 11–12

5.3. Adversarial Benign Flow Injection

  • Under benign-flow injection, STTP2STTP always outperforms S2STTP, and their performance gap grows steeply before becoming roughly constant or increasing only marginally.Both models’ scores tend to stabilize from a noise ratio of 2 (200% injected flows), when Subset Accuracy becomes almost constant; the authors attribute this to noise-robust samples being more numerous with TTP information.Source: §5.3 · p. 12

6. Discussion

The discussion argues that TTP-aware multi-label detection improves malware detection while identifying malicious TTP use, and highlights its configurability, performance under challenging conditions, and potential for finer-grained outputs.

Explainability and Precision in Detecting Malware

  • STTP2STTP identifies which specific TTPs contributed to a malicious classification, and its mean Accuracy exceeds S2STTP by 0.83%, STTP2S by 0.95%, and S2S by 1.81%.This per-TTP output makes detections more precise and explainable than a binary malicious-or-benign output.Source: §6 · p. 12

Recognition of rarely-occurring TTPs

  • With only 15 training datapoints for T1090, the system achieves an average F1-score of 0.946 when identifying malware using that TTP.The authors report that the approach can identify malicious behavior involving TTPs represented by few datapoints in training.Source: §6 · p. 12

System Configurability

  • Tuning individual TTP thresholds improves average Macro F1-score by 12.5%, Macro Precision by 12%, and Macro Recall by 47.6%.Users can lower thresholds for rare, high-risk TTPs and raise them for common, lower-risk TTPs to adjust detection to their threat model.Source: §6 · pp. 12–13

Utility of TTPs in Malware Detection

  • STTP2STTP improves average F1-score over S2STTP by 1.08% at sample level and 37.65% at TTP level, and outperforms the alternatives across every sample-level metric.The authors attribute this consistent advantage over binary DNN systems to the multi-label architecture, which can outperform even systems that use TTP features.Source: §6 · pp. 12–13

Data Scarcity

  • Across 30 experiments, STTP2STTP outperforms S2STTP on F1-score in 96.67% of cases, including when trained on less data.With 25% of training data, STTP2STTP has higher average F1-score than S2STTP trained on 100%; with 10%, it has higher average Recall.Source: §6 · p. 13

Resilience to Adversarial Attack

  • At a Sample Noise Ratio of 10.0, STTP2STTP improves over S2STTP by 28.93% in average F1-score, 62.17% in Subset Accuracy, 48.71% in Precision, 14.50% in Recall, and 20.23% in FPR.The study evaluates camouflaging attacks that inject benign flows into malicious samples; STTP2STTP performs better as noise increases, though performance declines.Source: §6 · p. 13

Minimal Additional Data

  • TTP features derived from open-source MITRE ATT&CK information can be created from existing data without additional training data.The authors present this as a performance benefit for data-greedy machine-learning algorithms.Source: §6 · p. 13

Different Levels of Granularity

  • The model reports maliciousness at sample and TTP levels, and the authors propose extending this hierarchy to tactics, techniques, sub-techniques, and procedures [3].The proposed levels could let the model identify malicious activity at a broader level when it cannot confidently identify the specific technique or procedure.Source: §6 · p. 13

7. Conclusions

The multi-label DNN detects malicious behavior at both TTP and sample levels, outperforming comparable systems overall and for malware using uncommon TTPs; it also provides precise TTP-level outputs with little additional data.

7. Conclusions

  • The multi-label DNN detects maliciousness per TTP and per sample, improving overall malware detection and TTP detection, including for malware using uncommon TTPs.Comparative analysis uses over 1.5 million real-world malware and benign samples; the system continues to outperform comparable systems with limited training data and under adversarial camouflage attacks.Source: §7 · p. 13
  • The system requires minimal additional data and identifies which TTPs are being used maliciously, enabling future hierarchical TTP representations for finer-grained detection.Source: §7 · p. 13
Loading 2609.06579v1…