PaperlayerView exact revision on arXiv

Source-linked AI summary

A TTP by TTP Approach: Precise Malware Detection via Malicious TTP Recognition

Yashovardhan Sharma

arXiv:2609.06579v1cs.CRcs.AI

TL;DR

Existing malware detectors often ignore TTP knowledge or fail to identify which TTPs are maliciously used. The paper addresses this gap with a multi-label DNN that incorporates automatically extracted TTP features and jointly predicts sample maliciousness and malicious TTPs. Its approach consistently outperforms alternatives, including for rare TTPs, limited training data, and adversarial camouflage, while TTP-specific threshold tuning further improves performance.

  • Problem

    Existing malware detectors may ignore TTP knowledge or fail to correlate malicious activity with, and explain, the specific TTPs used.

  • Method

    The paper uses a multi-label DNN that combines network-traffic features with automatically extracted TTP features to classify sample maliciousness and maliciously used TTPs.

  • Results

    The proposed STTP2STTP model has the best overall performance and improves rare-TTP detection by 37.55% Macro F1-score, 11.82% Macro Precision, and 46.05% Macro Recall on average.

  • Takeaways & Limitations

    The approach supports precise, TTP-specific malware detection and remains superior to alternatives under limited-data and adversarial-camouflage conditions.

  • Takeaways & Limitations

    TTP labels are highly unbalanced, and several TTPs do not occur in at least one average train, validation, or test split.

Abstract

from arXiv · show

Machine learning methods, and especially neural networks, are now routinely used for malware detection in network traffic. Though very effective, systems based on such methods often (i) are purely data-driven, ignoring the substantial body of available knowledge about the tactics, techniques, and procedures (TTPs) possibly used, and, consequently (ii) are not precise, since they either cannot correlate malicious activity with TTP usage, or if they do, they are unable to explain which TTP has been maliciously used. In this paper we demonstrate that it is possible to precisely detect malware by (i) providing the neural network model with information about the TTPs used by any given sample, and (ii) teaching the neural network to detect not just the malicious activity as a whole, but which specific TTPs are maliciously used. We show that our approach consistently outperforms the three alternative models, which either do not exploit TTP information, or which are not taught to detect the malicious usage of TTPs, or both. Moreover, we show that our approach (i) is particularly beneficial in detecting malware that utilises rarely-used TTPs, a scenario which is particularly challenging for the other systems; (ii) allows for TTP by TTP tuning, further improving its ability to detect the malicious usage of TTPs; (iii) consistently outperforms other systems across a wide-range of scenarios, including when relying on limited training data or when subjected to adversarial attack.

1. Introduction

The paper addresses malware detectors that ignore TTP knowledge or cannot explain which TTPs are maliciously used. It proposes a multi-label DNN that uses TTP features and jointly detects sample maliciousness and maliciously used TTPs, outperforming alternatives across challenging settings.

  • Motivation: The paper identifies data-driven malware detectors as insufficiently precise because they may not correlate malicious activity with TTP usage or explain which TTPs were maliciously used.Such methods may also require substantial training data and lack robustness to adversarial attacks.
  • Approach: The proposed architecture automatically extracts TTPs, combines their features with network-traffic features, and jointly classifies overall maliciousness and maliciously used TTPs.The task is formulated as multi-label classification with separate TTP and sample-maliciousness labels.
  • Evaluation: STTP2STTP, STTP2S, and S2STTP consistently outperform the standard S2S model, with STTP2STTP achieving the best overall performance.The four configurations differ in whether TTP features are used as inputs and whether TTP labels are produced as outputs.
  • Evaluation: 37.55%, 11.82%, and 46.05% are the average improvements in Macro F1-score, Macro Precision, and Macro Recall for detecting rarely-occurring malicious TTPs with STTP2STTP.The paper identifies rare-TTP detection as a particularly challenging scenario for alternative systems.
  • Further analysis: 12.5%, 12.0%, and 47.6% are the average gains in Macro F1-score, Macro Precision, and Macro Recall from tuning decision thresholds separately for each TTP.The system is evaluated on five balanced datasets containing over 1.5 million samples and under limited-data and adversarial conditions.

2. Background and Related Work

The paper situates its approach within MITRE ATT&CK, a knowledge base of adversarial behavior, and prior work using TTPs for automated malware detection. ATT&CK organizes adversarial behavior into tactics, techniques, and procedures, while earlier detection work used separate binary classifiers for individual TTPs.

  • MITRE ATT&CK: MITRE ATT&CK is a publicly available knowledge base of adversarial tactics, techniques, and procedures based on real-world observations.The framework is frequently updated to track evolving malware and adversarial approaches.
  • MITRE ATT&CK: A tactic is an adversary’s tactical goal, a technique is the method used to achieve it, and a procedure is a specific implementation of that method.The paper illustrates these levels with credential dumping, credential access, and a concrete tool or file-based implementation.
  • Prior uses: ATT&CK TTPs have been used to track malware evolution, locate behaviors in executable control-flow graphs, and automatically extract behaviors from cyber-threat-intelligence reports.These uses exploit domain knowledge about adversarial behavior in automated analysis.
  • Related detection work: Prior automated detection work created a separate dataset and decision tree for each TTP, labeling samples positive when they were malicious and contained that TTP.The paper models this state-of-the-art system with its STTP2S configuration for comparison.

3. System Architecture

The system converts network traffic into a Bag of Flows enriched with automatically extracted MITRE ATT&CK TTP information, then uses a multi-label DNN to classify both sample maliciousness and maliciously used TTPs. Its architecture supports four configurations that vary whether TTP information is provided as input and whether TTP-specific outputs are required.

  • System pipeline: Raw network traffic is converted into a Bag of Flows, enriched with automatically extracted TTP features, and passed to a multi-label DNN.The system outputs either overall maliciousness or both overall maliciousness and the maliciously used TTPs, depending on configuration.
  • System pipeline: Each sample is represented as network flows annotated with 13 MITRE ATT&CK TTPs, covering 11 tactics, 9 techniques, and 6 sub-techniques.Flow-level TTP matching follows the methodology described for the TTP annotations.
  • Bag of Flows: The Bag of Flows scales flow features and records normalized histograms of their values across five bins for each non-categorical feature.This produces a vectorial representation of the original network sample.
  • TTP features: TTP-aware inputs include per-TTP flow proportions and counts, plus aggregate statistics describing the number and distribution of matched TTPs.These features are included in the STTP2S and STTP2STTP configurations.
  • Multi-label DNN: The DNN has 14 output neurons: 13 for supported TTPs and one for the maliciousness of the complete sample.This output design supports both multi-label and binary classification settings.
  • Classification configurations: The four configurations are STTP2STTP, S2STTP, STTP2S, and S2S, differing in whether TTP features are inputs and whether TTP-specific predictions are outputs.The system formulates these tasks within multi-label classification and uses a Mal-Sample label to represent malicious samples that match no supported TTP.

4. Evaluation

The evaluation compares four models across five datasets to test whether TTP features and explicit detection of malicious TTP usage improve malware and sample classification. STTP2STTP, which uses both inputs and predicts both outputs, achieves the strongest overall results, including for rare TTPs.

  • Evaluation design: The study evaluates STTP2STTP, S2STTP, STTP2S, and S2S across five datasets to isolate the effects of TTP features and malicious-TTP detection.The models differ in whether they receive TTP features and whether they predict malicious TTPs, sample maliciousness, or both.
  • Datasets: The datasets track matched samples and TTP-level statistics across training, validation, and test sets, including the number and percentage of samples containing each TTP.The evaluation uses dataset statistics to characterize TTP support and class imbalance.
  • Datasets: Five rarely occurring TTP labels are excluded because they do not appear often enough in the corresponding training, validation, or test sets for reliable evaluation.The retained labels are T1135, T1124, T1071, T1105, T1090, T1550, T1571, T1021, plus Mal-Sample.
  • Malicious TTP detection: Providing TTP features consistently improves performance across metrics, with especially large gains for less frequent TTPs; for T1105, STTP2STTP reaches F1-score 0.686 versus 0 for S2STTP.The benefit is relatively modest for highly occurring TTPs such as T1124 and T1571.
  • Malicious TTP detection: Macro F1-score rises from 0.478 to 0.658, Macro Precision from 0.638 to 0.713, and Macro Recall from 0.422 to 0.617 for STTP2STTP relative to S2STTP.These correspond to increases of 37.55%, 11.82%, and 46.05%, respectively.
  • Overall findings: STTP2STTP has the best overall performance and better detects rarely occurring malicious TTPs, while TTP features and explicit malicious-TTP prediction each contribute to performance.The model also maintains its advantage under limited training data and adversarially camouflaged traffic.

5. Experimental Analysis

The experiments evaluate TTP-aware models under threshold tuning, reduced training data, and adversarial camouflage. STTP2STTP generally outperforms S2STTP, with the advantage especially visible for rare TTPs and noisy samples.

  • Threshold variation per TTP: 12.5%, 12.0%, and 47.6% average gains are obtained for Macro F1-score, Macro Precision, and Macro Recall through per-TTP threshold tuning.Thresholds can be varied independently for each TTP; the best threshold need not be 0.5.
  • Limited training data: STTP2STTP achieves better F1-score, Subset Accuracy, and Recall than S2STTP across training-set sizes, although the gap varies with the sampled percentage.S2STTP exceeds STTP2STTP at a few training percentages for Precision and False Positive Rate.
  • Limited training data: Low-support labels become especially difficult with fewer training samples, with most labels other than T1124, T1571, and Mal-Sample having support below 3.The analysis attributes much of the small-data behavior to reduced support for rare labels.
  • Adversarial camouflage: STTP2STTP consistently outperforms S2STTP as benign-flow noise increases, while the performance gap grows steeply before stabilizing.Both models show large standard deviations, with S2STTP usually more variable except for Recall.
  • Adversarial camouflage: From noise ratio 2 onward, Subset Accuracy is almost constant for both models, indicating some samples retain robust TTP-detection signals despite injected benign flows.More such robust samples are observed when TTP information is used.

6. Discussion

The discussion presents TTP-aware multi-label detection as more accurate, configurable, data-efficient, and resilient than the comparison systems. It also emphasizes that the architecture produces explanations at both sample and TTP levels.

  • Consistent improvement and explainability: STTP2STTP and S2STTP outperform the binary DNN systems, while TTP-level outputs identify which specific behaviors contributed to a malicious classification.The multi-label architecture enables both stronger detection and more explainable outputs than a binary malicious-benign decision.
  • Utility of TTPs: 1.08% and 37.65% average F1-score improvements occur at the sample and TTP levels, respectively, when comparing STTP2STTP with S2STTP.At sample level, STTP2STTP also outperforms S2STTP, STTP2S, and S2S across F1-score, Accuracy, Precision, Recall, and FPR.
  • Data scarcity: 96.67% of 30 limited-data experiments favor STTP2STTP over S2STTP by F1-score, including cases where 25% or 10% of training data beats S2STTP trained on 100%.The reported comparisons are higher average F1-score at 25% training data and higher average Recall at 10% training data.
  • Resilience to adversarial attack: 28.93% is the maximum average F1-score improvement at noise ratio 10.0, while STTP2STTP also improves every reported metric in the noisiest condition.At noise ratio 10.0, the improvements are 62.17% in Subset Accuracy, 48.71% in Precision, 14.50% in Recall, and 20.23% in FPR.
  • Data requirements and granularity: TTP features are created from open-source MITRE ATT&CK knowledge without requiring additional training data, providing performance benefits for data-greedy models.The approach can report maliciousness at both sample and TTP granularity, with possible future extensions to hierarchical TTP representations.

7. Conclusions

The paper concludes that its multi-label DNN detects malicious behavior at both sample and TTP levels and remains superior across challenging evaluation settings. It requires little additional data and can support more granular future representations.

  • Contributions: The multi-label DNN detects malicious behavior at two granularities: the overall network sample and the individual TTPs.The evaluation covers more than 1.5 million real-world malware and benign samples and challenging conditions including limited data and adversarial camouflage.
  • Implications: The system requires minimal additional data and provides precise outputs identifying maliciously used TTPs, supporting future hierarchical TTP representations.The proposed extensions could provide finer-grained malware-detection outputs.
Loading 2609.06579v1…