PaperlayerView exact revision on arXiv

Source-linked AI summary

CVE-driven Attack Technique Prediction with Semantic Information Extraction and a Domain-specific Language Model

Ehsan Aghaei, Ehab Al-Shaer

arXiv:2309.02785v1cs.CRcs.LG

TL;DR

The paper addresses the gap between CVE descriptions and the ATT&CK techniques that may result from exploiting vulnerabilities, especially with limited labeled data and semantic divergence. It combines SRL-based threat-action extraction, contextual SecureBERT classification, and functionality mapping, achieving approximately 98% accuracy with 95%–98% F1-scores and outperforming ChatGPT in the reported comparison.

  • Problem

    Mapping CVEs to ATT&CK TTPs is constrained by limited labeled datasets and semantic divergence between vulnerability descriptions and threat techniques.

    Source: §1 · p. 2, §8 · p. 13

  • Method

    The method uses SRL to extract threat actions, builds contextual content–context pairs, and fine-tunes SecureBERT to classify CVEs into functionality and ATT&CK categories.

    Source: §4 · p. 6, §5 · p. 8, §8 · p. 13

  • Results

    Approximately 98% accuracy and 95%–98% F1-scores were reported, while TTPpredictor averaged 93% accuracy versus ChatGPT’s 20% over randomly selected CVEs.

    Source: §8 · p. 13

  • Takeaways & Limitations

    The resulting annotated dataset and classifier support precise CVE characterization into corresponding attack techniques.

    Source: §8 · p. 13

    Performance is more confused for ambiguous content–context pairs in T_S1, whereas fuller CVE context in T_S2 improves prediction for dependency-related functionalities.

    Source: §6 · pp. 10–11

Abstract

from arXiv · show

This paper addresses a critical challenge in cybersecurity: the gap between vulnerability information represented by Common Vulnerabilities and Exposures (CVEs) and the resulting cyberattack actions. CVEs provide insights into vulnerabilities, but often lack details on potential threat actions (tactics, techniques, and procedures, or TTPs) within the ATT&CK framework. This gap hinders accurate CVE categorization and proactive countermeasure initiation. The paper introduces the TTPpredictor tool, which uses innovative techniques to analyze CVE descriptions and infer plausible TTP attacks resulting from CVE exploitation. TTPpredictor overcomes challenges posed by limited labeled data and semantic disparities between CVE and TTP descriptions. It initially extracts threat actions from unstructured cyber threat reports using Semantic Role Labeling (SRL) techniques. These actions, along with their contextual attributes, are correlated with MITRE's attack functionality classes. This automated correlation facilitates the creation of labeled data, essential for categorizing novel threat actions into threat functionality classes and TTPs. The paper presents an empirical assessment, demonstrating TTPpredictor's effectiveness with accuracy rates of approximately 98% and F1-scores ranging from 95% to 98% in precise CVE classification to ATT&CK techniques. TTPpredictor outperforms state-of-the-art language model tools like ChatGPT. Overall, this paper offers a robust solution for linking CVEs to potential attack techniques, enhancing cybersecurity practitioners' ability to proactively identify and mitigate threats.

1 Introduction

The paper addresses the difficulty of mapping CVEs to ATT&CK TTPs amid scarce labeled data and semantic gaps. It combines SecureBERT, SRL, SVO extraction, and contextual analysis to characterize threat actions and support automated classification.

1 Introduction

  • CVEs-to-TTP classification is hindered by scarce labeled datasets, semantic gaps, and limited domain-specific resources.The mapping is important for understanding threat techniques and informing defensive strategies.Source: §1 · p. 2
  • SecureBERT and semantic role labeling characterize contextual threat actions and map them to functionality classes and ATT&CK techniques.The approach treats functionalities as capabilities attackers acquire after exploiting vulnerabilities.Source: §1 · p. 2
  • SVO representations provide concise threat descriptions, while paradigmatic relations and surrounding context compensate for limited labeled training data.Context helps distinguish actions with similar wording but different intentions or impacts.Source: §1 · p. 2
  • The same SVO can represent different functionalities, so classification must integrate content with context rather than rely on context alone.For example, “attackers read memory” can indicate attack impact or credential theft depending on its surrounding statement.Source: §1 · pp. 2–3

2 Functionality

The paper organizes attacker capabilities into action and impact functionalities and models their dependencies. These dependencies capture inheritance and semantic commonality, guiding corpus construction, annotation, and classification.

2 Functionality

  • Functionalities comprise action capabilities, such as reading files, and impact capabilities, such as memory errors, within sixteen MITRE-defined classes.The paper notes that labeled data linking CVEs to these functionalities must be manually collected and annotated.Source: §2 · p. 3, Table 2 · p. 4
  • Inheritance means a child functionality retains its parent’s characteristics while adding attributes, such as out-of-bound reads inheriting from memory reads.These shared characteristics concern threat actions and MITRE techniques.Source: §2 · pp. 3–4
  • Commonality captures semantic similarity between functionality classes, distinguishing strong overlap from weak overlap between differently or similarly described concepts.Strong commonality can make classes difficult to distinguish, whereas weak commonality can let one phrase indicate distinct concepts.Source: §2 · pp. 4–5
  • Functionality dependencies guide corpus generation, dataset creation, and model training and evaluation.The paper formalizes inheritance, strong commonality, and weak commonality relationships for this purpose.Source: §2 · p. 5

3 Problem Definition

The research targets contextual CVE functionality classification when labeled data are unavailable. It asks how SRL can generate training data, whether context-oriented modeling improves classification, and how the method compares with models such as ChatGPT.

3 Problem Definition

  • The central problem is assigning CVEs to functionalities despite scarce labeled data and the context dependence of threat-action meaning.The proposed direction combines SRL, frequent verb and object extraction, and a domain-specific classifier.Source: §3 · p. 5
  • RQ.1 asks how SRL can generate labeled cybersecurity datasets for different tasks.Source: §3 · p. 6
  • RQ.2 asks whether a context-oriented classification design improves classification performance and robustness.Source: §3 · p. 6
  • RQ.3 examines the methodology’s strengths and limitations and possible improvement using state-of-the-art language models such as ChatGPT [14].Source: §3 · p. 6

4 Data Extraction, Annotation, and Assessment

The paper builds labeled functionality data from CVE and security-report text by extracting SVOs with SRL, then uses context and causal links to distinguish action and impact functionalities.

Data Extraction, Annotation, and Assessment

  • SRL converts sentences into semantic roles, enabling extraction of subjects, verbs, objects, and related arguments from security text.The AllenNLP SRL tool takes a sentence and assigns semantic roles to words or phrases; its outputs support SVO construction.Source: §4 · p. 6, Figure 2 · p. 6
  • The functionality taxonomy separates attacker actions, such as reading files or installing apps, from exploit impacts such as memory or buffer errors.This distinction captures both what an attacker does and the resulting consequence of exploitation.Source: §4 · p. 6
  • Action functionalities are represented as SVO triplets whose subjects, verbs, and objects are selected from manually annotated dictionaries.For each functionality, expert-selected verb and object sets guide AllenNLP extraction from CVE reports to generate an initially labeled dataset.Source: §4 · p. 6
  • Causal links extend SVO representations when an action or fault causes an impact, helping distinguish otherwise similar functionalities.For example, reading memory maps to Read From Memory, while reading memory that causes buffer errors maps to Memory Read (Memory Buffer Errors).Source: §4 · p. 7
  • Context-dependent threat actions are analyzed with SRL because identical SVOs can imply different functionalities in different CVE descriptions.The approach combines extracted phrases with surrounding document context to identify implicit patterns and distinguish interpretations such as authentication bypass or memory reading.Source: §4 · pp. 6–8
  • The assessment uses 110K CVE descriptions, 10,000 content-context pairs, 1,098 manually annotated SVOs, and two testing datasets.TS1 uses a 75%/25% train-test split, while TS2 contains 494 manually labeled CVEs whose context is the full CVE description.Source: §4 · pp. 7–8

5 Methodology

The methodology classifies threat-action content into functionalities by pairing concise SVO representations with broader CVE context. SecureBERT processes these pairs, while staged sampling and context-sensitive rules address semantic ambiguity and limited labeled data.

5 Methodology

  • The model classifies an SVO action using its surrounding CVE context, returning the content’s corresponding functionality.Content is a short action description, whereas context is a longer text containing related discussion.Source: §5 · p. 8
  • SecureBERT concatenates content and context with separator tokens, then fine-tunes a classification layer over the [CLS] representation.The output layer contains one neuron for each functionality.Source: §5 · p. 8
  • Training pairs combine SVOs with varied representations, manually labeled positive sentences, and full CVE descriptions to learn paradigmatic and contextual relationships.Pair generation uses multiple stages and limits samples per functionality to reduce oversampling and imbalance.Source: §5 · p. 8
  • Special labeling rules use causal objects in CVE descriptions to distinguish otherwise ambiguous memory-related functionalities.The same “attackers read memory” SVO can receive different labels depending on contextual objects.Source: §5 · p. 8
  • Context resolves cases where identical SVOs correspond to different functionalities, such as distinguishing memory reading from memory errors.The model links the action to contextual indicators such as denial-of-service impact or improper access checks.Source: §5 · p. 9, Table 4 · p. 9

6 Evaluation

Evaluation shows strong functionality classification across both testing datasets, including imbalanced and dependency-linked classes. A proof-of-concept CVE test also achieves complete top-five coverage, while comparison results show substantial ChatGPT errors.

6 Evaluation

  • The model shows high performance on both TS1 and TS2 functionality tests, according to the reported evaluation.Table 5 reports performance for the CVE-to-functionality classification model across all testing datasets.Source: §6 · p. 9, Table 5 · p. 10
  • Confusion matrices show high performance despite imbalance, with the greatest confusion among inheritance-related memory and file-reading functionalities.Read From Memory, Read Files, and Memory Read (Memory Errors) are especially difficult to separate.Source: §6 · pp. 9–10, Figure 4 · p. 10, Figure 5 · p. 10
  • Precise full-CVE context improves prediction for functionalities with dependencies in TS2.The TS2 evaluation uses entire CVE descriptions as context.Source: Figure 5 · p. 10, §6 · p. 11
  • 100% of the 66 proof-of-concept CVE tests are covered within the top five predictions.The reported hit rate is 87.88% at the exact number of correct classes and 96.97% with one extra prediction.Source: §6 · p. 11
  • ChatGPT failed on 80% of 20 CVE-to-ATT&CK technique predictions evaluated against MITRE classifications.The comparison is presented as evidence of a gap between general-purpose and domain-specific classification for this task.Source: §6 · p. 11

7 Related Works

Related work links vulnerability and threat information using classifiers, extraction pipelines, knowledge bases, and language models. The cited approaches differ in granularity, coverage, semantic extraction, labeled data, and classification performance.

7 Related Works

  • The related-work landscape is motivated by sustained efforts to extract threat information and connect vulnerability data with MITRE ATT&CK.The cited examples include automated action extraction, knowledge-base construction, and CVE labeling.Source: §7 · p. 11
  • Other extraction systems identify threat actions from reports but have semantic limitations or do not link vulnerability information to threats.The cited approaches use information retrieval, BERT-BiLSTM-CRF extraction, TF-IDF ontology mapping, or grammatical tagging.Source: §7 · pp. 11–12
  • A bias-correction approach classified threat information from large report collections with accuracy close to 60%.Its evaluation used threat-report datasets such as APT notes and Symantec data.Source: §7 · p. 12
  • BRON integrates ATT&CK, CWE, CVE, and CAPEC relationships but is too general or abstract for training CVE-to-technique classifiers.Its broad knowledge base does not provide sufficiently specific CVE and MITRE technique labels.Source: §7 · pp. 12–13
  • Prior CVE-to-ATT&CK mapping studies report constrained coverage or limited generalization, including 17 techniques and an F1-score of 47.84%.The cited limitations include small knowledge bases, small training sets, and mappings limited to 31 techniques.Source: §7 · p. 13
  • CVET labels CVEs with ten ATT&CK tactics using RoBERTa fine-tuning and self-knowledge distillation, achieving a 76.1% F1-score.The cited work maps at the tactic level rather than providing fine-grained CVE-to-technique classification.Source: §7 · p. 13

8 Conclusions

The paper addresses limited training data and semantic divergence between CVE descriptions and ATT&CK TTPs by extracting threat actions and building an annotated dataset with SRL and a cybersecurity language model. Evaluation reports approximately 98% accuracy, F1-scores of 95%–98%, and higher accuracy than ChatGPT, while future work targets defensive-action classification and effectiveness assessment.

Challenge

  • The approach confronts limited CVE-classification training data and semantic divergence between CVE information and the ATT&CK TTP framework.Source: §8 · p. 13

Method

  • SRL-based extraction from unstructured cybersecurity reports and refinement of a cybersecurity language model produce an annotated dataset for training and classifying CVEs into attack techniques.Source: §8 · p. 13

Evaluation

  • 93% average accuracy versus ChatGPT’s 20% shows TTPpredictor outperforming the language-model baseline across iterations on randomly selected CVEs.The comparison uses numerous iterations involving sets of 100 randomly selected CVEs (see Table 6).Source: §8 · p. 13
  • Approximately 98% accuracy and 95%–98% F1-scores support precise classification of CVE threat actions into corresponding ATT&CK techniques.Source: §8 · p. 13

Future work

  • Future work will classify CVEs and threat reports for defensive actions and assess their effectiveness.Source: §8 · p. 13
Loading 2309.02785v1…