PaperlayerView exact revision on arXiv

Source-linked AI summary

Unveiling Hidden Links Between Unseen Security Entities

Daniel Alfasi, Tal Shapira, Anat Bremler Barr

arXiv:2403.02014v1cs.CRcs.AI

TL;DR

Vulnerability scoping requires identifying weaknesses and affected products at scale, including for new vulnerabilities without additional training. VulnScopper combines knowledge-graph and text representations with ULTRA to handle unseen entities, reporting 76% and 71% Hits@10 accuracy on RedHat and NVD and an 11.7% improvement over ChatGPT4 for Red Hat CWE prediction.

  • Problem

    Existing methods do not satisfy the combined requirements of accurately identifying weaknesses and affected products, scaling to extensive vulnerability data, and handling new vulnerabilities without additional training.

    Source: §1 · p. 2

  • Method

    VulnScopper combines knowledge-graph relations with NLP embeddings of security-entity descriptions and uses ULTRA to generalize to unseen nodes.

    Source: §1 · p. 2, §2.3.2 · p. 4

  • Results

    76% and 71% Hits@10 accuracy were achieved on RedHat and NVD, respectively, and Red Hat CWE prediction improved 11.7% over ChatGPT4.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Ranking links to unseen vulnerabilities is a step toward AI-assisted analysis, prioritization, and response that can accelerate vulnerability scoping.

    Source: §8 · p. 12

    The inductive setup evaluates generalization using an inference graph that extends the dataset with links unseen during training.

    Source: §2.3.1 · p. 4

Abstract

from arXiv · show

The proliferation of software vulnerabilities poses a significant challenge for security databases and analysts tasked with their timely identification, classification, and remediation. With the National Vulnerability Database (NVD) reporting an ever-increasing number of vulnerabilities, the traditional manual analysis becomes untenably time-consuming and prone to errors. This paper introduces VulnScopper, an innovative approach that utilizes multi-modal representation learning, combining Knowledge Graphs (KG) and Natural Language Processing (NLP), to automate and enhance the analysis of software vulnerabilities. Leveraging ULTRA, a knowledge graph foundation model, combined with a Large Language Model (LLM), VulnScopper effectively handles unseen entities, overcoming the limitations of previous KG approaches. We evaluate VulnScopper on two major security datasets, the NVD and the Red Hat CVE database. Our method significantly improves the link prediction accuracy between Common Vulnerabilities and Exposures (CVEs), Common Weakness Enumeration (CWEs), and Common Platform Enumerations (CPEs). Our results show that VulnScopper outperforms existing methods, achieving up to 78% Hits@10 accuracy in linking CVEs to CPEs and CWEs and presenting an 11.7% improvement over large language models in predicting CWE labels based on the Red Hat database. Based on the NVD, only 6.37% of the linked CPEs are being published during the first 30 days; many of them are related to critical and high-risk vulnerabilities which, according to multiple compliance frameworks (such as CISA and PCI), should be remediated within 15-30 days. Our model can uncover new products linked to vulnerabilities, reducing remediation time and improving vulnerability management. We analyzed several CVEs from 2023 to showcase this ability.

1 Introduction

Vulnerability records require timely classification of weaknesses and affected products, but manual analysis is slow and product scoping can lag. VulnScopper combines knowledge-graph and text representations to predict links, including for unseen vulnerabilities, and is evaluated on two security datasets.

1 Introduction

  • More than 6% of NVD CVEs recorded from 2019 to 2023 received a new CPE configuration over 30 days after recording.The paper notes that Log4Shell received a new CPE configuration over 247 days after recording.Source: §1 · p. 1
  • The approach targets limits in prior NLP and graph methods: NLP can miss implicit relations, while shallow entity lookup embeddings cannot predict unseen entities.The paper also identifies scalability as an issue for previous graph-based methods and presents ULTRA as its solution.Source: §1 · p. 2
  • Current methods do not meet all stated requirements: accurately identifying weaknesses and products, scaling to extensive vulnerability data, and handling new vulnerabilities without additional training.The paper links delayed CPE updates to the difficulty of determining which products incorporate a vulnerable component.Source: §1 · pp. 1–2
  • VulnScopper combines knowledge-graph relations with NLP representations of entity descriptions and uses ULTRA to generalize to unseen nodes.Its evaluation covers the NVD and Red Hat CVE databases in transductive and inductive modes.Source: §1 · p. 2
  • 76% and 71% Hits@10 accuracy were achieved on the RedHat and NVD datasets, respectively; CWE prediction on RedHat improved 11.7% over ChatGPT4.The paper also reports evaluation in transductive and inductive modes and says VulnScopper outperformed previous work.Source: §1 · p. 2
  • The authors report linking unseen vulnerabilities without retraining and manually analyze newly predicted product links for cases including lib-webp, libcurl, and Log4shell.They describe this capability as allowing security teams to focus on vulnerabilities without having to retrain models.Source: §1 · p. 2

2 Preliminaries

This section establishes CVE, CWE, and CPE entities and the NLP and knowledge-graph background for link prediction, including ULTRA’s role in handling unseen vulnerabilities.

2.1 Security Entities

  • CVE identifiers standardize references to publicly known vulnerabilities, while CWE categories describe weakness types and CPE names identify affected platforms and software.The section presents these entities as the basis for understanding vulnerability weaknesses and scope.Source: §2.1 · p. 2, §2.1.1 · pp. 2–3, §2.1.2 · p. 3

2.1.1 Common Weakness Enumeration (CWE)

  • CWE organizes weakness types hierarchically, with broad parent categories and more specific child weaknesses.It also includes relations such as “can precede” and “can follow” to describe potential sequences between weaknesses.Source: §2.1.1 · pp. 2–3

2.1.2 Common Platform Enumeration (CPE)

  • CPE is a structured naming scheme for IT systems, software, and packages, specifying product attributes such as vendor, name, and version.The passage illustrates the scheme with Chrome and Fedora CPE records.Source: §2.1.2 · p. 3

2.2 Natural Language Processing

  • NLP methods range from rules and statistical text features to contextual embeddings and language models used for tasks including classification and link prediction.VulnScopper uses OpenAI’s Ada embeddings to represent security-entity descriptions.Source: §2.2 · p. 3

2.3 Knowledge Graphs Reasoning

  • Knowledge-graph reasoning methods use graph links and entity representations to support reasoning and link prediction.The section traces approaches from graph-ranking and neighborhood methods to embeddings, attention networks, and models that generalize to unseen entities.Source: §2.3 · pp. 3–4

2.3.1 Knowledge Graph Link Prediction

  • Link prediction infers a missing entity in a knowledge-graph triplet; inductive evaluation tests generalization to entities absent from training.Transductive evaluation assumes entities and relations are known during training, whereas the inductive setup uses an inference graph that may introduce new entities.Source: §2.3.1 · p. 4

2.3.2 ULTRA

  • ULTRA supports entity representations for unseen vulnerabilities through transferable relation representations and entity-level link prediction.Its described pipeline constructs a relation graph, conditions relation representations on the query, and uses an entity-level predictor.Source: §2.3.2 · pp. 4–5

3 VulnScopper Motivation

Delayed CPE updates and inconsistent CWE assignments complicate manual analysis of vulnerability scope and exposure time, motivating automated CVE-to-CWE and CVE-to-CPE linking.

3.1 CVE and CPE Association

  • 6.83%, 6.69%, 6.09%, and 4.83% of NVD CVEs received new CPE configurations after 1, 7, 30, and 180 days, respectively.The analysis covers more than 111,000 CVEs published from 2019 to 2023, averaging three CPEs per CVE; recent-year rates are lower as scope remains unrevealed.Source: §3.1 · p. 5, Table 1 · p. 5

3.2 CVE and CWE Association

  • 25.39% of NVD CVEs and 42.33% of Red Hat CVEs lack CWEs, while 44.3% of CVEs classified in both databases have different CWE assignments.The databases differ in scope, and CWE hierarchy permits broader or more specific labels; CVE-2023-38545 is labeled CWE-787 in NVD and CWE-119 in Red Hat.Source: §3.2 · p. 5

3.3 The Exposure Time of Software Systems

  • Exposure time spans CVE publication to security-team remediation, so the section argues that quickly clarifying vulnerability scope is important.The authors propose ranking related CWEs and CPEs to guide analysts toward understanding a vulnerability’s full scope.Source: §3.3 · p. 5

4 Related Work

Prior vulnerability-analysis work applies NLP to descriptions and knowledge graphs to capture relations, but existing graph methods cannot predict links to unseen nodes. VulnScopper combines ULTRA with textual embeddings.

4 Related Work

  • Prior work on vulnerability databases has been studied in industry and academia, while NLP and graph methods address different aspects of vulnerability analysis.Description-based methods treat CVEs separately; graph-based methods represent their relations.Source: §4 · pp. 5–6
  • NLP approaches predict weaknesses or products from individual CVE descriptions, which can omit relational information such as associated CPEs.For CVE-2023-4863, affected products including Fedora, Debian Linux, Firefox, Microsoft Edge, and Mozilla Thunderbird are not part of the description.Source: §4 · pp. 5–6
  • Knowledge-graph methods combine CVE, CWE, and CPE relations with graph and description embeddings, but cannot predict links to unseen nodes.Earlier approaches include TransE, TransCat, TransH, and graph attention networks.Source: §4 · p. 6
  • VulnScopper uses ULTRA for knowledge-graph reasoning and OpenAI Ada embeddings for textual descriptions.This combines structural and textual representations for link prediction.Source: §4 · p. 6

5 VulnScopper Design

VulnScopper predicts links among CVEs, CWEs, and CPEs by combining vulnerability-graph structure with entity descriptions. Its design constructs enriched security graphs and integrates ULTRA representations with text embeddings.

5 VulnScopper Design

  • VulnScopper uses structural and textual representations of vulnerability-graph entities to predict links between CVEs, CWEs, and CPEs.The design builds separate graphs from the NVD and Red Hat CVE databases.Source: §5 · p. 6

5.1 Vulnerability Graph Construction

  • The graphs parse NVD and Red Hat records through October 2023, enrich CWE hierarchy and properties, and retain CVEs linked to a CPE or CWE, yielding 12 relation types.CPEs connect to CVEs, vendors, and components; CWE links and properties add further graph relations.Source: §5.1 · p. 6

5.2 Security Entities Description Utilization

  • CVE descriptions come from NVD and Red Hat, CWE descriptions from MITRE, and entity labels substitute for descriptions when explicit text is unavailable.The system maps each entity label to its description for model training.Source: §5.2 · p. 7

5.3 Components of VulnScopper

  • Three VulnScopper components come from ULTRA, while a fourth intermediate component combines ULTRA’s graph representation with textual entity representations.The text component uses an LLM to generate description representations; this study uses OpenAI Ada.Source: §5.3 · p. 7

5.3.1 VulnScopper Language Model

  • The language model maps each entity’s description to a vector embedding, and a linear layer supports LLMs whose output dimension is at least ULTRA’s feature dimension.VulnScopper uses OpenAI Ada in this study.Source: §5.3.1 · p. 7

5.3.2 Combining ULTRA with Language Model

  • VulnScopper combines an entity’s textual representation with its relational representation from ULTRA’s second layer to form a node representation for a link query.The query also uses ULTRA’s conditional relation representation and a text vector with dimension at least d.Source: §5.3.2 · pp. 7–8

5.4 Implementation and Training

  • Training follows standard knowledge-graph embedding practice and uses ULTRA’s binary cross-entropy loss.The implementation uses Python 3.9 and PyTorch 2.1.0; hyperparameters for the relation- and entity-level GNNs follow ULTRA, with other learning parameters selected by grid search.Source: §5.4 · p. 8

6 Experimental Results

The evaluation compares VulnScopper with graph, language, and multimodal baselines on NVD and Red Hat, using transductive and temporally split inductive settings. Performance varies by link-prediction task and dataset.

6.1 Evaluation Metrics

  • MRR averages reciprocal ranks of first correct answers, while Hits@K measures the share of correct predictions ranked within the top K; ChatGPT's MRR was not comparable because it returned only ten results.For MRR, each query contributes the reciprocal of the rank of its first correct answer.Source: §6.1 · p. 10

6.2 Datasets

  • The inductive setup trained on triples through January 1, 2023, then allocated relations through October 18, 2023, to inference, validation, and test graphs.The transductive setup randomly selected validation and test triples while keeping all entities represented in training.Source: §6.2 · p. 10

6.3 Transductive Link Prediction Results

  • 76% and 71% Hits@10: VulnScopper outperformed other methods on CVE-to-CPE prediction for Red Hat and NVD, respectively.The transductive results are presented in Tables 8 and 9.Source: §6.3 · p. 10, Table 8 · p. 10, Table 9 · p. 11
  • 70% Hits@10: VulnScopper led CVE-to-CWE prediction on Red Hat; on NVD it had the best MRR and 71% Hits@10, while ChatGPT led Hits@3 and Hits@1 at 59% and 49%.ChatGPT's NVD advantage was limited to Hits@3 and Hits@1; VulnScopper led the reported MRR and Hits@10 measures.Source: §6.3 · p. 10, Table 8 · p. 10

6.4 Inductive Link Prediction

  • The inductive results varied by task: VulnScopper led NVD CPE prediction at 60% Hits@10, while fine-tuned ULTRA led Red Hat CWE and CPE at 66% and 78%.ChatGPT 4 had the best NVD CWE result at 74% Hits@10.Source: §6.4 · p. 11, Table 10 · p. 11

6.5 CWE Categorization per Dataset

  • ChatGPT 4 preferred NVD's CWE classification scheme when evaluated on Red Hat data, motivating a specialized tool trained on targeted datasets.Table 12 contrasts NVD and Red Hat classification styles.Source: §6.5 · p. 11, Table 12 · p. 11

7 Case studies: Link Prediction of Unseen Entities

The case studies use vulnerability descriptions and known CPEs to rank additional product links, then manually review and cross-check the top ten predictions. Two 2023 CVEs illustrate the approach.

7.1 CVE-2023–4863 - ’libwebp’

  • Eight of the top ten CPE predictions for CVE-2023-4863 (libwebp) were validated through cross-checking.The study removed known NVD links before reviewing predictions.Source: §7.1 · p. 12, §7 · p. 12, Table 13 · p. 12

7.2 CVE-2023-5217 - ’libvpx’

  • Seven of the top ten CPE predictions for CVE-2023-5217 (libvpx) were validated through cross-checking.The paper describes this as a high-impact heap buffer overflow vulnerability mainly affecting browsers using libvpx.Source: §7.2 · p. 12, Table 14 · p. 12

8 Discussion

The discussion presents VulnScopper as a scalable multimodal approach for ranking links to unseen vulnerabilities and assisting analysts with vulnerability scoping.

Discussion

  • ULTRA enables predictions for unseen vulnerabilities without constant retraining by a domain expert for each new report.The discussion connects unseen-entity handling with predictions across extensive graphs containing NVD and Red Hat entities through 2023.Source: §8 · p. 12
  • The multimodal approach combines KG structure with language-model context because language models can miss implicit product links, while KG models lack textual nuance important for CWE prediction.The paper frames this combination as addressing complementary shortcomings of the two model types.Source: §8 · p. 12
  • Ranking links to unseen vulnerabilities is presented as a step toward AI-assisted analyst prioritization and faster response to threats.The stated application is accelerating vulnerability scoping.Source: §8 · p. 12

9 Conclusions

The conclusion presents VulnScopper as a graph-foundation-model and LLM system for predicting links among vulnerability entities, with broader multimodal cybersecurity modeling as a future direction.

Conclusions

  • VulnScopper combines Graph Foundation Models and Large Language Models to predict relationships among CVEs, CWEs, and CPEs in transductive and inductive settings.The conclusion describes the system as supporting efficient vulnerability analysis by analysts.Source: §9 · p. 13
  • The authors identify a general-purpose cybersecurity model combining graph and text as a future research direction.They envision a unified framework for cyber threat intelligence.Source: §9 · p. 13

.1 Transductive link-prediction results

The section identifies tables reporting transductive and inductive CVE-to-CWE and CVE-to-CPE link-prediction results across the NVD and Red Hat datasets. The supplied captions also note reporting constraints for some NVD transductive results.

.1 Transductive link-prediction results

Inductive link-prediction results

.1 Transductive link-prediction results

  • For NVD transductive tables, TransE + Gat + W2V results are omitted where the model did not scale, and ChatGPT 4 MRR is unavailable because it returns only top-10 results.Source: Table 15 · p. 15, Table 16 · p. 15

Vulnerable CPE configurations

For CVE-2023-5217, the paper lists vulnerable CPE configurations based on NVD information, spanning software products and operating systems.

Vulnerable CPE configurations

Related papers

Loading 2403.02014v1…