A Survey on Data-driven Software Vulnerability Assessment and Prioritization

• Assessment helps practitioners prioritize critical vulnerabilities for remediation amid increasing numbers and limited resources.The introduction notes that more than 20,000 vulnerabilities were reported in the NVD in 2021.Source: §1 · p. 1
• Despite growing research using NLP, machine learning, and deep learning, the authors identify no comprehensive survey of data-driven vulnerability assessment and prioritization.Earlier reviews covered vulnerability discovery, vulnerable-code detection, OSINT, static analysis, rule-based methods, or patch management, rather than this survey’s full focus.Source: §1 · pp. 1–2
• The survey reviews studies that automate data-driven vulnerability assessment and prioritization tasks using vulnerability data and NLP, machine learning, or deep learning.Source: §1 · p. 2
• It categorizes key tasks, synthesizes practices and trade-offs in data, features, models, evaluation, and metrics, and discusses challenges and potential solutions.The authors intend the findings to guide researchers and practitioners.Source: §1 · p. 2

2.1 Background and Scope of the Survey

• The survey covers data-driven models that automate vulnerability assessment and prioritization, excluding manual or descriptive-only analyses and absolute performance comparisons across differing experimental setups.It places assessment and prioritization between vulnerability discovery and remediation in the vulnerability-management lifecycle.Source: §2.1 · p. 3

2.2 Methodology

• Following systematic-review-inspired selection, the authors included 84 studies after database searches, screening, and backward and forward snowballing.They used thematic analysis, with two authors independently analyzing and reviewing one another’s work and resolving disagreements through discussion.Source: §2.2 · pp. 3–4

2.3 Taxonomy of Data-driven Software Vulnerability Assessment and Prioritization

• The taxonomy groups related automated tasks into five themes, and a study may appear in multiple themes when it addresses cross-theme tasks.Source: §2.3 · p. 4
• The authors acknowledge alternative categorizations but note that all themes except theme 5 align with security standards such as CVSS and CWE.They propose that this alignment can help bridge academic literature and industrial practices.Source: §2.3 · p. 4

3.1 Summary of Primary Studies

• The survey groups exploitation prediction studies into likelihood, time, and characteristics sub-themes, covering whether exploits appear, when they occur, and how they operate.The theme aims to help practitioners react to vulnerabilities that are more exploitable or already exploited.Source: §3 · pp. 4–5

3.1.1 Exploit Likelihood.

• Exploit-likelihood studies predict PoC or real-world exploit appearance, drawing on expanded exploit sources, ensemble models, transfer learning, and vulnerability-related data.Twitter information supported earlier exploit detection than expert-verified sources in Sabottke et al.; code- and crash-based approaches also appear in the literature.Source: §3.1.1 · pp. 6–7

3.1.2 Exploit Time.

• Exploit-time studies predict broad windows or exact timing after disclosure; Chen et al. predict whether exploitation occurs within 1–12 months and the number of days to exploit.The survey reports Chen et al. as the only studies then pinpointing exact exploit time, and says their model outperformed many baselines.Source: §3.1.2 · p. 7

3.1.3 Exploit Characteristics.

• Exploit-characteristics studies predict CVSS v2 and v3 exploitability metrics or other properties, including vulnerability location, causes, privileges, exploit type, and relevant attack patterns.Predicted characteristics can inform potential scale and countermeasure selection; one unified multi-task model outperformed single-task counterparts and required less retraining time.Source: §3.1.3 · pp. 7–9

3.2 Theme Discussion

• Real-world exploit and CVSS v3 prediction models have generally performed worse than PoC and CVSS v2 models, despite the practical interest in real attacks and the current CVSS standard.The survey suggests collecting more exploit-related data and adapting patterns learned from PoC exploits and older CVSS versions, for example with transfer learning.Source: §3.2 · pp. 9–10
• VEST combines information about whether, when, and how exploitation may occur, but currently draws only on NVD/CVE and Twitter data.The survey identifies adding exploit sources and more sophisticated data-driven techniques as possible extensions.Source: §3.2 · p. 10
• Most exploitation-metric studies use vulnerability descriptions, while code-based prediction remains limited and one cited approach requires manual identification of dangerous C/C++ calls.The survey calls for data-driven methods that reduce reliance on manually defined rules.Source: §3.2 · p. 10

4 IMPACT PREDICTION

• Impact assessment covers confidentiality, integrity, availability, scope, and custom consequences; CVSS versions 2 and 3 share the first three metrics, while v3 adds scope.Scope indicates whether an exploited vulnerability affects only its containing system; Table 5 lists surveyed impact studies and groups the first four sub-themes because they were mostly predicted together.Source: §4 · p. 10, §4.1.1 · p. 10, Table 5 · p. 11

4.1.1 Confidentiality, Integrity, Availability, and Scope.

• Studies commonly classify CVSS v2 and v3 impact metrics using single-task, multi-target, or multi-task learning, with methods ranging from traditional classifiers to deep-learning architectures.Multi-target learning predicts concatenated outputs, whereas multi-task learning shares feature extraction and uses task-specific softmax layers.Source: §4.1.1 · p. 10
• Ensemble and deep-learning models usually outperform single models for CVSS impact metrics, but direct comparisons between those two model types are lacking.Reviewed approaches include single-task, multi-target, and multi-task learning; multi-target and multi-task methods use one model across tasks.Source: §4.1.1 · pp. 10–11
• One study used a Linear SVM for multi-label classification of 11 custom vulnerability consequences, allowing a vulnerability to have multiple consequences.The survey identifies this as the only study pursuing that direction at the time of review.Source: §4.1.2 · p. 11

4.2 Theme Discussion

• CVSS v3 impact models still need more community attention to reach the performance level of CVSS v2 models.The survey notes that impact metrics are often predicted alongside exploitability metrics, using task-wise or unified models.Source: §4.2 · p. 11
• Multi-target and multi-task learning are promising because they can reduce continuous retraining and maintenance time when deployed in production.Source: §4.2 · p. 12
• Fine-grained vulnerability consequences lack a widely accepted taxonomy and are less adopted than CVSS metrics, despite potentially giving more concrete information about compromised assets or components.The survey proposes developing a systematic taxonomy to support further data-driven research in this direction.Source: §4.2 · p. 12

5 SEVERITY PREDICTION

• Severity can be classified as severe versus non-severe, assigned a severity level, or predicted as a score; CVSS v2 and v3 scores range from 0 to 10 in 0.1 increments.Source: §5 · p. 12

5.1.1 Severe vs. Non-Severe.

• Binary severity studies commonly label vulnerabilities with CVSS severity scores of at least 7.0 as severe; term-frequency features with inverse gravity moment weighting enhanced several ML classifiers.Later work confirmed the feature method’s effectiveness across more projects and classifiers.Source: §5.1.1 · p. 12

5.1.2 Severity Levels.

• Severity-level studies span NVD’s Low, Medium, and High categories and other platform-specific schemes; ensemble methods outperformed single models, while deep learning showed potential gains over traditional ML.The survey also describes multi-task prediction of severity levels alongside other CVSS metrics, demonstrated to increase effectiveness and efficiency.Source: §5.1.2 · pp. 13–14, Table 7 · p. 13
• 86-88% Accuracy was obtained for ten-bin CVSS severity-score prediction using Linear SVM, Random forest, and Fuzzy system.Source: §5.1.2 · p. 14
• Deep-learning models generally outperformed traditional ML counterparts for severity-score prediction; studies comparing versions usually reported better performance for CVSS v3 and WIVSS than CVSS v2.The survey notes that the latter finding may indicate that improvements in CVSS v3 and WIVSS make severity-score patterns easier for ML models to capture.Source: §5.1.3 · p. 14

5.2 Theme Discussion

• Severity-level prediction is the most prevalent task, followed by severity-score prediction and binary classification; score prediction offers finer-grained prioritization but is harder to learn.Most severity-score studies did not evaluate continuous deployment under changing patterns of new vulnerabilities.Source: §5.2 · p. 15

6.1.1 Common Weakness Enumeration (CWE).

• CWE classification studies used Naïve Bayes, SVM, information retrieval, and feature selection; base CWEs provide finer remediation information than categorical CWEs.Other approaches used description features, tf-idf with cosine similarity, or Boruta with Random Forest.Source: §6.1.1 · p. 15
• ∼95% F1-score was achieved for CWE classification on vulnerable functions, but the model covers only C/C++ functions and 40 selected CWE classes.The model combines global and local code features.Source: §6.1.1 · pp. 15–16
• Unsupervised CWE studies identified temporal sequences, cross-aspect association rules, and relationships among CWE types.Reported patterns include buffer overflow appearing during implementation in C/C++ and affecting availability.Source: §6.1.1 · pp. 16–17

6.1.2 Custom Vulnerability Types.

• Custom types were derived through clustering and topic modeling; LDA topics overlapped with CWEs, but agreement with OWASP descriptions was below 40%.One study identified 28 prevalent types with up to 98% precision and 95% recall against CWEs.Source: §6.1.2 · p. 17
• Other studies predicted manually selected or platform-specific types, including predefined categories and China’s vulnerability database types.Ensemble models achieved the highest average performance for the China database task.Source: §6.1.2 · p. 17

6.2 Theme Discussion

• CWE type assessment commonly targets coarse- and fine-grained classes; Siamese networks address class imbalance, while graph networks capture hierarchy, motivating future combinations.The survey identifies the large, hierarchical CWE class set as a central challenge.Source: §6.2 · p. 17
• Custom vulnerability types lack a universally accepted taxonomy, so the survey recommends focusing on types commonly encountered and discussed by developers [4.1.2].The survey also calls for more solutions to CWE data imbalance.Source: §6.2 · p. 18

7 MISCELLANEOUS TASKS

• The theme also includes cross-source patterns and fixing-effort estimation alongside vulnerability information retrieval.These are the three main sub-themes identified for Miscellaneous Tasks.Source: §7 · p. 18

7.1.1 Vulnerability Information Retrieval.

• Retrieval methods extract vulnerability entities and relationships stated in vulnerability data, often using named-entity recognition and increasingly deep-learning models.Targets include vulnerable products and versions, attack information, and other assessment characteristics.Source: §7.1.1 · p. 19

7.1.2 Cross-source Vulnerability Patterns.

• Cross-source studies found a positive correlation between development activity and vulnerability mentions on social media, and modeled activity or links among vulnerabilities and attack patterns.The studies used GitHub, Reddit, Twitter, NVD, CWE, and CAPEC.Source: §7.1.2 · p. 19

7.1.3 Vulnerability Fixing Effort.

• Fixing-effort prediction used 65 code-, process-, and developer-based factors identified in an SAP qualitative study, and code components mattered more than vulnerability types.The later study used regression models to predict time to fix.Source: §7.1.3 · pp. 19–20

7.2 Theme Discussion

• NVD descriptions lack consistent templates, and affected-version and vendor/product names are inconsistent, prompting a recommendation to normalize and clean data before labeling and modeling.The survey connects those inconsistencies to challenges in labeling entities and reliable retrieval results.Source: §7.2 · p. 20
• Linking sources, extracting cross-source patterns, and estimating fixing effort can enrich assessment information, but these tasks remain in early stages.The survey describes multi-source linkage as a first step toward a vulnerability knowledge graph for queries about affected systems, exploitation, fixes, and similar vulnerabilities.Source: §7.2 · p. 20

8.1 Data sources

• NVD and CVE were the most prevalent sources across the five themes, but their inconsistencies and missing information motivate validation, cleaning, and use of additional sources.Their expert-verified content supplies model inputs and assessment outputs, while external sources can add exploit or code information.Source: §8.1 · pp. 20, 22
• Informal sources such as social media and darkweb are increasingly mined, but their unstructured, unverified, and potentially fake content calls for integrity checks.The survey suggests checking poster reputation to avoid unreliable model inputs and potentially misleading findings.Source: §8.1 · p. 22

8.2 Model features

• Features include term-frequency representations, learned neural features, and source- or expert-defined metadata; tf-igm outperformed BoW and tf-idf for severity classification.Neural features can capture context and transfer from data-rich domains, but their interpretability is limited.Source: §8.2 · p. 22

8.3 Prediction models

• Classification is more common than regression or unsupervised modeling; Linear SVM is frequent, while boosting and deep-learning models have increasingly been used [34] [96] [9].The survey recommends retaining simple baselines because they can perform on par with advanced methods [123].Source: §8.3 · p. 23

8.4 Evaluation techniques

• Studies commonly used validation or test sets, including k-fold cross-validation and single train/test splits, but single-split results may be unstable.The survey recommends multiple train/test splits with time-based validation and statistical analyses to assess reliability.Source: §8.4 · p. 24

8.5 Evaluation metrics

• Accuracy, Precision, Recall, and F1-score are common classification metrics, but Accuracy can mislead on imbalanced data and MCC is suggested as an alternative to ROC-AUC.The survey also recommends multiple suitable metrics and evaluation of efficiency and robustness, not effectiveness alone.Source: §8.5 · p. 24

9 OPEN RESEARCH CHALLENGES AND FUTURE DIRECTIONS OF DATA-DRIVEN SOFTWARE VULNERABILITY ASSESSMENT AND PRIORITIZATION

• The survey discusses three main open challenges in data-driven vulnerability assessment and prioritization.Source: §9 · p. 25
• It presents nine potential directions intended to address the identified challenges.Source: §9 · p. 25
• The section frames its future directions as responses to the challenges identified in the reviewed studies.Source: §9 · p. 25

9.1 SV Data Availability

• The section identifies missing developer concerns, uncollected issue-tracker bugs, and limited or imbalanced labels as constraints on SV assessment data.These constraints may make fully supervised model performance unreliable for some tasks, including real-world exploit prediction.Source: §9.1 · p. 25

9.1.1 Utilization of developer Q&A platforms and version control systems.

• Developer Q&A posts reveal challenges that SV databases may miss, while GitHub discussions and code can add earlier or project-level evidence for assessment.Developers’ most difficult SV types do not always match those reported by CWE or OWASP [110]; findings on Heartbleed concern one project, so cross-project generalization remains an open question.Source: §9.1.1 · pp. 25–26

9.1.2 Integration of SV data on issue tracking systems.

• Issue trackers contain SV-related reports and artifacts beyond CVE/NVD descriptions, but integration requires retrieval, filtering non-SV reports, and deduplication.Although automated security-bug retrieval has been studied, the manually verified dataset from Wu et al. came from data collected in 2014, making updated data important.Source: §9.1.2 · p. 26

9.1.3 Relaxation of fully-supervised learning.

• Low-shot and semi-supervised learning are proposed for tasks with limited labels, but evidence for their effectiveness in SV assessment remains limited.One study used a deep Siamese network for few-shot CWE prediction and generalized to unseen classes [40]; semi-supervised applications include patch collection and retrieval of Q&A discussions [167] [111].Source: §9.1.3 · p. 26

9.2 Real-world Application and Evaluation

• Promising experimental performance does not settle real-world applicability, which is challenged by delayed inputs, coarse outputs, black-box models, and optimistic evaluation settings.The section groups these concerns into practical usefulness, interpretability, and evaluation conditions.Source: §9.2 · p. 27

9.2.1 More timely and fine-grained prediction.

• Commit-level prediction could assess SVs when introduced, while report-level prediction remains useful for third-party software, unavailable code, and missed cases.CVSS may assign identical metrics to the same SV across different contexts; the authors call for domain/business knowledge and practitioner case studies to evaluate practical usefulness.Source: §9.2.1 · p. 27

9.2.2 Enhancing model interpretability.

• Few reviewed papers explain important features or when predictions work or fail, and the applicability of detection-area interpretation techniques remains unclear for assessment models.Deployment evaluations also often omit handling new data and adversarial inputs from informal sources such as social media or darkweb.Source: §9.2.2 · p. 27, §9.2.3 · p. 28

9.2.3 Realistic evaluation settings.

• Realistic evaluation should account for new inputs and adversarial robustness, which only three reviewed studies evaluated.The authors also recommend accommodating out-of-vocabulary words, considering online or incremental training, and using time-based rather than random splits when evaluating online training.Source: §9.2.3 · p. 28

9.3 Data-driven SV Assessment and Prioritization of Data-driven Systems

• Assessment of vulnerabilities in data-driven systems is still in its infancy, with framework compatibility, real-world data, and model representation identified as major challenges.The systems’ cores are powered by machine-learning or deep-learning models built on data rather than human-defined instructions [103].Source: §9.3 · p. 28

9.3.1 Designing a compatible SV assessment framework.

• CVSS guidance for data-driven systems is unclear, and current CWE categories largely omit vulnerabilities arising from training data or trained models.Automatically updated models complicate static CVSS metrics because adversarial examples may change after updates; the authors suggest studying a new vulnerability category for possible inclusion in CWE.Source: §9.3.1 · p. 28

9.3.2 Collection of SVs in data-driven systems.

• The paper reports no large-scale dataset of real-world data-driven-system vulnerabilities and proposes matching deployed models to published or repository models for attack testing.The authors note that few such vulnerabilities have been reported, including CVE-2019-20634; defining and labeling assessment outputs would still require significant effort. They also recommend representing data properties, model characteristics, and development frameworks in assessment models.Source: §9.3.2 · pp. 28–29, §9.3.3 · p. 29

10 CONCLUSIONS

• The survey synthesizes five research directions, reviews practices in data-driven model development, and identifies open challenges with proposed solutions.The authors see enriched data and more sophisticated models, especially deep learning, as likely avenues for improving task effectiveness, while emphasizing practitioner concerns and real-world usage as key to bridging academia and production.Source: §10 · p. 29