PaperlayerView exact revision on arXiv

Source-linked AI summary

Vulnerability Detection with Code Language Models: How Far Are We?

Yangruibo Ding, Yanjun Fu, Omniyyah Ibrahim, Chawin Sitawarin, Xinyun Chen, Basel Alomair, David Wagner, Baishakhi Ray, Yizheng Chen

arXiv:2403.18624v2cs.SEcs.CL

TL;DR

The paper asks whether code LMs can detect vulnerabilities reliably under realistic conditions, given weaknesses in existing datasets and evaluation methods. It introduces PrimeVul with higher-quality labeling, de-duplication, chronological splits, and practical evaluation criteria, then finds that current models perform poorly, including large drops from prior benchmark results. These findings indicate that existing code LMs remain inadequate for practical vulnerability detection.

  • Problem

    Existing vulnerability datasets and benchmarks have low label accuracy, duplication, and evaluation practices that do not reflect developers’ practical concerns.

  • Method

    PrimeVul combines rigorous data collection and labeling with de-duplication, chronological splitting, VD-S, and pair-wise evaluation.

  • Results

    Code LMs consistently performed poorly on PrimeVul; StarCoder2 scored 68.26% F1 on BigVul but 3.09% F1 on PrimeVul.

  • Takeaways & Limitations

    Current models fall short of the requirements for real-world vulnerability-detection deployment, motivating fundamentally new approaches.

  • Takeaways & Limitations

    OpenAI-model results used default settings, which could vary slightly with hyperparameter changes.

Abstract

from arXiv · show

In the context of the rising interest in code language models (code LMs) and vulnerability detection, we study the effectiveness of code LMs for detecting vulnerabilities. Our analysis reveals significant shortcomings in existing vulnerability datasets, including poor data quality, low label accuracy, and high duplication rates, leading to unreliable model performance in realistic vulnerability detection scenarios. Additionally, the evaluation methods used with these datasets are not representative of real-world vulnerability detection. To address these challenges, we introduce PrimeVul, a new dataset for training and evaluating code LMs for vulnerability detection. PrimeVul incorporates a novel set of data labeling techniques that achieve comparable label accuracy to human-verified benchmarks while significantly expanding the dataset. It also implements a rigorous data de-duplication and chronological data splitting strategy to mitigate data leakage issues, alongside introducing more realistic evaluation metrics and settings. This comprehensive approach aims to provide a more accurate assessment of code LMs' performance in real-world conditions. Evaluating code LMs on PrimeVul reveals that existing benchmarks significantly overestimate the performance of these models. For instance, a state-of-the-art 7B model scored 68.26% F1 on BigVul but only 3.09% F1 on PrimeVul. Attempts to improve performance through advanced training techniques and larger models like GPT-3.5 and GPT-4 were unsuccessful, with results akin to random guessing in the most stringent settings. These findings underscore the considerable gap between current capabilities and the practical requirements for deploying code LMs in security roles, highlighting the need for more innovative research in this domain.

I. INTRODUCTION

The paper examines whether code LMs can detect vulnerabilities under realistic conditions and identifies weaknesses in existing datasets and evaluation practices. It introduces PrimeVul and stricter evaluation methods, finding that current models perform poorly on realistic vulnerability detection.

  • Challenges: Existing vulnerability datasets suffer from poor data quality, low label accuracy, and substantial duplication, undermining realistic training and evaluation.The analysis links duplication across training and testing data to unrepresentative performance measurements.
  • PrimeVul: PrimeVul uses expert-informed labeling, rigorous filtering, and de-duplication to provide higher-quality, more realistic vulnerability-detection data.Its labeling techniques use CVE analysis and unique commit changes to improve label accuracy.
  • Evaluation: Chronological splitting reduces data-leakage risk by reflecting the evolution of vulnerabilities and coding patterns.The proposed split is intended to make evaluation results more predictive of real-world performance.
  • Evaluation: VD-S measures false-negative rates after tuning detectors to keep the false-positive rate below a fixed threshold, such as 0.5%.The paper also introduces pair-wise evaluation using vulnerable and benign fixed counterparts.
  • Results: 68.26% F1 on BigVul fell to 3.09% F1 on PrimeVul for StarCoder2, contrasting prior benchmark results with realistic evaluation.Across models and settings, performance on PrimeVul was consistently poor.
  • Results: GPT-3.5 and GPT-4, advanced training methods, and chain-of-thought reasoning failed to produce useful vulnerability detection in stringent evaluations.GPT-4 with chain-of-thought reasoning did not outperform random guessing in pair-wise evaluation.

II. BACKGROUND & CHALLENGES

The paper frames vulnerability-detection datasets as balancing scale against label reliability. It reviews automated and manual collection practices and describes manual verification procedures used to assess label accuracy.

  • A. Existing Data Collection Methods: Existing real-world vulnerability datasets trade off scalable automated labeling against more reliable but labor-intensive manual labeling.Automated methods are coarse-grained, while manual verification limits dataset size and diversity.
  • A. Existing Data Collection Methods: Manual verification is challenging and time-consuming even for experienced security experts.This limits the practicality of manually verified datasets for training deep neural networks.
  • Label Accuracy Analysis: The label-accuracy analysis samples 50 vulnerable functions from each benchmark for manual assessment.Three authors performed the analysis, including two experienced security researchers and one senior security expert.
  • Label Accuracy Analysis: Annotators review commit histories, before-and-after functions, CVE and NVD descriptions, and available security-issue discussions.These sources support judgments about whether sampled functions contain vulnerabilities.
  • Label Accuracy Analysis: A function is labeled vulnerable when at least two of three annotators agree, with an expert-led discussion when an annotator is unsure.The final decision is made through majority vote.

2) Results:

Existing vulnerability benchmarks show substantial label-accuracy problems and data-leakage concerns. PrimeVul’s labeling techniques reportedly match SVEN’s accuracy while providing 16.7× as many vulnerable C/C++ functions.

  • 16.7× as many vulnerable C/C++ functions are included in PrimeVul as in SVEN, with labeling accuracy on par with SVEN.
  • 25%–60% vulnerable-function label accuracy occurs in benchmarks without manual verification, while SVEN reaches 94%.
  • 24% vulnerable-function label accuracy in CodeXGLUE reflects inaccurate security-commit annotations and automated labeling of changed functions.
  • SVEN is the most accurate prior dataset but contains only 803 vulnerable functions across nine CWEs.Its total size is approximately 1.6k vulnerable and non-vulnerable functions.
  • Training on noisy datasets is concerning because nearly half of vulnerable training samples may have incorrect labels.
  • Realistic vulnerability-detection evaluation requires controlling code-copy and time-travel leakage through appropriate data splits.The study examines leakage in BigVul, CVEFixes, CodeXGLUE, and DiverseVul; split construction varies across these benchmarks.

1) Code Copy:

Existing vulnerability benchmarks suffer from code duplication and leakage, making reported performance unrepresentative of generalization to unseen code. Random splits can also create time-travel and same-commit leakage, while standard metrics may not reflect deployment priorities.

  • Code duplication and leakage: Exact copies across training and evaluation inevitably inflate performance because code language models can memorize repetitive code text.
  • Code duplication and leakage: Up to 18.9% of test samples are leaked from training sets in some vulnerability benchmarks through duplicated code.Exact copies and cloned vulnerabilities can appear across training and testing, inflating evaluation performance.
  • Code duplication and leakage: Contradicting labels can arise when the same function is sampled before and after a vulnerability-fixing commit, confusing model training.
  • Temporal leakage: Random dataset splits permit training on future data, testing on past data, and placing fixed benign versions in training alongside older vulnerable versions in testing.
  • Temporal leakage: Samples from one commit can be scattered across splits, leaking information because developers often fix multiple similar instances of an issue together.
  • Evaluation limitations: Accuracy and F1 do not fully capture deployment concerns because vulnerability detection must balance missed vulnerabilities against excessive false alarms.

2) Textually Similar Pairs of Vulnerable & Corresponding Patch:

PRIMEVUL addresses unreliable vulnerability labels and duplicate code with a filtered construction pipeline and two complementary labeling techniques. The resulting dataset expands accurately labeled examples while preventing vulnerable training functions from leaking into the test set.

  • Dataset construction: PRIMEVUL consolidates security-related data from several benchmarks while excluding Devign/CodeXGLUE commits found to be largely unrelated to security issues.
  • Dataset construction: Formatting normalization and MD5 hashing remove unchanged functions, code copies, and functions differing only in formatting.
  • Dataset construction: PRIMEVUL’s thorough deduplication ensures that no vulnerable training function leaks into the test set.
  • Data labeling: PRIMEVUL-ONEFUNC labels a function vulnerable only when it is the sole function changed by a security-related commit.
  • Data labeling: PRIMEVUL-NVDCHECK uses CVE-linked NVD descriptions and labels functions based on explicit function-name or file-name criteria.

IV. NEW EVALUATION GUIDELINES

The paper proposes evaluation guidelines designed to better match deployment conditions: chronological splits, false-negative evaluation under a false-positive constraint, and paired testing on vulnerable and fixed code.

  • Chronological splitting: Chronological splitting assigns the oldest 80% of samples to training, the middle 10% to validation, and the newest 10% to testing.Samples are ordered by the dates of their original commits to reduce data leakage and better represent future deployment.
  • Vulnerability Detection Score: The evaluation emphasizes minimizing missed vulnerabilities while keeping false positives within a tolerable level.
  • Vulnerability Detection Score: VD-S measures false negative rate at a configurable false-positive-rate threshold, using FNR @ (FPR ≤r) with r = 0.5% in the paper’s evaluation.
  • Paired functions: PRIMEVUL contains 5,480 vulnerable–patch pairs, including challenging pairs sharing at least 80% of their strings.Not every vulnerable function has a patch, so paired functions are fewer than all vulnerable functions.
  • Paired functions: Pair-wise evaluation treats each vulnerable–benign pair as one entity and records whether both elements receive the correct labels in a textually similar context.
  • Paired functions: The four pair-wise outcomes are correct, both-vulnerable, both-benign, and reversed predictions.

V. EXPERIMENTAL RESULTS

The experiments reassess vulnerability detection under PRIMEVUL’s more realistic data and evaluation setup, testing open-source code models, advanced training techniques, and larger language models. The study uses BigVul as an existing-benchmark comparison and includes GPT-3.5 and GPT-4 evaluations.

  • Research questions: Three research questions examine open-source code-LM performance, advanced training techniques, and whether larger language models improve vulnerability detection.
  • Datasets: PRIMEVUL is the main experimental dataset, while BigVul provides a case study illustrating how existing-benchmark limitations affect training and evaluation.
  • Models: Seven code language models with varied sizes are studied, with all open-source models fine-tuned for the first research question.
  • Models: UnixCoder is used to study advanced training techniques, while GPT-3.5 and GPT-4 are tested with two-shot and chain-of-thought prompting settings.

B. RQ1: Performance of Open-Source Code LMs on PRIMEVUL

Open-source code LMs perform poorly on PRIMEVUL, revealing that prior benchmarks substantially overestimate vulnerability-detection ability. Pair-wise evaluation further shows weak discrimination between vulnerable functions and benign counterparts.

  • Open-source models were fine-tuned and evaluated on PRIMEVUL, with BigVul used for direct comparison.The experiments distinguish models trained and tested on PRIMEVUL from models trained on BigVul.
  • 68.26% F1 on BigVul fell to 3.09% F1 on PRIMEVUL for StarCoder2, illustrating benchmark-driven overestimation.The performance drop was observed across models, not only for StarCoder2.
  • 18.05% F1 was achieved by StarCoder2 when trained on PRIMEVUL, still far below its 68.26% F1 on BigVul.Training directly on PRIMEVUL produced only a modest improvement over the 3.09% PRIMEVUL result.
  • VD-S can diverge from accuracy and F1, providing a more deployment-oriented view by emphasizing false negatives under a false-positive constraint.CodeBERT’s high accuracy and F1 on BigVul did not establish acceptable realistic vulnerability-detection performance.
  • 2.30% of StarCoder2 pairs were correctly labeled on both elements, while 88.30% of pairs were classified as both benign.The pair-wise results indicate frequent misclassification of both members rather than reliable vulnerable-versus-benign discrimination.
  • Code LMs’ underperformance on PRIMEVUL exposes limitations on realistic, diverse, and challenging vulnerabilities.The findings characterize existing benchmark performance as misleading for real-world vulnerability detection.

C. RQ2: Exploring to Improve the Performance on PRIMEVUL

The study tests class weighting and contrastive learning as ways to improve PRIMEVUL performance. Both techniques produce, at best, marginal gains and do not overcome the benchmark’s difficulty.

  • 1) Exploration-1: Class Weights: Class weights were tested because PRIMEVUL contains a much lower vulnerable-sample ratio than BigVul.The authors suspected that the stronger class imbalance might hinder learning.
  • 1) Exploration-1: Class Weights: Class weights did not fundamentally improve performance on PRIMEVUL, with false negative rates remaining around 90%.Although F1 increased, VD-S indicated that the improvement was not promising in realistic settings.
  • 2) Exploration-2: Contrastive Learning: Contrastive learning was added alongside the classification objective to separate representations with distinct semantics.The approach was implemented through the CLR objective during fine-tuning.
  • 2) Exploration-2: Contrastive Learning: Contrastive learning failed to significantly improve code-LM performance on PRIMEVUL because the CLR objective was misaligned with classification labels.CLR distinguishes samples regardless of whether their labels match, while CA-CLR produced only marginal gains.
  • Advanced training techniques, including class weights and contrastive learning, could not particularly improve PRIMEVUL performance.The results provide evidence that PRIMEVUL’s difficulty is not explained solely by class imbalance or representation similarity.

D. RQ3: Larger Code LMs on PRIMEVUL

The study evaluates larger GPT models on PRIMEVUL paired functions and finds that their apparent advantages do not yield reliable vulnerability discrimination. The discussion identifies missing context and security reasoning as important boundaries of the task.

  • D. RQ3: Larger Code LMs on PRIMEVUL: GPT-3.5 and GPT-4 were evaluated on PRIMEVUL paired functions using prompting and, for GPT-3.5, fine-tuning.The settings included two-shot prompting, chain-of-thought reasoning, and GPT-3.5 fine-tuning.
  • D. RQ3: Larger Code LMs on PRIMEVUL: GPT-3.5 and GPT-4 outperformed open-source models in pair-wise evaluation, but their performance was no better than random guessing overall.Most PRIMEVUL pairs still could not be distinguished by these models.
  • D. RQ3: Larger Code LMs on PRIMEVUL: State-of-the-art OpenAI models did not achieve reliable PRIMEVUL performance, motivating fundamentally novel approaches.This result extends the underperformance observed for smaller code LMs.
  • VI. DISCUSSIONS & THREATS TO VALIDITY: Single-function vulnerability classification is constrained because vulnerability status can depend on sanitization, downstream use, invariants, and interprocedural flows.PRIMEVUL retains commit metadata to support extraction of broader context.
  • VI. DISCUSSIONS & THREATS TO VALIDITY: Pair-wise shortcomings suggest that code LMs rely primarily on textual similarity rather than vulnerability root causes or fixes.The paper proposes augmenting models with security awareness or hybridizing them with traditional analysis tools.
  • VI. DISCUSSIONS & THREATS TO VALIDITY: PRIMEVUL labels are not perfectly accurate, leaving a small portion of mislabeled data despite stringent labeling methods.The authors expect the reported conclusions to remain valid given the small mislabeling percentage.

VII. RELATED WORK

Prior work shows that vulnerability detection with code LMs relies on fine-tuning or prompting, while existing datasets and evaluation metrics have substantial practical limitations. PRIMEVUL addresses these concerns with updated data and evaluation criteria, and experiments show current models still fail in realistic settings.

  • Code LMs for vulnerability detection are primarily used through fine-tuning or prompting.
  • Existing vulnerability detection datasets have poor data quality, low label accuracy, and high duplication rates, limiting their practical utility.
  • PRIMEVUL introduces updated evaluation criteria intended to measure the practical effectiveness of code-LM vulnerability detectors more accurately.
  • Experiments on PRIMEVUL found that sophisticated training methods and larger models consistently failed to meet practical vulnerability-detection requirements.

A. Detailed Label Error Analysis

The detailed analysis finds substantial mislabeling in existing vulnerability datasets, often because security-related commits affect irrelevant, multi-function, or merely caller functions. The paper’s stricter function-level criterion distinguishes independently vulnerable functions from benign functions changed alongside security fixes.

  • Sources of labeling errors: Labeling errors arise when commits modify irrelevant functions, spread vulnerabilities across multiple functions, or change benign functions during security fixes.
  • Label accuracy: Only 24% of sampled CodeXGLUE functions and 36% of sampled VulnPatchPairs functions were genuinely vulnerable.
  • Irrelevant functions: 58% of CodeXGLUE samples and 40% of VulnPatchPairs samples were irrelevant to security, often originating from commits unrelated to security issues.
  • Function-level criterion: The methodology labels a function as vulnerable only when it independently constitutes a security risk, excluding cases requiring system-level context such as race conditions.
  • Caller functions: Callers of vulnerable functions are classified as benign when they are changed only to align with security updates; wma_decode_init is one example.
Loading 2403.18624v2…