PaperlayerView exact revision on arXiv

Source-linked AI summary

Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG

Xueying Du, Geng Zheng, Kaixin Wang, Yi Zou, Yujia Wang, Wentai Deng, Jiayi Feng, Mingwei Liu, Bihuan Chen, Xin Peng, Tao Ma, Yiling Lou

arXiv:2406.11147v3cs.SEcs.AI

TL;DR

LLMs have limited ability to distinguish vulnerable code from similar patched code, raising questions about whether they capture vulnerability root causes. The paper introduces VUL-RAG, which retrieves multi-dimensional knowledge distilled from historical vulnerabilities and fixes to guide detection. VUL-RAG improves pair accuracy by 16% - 24%, improves manual confirmation accuracy from 60% to 77%, and identifies previously unknown Linux kernel bugs.

  • Problem

    LLMs struggle to distinguish vulnerable code from similar patched code, limiting evidence that they capture vulnerability root causes rather than superficial features.

  • Method

    VUL-RAG uses knowledge-level retrieval of multi-dimensional vulnerability knowledge distilled from historical vulnerabilities and fixes to guide LLM vulnerability assessment.

  • Results

    16% - 24% improvements in pair accuracy are reported, alongside manual confirmation accuracy increasing from 60% to 77% and 10 previously unknown Linux kernel bugs detected.

  • Takeaways & Limitations

    VUL-RAG-generated knowledge supports vulnerable-patched discrimination, manual vulnerability confirmation, and detection of previously unknown Linux kernel bugs.

  • Takeaways & Limitations

    An incomplete knowledge base may limit VUL-RAG when no relevant historical vulnerability exists, and generalizability beyond the four evaluated LLMs requires further investigation.

Abstract

from arXiv · show

Although LLMs have shown promising potential in vulnerability detection, this study reveals their limitations in distinguishing between vulnerable and similar-but-benign patched code (only 0.06 - 0.14 accuracy). It shows that LLMs struggle to capture the root causes of vulnerabilities during vulnerability detection. To address this challenge, we propose enhancing LLMs with multi-dimensional vulnerability knowledge distilled from historical vulnerabilities and fixes. We design a novel knowledge-level Retrieval-Augmented Generation framework Vul-RAG, which improves LLMs with an accuracy increase of 16% - 24% in identifying vulnerable and patched code. Additionally, vulnerability knowledge generated by Vul-RAG can further (1) serve as high-quality explanations to improve manual detection accuracy (from 60% to 77%), and (2) detect 10 previously-unknown bugs in the recent Linux kernel release with 6 assigned CVEs.

1 Introduction

The study finds that LLMs poorly distinguish vulnerable functions from highly similar patched versions, motivating VUL-RAG, which augments detection with multi-dimensional knowledge distilled from vulnerabilities and fixes.

  • Motivation: LLMs struggle to distinguish vulnerable code from corresponding patched code, leaving unclear whether they capture vulnerability root causes rather than superficial features.The paired-code problem also tests robustness because vulnerable and patched functions often have high textual similarity.
  • Empirical gap: 86% - 94% of cases are not correctly classified as both vulnerable and patched by existing LLMs.Advanced prompts and fine-tuning provide only 0.05 - 0.20 accuracy on this paired-identification task.
  • Approach: VUL-RAG distills generalizable vulnerability knowledge from historical vulnerabilities and fixes using functional semantics, vulnerability causes, and fixing solutions.Its knowledge-level representation emphasizes high-level vulnerability features rather than lexical code details.
  • Approach: VUL-RAG retrieves relevant knowledge and uses LLM reasoning about vulnerability causes and fixes to assess new code.The framework builds a knowledge base, retrieves by similar functional semantics, and guides vulnerability assessment.
  • Evaluation: 16% - 24% improvements in pair accuracy show that VUL-RAG enhances LLM distinction between vulnerable and patched code.Generated knowledge also improves manual confirmation accuracy from 60% to 77%.

2 Related Work

Prior work evaluates LLM vulnerability detection, prompt and fine-tuning enhancements, and code-level RAG; VUL-RAG instead augments source-code analysis with high-level generalized vulnerability knowledge.

  • Empirical studies: Existing empirical studies assess LLM vulnerability detection across diverse benchmarks, models, and metrics, while this work focuses on vulnerable-patched discrimination.The paper contrasts its focus with studies of smaller pretrained models and smaller paired samples.
  • Enhancement methods: Prior enhancement methods include prompt engineering, fine-tuning, and integration with static analysis for vulnerability detection.The paper notes that fine-tuning often targets small models with high-quality data, whereas static analysis often targets specific bug types.
  • Retrieval-augmented generation: Existing code-related RAG retrieves and augments with code, whereas VUL-RAG uses high-level, generalizable knowledge for source-code vulnerability detection.The distinction is between code-level retrieval and knowledge-level augmentation.

3 Empirical Study

The empirical study benchmarks LLMs on vulnerable-patched pairs, tests prompting strategies, and analyzes their errors, finding low discrimination accuracy, unstable bias, and difficulty recognizing subtle code changes.

  • Benchmark: PairVul evaluates vulnerable-patched discrimination using high-quality function pairs from real-world CVEs, addressing missing or unreliable patched versions in existing benchmarks.The benchmark includes 586 pairs across 420 CVEs after collection, verification, and selection procedures.
  • Basic capabilities: 0.06 - 0.14 pairwise accuracy shows that basic-prompt LLMs fail to correctly identify both members of most vulnerable-patched pairs.Balanced recall and precision are not more than 0.52, similar to random guessing.
  • Prompting impact: 0.20 pairwise accuracy is the best result reported for advanced prompting, while balanced precision and accuracy remain lower than 0.55.CoT-1 for GPT-4o achieves the best advanced-prompt result, and CWE-enhanced prompting can reduce pair accuracy.
  • Bias analysis: LLMs show unstable bias across neutral prompts, with some prompts causing most models to label code as benign and others causing most code to be labeled vulnerable.The study attributes this pattern to failure to capture the semantic difference between vulnerable and patched code.
  • Case analysis: LLMs struggle with subtle differences such as relocating or replacing method invocations and adding conditional checks.The case analysis samples pairs where all studied LLMs fail to distinguish opposing labels.

4 Enhancement Framework VUL-RAG

VUL-RAG enhances vulnerability detection by distilling multi-dimensional, generalizable knowledge from historical vulnerable–patched code pairs and retrieving it to guide LLM reasoning. Its pipeline constructs a knowledge base, retrieves relevant items, and iteratively checks vulnerability causes against fixing solutions.

  • Knowledge representation: VUL-RAG represents each vulnerability using functional semantics, vulnerability causes, and fixing solutions across seven elements.The representation covers code purpose and behavior, triggering actions and causal descriptions, and patch-based solutions.
  • Knowledge-base construction: The framework extracts knowledge from vulnerable–patched pairs with LLMs, then abstracts concrete methods, variables, and types into more general descriptions.Abstraction reduces dependence on implementation-specific details so related vulnerability instances can share knowledge.
  • Knowledge-base construction: VUL-RAG builds its knowledge base from multi-dimensional items aggregated from existing vulnerability instances, using data separated from the PairVul benchmark.The knowledge base contains items generated through extraction and abstraction rather than raw code pairs alone.
  • Knowledge retrieval: For detection, VUL-RAG combines code with extracted functional semantics, retrieves candidates for each query element, and reranks them with Reciprocal Rank Fusion.The retrieval process uses code, abstract purpose, and detailed behavior; it keeps the highest-ranked candidates after deduplication and reranking.
  • Knowledge-augmented detection: VUL-RAG sequentially checks retrieved knowledge items and labels code vulnerable when it exhibits the corresponding cause without the associated fix.If no retrieved item supports that judgment, the code is identified as non-vulnerable.

5 Evaluation for VUL-RAG

The evaluation examines VUL-RAG’s detection effectiveness, comparison with code-level retrieval and fine-tuning baselines, usability of generated knowledge, and discovery of previously unknown Linux kernel bugs. Results show improved detection metrics, higher manual identification accuracy with knowledge, and confirmed bugs found in a recent kernel release.

  • Evaluation design: The evaluation compares VUL-RAG with basic and advanced prompts, code-level RAG, and fine-tuning-based baselines across detection and usability studies.The experiments include PairVul comparisons, a developer user study, and a Linux kernel case study.
  • Overall improvements: VUL-RAG substantially outperforms the baselines, increasing pair accuracy by 16%-24% and balanced precision and recall by 9%-14% and 7%-11%.The reported comparisons cover PairVul evaluation results and all listed metrics.
  • Overall improvements: Compared with code-level RAG, VUL-RAG improves all metrics because its knowledge representation retrieves more semantically relevant vulnerabilities and guides inference beyond textual code similarity.The case analysis identifies both retrieval relevance and high-level inference guidance as reasons for the observed advantage.
  • Usability for developers: 77% detection accuracy with generated knowledge versus 60% without knowledge was observed in the user study, with the difference statistically significant at p = 0.01.Participants also rated helpfulness, preciseness, and generalizability at average scores of 3.00, 3.20, and 2.97.
  • Previously unknown vulnerabilities: VUL-RAG detected 10 previously unknown vulnerabilities among 1,568 sampled Linux kernel functions, and six confirmed bugs received accepted patches.The case study used Linux kernel v6.9.6 and GPT-4; the generated knowledge supported bug-reporting and patch development.

6 Conclusion

The paper identifies limitations in LLMs’ ability to distinguish vulnerable from patched code and proposes knowledge-level VUL-RAG to address them. VUL-RAG improves vulnerability detection, supports manual analysis, and finds previously unknown Linux kernel bugs, including six with assigned CVEs.

  • Conclusion: The study reveals limited LLM capability for distinguishing vulnerable code from patched code and introduces VUL-RAG with multi-dimensional knowledge distilled from historical vulnerabilities and fixes.The framework is presented as a knowledge-level RAG approach for vulnerability detection.
  • Conclusion: VUL-RAG outperforms all baselines in vulnerability detection and improves manual vulnerability detection by 17% accuracy.The conclusion summarizes both automated detection and the user-study outcome.
  • Conclusion: VUL-RAG detects 10 previously unknown Linux kernel bugs, six of which are confirmed by the Linux community with assigned CVEs.These findings extend the evaluation beyond benchmark performance to real-world software analysis.

A Dataset

The paper constructs PairVul from verified vulnerable–patched function pairs and develops prompts and extraction procedures for building and using a vulnerability knowledge base.

  • Benchmark construction: PairVul is constructed from function-level vulnerable and patched code pairs extracted from real-world CVEs and enriched with vulnerability descriptions.
  • Benchmark construction: Patched-code verification filters pairs to ensure patches were not subsequently reverted or modified by later commits.
  • Dataset statistics: The dataset contains 2,903 vulnerable–patched pairs across 1,325 CVEs, with PairVul comprising 586 pairs across 420 CVEs.
  • Evaluation prompts: The study evaluates basic, chain-of-thought, and CWE-enhanced prompts for distinguishing vulnerable code from patched code.
  • Knowledge construction: VUL-RAG extracts abstract purpose, detailed behavior, vulnerability causes, and fixing solutions, then abstracts these into generalizable knowledge.

D Retrieval Implementation

VUL-RAG retrieves vulnerability knowledge using BM25 and reranks candidates with Reciprocal Rank Fusion over code and extracted semantic queries.

  • Similarity retrieval: BM25 computes similarity between a query and documentation using term frequency, inverse document frequency, and document-length normalization.The implementation uses k=1.2 and b=0.75.
  • Candidate reranking: Retrieved knowledge items are reranked with Reciprocal Rank Fusion across query elements.The query elements are the code, abstract purpose, and detailed behavior.
  • Evaluation: The evaluation compares VUL-RAG and baselines across the 10 CWE categories.
  • Failure analysis: Manual analysis examines cases where studied LLMs and advanced techniques fail to distinguish vulnerable from patched code.The analyzed examples emphasize subtle textual differences and show patch diffs.

F.2 Case Study in Overall Improvements

Case studies compare VUL-RAG with code-level retrieval and basic LLMs, showing how its knowledge representation and retrieval strategy support vulnerability detection and previously-unknown bug discovery.

  • Overall improvements: VUL-RAG successfully detects examples that code-level RAG misses, motivating comparisons of knowledge representation and retrieval strategy.
  • Knowledge representation: For CVE-2023-30772, basic GPT-4 misses a race-condition-driven use-after-free and instead flags an unnecessary return-value check.Code-based RAG also fails to associate retrieved vulnerability knowledge with the target code.
  • Retrieval strategy: For CVE-2023-1989, the retrieval case compares code-based retrieval with retrieval using both code and extracted functional semantics.
  • Knowledge representation: The paired code example differs by adding cancel_work_sync(&charger->otg_work) before unregistering power supplies.
  • Previously-unknown vulnerability: VUL-RAG detects a previously-unknown Linux kernel v6.9.6 use-after-free caused by a race during switchtec_ntb_remove cleanup.The proposed mitigation cancels pending work before freeing sndev, aligning with knowledge retrieved from CVE-2023-30772.

G Usability for Developers

The user study evaluates whether VUL-RAG-generated knowledge helps developers understand and verify vulnerability detections, using balanced cases across CWE categories and structured quality ratings.

  • Study goal: The study is framed around assessing the quality of VUL-RAG-generated knowledge and its assistance in understanding and checking vulnerabilities.
  • Study design: The user study selects 10 PairVul cases, including true-positive and false-positive instances from five CWE categories.
  • Study procedure: Participants assess snippets with labels alone or with labels plus VUL-RAG-generated vulnerability knowledge.
  • Knowledge evaluation: The study surveys knowledge helpfulness, preciseness, and generalizability using a 4-point Likert scale.
  • Study procedure: The evaluation asks annotators to determine whether labeled C/C++ snippets are truly vulnerable and assess the usefulness and clarity of additional knowledge.

H Bad Case Analysis

VUL-RAG’s false negatives and false positives arise from shortcomings in vulnerability-knowledge descriptions, retrieval, and matching fixes to code. The analysis covers 19 false negatives and 21 false positives from CWE-119.

  • Analysis Scope: 19 false negatives and 21 false positives from CWE-119 were manually analyzed to identify VUL-RAG’s failure modes.The analysis categorizes false negatives and false positives by their reported causes.
  • False Negatives: 26.3% of false negatives involved relevant knowledge retrieval but imprecise vulnerability descriptions.The descriptions did not always specify the concrete bounds-checking required for the fix.
  • False Negatives: 15.8% of false negatives resulted from failing to retrieve relevant vulnerability knowledge.
  • False Positives: 52.4% of false positives involved retrieved knowledge whose fixing solution was judged inapplicable to the code.A vulnerability may have multiple alternative fixing solutions, creating mismatches with the retrieved solution.
  • False Positives: 47.6% of false positives were caused by retrieving irrelevant vulnerability knowledge.

I Limitations

VUL-RAG’s practical performance is constrained by the completeness of its vulnerability knowledge base, and its generalizability to other LLMs remains unexamined.

  • Knowledge-Base Scope: An incomplete knowledge base can limit VUL-RAG when no relevant historical vulnerability exists for the code under detection.The authors plan to open-source the knowledge base for continued community maintenance and extension.
  • Model Scope: The findings’ generalizability to other LLMs requires further investigation beyond the four evaluated models.The evaluation included both open-source and closed-source LLMs.
Loading 2406.11147v3…