PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning

Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Wei Ma, Lyuye Zhang, Yang Liu, Yingjiu Li

arXiv:2401.16185v4cs.CRcs.AIcs.SE

TL;DR

LLM vulnerability-detection results can reflect the model’s reasoning as well as external knowledge, context, and prompt choices. LLM4Vuln evaluates these factors using the UniVul benchmark across three programming languages. The study reports varied effects from the enhancements and identifies 14 vulnerabilities in pilot bug-bounty programs, earning $3,576.

  • Problem

    Prior evaluations leave unclear whether vulnerability-detection results reflect a model’s reasoning or external factors such as context, retrieved knowledge, and instruction-following.

    Source: §1 · p. 2

  • Method

    LLM4Vuln separates vulnerability reasoning from knowledge, context, and prompt enhancements, and evaluates them with UniVul’s retrievable knowledge and context-supplementable code across three languages.

    Source: §1 · pp. 2–3

  • Results

    Knowledge enhancement had heterogeneous effects: traditional models’ average F1 nearly doubled in Solidity, while gains in Java and C/C++ were limited or negative; a pilot confirmed 14 vulnerabilities and $3,576 in bounties.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The authors report that LLM4Vuln’s top-performing configuration helped uncover vulnerabilities in four Solidity-based bug-bounty projects.

    Source: §1 · p. 3

    Knowledge retrieval was uneven: relevant knowledge aligned with 68 of 100 sampled positive Solidity cases and 36 of 100 Java samples.

    Source: §7.2 · p. 13

Abstract

from arXiv · show

Large language models (LLMs) have demonstrated significant potential in various tasks, including those requiring human-level intelligence, such as vulnerability detection. However, recent efforts to use LLMs for vulnerability detection remain preliminary, as they lack a deep understanding of whether a subject LLM's vulnerability reasoning capability stems from the model itself or from external aids such as knowledge retrieval and tooling support. In this paper, we aim to decouple LLMs' vulnerability reasoning from other capabilities, such as vulnerability knowledge adoption, context information retrieval, and advanced prompt schemes. We introduce LLM4Vuln, a unified evaluation framework that separates and assesses LLMs' vulnerability reasoning capabilities and examines improvements when combined with other enhancements. To support this evaluation, we construct UniVul, the first benchmark that provides retrievable knowledge and context-supplementable code across three representative programming languages: Solidity, Java, and C/C++. Using LLM4Vuln and UniVul, we test six representative LLMs (GPT-4.1, Phi-3, Llama-3, o4-mini, DeepSeek-R1, and QwQ-32B) for 147 ground-truth vulnerabilities and 147 non-vulnerable cases in 3,528 controlled scenarios. Our findings reveal the varying impacts of knowledge enhancement, context supplementation, and prompt schemes. We also identify 14 zero-day vulnerabilities in four pilot bug bounty programs, resulting in $3,576 in bounties.

1. Introduction

LLM vulnerability-detection results can reflect more than a model’s reasoning, including knowledge, context, and instruction-following. LLM4Vuln and UniVul test these factors separately and reveal differing effects across model types and tasks.

1. Introduction

  • UniVul provides retrievable vulnerability knowledge and context-supplementable code for Solidity, Java, and C/C++.Its code corpus contains 51 vulnerable and 51 non-vulnerable Solidity functions, 46 of each for Java, and 50 of each for C/C++.Source: §1 · pp. 2–3
  • Knowledge augmentation nearly doubles average F1 for traditional models in Solidity, but gains in Java and C/C++ are limited or negative.Deep reasoning models benefit less from external knowledge, suggesting they may model vulnerability semantics more robustly internally.Source: §1 · pp. 2–3
  • Context supplementation is inconsistent: traditional models tend to reach higher peak F1 with context, while deep reasoning models often do better without it.The authors suggest context can distract when it is not aligned with the vulnerability.Source: §1 · p. 3
  • CoT prompting improves precision and reduces false positives for both model types, while recall effects vary by task.Deep reasoning models are more stable under CoT; traditional models show greater variance, and CoT can help them with multi-step analysis.Source: §1 · p. 3
  • In a pilot across four Solidity bug-bounty projects, 14 of 29 submitted issues were confirmed as vulnerabilities and earned $3,576 in bounties.The study used the top-performing configuration identified by LLM4Vuln.Source: §1 · p. 3
  • LLM4Vuln separates vulnerability reasoning from knowledge retrieval, context supplementation, and prompt design to evaluate their effects systematically.The framework also supports LLM-invoked tools for gathering target-code information and uses GPT-4.1 to refine less proficient models’ raw outputs.Source: §1 · pp. 2–3

2. Background

The background introduces general-purpose and open-source LLMs alongside newer models designed for deeper reasoning. It also describes tools and prompting approaches while noting that their effects on vulnerability detection remain uncertain.

2. Background

  • The background names GPT models, including GPT-4.1, and open-source families such as Llama 2/3 and Phi-3.Phi-3-mini-128k is described as a 3.8-billion-parameter model trained on 3.3 trillion tokens.Source: §2 · p. 3
  • Deep reasoning models such as o4, DeepSeek-R1, and QwQ-32B are presented as emphasizing analytical reasoning beyond general-purpose instruction following.The passage describes DeepSeek-R1 as trained with reinforcement learning to incentivize step-by-step reasoning.Source: §2 · p. 3
  • Tool use and chain-of-thought prompting can extend LLM capabilities, but their effects on vulnerability-detection precision, recall, and knowledge integration remain uncertain.The background notes that prompts differ in effectiveness and that the useful knowledge to integrate needs further investigation.Source: §2 · pp. 3–4

3. Problem Statement: A Formal Analysis

The formalization treats vulnerability detection as an LLM task whose output depends on more than the model’s pretrained reasoning. LLM4Vuln aims to isolate that reasoning and measure how auxiliary factors change results.

Detection task

  • Detection asks whether target code contains a vulnerability and identifies its type and cause, but outcomes also depend on externally enhanceable factors.These factors include retrieved vulnerability knowledge, surrounding code context, prompt schemes, and instruction-following capability.Source: §3 · p. 4

Decoupling objective

  • Because the model and auxiliary components often interact, their entanglement makes the contribution of inherent vulnerability reasoning difficult to isolate.The framework is designed to examine both the extent to which reasoning originates in the model and how external aids enhance it.Source: §3 · p. 4

Evaluation goals

  • LLM4Vuln’s evaluation framework measures isolated baseline performance, tests enhancements from auxiliary factors, quantifies their marginal utility, and supports controlled experiments across models and languages.Its modular design implements these components in a plug-and-play manner, and UniVul supplies the standardized benchmark.Source: §3 · p. 4

4. The LLM4Vuln Framework

LLM4Vuln uses replaceable components to assess and enhance vulnerability reasoning. Its framework separates knowledge retrieval, context supplementation, prompt schemes, and instruction following, with a separate annotation component for benchmarking.

4. The LLM4Vuln Framework

  • LLM4Vuln has four decoupled, replaceable components for evaluating and enhancing vulnerability reasoning, plus a result-annotation component used for benchmarking.Each component needs only one implementation to function.Source: §4 · p. 4

4.1. Vulnerability Knowledge Retrieval

  • Knowledge retrieval uses either code-matched original vulnerability reports or functionality-matched summaries of past reports to supply vulnerability knowledge.The first approach searches embeddings of code and reports but supplies report text; the second matches summarized functionality to retrieve summarized vulnerability knowledge.Source: §4.1 · pp. 4–5

4.2. Code Context Supplementation

  • Context supplementation adds related vulnerable code or callees before LLM analysis, with static analysis holding supplied contexts constant across models for fair benchmarking.Additional context can matter when a vulnerability trigger spans functions or when context clarifies function semantics.Source: §4.2 · p. 5

4.3. Prompt Schemes and Instruction Following

  • Prompt schemes combine model knowledge, raw reports, or summarized knowledge with either an unconstrained request or chain-of-thought instructions.The chain-of-thought sequence asks the model to summarize functionality, analyze possible vulnerability-causing errors, then determine vulnerability status.Source: §4.3 · pp. 5–6

4.4. LLM-based Result Annotation and Analysis

  • GPT-4.1 annotation converts LLM outputs into structured vulnerability decisions and rationales; manual checks found 100% binary-label accuracy and type-match accuracy of 81% for Solidity, 98% for Java, and 97% for C/C++.The reported manual check sampled 100 cases from each language.Source: §4.4 · p. 6, §4.3 · p. 6

5. The UniVul Benchmark

UniVul is designed to support controlled evaluation with retrievable vulnerability knowledge and context-supplementable code across Solidity, Java, and C/C++. Its construction separates knowledge sources from testing data and rewrites test samples to reduce leakage risk.

Benchmark scope

  • UniVul addresses benchmark gaps in retrievable vulnerability knowledge and surrounding code context, spanning Solidity, Java, and C/C++ vulnerability domains.The language selection covers traditional software and smart contracts, whose vulnerability profiles include both language-level issues and business-logic bugs.Source: §5 · p. 7

Knowledge synthesis

  • For each language, UniVul separates a knowledge set from a testing set and draws on developer-facing resources, including audit reports for Solidity and CWE/CVE sources for Java and C/C++.Knowledge is drawn from past vulnerabilities rather than the corresponding test case’s own vulnerability knowledge.Source: §5 · p. 7

Detailed collection and synthesis

  • The Java and C/C++ knowledge sets contain 770 and 860 synthesized items, respectively, generated from 77 and 86 CWE categories.For each CWE, GPT-4.1 generated ten representative code examples and corresponding audit-style vulnerability reports.Source: §5 · p. 7

Knowledge retrieval and evaluation scale

  • Across 294 vulnerable and non-vulnerable cases, retrieval, prompt, and context combinations yield 3,528 scenarios and 8,232 test cases per model.The benchmark uses top-three retrieval and repeats no-knowledge cases three times to match knowledge-enabled cases.Source: §5 · p. 7

Minimizing data leakage

  • Testing samples are semantically preserved but have function names, variables, and comments rewritten to reduce lexical overlap and memorization risk.Underlying code statements remain unchanged, and reported results use sanitized data.Source: §5 · p. 7

6. Evaluation

The evaluation separates LLM vulnerability reasoning from external enhancements and tests how knowledge, context, and prompting affect results across models and languages. The effects vary by model and language, while a Solidity pilot confirms vulnerabilities in bug-bounty projects.

6.1. RQ1: Effects of Knowledge Enhancement

  • Knowledge effects depend on model and language: summarized knowledge helps traditional models on Solidity, while deep-reasoning models and Java/C/C++ often see degraded performance.The authors attribute Solidity gains to retrieval of functionality-based knowledge for logic-heavy vulnerabilities; mismatched knowledge can mislead reasoning.Source: Table 3 · p. 8, Table 4 · p. 8, §6.1 · pp. 9–10

6.2. RQ2: Effects of Context Supplementation

  • Context supplementation offers slight, inconsistent improvements; traditional foundation models often reach their highest F1 with context, while deep reasoning models tend to perform best without it.In C/C++, context increased true positives in 28 of 36 combinations, but the highest absolute F1 was achieved by QwQ-32B without context.Source: Table 3 · p. 8, §6.2 · p. 10

6.3. RQ3: Effects of Different Prompt Schemes

  • CoT prompts do not reliably increase true positives or recall, but tend to reduce false positives and increase true negatives across most models and languages.Phi-3 is an exception: CoT degrades its false-positive, true-negative, and false-negative metrics across all three languages.Source: §6.3 · p. 10

6.4. RQ4: Testing for Zero-Day Vulnerabilities

  • Across four audited Solidity bug-bounty projects, 29 issues were submitted, 14 were community-confirmed, and the combined audits yielded $3,576 in bounties.Among the four cases discussed, only the first was detectable by static analysis; the other three were logic bugs closely tied to business logic.Source: §6.4 · pp. 11–12

7. Discussion and Future Work

The authors recommend matching enhancements to model and task characteristics, improving knowledge retrieval, and carefully filtering context. UniVul currently covers three languages, while extending it requires language-specific knowledge and tool adaptations.

7.1. Lessons Learned

  • Knowledge enhancement helps traditional foundation models on logic-heavy Solidity vulnerabilities but may degrade performance on Java and C/C++; reasoning models can also suffer from noisy external knowledge.The authors recommend retrieval methods that preserve semantics and model selection based on both audited language and sensitivity to enhancements.Source: §7.1 · pp. 12–13

7.2. More Accurate Knowledge Retrieval

  • Retrieval quality is limited: relevant knowledge matched 68 of 100 sampled Solidity cases and 36 of 100 Java cases, with low-level vulnerabilities underrepresented.Java retrieval was largely limited to XSS and SQL injection; SSRF and CSRF were rarely matched due to weak functional correspondence.Source: §7.2 · p. 13

7.3. Covering More Programming Languages

  • UniVul currently covers Solidity, Java, and C/C++; extending it requires tailored vulnerability knowledge bases and language-adapted tools.The framework is designed to be language-agnostic in principle, with possible additions including AST parsing, call-graph construction, or symbolic execution.Source: §7.3 · p. 13

8. Related Work

Prior work applies LLMs to vulnerability detection through detectors, fuzzing, and static-analysis combinations, while benchmark studies measure performance across tasks. LLM4Vuln instead targets reasoning behind assessments and evaluates it separately from external enhancements.

LLM-based Vulnerability Detection

  • LLM-based vulnerability work includes fine-tuned classifiers, graph-assisted function detection, smart-contract role-playing, and LLM-supported fuzzing,, [20],.Source: §8 · p. 13

Benchmarking LLMs’ Vulnerability Detection Capabilities

  • LLM4Vuln targets reasoning behind vulnerability assessments, unlike raw-performance-focused evaluations, and UniVul supports modular evaluation with retrievable knowledge and context-supplementable code.UniVul covers Solidity, Java, and C/C++.Source: §8 · p. 13
  • Benchmark studies span Solidity [28], DeFi [27], Java and C++ [30], cross-language datasets [29], [31],, configuration testing, and vulnerability repair, [35].Source: §8 · p. 13

LLM-based Vulnerability Detection

  • Static-analysis integrations pair GPT with symbolic execution or traditional analysis pipelines, but focus on detection rather than reasoning.Source: §8 · p. 13

Security-oriented LLMs

  • Security-oriented models address specialized tasks, while this work evaluates mainstream LLMs without task-specific retraining or model customization.Source: §8 · p. 14

9. Conclusion

The paper introduces LLM4Vuln and UniVul to evaluate vulnerability reasoning and enhancement strategies across three programming languages. Its study examines 294 vulnerable and non-vulnerable cases in 3,528 scenarios.

9. Conclusion

  • LLM4Vuln is a unified, modular framework designed to decouple and enhance LLMs’ inherent vulnerability reasoning.Source: §9 · p. 14
  • UniVul provides retrievable knowledge and context-supplementable code across Solidity, Java, and C/C++.Source: §9 · p. 14
  • The study examines how knowledge enhancement, context supplementation, and prompt schemes affect performance across 294 cases and 3,528 scenarios.Source: §9 · p. 14

1. Prompts Used in LLM4Vuln and UniVul

The appendix collects prompt templates for generating vulnerable code and reports, summarizing functionality and root causes, and supporting instruction following and annotation.

Prompts for Summarizing the Functionalities and Root Causes from Vulnerability Reports

  • Functionality and root-cause prompts seek concise business-function summaries and generalizable explanations of vulnerability mechanics without code identifiers.Source: §1 · p. 16

Prompt for Instruction Following and Auto-Annotation

  • Instruction-following and auto-annotation prompts route generated text through a report API and compare auditor descriptions with ground-truth vulnerabilities.Source: §1 · p. 16

Prompt for Data Augmentation

  • Data Augmentation includes a subsection labeled Vulnerable Code Generator.Source: §1 · p. 16

Vulnerable Code Generator

  • The Vulnerable Code Generator requests ten distinct vulnerable snippets from CWE information, aiming to cover varied business logic and return code only.Source: §1 · pp. 16–17

Vulnerability Report Generation

  • Vulnerability Report Generation requests a report with vulnerability description, code, root cause, impact, and mitigation.Source: §1 · p. 17

2. Detailed LLM Vulnerability Analysis Results

The detailed results are organized as raw counts and calculated precision, recall, and F1 scores under different knowledge, context, and prompt combinations. The tables cover Solidity separately from Java and C/C++.

2. Detailed LLM Vulnerability Analysis Results

Loading 2401.16185v4…