PaperlayerView exact revision on arXiv

Source-linked AI summary

VulDeeLocator: A Deep Learning-based Fine-grained Vulnerability Detector

Zhen Li, Deqing Zou, Shouhuai Xu, Zhaoxuan Chen, Yawei Zhu, Hai Jin

arXiv:2001.02350v2cs.CR

TL;DR

Existing vulnerability detectors lack both detection capability and locating precision sufficient for real-world adoption. VulDeeLocator combines intermediate-code representations with granularity refinement, and on 200 real-world files detects 18 confirmed vulnerabilities.

  • Problem

    Existing vulnerability detectors do not simultaneously provide high detection capability and high locating precision for real-world use.

  • Method

    VulDeeLocator links program information through intermediate-code representations and uses granularity refinement for fine-grained vulnerability localization.

  • Results

    VulDeeLocator detects 18 confirmed vulnerabilities in 200 files randomly selected from FFmpeg, Wireshark, and Libav.

  • Takeaways & Limitations

    The approach simultaneously targets vulnerability detection and precise localization in C programs.

  • Takeaways & Limitations

    VulDeeLocator focuses on C programs, requires compilable source code, and its four syntax characteristics cover 98.3% of collected vulnerable programs with stated data-selection caveats.

Abstract

from arXiv · show

Automatically detecting software vulnerabilities is an important problem that has attracted much attention from the academic research community. However, existing vulnerability detectors still cannot achieve the vulnerability detection capability and the locating precision that would warrant their adoption for real-world use. In this paper, we present a vulnerability detector that can simultaneously achieve a high detection capability and a high locating precision, dubbed Vulnerability Deep learning-based Locator (VulDeeLocator). In the course of designing VulDeeLocator, we encounter difficulties including how to accommodate semantic relations between the definitions of types as well as macros and their uses across files, how to accommodate accurate control flows and variable define-use relations, and how to achieve high locating precision. We solve these difficulties by using two innovative ideas: (i) leveraging intermediate code to accommodate extra semantic information, and (ii) using the notion of granularity refinement to pin down locations of vulnerabilities. When applied to 200 files randomly selected from three real-world software products, VulDeeLocator detects 18 confirmed vulnerabilities (i.e., true-positives). Among them, 16 vulnerabilities correspond to known vulnerabilities; the other two are not reported in the National Vulnerability Database (NVD) but have been "silently" patched by the vendor of Libav when releasing newer versions.

1 INTRODUCTION

VulDeeLocator addresses existing detectors’ inadequate detection capability and locating precision by combining intermediate-code representations with granularity refinement. Evaluated on real-world products, it reports stronger detection and locating outcomes and detects confirmed vulnerabilities.

  • Existing deep learning detectors report an F1-measure of 70.8%, a false-positive rate of 18.2%, and a false-negative rate of 32.0% for real-world programs.
  • Existing detectors struggle to capture semantically related statements across files and to accommodate accurate control flows and variable define-use relations.Header-defined types and macros may be used in source files, requiring cross-file dependence analysis.
  • 78.7% of program slices have at least 10 lines of code and 47.8% have at least 20 lines, limiting locating precision.
  • VulDeeLocator improves vulnerability detection F1-measure by 9.8%, false-positive rate by 7.9%, false-negative rate by 8.2%, and locating precision by 4.2X over the state-of-the-art detector.
  • On 200 files from FFmpeg, Wireshark, and Libav, VulDeeLocator detects 18 confirmed vulnerabilities, including 16 known vulnerabilities and two silently patched by Libav.
  • The paper contributes a vulnerability dataset containing 157,692 intermediate-code candidates, including 40,450 vulnerable and 117,242 non-vulnerable candidates.The dataset and experiment source code were made publicly available.

2 BASIC IDEAS

VulDeeLocator combines program analysis for candidate generation with deep learning for false-positive reduction and vulnerability localization. Intermediate code transforms semantically difficult source constructs into representations that support finer-grained outputs.

  • VulDeeLocator generates vulnerability candidates with program analysis and uses deep learning to eliminate false positives from that analysis.
  • Intermediate code accommodates semantic information that source-level detectors may miss, including implicit control flow and macro definitions.For the conditional expression involving “N”, the example explains why source-level analysis may fail to detect the vulnerability.

3 OVERVIEW OF VULDEELOCATOR

The overview defines source- and intermediate-code vulnerability candidates and organizes VulDeeLocator into candidate representation, labeling, neural-network training, and detection stages.

  • An sSyVC is one or more consecutive source-code tokens matching a vulnerability syntax characteristic.
  • An iSeVC is a sequence of intermediate-code statements data- or control-dependent on the intermediate-code representation of an sSyVC.
  • The learning input consists of C programs that can compile to platform-independent intermediate code and whose vulnerable locations are documented.
  • VulDeeLocator extracts sSyVCs from source code and generates corresponding iSeVCs from intermediate code.
  • Training labels iSeVCs and their vulnerability locations, then trains a neural network from their vector representations.
  • Testing applies source extraction, intermediate-code candidate generation, and the trained model to detect and locate vulnerabilities in target programs.

4 INTERMEDIATE CODE-BASED VULNERABILITY CANDIDATE REPRESENTATION

The candidate-representation design preserves semantic relations by linking intermediate representations across files and functions, slicing dependencies, and refining syntax-based candidates into iSeVCs.

  • Guiding Principles for Vulnerability Candidate Representation: Effective candidate representations should accommodate define-use relations across files and preserve ordered semantic relations across function boundaries.
  • Extracting sSyVCs: The method extracts sSyVCs using four AST-based vulnerability syntax characteristics: library/API calls, array definitions, pointer definitions, and arithmetic expressions.
  • Extracting sSyVCs: Syntax characteristics alone are inadequate because they cannot accommodate the semantic information related to vulnerabilities.
  • Generating iSeVCs: The pipeline generates linked IR files, IR slices corresponding to sSyVCs, and iSeVCs.
  • Generating iSeVCs: Linked IR files are produced by compiling each source file and linking the resulting IR files according to dependence relations.
  • Generating iSeVCs: IR slices are obtained from control- and data-dependence graphs, after which called-function statements are appended to preserve cross-function order.

5 FINE-GRAINED VULNERABILITY DETECTION

The paper defines fine-grained vulnerability detection around granularity refinement, easy output mapping, and attention taking, then implements these goals with BRNN-vdl over labelled intermediate-code representations. Its multiply and pooling layers emphasize vulnerable tokens, refine outputs, and map detected lines back to source code.

  • Requirements: Granularity refinement requires outputs finer than the input iSeVC code fragment, enabling precise identification of vulnerable source lines.The stated output granularity corresponds to statements, while inputs correspond to code fragments.
  • Requirements: Easy mapping requires token-sequence outputs whose consecutive tokens correspond to intermediate-code lines that map back to source programs.
  • Requirements: Attention taking assigns higher weight to vulnerable statements than contextual non-vulnerable statements within a vulnerable iSeVC.The paper motivates this because vulnerable iSeVCs may contain many non-vulnerable lines that provide context.
  • BRNN-vdl architecture: BRNN-vdl extends BRNN with multiply, κ-max pooling, and average pooling layers to achieve attention taking and granularity refinement.Its inputs are iSeVC vectors and vulnerability-location matrices, and training produces fine-tuned parameters.
  • BRNN-vdl operation: The multiply layer combines BRNN activation outputs with vulnerability-location matrices, while pooling selects κ largest values and averages them for each iSeVC.During detection, line-level averages above threshold ϑ identify vulnerable iSeVCs and lines, which are then mapped to source code.

6 EXPERIMENTS AND RESULTS

The experiments evaluate VulDeeLocator’s detection capability and locating precision on vulnerability candidates derived from C programs and represented using source and intermediate code.

  • Research questions: The experiments address whether intermediate code improves detection, whether BRNN-vdl improves locating precision, and how VulDeeLocator performs on known-ground-truth and real-world programs.
  • Evaluation metrics: Five standard metrics evaluate detection capability, while IoU measures overlap between detected and truly vulnerable lines.IoU approaches 1 as locating precision increases.
  • Dataset: 14,511 C programs were collected, including 2,182 real-world and 12,329 synthetic or academic programs, with vulnerable and corresponding patched versions.
  • Candidate construction: 157,692 vulnerability candidates were extracted, including 40,430 FC-kind, 37,692 AD-kind, 50,266 PD-kind, and 29,304 AE-kind candidates.Candidates were extracted from source syntax characteristics and LLVM-based intermediate-code slices.

6.4 Fine-grained Vulnerability Detection

RQ1 examines whether intermediate-code vulnerability candidates improve fine-grained detection. The results favor iSeVCs over source-code sSeVCs and attribute the improvement to SSA information and cross-file semantic relations.

  • Candidate representations: iSeVCs represent data- or control-dependent intermediate-code statements in SSA form, where each variable is defined and used and assigned exactly once.
  • Detection results: 6.7% improvement in F1-measure, 5.9% improvement in precision, and 6.5% improvement in accuracy favor iSeVCs over sSeVCs.The comparison also reports 4.6% and 7.4% improvements in false-positive and false-negative rates, respectively.
  • Explanation: Intermediate code exposes more control-flow and variable define-use information and captures semantic relations between definitions and uses across files.These relations include definitions of types or macros and their uses.
  • Examples: Source-code parsing misses examples involving macros across files and global-variable relationships, whereas intermediate-code candidates identify the semantically related statements.The examples include command injection, double free, and buffer under-read vulnerabilities.
  • Imbalanced data: 39.4% lower F1-measure results from under-sampling non-vulnerable iSeVCs compared with no imbalanced data processing.Over-sampling the vulnerable class produces a 0.3% higher F1-measure than no processing.

6.6 Experiments for Answering RQ2

RQ2 compares BRNN-vdl with BRNN for vulnerability locating. BRNN-vdl refines coarse candidate outputs into smaller vulnerable regions while retaining somewhat stronger detection capability.

  • Locating precision: 21.5% higher IoU and detected vulnerable regions averaging 2.6 lines versus 18.8 lines favor BRNN-vdl over BRNN.IoU is averaged over detected and ground-truth vulnerable-code overlap.
  • Granularity refinement: BRNN-vdl reduces output granularity because BRNN preserves input granularity while the vdl component outputs much smaller code regions.
  • Detection capability: With sSeVCs, BRNN-vdl reports 4.2% higher F1-measure and 5.0% lower false-positive rate than BRNN.
  • Detection capability: With iSeVCs, BRNN-vdl reports 3.9% higher F1-measure and 1.8% lower false-positive rate than BRNN.
  • Overall comparison: VulDeeLocator-BGRU achieves 4.2X higher locating precision than SySeVR on average.

6.8 Experiments for Answering RQ4

VulDeeLocator-BGRU was evaluated on 200 files from three real-world software products and detected confirmed vulnerabilities with substantially finer localization than SySeVR.

  • Experimental setup: 200 files from FFmpeg 2.8.2, Wireshark 2.0.5, and Libav 9.10 were selected for real-world vulnerability detection experiments.The files were randomly selected from the products’ main modules.
  • Score comparison: Figure 9 compares VulDeeLocator and SySeVR using boxplots of attribute and overall scores, with medians shown as solid red lines and means as dashed blue lines.The figure’s supplied caption specifies the score types and line encodings.
  • Detection results: VulDeeLocator-BGRU detected 22 vulnerabilities, including 18 confirmed vulnerabilities and 4 false positives, while missing 5 vulnerabilities.The confirmed vulnerabilities were identified after manually examining detector outputs.
  • Localization results: VulDeeLocator-BGRU detected an average of 3.8 vulnerable lines per vulnerability, compared with SySeVR’s 76.2 lines.The difference corresponds to VulDeeLocator’s substantially more precise vulnerability locations in this experiment.
  • Confirmed vulnerabilities: Among the 18 confirmed vulnerabilities, 2 belonged to Libav 9.10 and were absent from the NVD but silently patched in newer vendor releases.One example was a use-after-free vulnerability involving the pointer “tracks” in matroskadec.c.

7 LIMITATIONS

The study’s scope is limited to compilable C source programs, four vulnerability syntax characteristics, and a BRNN-vdl case-study instantiation.

  • Scope: VulDeeLocator focuses on C source programs because the study aims to demonstrate feasibility and its leveraged tools support C.Extending the approach to other programming languages is identified as future work.
  • Compilation requirement: VulDeeLocator cannot be used when program source code cannot be compiled because it requires compilation into intermediate code.This is a direct operational boundary of the approach.
  • Syntax coverage: The four vulnerability syntax characteristics cover 98.3% of vulnerable programs in the collected NVD and SARD data, but that coverage should be interpreted cautiously.The authors note limitations in the NVD diff-based locations and the representativeness of SARD data.
  • Instantiation: The case study instantiates VulDeeLocator with BRNN-vdl to demonstrate feasibility.The authors present broader vulnerability syntax coverage as an open problem.

8 RELATED WORK

Prior work includes static code similarity, rule-based, machine-learning, and deep-learning detectors, but related slice-based methods have limited semantic modeling and localization precision.

  • Static detection: Static vulnerability detectors include code similarity-based methods and pattern-based methods, with the latter divided into rule-based and machine-learning-based approaches.The related-work discussion distinguishes these method families by their analysis strategy.
  • Code similarity: Code similarity-based methods can localize vulnerabilities precisely when they detect them, but many vulnerabilities are not caused by code cloning.Their high false-negative rate is linked to this mismatch between method and vulnerability cause.
  • Slice-based detectors: VulDeePecker, SySeVR, and µVulDeePecker are closely related because they also operate on program slices.These detectors represent the slice-based deep-learning line of prior work discussed by the paper.
  • Open gaps: Related slice-based detectors have limited capability to accommodate cross-file type and macro relations, control flows, and variable define-use relations.They also cannot precisely pin down vulnerability locations.
  • Bug detection: Prior bug-detection studies use both static and dynamic methods, including information retrieval, spectrum-based analysis, and execution-trace techniques.The paper reviews bug detection because vulnerabilities are treated as a special kind of bugs.

9 CONCLUSION

The paper frames vulnerability detection as requiring both high detection capability and high locating precision, and presents VulDeeLocator to address both requirements for C programs.

  • Requirements: VulDeeLocator targets the simultaneous requirements of high vulnerability detection capability and high locating precision in C programs.The conclusion explicitly identifies these as the detector requirements addressed by the study.
  • Technical approach: The approach addresses semantic information challenges by using intermediate-code-based representations and granularity refinement.The targeted relations include cross-file type and macro uses, control flows, and variable define-use relations.
  • Application: VulDeeLocator detected four vulnerabilities that were not reported in the NVD.The conclusion presents this as an application of the detector.
  • Future research: The study’s limitations identify open problems for future research.The conclusion connects the stated limitations with future work without specifying a broader claim.

APPENDIX

Algorithm 1 automatically generates intermediate-code vulnerability candidates by compiling, dependency-linking, slicing, and refining representations across functions.

  • Algorithm 1: Algorithm 1 generates iSeVCs from sSyVCs and intermediate code through three stages: linked IR-file generation, IR slicing, and iSeVC generation.These stages correspond to lines 4–12, 13–20, and 21–26, respectively.
  • Linked IR files: Each source-program file is compiled into an IR file, and IR files are grouped by dependency relationships before being linked.The linked IR files provide the basis for subsequent slice generation.
  • IR slices: For each sSyVC, the algorithm generates a corresponding IR slice from every linked IR file containing its source file and adds the slice to the iSeVC set.The source-file membership condition determines which linked IR file is sliced.
  • iSeVC generation: For each generated slice, statements from called functions are appended at their call sites, and local-variable numeric values are renamed to unused values in the caller.This refinement incorporates called-function statements while avoiding numeric-value reuse within the calling function.
Loading 2001.02350v2…