PaperlayerView exact revision on arXiv

Source-linked AI summary

$μ$VulDeePecker: A Deep Learning-Based System for Multiclass Vulnerability Detection

Deqing Zou, Sujuan Wang, Shouhuai Xu, Zhen Li, Hai Jin

arXiv:2001.02334v1cs.CRcs.LG

TL;DR

Because vulnerabilities are inevitable, detecting them early is important, including identifying their types rather than only their presence. µVulDeePecker introduces deep learning for multiclass vulnerability detection using code attention, and experiments find it effective, with higher detection capabilities when control-dependence is included.

  • Problem

    Vulnerabilities are inevitable, motivating earlier detection and patching, while the paper investigates identifying vulnerability types through multiclass detection.

  • Method

    µVulDeePecker uses code attention and dependence information in a deep learning system for multiclass vulnerability detection, evaluated on a dataset organized into 40 vulnerability types.

  • Results

    µVulDeePecker is effective for multiclass vulnerability detection, and accommodating control-dependence can lead to higher detection capabilities.

  • Takeaways & Limitations

    Pinpointing vulnerability types in code gadgets helps human analysts recognize vulnerabilities, while combining µVulDeePecker with other systems can capture more useful information.

  • Takeaways & Limitations

    µVulDeePecker cannot locate vulnerabilities more precisely than the code-gadget granularity and its current design targets C/C++ programs.

Abstract

from arXiv · show

Fine-grained software vulnerability detection is an important and challenging problem. Ideally, a detection system (or detector) not only should be able to detect whether or not a program contains vulnerabilities, but also should be able to pinpoint the type of a vulnerability in question. Existing vulnerability detection methods based on deep learning can detect the presence of vulnerabilities (i.e., addressing the binary classification or detection problem), but cannot pinpoint types of vulnerabilities (i.e., incapable of addressing multiclass classification). In this paper, we propose the first deep learning-based system for multiclass vulnerability detection, dubbed $μ$VulDeePecker. The key insight underlying $μ$VulDeePecker is the concept of code attention, which can capture information that can help pinpoint types of vulnerabilities, even when the samples are small. For this purpose, we create a dataset from scratch and use it to evaluate the effectiveness of $μ$VulDeePecker. Experimental results show that $μ$VulDeePecker is effective for multiclass vulnerability detection and that accommodating control-dependence (other than data-dependence) can lead to higher detection capabilities.

1 INTRODUCTION

The paper addresses the gap between binary vulnerability detection and the finer-grained task of identifying vulnerability types. It introduces μVulDeePecker, whose code attention, control-dependence, fused architecture, and purpose-built dataset support multiclass detection.

  • Research gap: Existing deep learning systems detect whether code is vulnerable but cannot pinpoint the vulnerability type.Vulnerability type information can help identify vulnerability principles, locate vulnerabilities, and reduce analyst workload.
  • Contribution: μVulDeePecker is introduced as a deep learning system for multiclass vulnerability detection.The system investigates detection that both determines whether code is vulnerable and identifies its vulnerability type.
  • Core innovations: Code attention captures local information useful for learning features and pinpointing vulnerability types.It refines the code gadget concept, which groups semantically related statements.
  • Core innovations: The system incorporates control-dependence into code-gadget extraction and uses a neural architecture that fuses different code-gadget and code-attention features.The architecture is constructed mainly from building-block BLSTM networks.
  • Evaluation: The authors create a dataset containing 181,641 code gadgets from 33,409 programs, including 43,119 vulnerable gadgets spanning 40 vulnerability types.The remaining 138,522 gadgets are non-vulnerable, and the dataset is made available to researchers.
  • Evaluation: The paper evaluates μVulDeePecker against multiclass extensions and explains that its dataset and methods address missing vulnerability-type, dependence, and statement information.VulDeePecker+ modifies the activation and loss functions, while another per-type variant has scalability and small-sample weaknesses.

2 BASIC IDEAS AND TERMINOLOGY

The paper represents vulnerability-relevant code through refined code gadgets and code attention. Code gadgets capture dependence relations among statements, while code attention selects syntax-characteristic statements for vulnerability-type detection.

  • Code attention: Code attention is proposed to capture localized, discriminative code information for detecting specific vulnerability types.The idea is inspired by region attention in image processing, where informative regions help classify an image.
  • Code gadgets: A code gadget is a set of statements related through data-dependence, and the refined version also captures control-dependence.The refined representation is intended to capture broader semantic information across program statements.
  • Model inputs: The multiclass model trains on both refined code gadgets and code attention.The former provides more global dependence-based information, while the latter provides localized information within statements, such as function-call arguments.
  • Terminology: A program is modeled as an ordered set of statements, and each statement is modeled as an ordered set of tokens.Tokens may include variable identifiers, function identifiers, constants, keywords, or operators.
  • Dependence relations: A statement is control-dependent on another when the first statement’s execution outcome affects whether the second will execute.This relation complements data-dependence in the refined code-gadget representation.
  • Code gadgets: A code gadget for a library/API call contains statements recursively data-dependent on its arguments or recursively control-dependent on the call statement.The gadget is represented as an ordered set of statements associated with the function call.
  • Code attention: A code attention is the subset of a code gadget whose statements match vulnerability-specific syntax rules.The rules are represented as a set of syntax-characteristic rules and are elaborated later in the paper.

3 DESIGN OF µVULDEEPECKER

µVulDeePecker designs multiclass vulnerability detection around code gadgets, code attentions, vector representations, and a BLSTM-based detector. Its extraction process incorporates both data- and control-dependence, while its current scope is C/C++ library/API-call vulnerabilities.

  • The system targets multiclass vulnerability detection for C/C++ vulnerabilities related to library/API function calls, using type-0 for non-vulnerable code and type-i for CWE-ID i.
  • µVulDeePecker comprises a parser, vector representation extractor, and detector, with the parser covering Steps I–IV, representation extraction Step V, and detection Step VI.
  • Code gadget extraction: The parser builds SDGs whose nodes represent statements or control predicates and whose edges encode data- or control-dependence relations.
  • Code gadget extraction: Code gadgets are extracted around library/API calls using both forward and backward slices while accommodating data-dependence and control-dependence.
  • Normalization and code attention: Code gadgets are normalized by mapping equivalent variables and functions consistently and renaming different ones by appearance order, while preserving reserved words, library/API names, and constants.
  • Normalization and code attention: Code attentions select statements matching vulnerability syntax characteristics, including relevant definitions, control statements, and usage-related patterns, before vectorization and neural-network processing.
  • Neural architecture: The detector uses three BLSTM-based components to learn global features from code gadgets, local features from code attentions, and fused features from both.
  • Scope: The current system is limited to vulnerabilities caused by C/C++ library/API function calls, although its extraction techniques are described as extensible under stated generalization premises.

4 IMPLEMENTATION AND EVALUATION METRICS

The paper constructs and labels a multiclass vulnerability dataset, processes code into normalized gadgets and attentions, and evaluates detections with macro- and weighted-average metrics.

  • 4.1 Dataset Preparation: The dataset combines SARD and NVD programs, assigning vulnerability labels from aggregated third-level CWE-IDs.SARD supplies labeled vulnerable, non-vulnerable, and mixed programs; NVD supplies vulnerability data from its corpus.
  • 4.1 Dataset Preparation: 40 vulnerability types plus a non-vulnerability type are used, with 80% of SARD and NVD programs randomly selected for training.The remaining programs form the testing set.
  • 4.2 Implementation: Training gadgets are extracted from program dependence graphs and system dependence graphs around security-related C/C++ library or API calls.Joern constructs the graphs, and the method traverses graph nodes to identify relevant calls and extract arguments.
  • 4.2 Implementation: Code gadgets containing vulnerable statements receive vulnerability-type labels, while others receive the non-vulnerable type-0 label.SARD provides vulnerable statements directly, whereas NVD statements are identified from deleted patch-file statements.
  • 4.2 Implementation: The detection model uses code-gadget and code-attention vectors as inputs and outputs whether gadgets are vulnerable and, when applicable, their classes.The architecture uses BLSTM-based global- and local-feature learning models, with token vectors generated using skip-gram embeddings.
  • 4.4 Evaluation Metrics: Evaluation uses macro and weighted false-positive rate, false-negative rate, and F1 metrics for multiclass vulnerability detection.Macro metrics average across vulnerability classes, while weighted metrics use each class’s proportion among vulnerabilities; macro metrics reduce to binary counterparts when there is one class.

5 EXPERIMENTS AND RESULTS

The experiments evaluate μVulDeePecker for multiclass vulnerability detection, compare it with VulDeePecker+, and test control-dependence, real-world software, and combined use with another detector.

  • Testing-set evaluation: The testing-set experiments compare μVulDeePecker with VulDeePecker+ for multiclass vulnerability detection.The evaluation also considers real-world software and combined use of both systems.
  • Testing-set evaluation: 94.22% M F1, 0.02% M FPR, and 5.73% M FNR were achieved by μVulDeePecker on the testing set.Compared with VulDeePecker+, μVulDeePecker had 8.72% higher M F1, 0.01% lower M FPR, and 10.75% lower M FNR.
  • Testing-set evaluation: μVulDeePecker was overall more effective than VulDeePecker+ across the 40 vulnerability types, especially for several types with small vulnerability counts.The highlighted types include CWE-673, CWE-362, CWE-170, a CWE-662 and CWE-573 subtype, and CWE-467.
  • Control-dependence: Adding control-dependence improved M F1 by 12.63% and W F1 by 11.13%, while decreasing M FNR by 12.27% and W FNR by 6.9%.The comparison uses a dataset containing only data-dependence against the original setting accommodating both relations.
  • Combining detectors: Combining μVulDeePecker with VulDeePecker+ increased M F1 by 2.65% and W F1 by 1.59%.The combination selects the vulnerability type associated with the maximum probability across both systems.

6 RELATED WORK

Prior source-code vulnerability detection approaches include rule-based, similarity-based, and pattern-based methods. The present study extends deep-learning pattern-based detection from binary vulnerability classification to multiclass detection using enhanced code gadgets, code attention, and a new neural architecture.

  • Source-code vulnerability detection is commonly divided into rule-based, similarity-based, and pattern-based approaches.
  • Rule-based tools characterize vulnerabilities with human-defined rules, with commercial systems generally more capable than open-source tools.
  • Similarity-based methods exploit vulnerability replication through code cloning and compare tokens, strings, trees, graphs, or hybrids.
  • Pattern-based methods use machine learning to learn vulnerability patterns, while deep-learning systems reduce reliance on manually defined features.
  • µVulDeePecker extends binary deep-learning detection to multiclass vulnerabilities through enhanced code gadgets, code attention, and a novel neural architecture.

7 LIMITATIONS

µVulDeePecker detects vulnerability types at code-gadget granularity but does not identify the precise vulnerable statements. Its current design is limited to C/C++ programs and vulnerabilities related to library/API calls.

  • µVulDeePecker cannot locate vulnerabilities more precisely than a code gadget containing multiple statements.Precisely identifying vulnerable statements is identified as future work.
  • The current implementation targets programs written in C/C++ rather than other programming languages.
  • The current implementation focuses on vulnerabilities associated with library/API function calls.Vulnerabilities not associated with library/API functions remain a future research direction.

8 CONCLUSION

µVulDeePecker is presented as the first deep-learning system for multiclass vulnerability detection, identifying vulnerability types within code gadgets. Experiments report effective detection, improved capabilities when control-dependence is accommodated, and benefits from combining detectors.

  • µVulDeePecker is the first deep-learning-based system for multiclass vulnerability detection.
  • The system pinpoints vulnerability types within code gadgets, helping human analysts recognize vulnerabilities.
  • Systematic experiments show that µVulDeePecker is effective for multiclass vulnerability detection.
  • Accommodating control-dependence can lead to higher detection capabilities.
  • Combining µVulDeePecker with other systems, such as VulDeePecker+, can capture more useful information for multiclass vulnerability detection.
Loading 2001.02334v1…