PaperlayerView exact revision on arXiv

Source-linked AI summary

VulDec: Vulnerability Detection in Binaries via Neural Decompilation

Francesco Crescenzo Grasso, Pietro Liguori, Roberto Natella, Domenico Cotroneo

arXiv:2610.33716v1cs.CRcs.LGcs.SE

TL;DR

Binary vulnerability detection is challenging when source code is unavailable and assembly offers limited semantic information. VulDec pairs real-world C–Assembly functions with CWE annotations and uses neural decompilation with LLM-based detection. Its tuned models report source-level recovery across architectures and macro F1-score 0.61 on an in-distribution realistic test set.

  • Problem

    Binary vulnerability detection lacks source-level semantic information, and existing resources do not jointly provide realistic source–binary pairs with vulnerability annotations.

    Source: §1 · p. 2

  • Method

    VulDec combines a curated, multi-architecture dataset of real-world C–Assembly function pairs with CWE annotations and an end-to-end neural-decompilation and LLM-based detection framework.

    Source: §1 · p. 3

  • Results

    Tuned neural decompilers achieve Edit Similarity ≈56% and BLEU-4 ≈43% across architectures, while tuned detectors reach macro F1-score 0.61 on an in-distribution realistic test set with a 3% vulnerability rate.

    Source: §1 · p. 4

  • Takeaways & Limitations

    The findings provide evidence that neural decompilation is an effective approach for binary vulnerability detection.

    Source: §1 · p. 4

    VulDec inherits DiverseVul’s documented 5−15% label noise and class imbalance across vulnerability categories.

    Source: §8 · p. 25

Abstract

from arXiv · show

Automated vulnerability detection in binary executables is challenging due to the lack of source code and the limited semantic information available at the assembly level. Despite significant progress in machine-learning-based code analysis, the field still lacks methodologies and large, high-quality datasets that provide vulnerable code jointly at assembly- and source-level with precise vulnerability labels. To address this gap, we introduce VulDec, consisting of (i) a new large-scale, carefully curated dataset aligning real-world Assembly-C function pairs with detailed CWE annotations across multiple architectures, and (ii) the first end-to-end framework that leverages neural decompilation for binary vulnerability detection. Tuned LLM-based neural decompilers trained on our dataset achieve strong source-level recovery across architectures (Edit Similarity of approximately 56%, BLEU-4 of approximately 43%), while our tuned LLM-based detectors reach a macro F1-score of 0.61 on an in-distribution realistic test set (3% vulnerability rate). Our findings provide evidence that neural decompilation offers a powerful alternative to conventional low-level binary representations, facilitating robust binary vulnerability detection.

1 Introduction

The introduction motivates binary vulnerability detection as a need in settings without source code and proposes VulDec, which combines neural decompilation with LLM-based detection and a paired, CWE-annotated dataset. It reports source-level recovery and detection results from evaluation on that dataset.

1 Introduction

  • Binary vulnerability detection matters when source code is unavailable, while existing low-level representations retain assembly-level structure and semantics that limit high-level analysis.Traditional decompilers can also produce code that remains close to raw assembly; neural decompilation translates binaries into source code.Source: §1 · p. 2
  • Existing datasets separate decompilation pairs from vulnerability labels, and many vulnerability datasets are source-level or synthetic rather than realistic binary-linked examples.Training both pipeline stages on a shared data distribution is presented as a way to ensure semantic consistency and mitigate distribution shift.Source: §1 · p. 3
  • VulDec combines neural decompilation and LLM-based vulnerability detection with a curated dataset of real-world C–Assembly function pairs and detailed CWE annotations across multiple architectures.The framework translates binaries into high-level code before vulnerability detection.Source: §1 · p. 3
  • Edit Similarity ≈56%, BLEU-4 ≈43%, and macro F1-score 0.61 on an in-distribution realistic test set with a 3% vulnerability rate.The recovery figures are for tuned neural decompilers across architectures; the F1-score is for tuned LLM-based detectors.Source: §1 · p. 4

2 Background

The background explains why recovering high-level representations from binaries is difficult and reviews traditional and neural approaches to decompilation. It also situates VulDec amid gaps in binary vulnerability detection and datasets that jointly provide source, binary, and security annotations.

2 Background

  • Decompilation reverses compilation to recover high-level representations, but no deterministic inverse exists because compilation discards semantic information and binaries encode structure through low-level operations.Architecture-specific instructions, calling conventions, and register models further complicate recovery.Source: §2 · p. 4
  • Traditional decompilers reconstruct high-level representations using intermediate representations and control-flow graphs, sometimes enriched with data-flow and program-dependence graphs.Their accuracy depends on complete functions and closed control-flow graphs, and their outputs can remain semantically approximate and assembly-like.Source: §2 · pp. 4–5
  • Neural decompilation frames recovery as translation over paired low- and high-level code, learning semantic patterns that can produce more structured, readable C-like representations.It can operate on code fragments or isolated functions, but output correctness, paired-data quality, generalization, and interpretability remain challenges.Source: §2 · p. 5
  • Binary vulnerability detection lacks high-level abstractions, while decompilation can recover representations that support applying source-level detection techniques to binaries.The paper motivates tightly integrating neural decompilation with automated vulnerability detection rather than using it only for readability or manual reverse engineering.Source: §2 · p. 6
  • Existing datasets generally lack a unified combination of source code, corresponding binaries, and vulnerability annotations, limiting study of how source vulnerabilities manifest in binaries.Decompilation datasets often lack security labels; vulnerability datasets are primarily source-level and may be synthetic, narrowly patterned, or not fully compilable.Source: §2 · p. 6

3 Methodology

VulDec constructs a C–Assembly dataset from real-world projects and uses a neural-decompilation pipeline to detect vulnerabilities from binaries. Its methodology combines reproducible source-to-binary alignment with separate decompilation and vulnerability-classification tasks.

3.1 Dataset Construction

  • VulDec aligns DiverseVul source functions with Debian-built assembly by selecting a supported release with high vulnerability-sample density, then matching assembly symbols to source function names.The construction process compiles the selected package, disassembles its binary, and extracts function boundaries for alignment.Source: §3.1 · pp. 7–8, Figure 2 · p. 7
  • The dataset is normalized to reduce syntactic variation while preserving functional structure, and the automated process yields high-fidelity cross-domain pairs.Normalization abstracts literal strings and removes comments, preprocessor directives, call-frame information, and irregular spacing.Source: §3.1 · p. 9

3.2 Proposed Framework

  • VulDec processes a binary through architecture-aware disassembly, neural reconstruction of high-level code, and neural vulnerability detection.Figure 3 depicts the overall pipeline and its intermediate representations.Source: §3.2 · p. 9, Figure 3 · p. 9
  • Decompilation is trained as assembly-to-C/C++ sequence translation, while vulnerability detection is trained as binary classification on source code and applied to decompiled pseudo-C/C++ at inference.The decompilation setting uses function-level samples and excludes compiler optimizations and code obfuscation.Source: §3.2 · p. 9, Figure 4 · p. 10

4 Experimental Setup

The experimental setup evaluates the dataset and model configurations across x86 and ARM, then trains neural decompilers and vulnerability detectors under constrained sequence lengths and realistic class imbalance.

4 Experimental Setup

  • Experiments used a GTX 1660 with 6 GB VRAM and 32 GB RAM, adding a T4 with 16 GB VRAM and LoRA for models exceeding 1 billion parameters.The authors report that this setup supported training and inference across the experimental phases.Source: §4 · p. 10

4.1 Dataset Overview

  • 23,163 functions are shared across the x86 and ARM subsets, which were compiled with GCC 12.2.0 at -O2; each subset has an approximately 6–7% vulnerability rate.The dataset statistics report no duplicate entries or missing values.Source: §4.1 · pp. 10–11, Table 1 · p. 11

4.2 Decompilation Setup

  • Decompilers were evaluated across representative model architectures using project-level stratified 90/5/5 splits, with 512-token assembly inputs and 256-token C outputs.These sequence limits reduced the dataset by approximately 50%.Source: §4.2 · pp. 11–12

4.3 Vulnerability Detection Setup

  • Detection models were fine-tuned on VulDec plus unused DiverseVul samples, with sequence filtering reducing the data by approximately 25% and leaving a 3% vulnerability rate.Raw training, class weighting, and undersampling were compared on an imbalanced validation set to select the mitigation strategy.Source: §4.3 · pp. 12–13

5 Experimental Evaluation

The evaluation tests neural decompilation and vulnerability detection separately and in combined pipelines. Neural decompilers recover source structure across architectures, while detection performance varies with class imbalance, distribution shift, vulnerability category, and input representation.

5.1 Decompilation Task

  • The decompilation evaluation uses Edit Similarity, BLEU-4, and ROUGE-L, while noting that BLEU-style metrics may not be well suited to code translation.Edit Similarity is normalized to [0,1] and measures structural fidelity; the reported results cover x86 and ARM.Source: §5.1 · pp. 13–14, Table 2 · p. 14
  • Qwen2.5-Coder achieves Edit Similarity of 55.83% on x86 and 56.60% on ARM, while CodeT5+ leads BLEU-4 with 42.45% on x86 and 44.41% on ARM.Small autoregressive CodeGPT and CodeParrot fail on translation, while CodeBERT retains Edit Similarity above 43%.Source: Table 2 · p. 14, §5.1 · p. 14

5.2 Vulnerability Detection Task

  • PLBART leads in-distribution vulnerability detection with Vul-Recall of 0.444 and Vul-F1 of 0.257, while neural architectures’ highest Vul-F1 remains modest under 3% vulnerability prevalence.The DiverseVul baseline’s Accuracy of 0.923 accompanies a Macro F1-score of 0.472, illustrating why Accuracy is uninformative in this imbalanced setting.Source: §5.2 · pp. 16–17, Table 5 · p. 17

5.3 End-to-End Evaluation

  • VulDec’s fully neural pipeline achieves the best out-of-distribution performance, with Vul-Recall of 0.28 and the highest global F1-score; minority-class recall falls from 0.44 in-distribution to 0.28.The traditional Ghidra → Flawfinder pipeline has a Vul-F1 of 0.00, while replacing Ghidra with the neural decompiler raises Flawfinder’s Vul-F1 to 0.05.Source: Table 6 · p. 19, §5.3 · p. 19

5.4 Impact of decompilation on vulnerability prediction

  • Accuracy rises monotonically with Edit Similarity, but vulnerability F1-score has no statistically meaningful dependency on Edit Similarity and peaks at 61.5% near ES 0.3.Evaluating the detector on clean OOD source code yields nearly identical results, with vulnerable-class recall of 0.28 and precision of 0.14, suggesting decompilation noise is not the primary cause of the performance drop.Source: Figure 6 · p. 20, §5.4 · pp. 20–21

5.5 Effectiveness of VulDec Across Vulnerability Categories

  • Validation and Authorization Issues achieve Macro Recall of 0.72 and Macro F1-score of 0.63, while the dominant Memory Management and Concurrency group reaches 0.72 Recall and 0.61 F1-score.Uncategorized functions account for 48.47% of samples and Memory Management and Concurrency for 32.30%; Insecure Configuration Issues account for 0.48%.Source: Table 7 · p. 21, §5.5 · p. 22

5.6 Assembly-Based Vulnerability Detection

  • Direct x86 assembly detection performs well on Safe samples (F1-score 0.92) but yields zero precision, recall, and F1-score for vulnerable functions.The authors attribute this failure to assembly’s lack of higher-level semantic cues and conclude that decompilation remains necessary for effective detection within VulDec.Source: Table 8 · p. 22, §5.6 · pp. 22–23

6 Lessons Learned

The authors see neural decompilation as promising for cybersecurity and the VulDec dataset as a benchmark for training decompilation and detection models. They frame the framework as a first-pass aid for human review, not an autonomous triage system.

VulDec Framework Deployment Context

  • VulDec is best suited as a first-pass screening tool that highlights potential high-risk regions for manual inspection, rather than as an autonomous triage system.The authors describe this lightweight decision-support role as a way to reduce the search space for human experts.Source: §6 · p. 23

Effectiveness of VulDec Dataset

  • The dataset’s joint low-level, source-level, and vulnerability annotations open research directions beyond the paper’s contributions.The authors also report encouraging outcomes on an out-of-distribution test set, while noting performance degradation during training due to computational-resource and model-capacity limits.Source: §6 · p. 23

Applicability of neural decompilation to cybersecurity

  • Despite limited syntactic fidelity, decompiled C/C++ code preserves semantic content that learning-based approaches can use through recurring patterns and structural features.The authors describe neural decompilation for cybersecurity tasks, including binary malware analysis, as a promising direction for future research.Source: §6 · p. 23

7 Related Work

VulDec uses neural decompilation as an intermediate stage for binary vulnerability detection and introduces a dataset aligning assembly, source semantics, and fine-grained vulnerability annotations. It extends an earlier exploratory pipeline to real-world projects and multiple architectures, while distinguishing its representation and evaluation setting from prior work.

7 Related Work

  • The present work replaces the earlier pipeline’s synthetic Juliet data with a dataset built from real-world projects and extends support from x86 to x86 and ARM binaries.The earlier exploratory study demonstrated feasibility but was limited by synthetic data and x86-only binaries (Cotroneo et al. 2025).Source: §7 · pp. 23–24
  • Prior neural decompilers target decompiled-code quality, including Beyond The C’s 54% ED for C/C++ and Katz et al.’s 30% ED; LLM4Decompile reports improved readability and re-executability over traditional tools.Beyond The C was trained on 2,000,000 samples, while Katz et al.’s approach used about 700,000 samples.Source: §7 · p. 24
  • VulDec differs from prior neural decompilation work by using decompilation for downstream black-box binary vulnerability detection and by jointly aligning assembly, source semantics, and fine-grained vulnerability annotations.The paper notes that direct experimental comparison across binary-detection studies is inherently challenging because their data sources, representations, learning paradigms, and evaluation methods differ.Source: §7 · p. 24
  • The pipeline is described as modular, allowing multi-agent verification systems to operate as downstream backends on decompiled pseudo-C.Examples cited are VulTrial (Widyasari et al. 2025) and AEGIS (Fang et al. 2026).Source: §7 · p. 25
  • Earlier binary vulnerability tools use assembly tokens, LLVM IR, call graphs, or combinations; the paper identifies decompiled C/C++ pseudo-code as a distinct primary representation that also approximates original source.Prior work includes VDISCOVER, which detects four types of memory corruption with 55% accuracy, and tools focused on particular vulnerabilities or architectures.Source: §7 · p. 25
  • The paper argues that strong classification results in prior work, particularly in multi-class settings, often rely on synthetic benchmarks, and contrasts them with the effects of dataset realism.Its earlier Juliet-based study reported up to 90% accuracy (Cotroneo et al. 2025).Source: §7 · p. 25

8 Threats To Validity

The validity of VulDec’s results is qualified by imperfect function alignment and inherited label noise, heterogeneous model adaptation, similarity-based decompilation metrics, and a small vulnerable OOD test sample.

Dataset Construction and Integrity

  • A manual audit found correct symbol-name alignment in 95% of 100 sampled function pairs, while the dataset inherits DiverseVul’s documented 5–15% label noise and class imbalance.The reported 95% Wilson confidence interval for alignment accuracy is [88.7%, 98.0%].Source: §8 · p. 25

Model selection and adaptation

  • Different adaptation regimes may confound model comparisons: smaller baselines were fully fine-tuned, while larger generative LLMs used LoRA for three epochs under resource constraints.The authors note that more extensive hyperparameter optimization could influence model capacity.Source: §8 · p. 26

Metrics

  • BLEU, Edit Similarity, and ROUGE-L may not capture whether decompiled code is correct, creating a construct-validity concern despite their common use for translation evaluation.The cited code-to-code and natural-language-to-code evaluation studies include Hosseini and Dolan-Gavitt (2022), Katz et al. (2018), and Liguori et al. (2023).Source: §8 · p. 26

OOD Vulnerable Sample Size

  • Only 29 vulnerable functions appear in the OOD test set, yielding wide 95% confidence intervals for Vul-Recall [0.12, 0.44] and Vul-F1 [0.07, 0.29].The authors caution that individual prediction shifts under extreme imbalance significantly affect point estimates.Source: §8 · p. 26

9 Conclusion

VulDec combines a dataset aligning vulnerability annotations with assembly representations and an LLM-based neural-decompilation framework for binary vulnerability detection. The authors identify robustness to OOD samples, decompiler hallucinations, and behavioral correctness as future-work priorities.

9 Conclusion

  • VulDec introduces a large-scale dataset aligning C/C++ vulnerability annotations with assembly representations and an end-to-end LLM-based neural-decompilation framework for binary vulnerability detection.The framework translates binaries into source-level pseudo-code and treats both representations as textual artifacts.Source: §9 · p. 26
  • Future work will target detector robustness on OOD samples, decompiler hallucinations, and behavioral correctness through semantic constraints and data-flow representations.The conclusion identifies these as priorities for neural binary analysis.Source: §9 · p. 26

Appendix A Additional Details on VulDec Dataset

The dataset spans identical project versions across ARM and x86 and includes projects from diverse software domains. Its function distribution is heavily weighted toward the Linux kernel, which may limit immediate generalizability to smaller and embedded systems.

Appendix A Additional Details on VulDec Dataset

  • Project versions are identical across ARM and x86, while the selected open-source projects cover diverse application domains and software types.The project-level function counts are reported by architecture in Table A1.Source: Appendix A · p. 27, Table A1 · p. 27
  • Linux alone contributes over 60% of dataset functions, potentially limiting immediate generalizability to smaller projects, embedded systems, and IoT firmware.The dataset is dominated by large-scale systems.Source: Appendix A · p. 28

Appendix B Experimental Setup Details

The appendix reports dataset statistics and training configurations for neural decompilation and vulnerability detection. The setup uses 90/5/5 splits and adapts model training budgets to architecture and pre-training capacity.

Appendix B Experimental Setup Details

  • Assembly sequences are longer on average than their corresponding C outputs, and Table B2 also reports statistics for the unbalanced vulnerability-detection dataset.The table covers training, development, and test distributions for the x86 and ARM decompilation datasets.Source: Appendix B · p. 28, Table B2 · p. 28
  • Both tasks use 90/5/5 dataset splits, with model parameters and training budgets adapted by architecture and task.Encoder-decoder models were trained for up to 10 epochs, large decoder-only LLMs for 3, and smaller decoder-only models for 5–8.Source: Appendix B · pp. 28–29, Table B3 · p. 29
  • The pipeline caps assembly inputs at 512 tokens and generated C/C++ outputs at 256 tokens before encoding vulnerability information as 0/1.Decision threshold, batch size, gradient accumulation, epochs, and beam size were adapted for each model and task.Source: Appendix B · pp. 28–29

Appendix C Class-Imbalance Strategy Optimization across Model Architectures

The appendix compares class-imbalance strategies across model families and describes a mapping from vulnerability macro-categories to CWE identifiers.

Appendix C Class-Imbalance Strategy Optimization across Model Architectures

  • Standard encoder-based and small decoder-only models perform best with US 80-20 + CW, whereas modern decoder-only LLMs favor US 50-50.At 50-50, inverse-frequency class weights become 1:1 and are mathematically redundant, so class balance is controlled by resampling alone.Source: Appendix C · p. 29, Table C4 · p. 30

Appendix D CWE Macrocategory Mapping

  • The macro-category mapping groups vulnerabilities with shared security characteristics and links each category to corresponding CWE identifiers.The mapping is presented in Table D5.Source: Appendix D · p. 30, Table D5 · p. 30

Appendix E Decompiled Code: Qualitative Analysis

The qualitative analysis presents four test-set pairs sorted by Edit Similarity, illustrating outcomes from structurally faithful recovery to severe hallucination. Similarity scores can reflect syntactic or semantic errors, while some failures produce incomplete or non-compilable output.

Tier 1: Perfect Structural Match and Contextual Renaming

  • At 95.21% ES, the Linux routine's control-flow graph and syntactic blocks match the reference, with the score reduction attributed to contextual variable renaming.The identifier j becomes delay, reflecting context from the function name and jitterDelay field.Source: Appendix E · p. 31, Table E6 · p. 32

Tier 2: Syntactic Shift with Functional Equivalence

  • At 93.15% token-level similarity, a conditional expression is expanded into two returns while the decompiled function remains 100% functionally equivalent.The example is the i2c smbus read byte data function.Source: Appendix E · p. 31, Table E6 · p. 32

Tier 3: Semantic and Type Shift

  • At 58.15% ES, the word “reverse” biases the generated comparison toward descending order, while the underlying conditional structure is preserved.The output also has a type misalignment and swaps the 1 and -1 return values.Source: Appendix E · p. 31, Table E6 · p. 32

Tier 4: Catastrophic Hallucination and Loop

  • At 17.62% ES, the lowest tier includes premature termination, syntactic noise or repeated code, and fluent generation of a structurally correct but unrelated function.The reported severe hallucinations also include infinite repetition caused by collapse of the model's self-attention stopping criteria.Source: Appendix E · pp. 31, 33, Table E6 · p. 32
  • In the xmlRegNewState example, repeated allocation code continues until the token-generation limit, leaving an incomplete, non-compilable artifact.The loop follows hallucination of a non-existent error-tracking structure.Source: Appendix E · p. 33
Loading 2610.33716v1…