VulBinLLM: LLM-powered Vulnerability Detection for Stripped Binaries

• Compilation and decompilation lose high-level constructs, optimization context, symbols, and debug information, making binary vulnerabilities ambiguous and costly to identify manually.Source: §1 · p. 1
• LLMs face additional difficulties because binary code is abstract, semantic clues are sparse, long inputs increase hallucinations, and binaries exceed context windows.Source: §1 · pp. 1–2
• The framework enriches decompiled code with vulnerability-specific comments, uses prompt-based vulnerability reasoning, and combines extended memory with a function queue to reduce hallucinations.Source: §1 · p. 2
• Vul-BinLLM outperformed state-of-the-art tools and frameworks on stripped synthetic Juliet binaries and provided justification for detected CWEs.Source: §1 · p. 2
• The work contributes an LLM-assisted tool for CWE detection on decompiled binaries and investigates whether LLMs can reconstruct vulnerability-relevant behavior with limited contextual knowledge.Source: §1 · p. 2
• Approximately 10% higher accuracy was achieved for detecting vulnerabilities in stripped synthetic code.Source: §1 · p. 2

2.1 Reserve Engineering for Vulnerability Analysis

• Binary reverse engineering reconstructs program logic and execution conditions to reveal bugs or suspicious behavior, but remains time-consuming and dependent on scarce expert skill.Source: §2.1 · p. 2

2.2 Binary Analysis and Program Analysis

• Static binary analysis examines code without execution and typically combines disassembly, decompilation, control-flow, data-flow, call-graph, symbolic-execution, and pattern-matching techniques.Source: §2.2 · pp. 2–3
• Vul-BinLLM’s workflow decompiles binaries, enriches the result with contextual information, analyzes it, stores prior analyses, and queues functions for managed processing.Source: Figure 1 · p. 3
• A buffer-overflow example illustrates that subtle vulnerabilities detectable by human security experts may remain difficult for an LLM to identify.Source: Figure 2 · p. 3

2.3 CVE and CWE

• CVE identifies specific real-world vulnerability instances, whereas CWE organizes broader classes of software and hardware weaknesses.Source: §2.3 · p. 3
• Widely used datasets span broad CWE coverage, public-CVE-linked vulnerabilities, open-source critical flaws, memory weaknesses, and Java-specific vulnerabilities.Source: Table 1 · p. 3

3.1 Overview

• Vul-BinLLM first reconstructs syntactic information from binaries and optimizes the recovered code to make vulnerable features more prominent for LLM detection.The framework targets stripped binaries whose low-level representations lack source-level abstractions and semantic information.Source: §3.1 · p. 4

3.2 Design of Vul-BinLLM

• An extended-memory workflow stores summarized function analyses in archival SQL storage and uses a queue to process functions across the binary.This design addresses binaries that exceed the LLM context window while supporting context-sensitive analysis of control-flow-related code segments.Source: §3.2 · p. 4

3.2.1 Vul-BinLLM - Vulnerability Prominence.

• The framework restores vulnerability-related comments, variable information, code structure, and contextual relationships rather than merely translating binary code verbatim.Its optimization highlights dangerous calls, memory-corruption patterns, logic errors, and broader program context.Source: §3.2.1 · pp. 4–5
• Table 2 reports that no evaluated LLM intuitively identified CWE-78 OS Command Injection from binary-level code, although some understood a simple synthetic matrix-multiplication weakness.The table compares Vul-1, a synthetic buffer-overflow weakness, with Vul-2, a Juliet CWE-78 case; processing CodeLLaMA’s 100k-token context required substantial computation.Source: Table 2 · p. 5
• In-context learning and few-shot Chain-of-Thought prompting guide analysis of functionality, root causes, and potential impacts across vulnerability patterns.The prompting strategy decomposes vulnerability reasoning into smaller steps aligned with systematic security analysis.Source: §3.2.1 · p. 5
• A memory-management agent creates knowledge documents from binary functions and places them in an archived analysis store for clearer LLM reasoning.The VulBinQ manages which functions are analyzed and organizes the resulting binary-derived knowledge documents.Source: §3.2.1 · p. 5

4.1 Implementation

• GPT-4o enhances Ghidra-derived output by appending vulnerability and functionality comments, simplifying structures, and renaming variables to make vulnerability features more prominent.The implementation extracts decompiled code, control-flow graphs, import tables, stack frames, and related information before enhancement.Source: §4.1 · p. 5, Figure 3 · p. 6
• Figure 3 illustrates a progression from original source code to Ghidra output, GPT-4o-enhanced decompilation, and Vul-BinLLM output with vulnerability-specific annotations.The example highlights pointer-arithmetic warnings associated with possible buffer overflows or memory-access vulnerabilities.Source: Figure 3 · p. 6
• Figure 4 shows an Optimization Decision Agent and three action agents that check raw decompilation before renaming identifiers, reorganizing structures, and appending vulnerability explanations.The decompiler can also use in-context examples during optimization.Source: Figure 4 · p. 7

4.2 Research questions and evaluation setup

• The evaluation asks whether LLMs can detect vulnerabilities in stripped and unstripped binaries, analyze binaries beyond their context windows, and compete with LLM-powered tools on stripped synthetic tests.The study benchmarks decompiled Juliet code and examines how the extended-context approach affects accuracy for smaller binaries.Source: §4.2 · p. 7

4.3 Results

• Vul-BinLLM is evaluated on compiled, stripped Juliet C/C++ binaries, with more than 20,000 samples generated after removing debug information and symbol tables.Constant-value cases were excluded because LATTE targets vulnerabilities introduced by external inputs.Source: §4.3 · pp. 7–8
• The evaluation compares Vul-BinLLM with LATTE, a state-of-the-art LLM approach for binary taint analysis with coverage limited to certain detectable CWE classes.The comparison is reported in Table 3, which evaluates Vul-BinLLM’s abilities on the synthetic dataset.Source: §4.3 · p. 8
• Vul-BinLLM performs CWE classification by presenting optimized decompiled code with multiple MITRE-derived weakness descriptions and directing GPT-4o to examine targeted aspects such as data flow.Using multiple CWEs in each query reduces reliance on memorized patterns and supports more nuanced vulnerability assessment.Source: §4.3 · p. 8

Limitations

• Compiled Juliet binaries were used because binary vulnerability datasets are scarce and source-code datasets are difficult to compile consistently for binary analysis.The authors cite inconsistent build environments and dependencies as barriers to using datasets such as Devign and BigVul.Source: §5 · p. 8

CWE classification

• Figure 5 illustrates binary CWE classification in which the LLM answers yes or no while being prompted with multiple CWE descriptions.The example combines CWE-78 with CWE-121 and CWE-787 to reduce memorization effects.Source: Figure 5 · p. 8

Future directions

• The paper leaves vulnerability-reasoning explainability, architecture-specific vulnerability taxonomies, and direct assembly-level detection as future research directions.The discussion notes that assembly representations can be highly similar despite different vulnerability semantics.Source: §5 · p. 8

Evaluation

• Table 3 reports vulnerability-inspection evaluation results on the Juliet test suite, with LATTE and Vul-BinLLM represented using different test-case conventions.The accompanying notes distinguish bad/good test-case counts, files containing vulnerability types, and correctly undetected cases.Source: Table 3 · p. 9, §5 · p. 9

6.1 Semantic Recovery from Binary files

• Prior binary-analysis LLM work mainly regenerates readable code, function names, or summaries rather than directly detecting vulnerabilities.These approaches improve human understandability but remain limited by missing high-level abstractions and semantic information.Source: §6.1 · p. 9

6.2 Reverse Engineering and Program Analysis for Vulnerability Detection

• Reverse-engineering research covers vulnerability assessment, malware analysis, software repair, and optimization, with decompilation serving as a central technique.Decompilation optimization has also used machine learning to recover variable names, types, and readable code from stripped binaries.Source: §6.2 · p. 9
• Vul-BinLLM differs from prior decompilation work by annotating potential vulnerabilities, simplifying code, and renaming variables to provide concrete reasoning cues.The paper contrasts this focus with methods centered on recovering readable code or debugging information.Source: §6.2 · p. 9

6.3 Learning-based Vulnerability Detection

• GNNs exploit graph structure but can struggle with lexically similar, semantically different code, while LLMs face difficulties with complex low-level vulnerabilities and intricate program flows.Rule-based static analyzers are efficient and interpretable but can produce false positives and fail to generalize beyond predefined rules.Source: §6.3 · pp. 9–10
• PLM and LLM approaches use semantic encoding, prompt engineering, chain-of-thought, few-shot learning, or fine-tuning to identify vulnerabilities in code.Prompt engineering avoids modifying model parameters, whereas fine-tuning updates them using vulnerability datasets.Source: §6.3 · p. 10

6.4 Code Large Language Models (CodeLLMs)

• CodeLLM research has largely emphasized generation, debugging, and summarization, whereas this work focuses on vulnerability and weakness analysis.The related work includes encoder-only, decoder-only, and newer long-sequence model families.Source: §6.4 · p. 10
• LATTE is the closest LLM-based binary-analysis work but targets specific taint analysis, while other binary methods use traditional machine-learning techniques.The paper presents Vul-BinLLM as complementary to LATTE and distinct from fine-tuned DeBinVul because it requires no fine-tuning.Source: §6.4 · p. 10

Conclusion

• Vul-BinLLM integrates neural decompilation optimization, extended context, and memory management to enable vulnerability analysis from binaries.The conclusion presents these components as addressing longstanding challenges in binary vulnerability detection.Source: §7 · p. 10