Source-linked AI summary
LLM4Decompile: Decompiling Binary Code with Large Language Models
Hanzhuo Tan, Qi Luo, Jing Li, Yuqun Zhang
TL;DR
Traditional decompilers can turn binaries into hard-to-read, non-executable pseudo-code. LLM4Decompile trains open-source models for direct binary decompilation and Ghidra-output refinement, with reported re-executability gains over existing tools and an additional gain from refinement.
Problem
Traditional decompilers can obscure high-level structure and produce syntax errors, limiting readability and re-executability.
Source: §1 · p. 1Method
LLM4Decompile trains open-source models for direct binary decompilation and fine-tunes Ref models to refine Ghidra output.
Source: §1 · p. 2Results
Over 100% higher re-executability than GPT-4o and Ghidra on HumanEval and ExeBench is reported for LLM4Decompile-End, with Ref models adding 16.2% over End.
Source: §1 · p. 2Takeaways & Limitations
LLM4Decompile can complement Ghidra by refining its output, while neither approach effectively decompiles obfuscated code.
Source: §6 · p. 9, Ethic Statement · p. 9The study is limited to C targeting x86, and full fine-tuning reached 6.7B models; larger-scale exploration remains future work.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Decompilation aims to convert binary code to high-level source code, but traditional tools like Ghidra often produce results that are difficult to read and execute. Motivated by the advancements in Large Language Models (LLMs), we propose LLM4Decompile, the first and largest open-source LLM series (1.3B to 33B) trained to decompile binary code. We optimize the LLM training process and introduce the LLM4Decompile-End models to decompile binary directly. The resulting models significantly outperform GPT-4o and Ghidra on the HumanEval and ExeBench benchmarks by over 100% in terms of re-executability rate. Additionally, we improve the standard refinement approach to fine-tune the LLM4Decompile-Ref models, enabling them to effectively refine the decompiled code from Ghidra and achieve a further 16.2% improvement over the LLM4Decompile-End. LLM4Decompile demonstrates the potential of LLMs to revolutionize binary code decompilation, delivering remarkable improvements in readability and executability while complementing conventional tools for optimal results. Our code, dataset, and models are released at https://github.com/albertan017/LLM4Decompile
1 Introduction
Traditional decompilers can make source difficult to read and re-execute, while LLM4Decompile trains models to decompile binaries directly and refine Ghidra output. The authors report higher re-executability than GPT-4o and Ghidra, with further gains from refinement.
1 Introduction
- Traditional tools can produce pseudo-code with obscured high-level structure, pointer arithmetic, and syntax errors, making algorithm reconstruction difficult.The stated applications include legacy software migration and security instrumentation.Source: §1 · pp. 1–2
- Neither LLM4Decompile nor Ghidra effectively decompiles code protected by obfuscation methods, according to the authors’ experiments.The authors connect this result to concerns about unauthorized intellectual-property infringement.Source: §1 · p. 2
- LLM4Decompile is an open-source series of 1.3B–33B-parameter models trained for binary decompilation.The contribution passage also states that the models are fine-tuned on 15 billion tokens.Source: §1 · p. 2
- LLM4Decompile-End models set a new standard for direct binary decompilation, surpassing GPT-4o and Ghidra by over 100% in re-executability on HumanEval and ExeBench.Source: §1 · p. 2
- LLM4Decompile-Ref models refine Ghidra’s decompilation results and improve re-executability by 16.2% over LLM4Decompile-End.The authors present refinement as complementing direct binary decompilation.Source: §1 · p. 2
2 Related Work
Prior decompilation work spans traditional control- and data-flow tools, neural translation, and LLM-based refinement or direct decompilation. The paper positions LLM4Decompile as expanding open-source direct decompilation to larger models and integrating refinement with Ghidra.
2 Related Work
- Traditional tools often produce low-level, source-like output that is difficult to understand and may not be re-executable.Their methods use control- and data-flow analysis and pattern matching to identify constructs such as loops and conditionals.Source: §2 · p. 2
- Early neural decompilation work treated machine-level instructions as a translation task and used recurrent neural networks with error correction.The translation framing converts machine-level instructions into readable source code.Source: §2 · p. 3
- LLM-based work includes refinement of traditional outputs and direct binary decompilation, but open-source direct models were limited to about 200M parameters.Reported refinement examples include DeGPT’s 24.4% reduction in cognitive load and DecGPT’s re-executability above 75%; Nova’s 1B-parameter model was not open-sourced.Source: §2 · p. 3
- The paper aims to advance LLM decompilation through direct binary models and an enhanced Ghidra-integrated refinement framework.The stated goal is to combine LLMs with Ghidra to augment conventional tools.Source: §2 · p. 3
3 LLM4Decompile
LLM4Decompile has two tracks: End models learn direct decompilation from assembly-derived inputs, while Ref models refine Ghidra pseudo-code. The training pipeline expands and filters data, then stages training from compilable code to executable code.
3 LLM4Decompile
- LLM4Decompile comprises End models for direct decompilation and Ref models that refine Ghidra’s results.The section introduces training optimization for End and a separate enhancement strategy for Ref.Source: §3 · p. 3
3.1.1 The End2End-Decompile Framework
- End2end-Decompile converts binary to assembly before the LLM generates source, and computes training loss between generated and original source.The text treats binary and disassembled assembly as interconvertible inputs.Source: §3.1.1 · pp. 3–4
3.1.2 Optimize LLM4Decompile-End
- Training pairs come from ExeBench’s five million C functions, compiled at O0, O1, O2, and O3 to augment optimization-state coverage.Each compiled version is paired with its source code.Source: §3.1.2 · p. 4
- The End training set is cleaned by removing MinHash/LSH-detected duplicates and samples shorter than 10 tokens.Source: §3.1.2 · p. 4
- Two-stage training first uses compilable but non-linkable data to teach binary knowledge, then executable code to support practical applicability.The authors state that compilable, non-linkable code is easier to extract and resembles its executable version except for linked external-symbol addresses.Source: §3.1.2 · p. 4
3.2.1 The Refined-Decompile Framework
- Refined-Decompile changes the LLM input to Ghidra’s pseudo-code, which the model fine-tunes to improve.The authors say Ghidra output can have readability issues and syntax errors while preserving underlying logic.Source: §3.2.1 · p. 4
3.2.2 Refine Ghidra by LLM4Decompile-Ref
- For Ref training, the authors decompile non-executable files to reduce Ghidra processing time, then augment and filter the data, including overlength outputs.Ghidra Headless takes 2 seconds per executable sample versus 0.2 seconds for non-executable files; the text cites high similarity between the two binary types.Source: §3.2.2 · pp. 4–5
4 Experiments
The experiments evaluate direct binary decompilation and Ghidra-output refinement on HumanEval-Decompile and ExeBench, measuring re-executability and readability. Results show gains from LLM4Decompile models, with outcomes also varying by model scale and training-data composition.
4.1.1 Experimental Setups
- Training data comprise 7.2 million compilable samples (~7 billion tokens) and 1.6 million executable samples (~572 million tokens), constructed from ExeBench assembly-source pairs.Models use sequence-to-sequence prediction, minimizing negative log likelihood on source-code output tokens.Source: §4.1.1 · p. 5
- HumanEval-Decompile converts 164 Python challenges and assertions into C, while ExeBench covers 5,000 real-world C functions with IO examples.Re-executability is checked by compiling the reconstructed C and testing whether it passes the assertions.Source: §4.1.1 · p. 5
4.1.2 Experimental Results
- 45.4% average success rate for End-6.7B versus 27.3% for End-1.3B; GPT-4o falls from 30.5% at O0 to about 11% at O1-O3.The 33B model outperforms the 1.3B model despite training being restricted to 200M tokens.Source: §4.1.2 · p. 6
4.2.2 Experimental Results
- 160% increase in re-executability for Ref-6.7B over Ghidra; versus End-6.7B, Ref-6.7B shows a 16.2% performance increase.The 16.2% comparison is reported despite Ref-6.7B using only 10% of the data used to train the other Ref models.Source: Table 3 · p. 7, §4.2.2 · p. 7
4.1.2 Experimental Results
- 37.5% HumanEval-Decompile success for compilable-only 6.7B versus 6.8% on ExeBench; executable-only reaches 26.7% on ExeBench and 23.8% on HumanEval-Decompile.The compilable-only data resemble single-function HumanEval-Decompile inputs, whereas ExeBench functions include extensive user-defined functions.Source: §4.1.2 · pp. 6–7
4.2.1 Experimental Setups
- The refinement experiments train on 1.6 million samples across optimization levels O0-O3 and evaluate on HumanEval-Decompile using Edit Similarity for readability.Resource constraints limited the training set to 400K functions at each optimization level.Source: §4.2.1 · p. 7
4.2.2 Experimental Results
- 5.7%-14.0% Edit Similarity across decompilers; Ghidra averages 6.2%, while LLM4Decompile-End and -Ref outputs are more source-aligned and easier to comprehend.GPT-4o refinement slightly decreases Edit Similarity; GPT-based readability scores are subjective, so the authors suggest considering them alongside Edit Similarity.Source: §4.2.2 · p. 8, Figure 4 · p. 8
- GPT-4o readability assessment scores syntax similarity and structural integrity on a 1-to-5 scale across models and optimization levels.The paper describes GPT’s judgments as subjective and suggests combining them with Edit Similarity for a fuller readability assessment.Source: §4.2.2 · p. 8, Table 4 · p. 8
5 Obfuscation Discussion
The section evaluates whether common obfuscation techniques constrain decompilation and frames that test as relevant to intellectual-property protection. It reports that both Ghidra and LLM4Decompile are prevented from decoding obfuscated binaries.
5 Obfuscation Discussion
- The section frames decompilation as a potential intellectual-property risk because it aims to reveal source code from binaries distributed by developers.It assesses possible misuse of the models in that context.Source: §5 · p. 8
- 90.2% (0.5274 to 0.0519) under CFF and 78.0% (0.5274 to 0.1159) under BCF were the success-rate drops for LLM4Decompile-Ref-6.7B; basic obfuscation prevented both it and Ghidra from decoding binaries.Table 5 summarizes re-executability rates under obfuscation and reports drops exceeding 70% compared with Table 3.Source: Table 5 · p. 9, §5 · p. 9
- Control Flow Flattening and Bogus Control Flow are designed to disguise software logic and make decompilation more challenging.The study tests these two fundamental obfuscation techniques from Obfuscator-LLVM.Source: §5 · p. 9
6 Conclusions
LLM4Decompile introduces an open-source model series for direct binary decompilation and refinement of Ghidra output. The reported End and Ref results exceed existing tools and improve on the preceding model stage.
6 Conclusions
- The open-source LLM4Decompile series ranges from 1.3B to 33B parameters and is trained for binary-code decompilation.Source: §6 · p. 9
- The 6.7B LLM4Decompile-End model achieved 45.4% accuracy on HumanEval and 18.0% on ExeBench, surpassing Ghidra and GPT-4o by over 100%.The reported comparison is in terms of re-executability.Source: §6 · p. 9
- LLM4Decompile-Ref models refine Ghidra output and achieve a 16.2% improvement over LLM4Decompile-End.Source: §6 · p. 9
Limitations
The study is limited to C compilation and decompilation for x86, and compute constraints limited full fine-tuning to 6.7B models. Larger-model exploration remains preliminary.
Limitations
- The research covers compilation and decompilation of C targeting the x86 platform, leaving other languages and platforms for future investigation.Source: Limitations · p. 9
- A budget equivalent to 8 × A100 GPUs for one year limited full fine-tuning to 6.7B models.The budget included all trials and iterations.Source: Limitations · p. 9
- Exploration of 33B models used a small dataset, while models of 70B and larger remain for future study.Preliminary tests suggest potential advantages from scaling model size.Source: Limitations · p. 9
Ethic Statement
The statement presents basic obfuscation as a tested protection against unauthorized decompilation and describes intended use in permitted or uncopyrighted settings.
Ethic Statement
- Control Flow Flattening and Bogus Control Flow were empirically tested and protected against unauthorized decompilation by both Ghidra and LLM4Decompile.Source: Ethic Statement · p. 9
- Industry developers use complex obfuscation before release as an additional layer of security and intellectual-property protection against decompilation.Source: Ethic Statement · p. 9
- The models are intended for permission-granted or uncopyrighted software, including academic research, debugging, learning, and recovering a company's lost source code.Source: Ethic Statement · p. 10
A ExeBench Setups
ExeBench evaluation uses a realistic binary-only setting, with evaluation limited to samples the authors could compile; this differs from prior results based on assembly intermediates. The section also distinguishes compilable from executable binaries by how jump addresses are resolved.
A ExeBench Setups
- Only 443K of 797K training samples and 2621 of 5000 test samples could be compiled using standard settings, so evaluation used those subsets.The authors report that ExeBench did not provide the original compilation settings and processing details.Source: A ExeBench Setups · p. 12
- GPT-4o achieved 4.4% re-executability and 7.9% edit similarity on ExeBench, versus Slade’s ChatGPT results of 22.2% and 44.0% using assembly intermediates.The authors describe binary-only evaluation as the more realistic setting because such intermediate output is rarely released by software developers.Source: A ExeBench Setups · pp. 12–13, Table 6 · p. 13
- GPT-4o could not recover lost user-defined types from binary data, instead using default types and producing non-executable code, a pervasive ExeBench failure.Figure 5 illustrates this issue with the types Ltc4151State, Ltc4151, and device.Source: A ExeBench Setups · p. 13, Figure 5 · p. 13
B Compilable and Executable Binary
- In compilable binaries, jump addresses are relative-offset placeholders; in executable binaries, linking resolves them to memory locations.The section summarizes training and test dataset statistics for these two dataset types.Source: B Compilable and Executable Binary · p. 13, Figure 6 · p. 13, Table 7 · p. 13
C Further Analysis of LLM4Decompile-End
Further analysis finds that re-executability declines with longer inputs and higher optimization, while the larger model is more robust to input length. Error patterns differ across HumanEval-Decompile and ExeBench, with user-defined components a major ExeBench challenge.
C Further Analysis of LLM4Decompile-End
- The 6.7B model is more robust to increasing input length than the 1.3B model, and the analysis reports an overall advantage for the larger model.Figure 7 compares re-executability rates as input length grows.Source: Figure 7 · p. 13, C Further Analysis of LLM4Decompile-End · p. 14
- Re-executability falls as input length increases, with a marked decline at higher optimization levels.The findings highlight difficulties in decompiling long and highly optimized sequences.Source: C Further Analysis of LLM4Decompile-End · p. 13
- In HumanEval-Decompile, 64% of errors are failed assertions; in ExeBench, 50% involve undeclared functions and 28% improper structure use.The authors note that user-defined details are typically lost during compilation and suggest retrieval-augmented generation as a possible source of external information.Source: C Further Analysis of LLM4Decompile-End · p. 14, Figure 8 · p. 14
D Data Quality, Volume and Model
The analysis examines data quality, training volume, model choice, and obfuscation. It reports that data selection and training choices affect performance, while describing two control-flow obfuscation techniques.
D Data Quality, Volume and Model
- Selective removal of data incompatible with standard C libraries can improve performance on Decompile-Eval, while the authors retain a broad, minimally processed baseline.Their preprocessing filters short texts and duplicates; the baseline is intended to reflect diverse scenarios and support future refinement.Source: D Data Quality, Volume and Model · p. 14, Table 8 · p. 14
- Two epochs optimize LLM4Decompile-Ref-1.3B performance, whereas additional epochs tend toward overfitting and diminished results.A single epoch serves as a strong baseline; the training epoch corresponds to 2B tokens.Source: D Data Quality, Volume and Model · p. 14, Table 9 · p. 14
- The 6.7B model achieves performance comparable to the 1.3B model with 20% of the data, and the 33B model matches the 6.7B outcomes with 10%.The authors caution that these ratios may vary across datasets and say scaling can enhance performance when training data is sufficiently large (100M tokens).Source: D Data Quality, Volume and Model · p. 14
- On HumanEval-Decompile, LLM-Compiler-7B improves average re-executability by 3.5% over Deepseek-Coder-6.7B, Yi-Coder-9B by 23.1%, and CodeStral-22B by 21.7% over smaller models.Deepseek-Coder-6.7B’s reported average re-executability rate is 52.74%.Source: D Data Quality, Volume and Model · p. 15, Table 11 · p. 15
E Obfuscation Techniques
- The section describes control-flow flattening, which puts basic blocks at one level in a loop-contained switch, and bogus control flow, which adds opaque-predicate blocks and meaningless instructions.Both are presented as classic obfuscation techniques suggested in Obfuscator-LLVM.Source: E Obfuscation Techniques · p. 15