Source-linked AI summary
CHISEL-ing Back Source Code with AI-enabled Iterative Recovery
Varun Kohli, N Raghava, Biplab Sikdar, Dinil Mon Divakaran
TL;DR
Decompilation must recover readable, compilable, and semantically faithful source, while existing decompilers and LLM-assisted methods have important shortcomings. CHISEL addresses this with test-suite-free iterative recovery using compiler and coverage-guided-fuzzer feedback, achieving strong compilation and execution recovery while reducing false acceptance. Its evaluation remains bounded by reliance on Ghidra pseudo-C and approximate I/O sampling.
Problem
Decompilation seeks readable, compilable, and semantically faithful source from binaries, but traditional decompilers produce difficult-to-read, rarely compilable pseudo-C without guaranteed semantic fidelity.
Method
CHISEL is a test-suite-free iterative decompiler that uses compiler and coverage-guided-fuzzer signals from the original binary, augmented by rich observables, divergence memory, and best-candidate retention.
Results
CHISEL reports 96.1% re-compilability, 79.8% re-executability, 9.4% false acceptance, and 26.2% recovery after first-generation failure, outperforming LLM4Decompile and Agent4Decompile.
Takeaways & Limitations
Richer observation provides the decisive gain in feedback soundness, while CHISEL substantially improves recovery over one-shot and prior LLM-assisted decompilation approaches.
Takeaways & Limitations
The evaluation relies on Ghidra pseudo-C as the source of truth, and I/O sampling approximates functional similarity rather than establishing semantic equivalence.
Abstract
from arXiv · showhide
Decompilation aims to recover high-level, compilable, and semantically equivalent code from binaries. Traditional decompilers produce pseudo-C that is difficult to read and does not compile, while the recent LLM-assisted approaches generate readable, but semantically incorrect code. LLM-aided iterative recovery is an emerging branch of research, but prior works rely on supplied test suites for semantic recovery. In this work, we present CHISEL, a test suite-free framework to iteratively recover source code from Ghidra-derived pseudo-C. CHISEL uses simple yet effective feedback from a compiler (static analysis) and a coverage-guided fuzzer (differential analysis), augmented by rich observables for grounded divergence detection and feedback, cross-iteration divergence memory, and best candidate retention. We systematically evaluate CHISEL for compilation and semantic recovery, feedback oracle soundness, and iteration overhead on 120 ExeBench functions compiled for the x86-64 architecture, across four optimizations (O0-O3), in both stripped and unstripped variants, using the open-weight Gemma4:31b LLM. CHISEL, with all recommended features, achieves an average of 96.1% re-compilability and 79.8% re-executability rates at an average of 2.1 iterations. Significantly, CHISEL recovers 26% of first-generation execution errors. At the same time, CHISEL feedback oracle falsely accepts only 9.4% candidates. Lastly, CHISEL performs significantly better than two recent prior work on LLM-assisted decompilation.
N Raghava Coimbatore Institution of Technology
The passage identifies Dinil Mon Divakaran with the A*STAR Institute of Advanced Intelligence and Computing.
- Dinil Mon Divakaran is affiliated with the A*STAR Institute of Advanced Intelligence and Computing.
- The listed affiliation is A*STAR Institute of Advanced Intelligence and Computing.
- The passage names Dinil Mon Divakaran alongside the A*STAR Institute of Advanced Intelligence and Computing.
1 Introduction
Decompilation seeks readable, compilable, semantically faithful source, but prior iterative approaches often depend on unavailable test suites. CHISEL addresses this gap with test-suite-free compiler and fuzzer feedback, and reports strong recovery across a broad evaluation.
- Motivation: Traditional decompilers produce pseudo-C that is difficult to read, rarely compilable, and not guaranteed semantically faithful.
- Motivation: Prior iterative methods often rely on native test suites for semantic-divergence feedback, which may be unavailable for binaries with unknown source.
- Approach: CHISEL is a test-suite-free iterative framework whose in-loop signals come from the original binary through a compiler and coverage-guided fuzzer.
- Approach: CHISEL augments feedback with rich observables, cross-iteration divergence memory, and best-candidate retention.
- Evaluation: 96.1% RC, 79.8% RE, 9.4% FA, and 26.2% recovery after first-generation failure were reported across the evaluated settings.
2 Related Work
LLM-assisted decompilers range from one-shot generation to iterative systems using compiler, test-suite, sanitizer, or fuzzer feedback. CHISEL removes the supplied-test-suite requirement and broadens differential observability while retaining iterative recovery features.
- Prior approaches: Non-iterative methods generate candidate source without behavioral feedback, relying on the LLM’s inherent recovery capability.
- Prior approaches: DecLLM uses compiler repairs, AFL++ divergence feedback seeded by a supplied test suite, and candidate sanitization.
- Prior approaches: PCodeTrans validates against an official test suite and localizes failures through breakpoint-matched differential tracing.
- CHISEL: CHISEL derives in-loop signals solely from the original binary and compares rich observables including outputs, exit codes, timeouts, and crashes.
3 Proposed Framework: Chisel
Chisel iteratively transforms Ghidra pseudo-C into compilable C using compiler and differential-fuzzer feedback, with observability, divergence memory, and best-candidate retention supporting recovery.
- Iterative Enhancement Algorithm: Chisel accepts Ghidra pseudo-C, a reference object, an iteration budget, feedback oracles, feedback budgets, and optional features, returning a compilable C function.The iterative algorithm is designed to nudge generation toward compilable and functionally similar code.
- Iterative Enhancement Algorithm: Each round samples one zero-temperature LLM candidate from the pseudo-C, previous candidate, and oracle-derived feedback.The compiler oracle supplies parsed diagnostics when compilation fails.
- Iterative Enhancement Algorithm: The differential oracle uses coverage-guided libFuzzer with type-aware seeds to find inputs that expose behavioral divergence from the original reference.Discriminating inputs are accumulated and sampled as feedback, subject to compilation and fuzzer feedback budgets.
- Iterative Enhancement Algorithm: Rich observables compare return values, buffers, stdout, timeouts, exit codes, crashes, and pointer contents rather than raw addresses.These observations provide grounded differential feedback for subsequent generations.
- Iterative Enhancement Algorithm: Cross-iteration memory retains deduplicated divergences, while best-candidate retention returns the candidate with minimum divergence.The retention feature selects the best-performing candidate from the loop.
- Prompts and Parameters: The prompt instructs the LLM to rewrite pseudo-C as compilable C named func0 while preserving control flow and operation order and removing decompiler scaffolding.Later rounds append the previous candidate and available feedback.
4 Experimental Setup
The evaluation measures recovery, feedback-oracle soundness, and iteration overhead through a six-arm ablation, cross-optimization tests, and comparisons with prior systems.
- Experimental Setup: Experiments use an x86-64 Linux host with four NVIDIA H200 GPUs, Gemma4:31 at int-4 quantization, Ghidra, gcc -c, and libFuzzer.The dataset draws long real-world functions from ExeBench’s real and val-real splits.
- Experiments and Evaluation Metrics: The study evaluates recovery, soundness, and overhead using a six-arm ablation, optimization variants, stripped and unstripped cases, and prior-work comparisons.Recovery includes compilation and execution behavior; soundness evaluates oracle decisions against tests; overhead measures average iterations.
- Experiments and Evaluation Metrics: The six-arm ablation incrementally adds the compiler and fuzzer oracles, observables, cross-iteration memory, and best-candidate retention to a one-shot LLM.Each arm retains previously added features, with the final arm representing complete Chisel.
- Evaluation Metrics: RC and RE measure returned candidates that compile and pass every test case, while Pass measures the average fraction of test cases passed.Additional metrics measure recovery from compilation or execution failures, regression, oracle errors, and iteration overhead.
- Evaluation Metrics: Oracle soundness is assessed with false-acceptance and false-rejection rates against the ground-truth test suite.High false rejection increases iterative-loop length and average iterations.
- Baselines: The comparison includes LLM4Decompile-9B-v2 and Agent4Decompile, with Agent4Decompile using Gemma4:31 for a fair harness comparison.No evaluated method receives the ground-truth test suite during iterations.
5 Results
CHISEL’s evaluation examines recovery, oracle soundness, and iteration overhead through ablations, optimization levels, strippedness, and comparisons with prior work. The full framework improves semantic recovery and oracle soundness while outperforming the reported baselines.
- Six-arm ablation: 96.6% re-compilation follows compiler-oracle feedback, but re-execution recovery remains 0 because compilation alone cannot distinguish behavioral correctness.The raw one-shot model compiles 90.0% and re-executes 73.0% of functions.
- Six-arm ablation: 79.3% RE follows differential-fuzzer feedback, which recovers 7.5% of compiled-but-incorrect first generations and reduces FA to 17.6%.The fuzzer supplies behavioral feedback absent from compiler-only checking.
- Six-arm ablation: 9.4% FA is achieved by the observe arm, halving false acceptance from 17.6% while preserving RE and FR.Rich observation provides grounded divergence information for feedback.
- Stripped versus unstripped: 16.4% FA occurs on stripped binaries versus 1.7% on unstripped binaries, with residual false acceptance dominated by signature recovery errors.Stripping removes type information needed for signature recovery.
- Stripped versus unstripped: 86.7% of unstripped and 72.9% of stripped functions re-execute on average, while RE declines from O0 to O3 in both variants.RE falls from 93.3% to 80.8% for unstripped binaries and from 78.3% to 68.3% for stripped binaries.
- Comparison with prior work: Chisel outperforms LLM4Decompile’s 17% RE and performs better than Agent4Decompile at lower iteration cost.The comparison uses the same 120 functions and test suite.
6 Limitations, Risks and Ethics
The evaluation is bounded by signature recovery difficulties, experimental settings, approximate behavioral equivalence, and dataset choices. The authors also identify potential misuse risks because the work uses open-weight models.
- Limitations: Signature recovery from stripped binaries remains a limitation for decompilers.The limitation is especially relevant because stripped binaries remove type information needed for signature recovery.
- Experimental scope: The experiments use temperature 0 and non-reasoning settings, so higher temperatures or LLM reasoning may improve recovery but increase variation or overhead.The authors identify these settings as constraints on the reported evaluation.
- Evaluation caveat: I/O sampling approximates functional similarity and does not establish semantic equivalence.This affects both the in-loop oracle and the evaluation instrument.
- Evaluation scope: Using Ghidra pseudo-C as the source of truth leaves decompiler resilience for further evaluation.The authors also note that future work may use newly released datasets to reduce possible training-data overlap.
- Ethics: Open-weight models create potential unauthorized-use risks for threat actors, motivating ethical use of the work.The authors frame the research as advancing reverse engineering while acknowledging possible misuse.
7 Conclusion
CHISEL is a test suite-free iterative decompiler whose compiler- and fuzzer-based feedback supports compilation and execution recovery. Across ExeBench variants, it achieves strong recovery rates, repairs first-generation failures, reduces false acceptance, and outperforms prior approaches.
- 96.1% RC and 79.8% RE were achieved across 120 ExeBench functions, four optimizations, and stripped and unstripped x86-64 variants.The evaluation used a test suite-free iterative decompiler with static and differential analysis.
- 26.2% of first-generation failures were repaired by CHISEL.
- 9.4% FA was achieved by the in-loop oracle, halving the 17.6% FA of a return-only differential fuzzer.
- Richer observation, rather than more search, provided the decisive gain in CHISEL’s performance.
- CHISEL achieved 79.8% RE versus 17.0% RE for LLM4Decompile-9B-v2 and outperformed Agent4Decompile on RE, FA, and overhead.These comparisons used the same evaluation subset and, for Agent4Decompile, the same LLM and test-suite assumptions.
A Generative AI Usage
The authors report using Claude and Gemini for manuscript language, verified plotting code, and prototype code, while producing experiments and numerical results through their own pipeline. They state that AI-assisted code was reviewed by the authors, but the supplied passage is truncated before completing that statement.
- Claude and Gemini were used to improve manuscript language, generate verified plotting code, and write prototype code.
- All experiments, datasets, and numerical results were produced by the authors’ own pipeline.
- Every figure and table was regenerated from raw experimental outputs and checked against underlying data.
- The authors reviewed all AI-assisted code, but the supplied statement is incomplete.