PaperlayerView exact revision on arXiv

Source-linked AI summary

When Coding Style Survives Compilation: De-anonymizing Programmers from Executable Binaries

Aylin Caliskan, Fabian Yamaguchi, Edwin Dauber, Richard Harang, Konrad Rieck, Rachel Greenstadt, Arvind Narayanan

arXiv:1512.08546v3cs.CR

TL;DR

Identifying programmers from executable binaries matters because authorship attribution can threaten privacy, while compilation obscures source-code features. The paper combines binary and decompiled-code representations with machine learning, achieving attribution across GCJ and real-world samples. Its scope does not include heavily obfuscated binaries or programs authored by multiple people.

  • Problem

    Executable-binary attribution is harder than source-code attribution because compilation removes distinguishing features and may alter program structure, while successful attribution threatens programmer privacy.

    Source: §I · p. 1

  • Method

    The method combines assembly and decompiled-code features, selects informative representations, and uses machine learning to attribute binary authorship.

    Source: B. Feature extraction via decompilation · p. 5, §VII · p. 13

  • Results

    Across GCJ and real-world GitHub and forum samples, the method attributes programmers; it reaches 96% accuracy for 100 programmers on GCJ.

    Source: §IX · p. 14

  • Takeaways & Limitations

    Coding style can survive compilation, so executable binaries may expose the identity of programmers seeking anonymity.

    Source: §I · p. 1, §IX · p. 14

    The study does not model multi-author programs or heavily obfuscated binaries designed to hinder reverse engineering.

    Source: §VIII · p. 14

Abstract

from arXiv · show

The ability to identify authors of computer programs based on their coding style is a direct threat to the privacy and anonymity of programmers. While recent work found that source code can be attributed to authors with high accuracy, attribution of executable binaries appears to be much more difficult. Many distinguishing features present in source code, e.g. variable names, are removed in the compilation process, and compiler optimization may alter the structure of a program, further obscuring features that are known to be useful in determining authorship. We examine programmer de-anonymization from the standpoint of machine learning, using a novel set of features that include ones obtained by decompiling the executable binary to source code. We adapt a powerful set of techniques from the domain of source code authorship attribution along with stylistic representations embedded in assembly, resulting in successful de-anonymization of a large set of programmers. We evaluate our approach on data from the Google Code Jam, obtaining attribution accuracy of up to 96% with 100 and 83% with 600 candidate programmers. We present an executable binary authorship attribution approach, for the first time, that is robust to basic obfuscations, a range of compiler optimization settings, and binaries that have been stripped of their symbol tables. We perform programmer de-anonymization using both obfuscated binaries, and real-world code found "in the wild" in single-author GitHub repositories and the recently leaked Nulled.IO hacker forum. We show that programmers who would like to remain anonymous need to take extreme countermeasures to protect their privacy.

I. INTRODUCTION

The paper argues that coding style survives compilation and can expose a programmer’s identity when candidate authors and known samples are available. It combines assembly and decompiled-code features with machine learning, reports attribution across controlled and real-world settings, and notes important limits to practical use.

I. INTRODUCTION

  • Coding style preserved in compiled binaries may make it possible to infer identity when candidate programmers and their labeled samples are known.The privacy implications apply to privacy-minded individuals as well as malicious actors.Source: §I · p. 1
  • 92% accuracy for 191 programmers, versus Rosenblum et al.’s 51%; accuracy reached 83% with 600 programmers.The approach also distinguishes 600 candidates versus 20 with higher accuracy, using 53 stylistic features per programmer.Source: §I · pp. 1–2
  • Basic obfuscation, compiler optimization, and stripped debugging symbols each cause only a modest decrease in attribution accuracy.The paper presents these results as evidence of robustness across realistic binary transformations.Source: §I · p. 2
  • Decompiled features need not resemble original-source features: the combined binary feature vector predicts them, with over 80% average cosine similarity.The authors also find that more skilled programmers are more fingerprintable, suggesting that individual style develops with experience.Source: §I · p. 2
  • On noisy GitHub data, the method attributes 50 programmers with 65% accuracy; in the Nulled.IO case study, it correctly identifies three members and rejects a fourth.The fourth member had only one sample, and the authors could not build a model for that person.Source: §I · p. 2
  • The method is not yet fully ready for practical use because multi-author programs and encrypted binaries are not modeled.The authors say these cases require different machine-learning and segmentation techniques.Source: §I · p. 2

II. PROBLEM STATEMENT

The paper frames executable-binary authorship attribution as a supervised task and reviews stylometry across text, source code, and binaries. It situates its contribution against earlier binary work conducted on controlled corpora.

II. PROBLEM STATEMENT

  • The main task is closed-world, multiclass classification among known candidate programmers, with separate experiments for open-world attribution.The analyst is assumed to have labeled binary samples from candidate authors; the open-world setup can reject an unfamiliar author.Source: §II · pp. 2–3
  • The experiments assume the compiler and optimization level are known, and that training uses the same toolchain as the binaries under analysis.The authors ground these assumptions in prior toolchain-identification results.Source: §II · p. 3

III. RELATED WORK

  • Stylometry applies authorship analysis to creative expression, including language, music, and source code,,.Source-code approaches range from n-grams and lexical features to AST structures, with reported GCJ accuracies of 77% for 10 programmers and 94% for 1,600,.Source: §III · p. 3
  • Several authors applied similar techniques to identify programmers from source code; Rosenblum et al. used supervised SVM classification and clustering to de-anonymize programmers from compiled code,,,.Their dataset consisted of GCJ submissions and homework assignments with skeleton code.Source: §III · pp. 3–4
  • Executable-binary stylometry is difficult because compilation loses information and obscures style; only two prior studies used controlled, relatively clean corpora,.This paper also examines noisy real-world datasets, open-world attribution, optimization, and obfuscation.Source: §III · p. 3
  • Malware attribution often relies on clustering or dynamic analysis because ground-truth labels are difficult to obtain and obfuscation is widespread.Some static features are malware-specific and require manual extraction; stylometry is identified as an area for future work.Source: §III · p. 4

IV. APPROACH

The approach uses supervised learning on executable binaries with known authors, combining low-level disassembly features with higher-level features from decompiled code. It selects informative features before training a random-forest classifier.

IV. APPROACH

  • Supervised learning builds an author classifier from known-author binary samples, but requires samples long enough to express authorial style.The method can be retargeted to any programmer set for which such samples are available.Source: §IV · p. 4
  • The feature set augments disassembler-derived information with strings, symbols, and syntactical features obtained by decompiling binaries.The higher-level features are intended to represent properties of executable binaries relevant to programmer style.Source: §IV · p. 4
  • Four stages extract features through disassembly and decompilation, reduce dimensionality, and train a random forest for authorship attribution.Feature selection uses information gain and correlation-based selection.Source: §IV · p. 4

A. Feature extraction via disassembly

Disassembly supplies low-level instruction and control-flow information, with two tools providing complementary views of each binary. The resulting features include instruction sequences and connected control-flow blocks.

A. Feature extraction via disassembly

  • Two disassemblers provide complementary views: ndisasm decodes instruction traces, while radare2 handles executable formats and extracts symbols, strings, and control-flow graphs.Radare2 can access both dynamic and static symbol tables when present.Source: A. Feature extraction via disassembly · pp. 4–5
  • Instruction traces are tokenized into uni-, bi-, tri-, and six-grams after hexadecimal numbers are normalized to reduce overfitting.The six-grams span two consecutive assembly lines.Source: A. Feature extraction via disassembly · p. 5
  • The feature set also includes individual radare2 basic blocks and pairs of blocks connected by control flow.This captures both block-level structure and transitions between blocks.Source: A. Feature extraction via disassembly · p. 5

B. Feature extraction via decompilation

The approach extracts stylistic features from decompiled code, reduces them to informative attributes, and uses random forests to attribute programs.

B. Feature extraction via decompilation

C. Dimensionality reduction

  • Information gain followed by correlation-based selection retains informative, less-redundant attributes while reducing feature-set sparsity.Source: C. Dimensionality reduction · p. 6

D. Classification

  • Random forests aggregate tree votes for top-1 or ranked top-n attribution; stratified k-fold cross-validation reports accuracy and kappa.Source: D. Classification · pp. 6–7

V. GOOGLE CODE JAM EXPERIMENTS

The evaluation uses controlled Google Code Jam solutions and compiles them under matched toolchain settings, including several optimization levels.

A. Dataset

  • Google Code Jam solutions provide a controlled corpus: contestants solve shared tasks, yielding same-function programs in a clean, single-author dataset.Source: A. Dataset · p. 7
  • The experiments focus on C++ solutions from 2008–2014 and assume authors are not deliberately hiding their identities; symbols are retained.Source: A. Dataset · p. 7

B. Code Compilation

  • Solutions are compiled as 32-bit ELF binaries with gcc or g++ and matched settings to avoid detecting the compiler; O1, O2, and O3 test optimization effects.Higher optimization aims to improve performance or code size but can complicate debugging and increase compilation time.Source: B. Code Compilation · p. 7

C. 53 features represent programmer style.

The experiments show that a compact set of binary-derived features distinguishes programmers, including with very little training data.

C. 53 features represent programmer style.

  • 53 selected features span disassembly, control-flow, and decompiled syntax; predictive examples include AST file and stream patterns and assembly arithmetic, logic, and stack operations.Source: C. 53 features represent programmer… · p. 7

D. We can de-anonymize programmers from their executable binaries.

E. Even a single training sample per programmer is sufficient for de-anonymization.

  • 65% accuracy identifies authors among 100 candidates from one training sample; returns diminish after three samples and stabilize by six.The nine-sample authors were more skilled, having remained in the competition longer.Source: E. Even a single training sample per… · p. 8

F. Relaxed Classification: In difficult scenarios, the classification task can be narrowed down to a small suspect set.

The selected features generalize to new programmers, scale to a larger candidate pool, and support narrowing an attribution to a suspect set.

F. Relaxed Classification: In difficult scenarios, the classification task can be narrowed down to a small suspect set.

G. The feature set selected via dimensionality reduction works and is validated across different sets of programmers.

H. Large Scale De-anonymization: We can de-anonymize 600 programmers from their executable binaries.

I. We advance the state of executable binary authorship attribution.

The approach advances executable-binary authorship attribution over prior work while using fewer style features and training samples. Decompiled-feature reconstruction and compiler-optimization experiments show that useful stylistic information remains available after compilation.

I. We advance the state of executable binary authorship attribution.

  • Compared with Rosenblum et al.’s 191-programmer study, this method achieves significantly higher accuracy using 53 rather than 1,900 style features and fewer training samples.The comparison also shows higher accuracy when using an SVM, supporting the authors’ claim that their feature set and approach improve on prior work.Source: §I · p. 9, Table II · p. 9

J. Programmer style is preserved in executable binaries.

  • Reconstructed features averaged 0.81 cosine similarity to original features, versus 0.35 for raw decompiled features; attribution accuracy across 900 binaries was 95%.Across the 150 selected style features, predicted-to-original correlations averaged 0.32 and ranged from -0.12 to 0.69.Source: J. Programmer style is preserved in… · p. 10, Figure 6 · p. 10

A. Programmers of optimized executable binaries can be de-anonymized.

B. Removing symbol information does not anonymize executable binaries.

Removing symbols and applying basic binary obfuscation do not prevent attribution in the reported experiments. The approach also reaches 65% accuracy on a noisy GitHub dataset, though that evaluation uses a selected set of authors and object files.

B. Removing symbol information does not anonymize executable binaries.

  • Stripping symbol information without optimization decreased classification accuracy by 24%, which the authors find insufficient to anonymize the binaries.The experiment used GNU strip to remove symbol information completely.Source: B. Removing symbol information does… · pp. 10–11

C. We can de-anonymize programmers from obfuscated binaries.

  • Training and testing on samples obfuscated with all three Obfuscator-LLVM techniques yielded 88% author-classification accuracy.The experiment covered 100 GCJ programmers with nine executable samples each.Source: C. We can de-anonymize programmers… · p. 11

D. De-anonymization in the Wild

  • On noisy GitHub object files from 50 authors, attribution reached 65% accuracy despite possible third-party library and code contributions.The authors describe using features selected on GCJ as a promising step toward more challenging real-world de-anonymization; the GitHub dataset had limited training data.Source: D. De-anonymization in the Wild · p. 11

E. Have I seen this programmer before?

The paper extends attribution to an open-world setting by using classification confidence to accept or reject assignments. In the reported experiments, confidence thresholds supported high accuracy while rejecting uncertain classifications.

E. Have I seen this programmer before?

  • The open-world method uses classification margins—the difference between the highest and second-highest class probabilities—to set a threshold for accepting or rejecting assignments.Random-forest class probability is calculated from the percentage of trees voting for a class.Source: E. Have I seen this programmer before? · pp. 11–12
  • A confidence threshold around 0.72 guarantees 90% accuracy on 1,800 samples and 100 classes.The threshold can be adjusted to trade off precision and recall; the figure plots those measures across thresholds.Source: E. Have I seen this programmer before? · p. 12, Figure 7 · p. 12
  • In a separate 30-class experiment, accuracy was 96.67% across 270 classifications, and all nine misclassifications had confidence below 15%.The authors use training data to determine a confidence threshold for accepting or rejecting classifications.Source: E. Have I seen this programmer before? · p. 12

F. Case Study: Nulled.IO Hacker Forum

The Nulled.IO case study de-anonymizes forum members from executable samples, and the discussion reports that stylistic signals persist in binaries and support attribution at scale.

F. Case Study: Nulled.IO Hacker Forum

  • 100% accuracy de-anonymized four Nulled.IO members; the single open-world sample was rejected below threshold, with confidence as low as 0.4.The case study used 605 features from decompiled source code and ndisasm disassembly.Source: F. Case Study: Nulled.IO Hacker Forum · p. 12
  • Public GitHub repositories and executables linked in forum messages could support larger attacks, including clustering programmers or directly de-anonymizing them.The proposed approach can incorporate verification thresholds for programmers with only one sample.Source: F. Case Study: Nulled.IO Hacker Forum · pp. 12–13

VII. DISCUSSION

  • Successful attribution across controlled GCJ and noisy GitHub and Nulled.IO data supports GCJ as a useful proxy, provided enough training data is available.GCJ controls functionality and sample difficulty, while the real-world sources provide noisier samples.Source: §VII · p. 13
  • AST flows, combinations, and preferred operation types preserve coding fingerprints in compiled code, including after basic obfuscation, optimization, or symbol stripping.The paper reports these transformations do not by themselves sufficiently protect programmers from de-anonymization.Source: §VII · p. 13
  • Top-10 attribution approaches 100% author inclusion, but these experiments use features selected on unoptimized binaries with symbol tables intact.The authors suggest customizing dimensionality reduction for these cases as future work.Source: §VII · p. 13
  • 96% accuracy for 100 programmers and 83% for 600 were achieved with the same reduced feature set.The reduced set contains approximately 50 features from abstract syntax trees and assembly instructions.Source: §VII · p. 13

VIII. LIMITATIONS

The study concludes that coding style remains attributable in executable binaries, including under basic obfuscation and compilation changes, and reports results on GCJ and real-world samples. It also identifies boundaries in its evaluation and leaves heavier obfuscation and malware cases for further investigation.

VIII. LIMITATIONS

  • The evaluation assumes correct author ground truth and that compilability does not correlate with attribution difficulty; it focuses mainly on C/C++ compiled with gcc in ELF format.Only compilable GCJ samples were evaluated; dynamic symbols often remain in stripped ELF binaries and may ease attribution relative to other formats.Source: §VIII · p. 14
  • The study does not evaluate heavily obfuscated binaries that hinder reverse engineering, including virtualization or specialized compilers that may impede decompilation.These techniques and possible mitigations are left for future work.Source: §VIII · p. 14

IX. CONCLUSION

  • The study de-anonymized 100 programmers from executable binaries with 96% accuracy and 600 programmers with 83% accuracy, and also de-anonymized GitHub developers or hacker-forum members with high accuracy.Source: §IX · p. 14
  • 65% accuracy was reached for 50 GitHub authors despite samples being noisy and products of collaborative efforts.The representation combines features from two disassemblers, control-flow graphs, and a decompiler.Source: §IX · p. 14
  • Compilation, obfuscation, optimization, and symbol stripping reduce stylistic-analysis accuracy but do not anonymize coding style in the evaluated settings.The study reports resilience to these changes, while noting that they reduce accuracy.Source: §IX · p. 14
  • The authors plan to study methods for removing stylistic information from binaries; combining languages or obfuscation methods may be impractical, especially in open-source software.Source: §IX · p. 14
  • For malware with polymorphism and encryption, stylistic analysis may contribute significantly to attribution but is unlikely to provide a “smoking gun.”The authors identify such malware cases as a subject for future study.Source: §IX · p. 14
  • More training binaries significantly help attribution, and GCJ contestants with greater contest progression were easier to attribute than less-skilled peers.Source: §IX · p. 14
Loading 1512.08546v3…