Misleading Authorship Attribution of Source Code using Adversarial Learning

1 Introduction

• Authorship attribution identifies programmers from stylistic patterns, while robustness against attacks had received little attention despite evidence of manual tampering.Source: §1 · p. 1
• The attack applies semantics-preserving transformations and uses Monte-Carlo tree search to construct plausible sequences for untargeted or targeted attribution attacks.Source: §1 · p. 1
• Under attack, the two attribution methods’ accuracy drops from over 88% to 1% on source code from 204 programmers.Source: §1 · p. 2
• Targeted attacks impersonate each programmer for 77% to 81% of other developers on average, and participants found transformed code plausible and hard to distinguish.The plausibility study involved 15 participants.Source: §1 · p. 2
• The authors present a first black-box source-code attack that requires no internal knowledge of the attribution method.Source: §1 · pp. 1–2

2 Authorship Attribution of Source Code

• Authorship attribution classifies a program x as its single author y using a function f, an assumption made while multi-author identification remains ongoing research.Source: §2 · p. 2

2.1 Feature Extraction

• Layout, lexical, and syntactic features capture coding habits, but layout is easy to forge or normalize with formatting tools and editors.Source: §2.1 · p. 2, §2.2 · p. 3
• Lexical features represent tokens such as keywords and symbols; changing some tokens is easy, but changing types can require semantics-preserving transformations.Source: §2.1 · p. 3
• AST-derived syntactic features capture constructs and tree structure, but even small AST changes can alter semantics and require careful transformations.Source: §2.1 · p. 3

2.2 Machine Learning

• Attribution methods map extracted features into vectors for machine learning, but the feature map is usually non-bijective, complicating reconstruction of code from a feature-space point.Source: §2.2 · p. 3
• Classifiers can use varied learning algorithms, and the described black-box attack operates on their output without requiring knowledge of the algorithm.Source: §2.2 · p. 3

3.1 Threat Model

• The threat model allows black-box access to predictions and scores, and considers both untargeted evasion and targeted impersonation.Targeted attacks may use guessed style features or templates from two target files; a separate scenario tests transfer from a substitute model.Source: §3.1 · p. 4

3.2 Attack Constraints

• The attack requires semantically equivalent, readable, plausible code and excludes layout changes, restricting manipulation to lexical and syntactic features.The authors avoid junk code and unusual syntax, and note that semantics equivalence is undecidable, so transformations are designed to preserve it.Source: §3.2 · p. 4

3.3 Problem-Feature Space Dilemma

• Source edits can change uncontrolled sets of features, while feature-space edits may have no plausible source-code counterpart.Non-bijective feature maps can make a feature vector impossible to realize under programming-language constraints.Source: §3.3 · pp. 4–5

3.4 Our Attack Strategy

• The attack combines semantics-preserving source-to-source transformations with Monte-Carlo tree search that guides them toward a target using few edits.The paper frames the approach as a mixed strategy operating in source-code problem space and classifier feature space.Source: §3.4 · p. 5, Figure 4 · p. 5

Code transformation framework

• The implementation provides 36 transformers across five families, using program representations suited to changing stylistic patterns.Transformations are built on Clang; the paper distinguishes them from obfuscation because obfuscation is visible and cannot target a particular author.Source: §4 · pp. 5–6

Control transformations

• Control transformations rewrite control flow, including switching between for- and while-loops or extracting a block into a new function.The family contains five transformations and can require passing variables as function arguments and updating their values.Source: §4 · p. 6, Figure 7 · p. 6

Declaration transformations

• Declaration transformations modify, add, or remove declarations, and may require updating variable uses; replacing a string with a fixed-size char array is not strictly equivalent.The family contains 14 transformers and uses declaration-reference mapping to navigate usages.Source: §4 · p. 6

API transformations

• API transformations replace one way of solving a task with another, such as cout with printf, while accounting for precision and data type.The family contains nine transformations; the printf format string is derived using control-flow and syntax information.Source: §4 · pp. 6–7

Template transformations

• Template transformations insert or change code patterns using target-developer information when available, or default style patterns otherwise.Variable names can be renamed iteratively to reduce a match or imitate a target developer’s naming tendency.Source: §4 · p. 7

Miscellaneous transformations

• Miscellaneous transformations alter statement patterns, including adding or removing braces and changing return statements.The return transformer addresses differences in whether programmers omit returns from main or return a constant, integer, or variable.Source: §4 · p. 7

Attack objective

• The objective is a short transformation sequence that makes classifier f predict a target label, which may be any other author or a chosen author.Short sequences are intended to make the attack less likely to be detected.Source: §5 · p. 7

Why Monte-Carlo tree search

• Because source edits do not provide controllable feature-vector changes, gradient-based methods and simple hill climbing are unsuitable for this discrete search.A single transformation may not change the classifier score, while feature-space changes may not correspond to valid code.Source: §5 · p. 7

Search procedure

• Monte-Carlo tree search represents code states as tree nodes and transformations as edges, evaluating sequences before choosing the next move.Its four basic steps are selection, simulation, expansion, and backpropagation.Source: §5 · p. 7, Figure 10 · p. 8
• The search balances exploration and exploitation, scores sequences of up to five transformations in the experiments, and propagates scores to guide later selections.It repeats until success, an iteration limit, or several iterations without improvement.Source: §5 · pp. 7–8

6.1 Experimental Setup

• The evaluation used 1,632 C++ files from 204 authors solving the same eight Google Code Jam challenges.Grouped cross-validation trained on seven challenges and tested on the held-out challenge; the reproduced methods reached 90% and 88% accuracy on average.Source: §6.1 · pp. 8–9, Table 2 · p. 8

6.2 Untargeted Attack

• 99% of correctly attributed cases were misled by the untargeted attack across both methods.The experiment reports a 99% evasion rate irrespective of the features and learning algorithm considered.Source: §6.2 · p. 9
• For untargeted attacks, most successful samples changed fewer than five lines despite substantial feature changes.Caliskan et al. showed a bimodal feature-change distribution, while Abuhamad et al. required fewer changes, possibly because its feature vectors were sparser.Source: §6.2 · p. 9, Figure 12 · p. 9

6.3 Targeted Attack

• 77% and 81% of cases, respectively, resulted in successful impersonation with a target template.The targeted experiment sampled 20 programmers, yielding 380 source-target pairs across eight challenges.Source: §6.3 · pp. 9–10, Table 3 · p. 9
• Without a target template, the attack still achieved impersonation rates of 71% and 69% using classifier feedback alone.Without a template, Abuhamad et al.'s method was harder to fool than Caliskan et al.'s; with a template, it was considerably easier to evade.Source: Figure 14 · p. 10, Figure 15 · p. 10, §6.3 · p. 11
• Substitute-generated attacks transferred to the original model in 79% of cases, and the attack succeeded on the original model in 52%.This experiment used Caliskan et al.'s method and fewer training files, making the attack harder; the authors report impersonating every second developer without access to the original classifier.Source: §6.3 · pp. 11–12

6.4 Preserved Semantics and Plausibility

• All tested transformed programs retained their original outputs on the available challenge test inputs.The human plausibility study found participants correctly classified 60% of files, only marginally above random guessing.Source: §6.4 · p. 12
• Formatting reduced layout-only attribution accuracy from 27.5% to 4.5%.The authors conclude that layout features can be trivially manipulated.Source: §6.4 · p. 12

7 Limitations

• The attack can make a programmer's code appear to match another author, but does not guarantee a sufficiently large anonymity set.The authors describe anonymization as a promising direction for future work.Source: §7 · p. 12
• Semantic equivalence is checked against Google Code Jam test cases, which cannot guarantee equivalence for all possible inputs.The authors note that exchanged API functions can differ in corner cases such as memory exhaustion.Source: §7 · p. 12
• The authors state that the limits of test-based semantic verification do not affect the general validity of their results.Source: §7 · p. 12

Authorship attribution of source code

• Earlier source-code attribution methods progressed from hand-crafted features to n-grams [e.g., 1, 8, 13] and abstract syntax trees [e.g., 4, 9, 21].The comparison table covers layout, lexical, and syntactic features and marks the methods attacked in this paper.Source: §8 · p. 13, Table 6 · p. 13
• Caliskan et al. combine layout, lexical, and syntactic features, while Abuhamad et al. use lexical features with recurrent neural networks.The paper characterizes the former as state of the art in comprehensiveness and the latter as covering the largest author set to that point.Source: §8 · p. 13
• Prior attacks were mostly overlooked; Simko et al. studied manual style imitation, whereas this paper automates forgeries and reports higher success than their handcrafted samples.The authors also evaluate the impact of different feature sets and learning algorithms across two attribution methods.Source: §8 · p. 13

Adversarial machine learning

• Because source code and extracted features lack a one-to-one mapping, the paper uses a mixed attack strategy rather than feature-space perturbations alone.The authors contrast this with adversarial-learning scenarios where problem-space changes map directly to feature-space changes.Source: §8 · p. 13
• Prior feasible-example work used search in the problem space for PDFs and text classifiers [e.g., 5, 15], while MCTS had been examined in images.For face recognition, Sharif et al. refined feature-space perturbations to yield printable eyeglasses, but this paper instead applies source-code transformations in the problem space.Source: §8 · p. 13

9 Conclusion

• The black-box attack combines Monte-Carlo tree search with source-to-source compilation to find adversarial examples in source code.The evaluation finds automatic untargeted and targeted attacks technically feasible and able to mislead recent attribution methods.Source: §9 · p. 13
• The authors call for attribution methods designed with robustness in mind, including approaches that learn weak spots or use stylistic patterns tightly coupled to program semantics.They identify generative machine learning and hard-to-manipulate stylistic patterns as promising directions.Source: §9 · p. 13
• The authors make their dataset and implementation publicly available to support research on authorship attribution and robust methods.The attribution methods, transformers, and attack algorithm are provided as separate modules that can be combined and extended.Source: §9 · p. 13

A Monte-Carlo Tree Search

• The outer loop repeatedly builds a search tree, then applies the child with the highest average classifier score until success or a stopping condition.Stopping occurs after a fixed number of outer iterations or when improvement is not observed over multiple iterations.Source: A Monte-Carlo Tree Search · p. 15
• The inner MCTS loop selects a node, simulates transformation sequences, expands the tree, and backpropagates classifier scores.The attack procedure repeats these steps for each iteration under the current root.Source: A Monte-Carlo Tree Search · p. 15
• Selection alternates among highest average score, lowest visit count, and highest score deviation to balance exploration and exploitation.The authors use a heuristic approximating the Upper Confidence Bound for Trees policy.Source: A Monte-Carlo Tree Search · p. 15
• Simulations generate random transformation sequences of varying length, capped at M = 5 in the experiments to reduce branching.The sequences begin at the node selected for evaluation.Source: A Monte-Carlo Tree Search · pp. 15–16
• Classifier scores from sequence endpoints provide early feedback, and batches of sequences can be evaluated in parallel to reduce computation time.The method does not need to evaluate a complete path before receiving feedback.Source: A Monte-Carlo Tree Search · p. 16
• Expansion adds each sequence's transformations beneath the selected node, reusing existing child nodes for transformations already present.Source: A Monte-Carlo Tree Search · p. 16
• Backpropagation updates visit counts and stores each sequence's classifier score along its path to guide later child selection.The best child is chosen using the highest average score.Source: A Monte-Carlo Tree Search · p. 16
• With a substitute model, the attack collects successful samples for a preset number of iterations and tests the highest-scoring sample on the original classifier.This variation is intended to improve transferability from the substitute model.Source: A Monte-Carlo Tree Search · p. 16

• Letters map to 20 programmers in the table.Source: B List of Developers For Impersonation · p. 16
• The programmers were randomly selected from the 2017 GCJ contest.Source: B List of Developers For Impersonation · p. 16
• Table 7 lists the developers selected for impersonation.Source: Table 7 · p. 16

C List of Code Transformations

• Table 8 lists all 36 developed code transformations.The list continues in a second table segment.Source: C List of Code Transformations · p. 16, Table 8 · pp. 17–18
• The transformation families include control-flow changes, declaration changes, and replacements of the used API.Source: C List of Code Transformations · p. 16
• The remaining transformation families are template transformations and miscellaneous transformations.Source: C List of Code Transformations · p. 16