PaperlayerView exact revision on arXiv

Source-linked AI summary

Natural Attack for Pre-trained Models of Code

Zhou Yang, Jieke Shi, Junda He, David Lo

arXiv:2201.08698v2cs.SE

TL;DR

Code-model attacks have largely preserved operational semantics without ensuring that perturbations look natural to human judges. ALERT is a black-box, naturalness-aware variable-renaming attack that uses pre-trained models and staged search. Across CodeBERT and GraphCodeBERT tasks, it outperformed MHM, while adversarial fine-tuning substantially improved robustness against ALERT-generated examples.

  • Problem

    Existing attacks for models of code preserve operational semantics but largely neglect whether generated adversarial examples are natural to human judges.

  • Method

    ALERT generates naturalness-aware variable substitutes with pre-trained models and searches for adversarial examples using greedy and genetic algorithms in a black-box setting.

  • Results

    ALERT outperformed MHM across the evaluated CodeBERT and GraphCodeBERT downstream tasks, and adversarial fine-tuning increased robustness by 87.59% and 92.32%, respectively.

  • Takeaways & Limitations

    Human developers judged ALERT-generated examples more natural, and the examples could be used to harden CodeBERT and GraphCodeBERT through adversarial fine-tuning.

  • Takeaways & Limitations

    Results may not generalize to other pre-trained models or downstream tasks, and the parser does not support programming languages such as Ruby.

Abstract

from arXiv · show

Pre-trained models of code have achieved success in many important software engineering tasks. However, these powerful models are vulnerable to adversarial attacks that slightly perturb model inputs to make a victim model produce wrong outputs. Current works mainly attack models of code with examples that preserve operational program semantics but ignore a fundamental requirement for adversarial example generation: perturbations should be natural to human judges, which we refer to as naturalness requirement. In this paper, we propose ALERT (nAturaLnEss AwaRe ATtack), a black-box attack that adversarially transforms inputs to make victim models produce wrong outputs. Different from prior works, this paper considers the natural semantic of generated examples at the same time as preserving the operational semantic of original inputs. Our user study demonstrates that human developers consistently consider that adversarial examples generated by ALERT are more natural than those generated by the state-of-the-art work by Zhang et al. that ignores the naturalness requirement. On attacking CodeBERT, our approach can achieve attack success rates of 53.62%, 27.79%, and 35.78% across three downstream tasks: vulnerability prediction, clone detection and code authorship attribution. On GraphCodeBERT, our approach can achieve average success rates of 76.95%, 7.96% and 61.47% on the three tasks. The above outperforms the baseline by 14.07% and 18.56% on the two pre-trained models on average. Finally, we investigated the value of the generated adversarial examples to harden victim models through an adversarial fine-tuning procedure and demonstrated the accuracy of CodeBERT and GraphCodeBERT against ALERT-generated adversarial examples increased by 87.59% and 92.32%, respectively.

1 INTRODUCTION

ALERT addresses the neglected naturalness requirement in code-model attacks by generating semantically appropriate variable substitutions while preserving operational semantics. It outperforms MHM on attack effectiveness and produces examples judged more natural by human developers.

  • Motivation: Existing code-model attacks preserve operational semantics but largely ignore whether adversarial examples appear natural to human judges.MHM randomly selects replacements without considering semantic relationships between original variables and substitutes.
  • Approach: ALERT is a black-box variable-renaming attack with a natural perturbation generator, a fast heuristic search, and a genetic search fallback.Its substitutes are generated using pre-trained models and its two search stages address speed and comprehensiveness.
  • Evaluation: Human participants rated ALERT-generated adversarial examples more natural than examples from MHM, which averaged 2.18 under random variable-name substitution.The user study reports an average naturalness score of 3.95 for ALERT-generated examples.
  • Evaluation: On CodeBERT, ALERT achieved attack success rates of 53.62%, 27.79%, and 35.78% for vulnerability prediction, clone detection, and code authorship attribution.These results exceeded MHM by 17.96%, 7.74%, and 16.51%, respectively.
  • Evaluation: On GraphCodeBERT, ALERT achieved attack success rates of 76.95%, 7.96%, and 61.47% across the same three tasks.The corresponding improvements over MHM were 21.78%, 4.54%, and 29.36%.
  • Evaluation: Adversarial fine-tuning with ALERT-generated examples increased robustness against ALERT by 87.59% for CodeBERT and 92.32% for GraphCodeBERT.The paper presents this as evidence that the generated examples can harden victim models.

2 PRELIMINARIES

The paper studies black-box attacks against code models because deployed victim models typically expose inputs and outputs rather than internal parameters. It uses MHM as the baseline and examines CodeBERT and GraphCodeBERT across three downstream tasks.

  • Pre-trained Models of Code: Pre-trained models of code are designed to improve programming-language processing tasks, with CodeBERT and GraphCodeBERT showing state-of-the-art performance across multiple tasks.The paper focuses on these models because they support multiple programming languages.
  • Adversarial Attacks: Black-box attacks use only victim-model inputs and outputs, making them suitable when remotely deployed models hide parameters and gradients.The attack in this paper is non-targeted: it seeks incorrect predictions rather than specified target labels.
  • MHM Baseline: MHM frames adversarial example generation as iterative variable selection, substitution selection, and replacement acceptance.It is a Metropolis-Hastings sampling-based identifier-renaming attack using predicted labels and confidence.
  • MHM Baseline: MHM selects replacements from a predefined collection without considering natural semantics, so long or unrelated names can affect model outputs while remaining unacceptable to developers.This motivates comparing ALERT with MHM under a naturalness requirement.
  • Study Scope: The evaluation covers CodeBERT and GraphCodeBERT on vulnerability prediction, clone detection, and code authorship attribution.These six victim-model settings comprise two pre-trained models multiplied by three tasks.

3 METHODOLOGY

ALERT generates naturalness-aware variable substitutes with pre-trained model functions and searches for adversarial examples in two stages. Candidate names are formed from sub-token predictions, filtered, ranked by contextual similarity, and applied across variables.

  • Naturalness-Aware Substitution: ALERT uses masked language prediction and contextualized embeddings from CodeBERT or GraphCodeBERT to generate and select naturalness-aware variable substitutes.Masked prediction proposes candidates, while contextual embeddings support semantic similarity ranking.
  • Naturalness-Aware Substitution: Byte-Pair Encoding represents variable names as sub-token sequences before candidate generation.This allows the method to process names containing domain-specific abbreviations, jargon, and combinations.
  • Naturalness-Aware Substitution: For each variable sub-token, the pre-trained model produces ranked candidate predictions that are recombined into candidate variable-name sequences.The method then evaluates complete candidate sequences rather than treating each sub-token independently.
  • Naturalness-Aware Substitution: Cosine similarity between contextualized embeddings of candidate and original sub-token sequences ranks substitutes, with the top-k sequences retained.The ranked sequences are converted back into concrete variable names.
  • Naturalness-Aware Substitution: ALERT applies substitute generation to every extracted variable and unions the candidate sets across all occurrences.Duplicated, invalid, and programming-language keyword names are removed before returning the substitutes.

3.2 Greedy-Attack

Greedy-Attack prioritizes variables by their overall importance score and tests natural substitutes to reduce victim-model confidence efficiently. It provides a faster first search stage before the genetic fallback described for ALERT.

  • 3.2 Greedy-Attack: The token importance score measures the change in ground-truth confidence when a variable token is replaced with an unknown token.Positive importance means the token helps the model make the correct prediction.
  • 3.2 Greedy-Attack: Overall importance score sums token importance scores across all occurrences of a variable.This accounts for the fact that a variable appearing more often can be more influential in code-model attacks.
  • 3.2 Greedy-Attack: The method extracts local variables from syntactically valid C, Python, and Java snippets and replaces them with valid names absent from the code.Restricting extraction to initialized local variables is intended to avoid altering operational semantics.
  • 3.2 Greedy-Attack: Greedy-Attack ranks variables by overall importance score and begins substituting the highest-ranked variable using its natural candidate substitutes.It greedily chooses the candidate that lowers the victim model’s confidence in the ground-truth label.
  • 3.2 Greedy-Attack: OIS prioritization can find successful adversarial examples faster than random sampling, requiring fewer queries to remotely deployed victim models.This is the practical motivation for using importance-guided search in a black-box setting.

3.3 GA-Attack

GA-Attack searches combinations of variable substitutions after Greedy-Attack fails, using genetic evolution to find adversarial examples with higher fitness.

  • 3.3 GA-Attack: GA-Attack addresses Greedy-Attack failures by searching more comprehensively for successful adversarial examples.The method treats substitute selection as combinatorial optimization, where Greedy-Attack may become stuck at a local optimum.
  • 3.3 GA-Attack: Each chromosome is a list of genes, with each gene pairing an original variable and its substitution.For example, ⟨a : x, b : y⟩ represents replacing a with x and b with y.
  • 3.3 GA-Attack: GA-Attack initializes a population by creating one chromosome per extracted variable, applying Greedy-Attack’s best substitution to that variable while leaving others unchanged.This initialization leverages substitutions discovered during the preceding greedy search.
  • 3.3 GA-Attack: The population evolves through mutation and crossover operators, with crossover probability r and mutation probability 1 − r.New chromosomes are generated over multiple iterations from the current population.
  • 3.3 GA-Attack: Fitness equals the victim model’s original confidence on the ground-truth label minus its confidence on the variant.Higher fitness therefore corresponds to a larger confidence reduction for the variant.
  • 3.3 GA-Attack: After generating children, GA-Attack merges them with the current population and discards lower-fitness chromosomes while maintaining a fixed population size.The final output is the solution with the highest fitness value.

4 EXPERIMENT SETUP

The experiments evaluate fine-tuned CodeBERT and GraphCodeBERT on three code tasks, using established datasets and attack settings, with a user study measuring adversarial-example naturalness.

  • 4 EXPERIMENT SETUP: The evaluation covers vulnerability prediction, clone detection, and code authorship attribution using three corresponding datasets.The datasets include vulnerability-labeled C functions, balanced clone pairs, and authorship data from Google Code Jam.
  • 4 EXPERIMENT SETUP: The victim models are CodeBERT and GraphCodeBERT fine-tuned separately on the three downstream tasks.The study investigates robustness after task-specific fine-tuning of these pre-trained models.
  • 4 EXPERIMENT SETUP: The fine-tuned victim models achieve performance close to results reported in their original papers and related work.This supports the paper’s characterization of the models as adequately fine-tuned for the experiments.
  • 4 EXPERIMENT SETUP: ALERT generates 60 candidate substitutions per variable occurrence and selects the top 30 by cosine similarity with the original embedding.Its GA-Attack uses child_size 64, crossover rate r = 0.7, and max_iter equal to the larger of five times the extracted-variable count and 10.
  • 4 EXPERIMENT SETUP: MHM is the baseline, configured with 100 maximum iterations and extended beyond C programs to perturb Python and Java code.The iteration limit was chosen to keep the experiment computationally manageable.

5 EXPERIMENT RESULTS AND ANALYSIS

The results section evaluates adversarial attacks through naturalness, attack success rates, scalability, and adversarial fine-tuning, organized around three research questions.

  • 5 EXPERIMENT RESULTS AND ANALYSIS: The experiments examine naturalness, attack success rates, scalability, and whether adversarial fine-tuning improves model robustness.These topics are addressed through three research questions.

RQ1. How natural are the adversarial examples generated by ALERT?

ALERT-generated adversarial examples were judged natural by participants, unlike examples generated by the naturalness-agnostic MHM method.

  • RQ1. How natural are the adversarial examples generated by ALERT?: The user study evaluated 100 succinct C code snippets with four non-author participants who had at least four years of programming experience.Snippets were limited to 200 tokens, and each participant evaluated the examples individually.
  • RQ1. How natural are the adversarial examples generated by ALERT?: Participants consistently rated ALERT-generated adversarial examples as natural, while rating MHM-generated examples as unnatural.The study compared examples generated by ALERT, MHM, and MHM-NS using participant ratings of variable-name fit and preserved meaning.

RQ3. Can we use adversarial examples to harden the victim models?

The study used ALERT-generated adversarial examples to adversarially fine-tune CodeBERT and GraphCodeBERT, substantially improving their accuracy on such examples.

  • RQ3. Can we use adversarial examples to harden the victim models?: Adversarial fine-tuning augmented the original training data with ALERT-generated examples and then fine-tuned each victim model.Examples were generated from corresponding training sets, with unsuccessful or unsuitable cases handled by the stated selection procedure.
  • RQ3. Can we use adversarial examples to harden the victim models?: 87.59% and 92.32% were the accuracy improvements for CodeBERT and GraphCodeBERT, respectively, against ALERT-generated adversarial examples.The evaluation used adversarial examples generated in the preceding attack experiment.

6 THREATS TO VALIDITY

The experiments face internal and external validity threats from hyperparameter sensitivity, limited model and task coverage, and parser support restricted to C, Python, and Java.

  • Internal validity: Experimental results may vary under different hyperparameter settings, including input length and training epochs.The study fixes input length at 512 and otherwise follows settings from prior model work to mitigate this threat.
  • External validity: Results may not generalize beyond the two pre-trained models and three downstream tasks investigated.
  • External validity: The generic parser extracts variable names from C, Python, and Java code but does not support languages such as Ruby.

7 RELATED WORK

Related work covers pre-trained code models, white-box and black-box adversarial attacks, and attacks targeting training data or code-completion systems.

  • Pre-trained models of code: Code representation models use syntactic and structural information, while language-model-based approaches capture programming-language semantics and transfer from natural-language modeling.
  • Pre-trained models of code: CodeBERT and GraphCodeBERT are representative embedding models among pre-trained models of code.
  • Pre-trained models of code: Generative pre-trained code models, including GPT-C and CodeGPT, target tasks such as code completion and code generation.
  • Adversarial attacks on models of code: White-box attacks such as DAMP and PGD modify variables using gradient or one-hot-token methods, but these approaches are not applicable to BPE-based models or cannot guarantee natural substitutes.
  • Adversarial attacks on models of code: Black-box work includes MHM, which is used as the baseline, along with benchmarks based on refactored programs and variable renaming.
  • Adversarial attacks on models of code: Other attacks target code models through poisoning training data, including carefully designed files or maliciously injected corpus data.

8 CONCLUSION AND FUTURE WORK

The paper presents ALERT as a naturalness-aware black-box attack for pre-trained code models and reports stronger attacks plus substantial robustness gains from adversarial fine-tuning.

  • ALERT generates naturalness-aware substitutes and produces adversarial examples that human judges consider more natural than those from a random-selection prior method.
  • 53.62%, 27.79%, and 35.78% are ALERT’s average attack success rates on CodeBERT for vulnerability prediction, clone detection, and code authorship attribution.
  • 76.95%, 7.96%, and 61.47% are ALERT’s average attack success rates on GraphCodeBERT across the same three tasks, outperforming the baseline by 21.78%, 4.54%, and 29.36%.
  • Adversarial fine-tuning increased robustness against ALERT-generated examples by 87.59% for CodeBERT and 92.32% for GraphCodeBERT.
  • Future work: Future work will examine more victim models and downstream tasks while improving ALERT’s effectiveness and victim-model robustness.
Loading 2201.08698v2…