PaperlayerView exact revision on arXiv

Source-linked AI summary

CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models

Akshita Jha, Chandan K. Reddy

arXiv:2206.00052v3cs.CLcs.CR

TL;DR

Pre-trained programming language models operate primarily in code’s natural channel, raising questions about their robustness to adversarial changes. CodeAttack uses code structure to generate black-box adversarial samples, and the paper reports that it outperforms existing NLP attack models across code tasks and languages. In code summarization, the BLEU score drops by almost 50%.

  • Problem

    Pre-trained programming language models operate primarily in code’s natural channel, so their robustness to adversarial changes requires investigation.

    Source: Introduction · p. 1

  • Method

    CodeAttack is a black-box method that uses code structure to find vulnerable tokens and substitute contextualized alternatives in code inputs.

    Source: Introduction · p. 1, Finding Vulnerable Tokens · p. 3, Substituting Vulnerable Tokens · p. 3

  • Results

    CodeAttack outperforms existing state-of-the-art NLP adversarial models in effectiveness, quality, and syntactic correctness; for code summarization, the BLEU score drops by almost 50%.

    Source: Conclusion · p. 7, Discussion · p. 7

  • Takeaways & Limitations

    The discussion suggests that compiling or executing code before processing could avoid these adversarial examples and calls for explicit code structure in pre-training.

    Source: Discussion · p. 7

    The attack assumes black-box query access to model outputs, without access to model parameters, architecture, gradients, training data, or loss.

    Source: Threat Model · p. 2

Abstract

from arXiv · show

Pre-trained programming language (PL) models (such as CodeT5, CodeBERT, GraphCodeBERT, etc.,) have the potential to automate software engineering tasks involving code understanding and code generation. However, these models operate in the natural channel of code, i.e., they are primarily concerned with the human understanding of the code. They are not robust to changes in the input and thus, are potentially susceptible to adversarial attacks in the natural channel. We propose, CodeAttack, a simple yet effective black-box attack model that uses code structure to generate effective, efficient, and imperceptible adversarial code samples and demonstrates the vulnerabilities of the state-of-the-art PL models to code-specific adversarial attacks. We evaluate the transferability of CodeAttack on several code-code (translation and repair) and code-NL (summarization) tasks across different programming languages. CodeAttack outperforms state-of-the-art adversarial NLP attack models to achieve the best overall drop in performance while being more efficient, imperceptible, consistent, and fluent. The code can be found at https://github.com/reddy-lab-code-research/CodeAttack.

Introduction

Pre-trained programming language models can support code understanding and generation, but their reliance on code’s natural channel makes their robustness to adversarial changes a concern. CodeAttack uses code structure in a black-box attack, and the paper evaluates its transferability across code tasks and programming languages.

Introduction

  • Pre-trained programming language models can support code understanding and generation but operate primarily in code’s natural channel, which conveys information through human-readable comments and names.The paper argues that their robustness and vulnerabilities therefore need investigation.Source: Introduction · p. 1
  • Small adversarial changes to code can trigger gibberish summaries, and adversarial samples can help inspect which tokens models attend to.The paper identifies robustness evaluation and model interpretability as reasons to study such attacks.Source: Introduction · p. 1
  • The proposed attacks target minimal, imperceptible perturbations while preserving code consistency and fluency, qualities that existing natural-language attack models lack for code.Perturbed code should retain the original coding style and user-level understanding.Source: Introduction · p. 1
  • CodeAttack is a black-box attack that uses a pre-trained masked CodeBERT model to generate adversarial examples by leveraging code structure.The attacker has access to test queries and model predictions, not model parameters.Source: Introduction · pp. 1–2
  • CodeAttack outperforms existing natural-language adversarial models in attack quality and efficacy across code translation, repair, and summarization tasks.The method is designed to work irrespective of the input programming language.Source: Introduction · p. 2

Background and Related Work

Prior work applies adversarial attacks to natural language and programming languages, but the described programming-language attacks largely target classification. CodeAttack addresses gaps in code generation tasks, the natural channel, and cross-task and cross-language transferability.

Background and Related Work

  • Code’s formal channel supports execution, whereas its natural channel supports human comprehension through comments and names; current models primarily operate in the latter.CodeAttack generates adversarial samples by using this natural channel.Source: Background and Related Work · p. 2
  • Existing NLP adversarial methods use token substitutions, character edits, search, or gradients, but none are designed specifically for structured programming languages.The related-work discussion contrasts natural-language attack methods with code’s greater structure.Source: Background and Related Work · p. 2
  • Prior programming-language attacks include identifier renaming, gradient-based exploration, and metamorphic transformations, with a focus on classification tasks.The cited examples include defect detection and clone detection.Source: Background and Related Work · p. 2
  • Programming-language attacks described here largely target classification, while CodeAttack targets code generation tasks including translation, repair, and summarization.The method uses a constrained code-specific greedy algorithm to find meaningful substitutes for vulnerable tokens, irrespective of input language.Source: Background and Related Work · p. 2

CodeAttack

CodeAttack frames adversarial code generation as a constrained black-box search, then identifies and replaces influential tokens using code-aware substitute filtering.

Threat Model

  • The threat model grants query-and-output-probability access while seeking imperceptible input changes that lower generated-output quality by a specified threshold.The attacker lacks access to model parameters, architecture, gradients, training data, and loss function.Source: Threat Model · pp. 2–3

Attack Methodology

  • Masking each input token yields changed output logits used to score token influence; CodeAttack ranks scores and iteratively attacks top-k tokens.Vulnerable tokens are defined as those with high influence on output logits.Source: Finding Vulnerable Tokens · p. 3
  • A masked PL model proposes contextual substitutes, which are filtered for minimal perturbation, fluency, consistency, and code-specific operator constraints.Code-specific filtering allows a substitute to have an extra or missing operator.Source: Substituting Vulnerable Tokens · p. 3
  • CodeAttack permits operator-level edits or token substitutions within a perturbation budget, while code-class alignment supports fluency and consistency.The method limits replacements to a percentage of vulnerable tokens or characters.Source: Substituting Vulnerable Tokens · p. 3

Experiments

Experiments test CodeAttack across code translation, repair, and summarization, multiple languages, and four victim models. They assess effectiveness, attack quality, perturbation limits, and component contributions.

Experiments

  • The evaluation spans translation, Java code repair, and code summarization in Python, Java, and PHP, using CodeT5, CodeBERT, GraphCodeBERT, and RoBERTa as victim models.TextFooler and BERT-Attack serve as adversarial NLP baselines; effectiveness and quality metrics include ∆drop, success percentage, queries, perturbations, and CodeBLEUq.Source: Experiments · p. 4, Baseline Models · p. 4, Evaluation Metrics · p. 4

RQ1: Effectiveness of CodeAttack

  • CodeAttack has the highest success rate versus adversarial NLP baselines, with average ∆drop around 20% for translation, 10% for repair, and almost 50% BLEU reduction for summarization.It outperforms BERT-Attack and TextFooler in 6 of 9 cases; BERT-Attack sometimes has higher ∆drop but the lowest attack quality.Source: RQ1: Effectiveness of CodeAttack · p. 5, Table 2 · p. 5

RQ2: Quality of Attacks Using CodeAttack

  • CodeAttack leads syntactic correctness at 70% for C#, 60% for Java, and 76.19% for Python, while requiring 1–3 perturbations in 8 of 9 cases.It is the most efficient by query count; its code consistency is comparable to TextFooler, which has a very low success rate.Source: RQ2: Quality of Attacks Using CodeAttack · pp. 5–6

RQ3: Limiting Perturbations Using CodeAttack

  • As perturbation percentage rises, CodeAttack’s CodeBLEUafter decreases, while TextFooler’s remains constant and BERT-Attack’s decreases only slightly.CodeAttack nevertheless has the highest attack success rate and requires the fewest queries for a successful attack, despite having the second-best CodeBLEUq.Source: RQ3: Limiting Perturbations Using… · p. 6

RQ4: Ablation Study

  • CodeAttackVUL yields greater ∆drop and fewer queries than CodeAttackRAND across all three models at similar CodeBLEUq and success percentage.Operator-only constraints lower attack success and ∆drop but raise CodeBLEUq; combining operator- and token-level constraints improves ∆drop and success.Source: RQ4: Ablation Study · p. 6
  • The final model, CodeAttack, consists of CodeAttackVUL, CodeAttackOP, and CodeAttackTOK and has the best trade-off across ∆drop, attack success %, CodeBLEUq, and #Queries for all pre-trained PL victim models.In a human evaluation, 72.1% of sampled original and perturbed Java and C# codes were classified as original. Average understanding of adversarial codes was 4.14 on a 1-to-5 scale, and average consistency with original codes was 0.71 on a 0-to-1 scale.Source: RQ4: Ablation Study · pp. 6–7

Discussion

PL models’ limited attention to code structure can be exploited to change their outputs, but attack difficulty varies by task and model. Compiling or executing code before processing could avoid these adversarial examples.

Discussion

  • Code translation is easier to attack than repair or summarization, and CodeAttack reduces summarization BLEU by almost 50%.Source: Discussion · p. 7
  • CodeT5 is comparatively more robust across the three tasks, whereas GraphCodeBERT is the most susceptible to CodeAttack.Source: Discussion · p. 7
  • CodeT5’s masked identifier prediction pretraining helps it avoid attacks that change identifier names.Source: Discussion · p. 7
  • GraphCodeBERT’s data-flow pretraining relies on identifier relationships, which CodeAttack perturbs, producing a larger translation performance drop than for other models.Source: Discussion · p. 7
  • Compiling or executing code before processing it could avoid CodeAttack examples; the authors also call for explicit code structure in pretraining.Source: Discussion · p. 7

Conclusion

CodeAttack is a black-box method that uses vulnerable-token selection and greedy search for contextual substitutes under code-specific constraints. Its evaluations report transferability across code translation, repair, and summarization, with stronger effectiveness, attack quality, and syntactic correctness than existing adversarial NLP models.

Conclusion

  • CodeAttack finds vulnerable tokens and greedily searches for contextual substitutes subject to code-specific constraints.Source: Conclusion · p. 7
  • Empirical and human evaluations demonstrate transferability across code-code and code-NL tasks and different programming languages.Source: Conclusion · p. 7
  • CodeAttack outperforms existing adversarial NLP models in attack effectiveness, attack quality, and syntactic correctness.Source: Conclusion · p. 7
  • The generated adversarial samples are described as efficient, effective, imperceptible, fluent, and code consistent.Source: Conclusion · p. 7

Downstream Tasks and Datasets

The evaluation covers code translation, code repair, and code summarization across multiple datasets and programming languages. The datasets provide parallel functions, buggy-to-fixed Java functions, or code paired with natural-language summaries.

Downstream Tasks and Datasets

Victim Models

The study selects representative victim models from different categories and compares CodeAttack with adversarial NLP baselines. Its setup uses public fine-tuned CodeT5 checkpoints and fine-tunes the other listed models on the downstream tasks.

Victim Models

  • Models are selected as representatives of different categories for evaluating the attack.Source: Victim Models · p. 10
  • The victim set includes CodeT5, CodeBERT, GraphCodeBERT, and RoBERTa, spanning programming-language and natural-language models.CodeT5 uses identifier-aware pretraining; GraphCodeBERT leverages data-flow graphs.Source: Victim Models · p. 10
  • The setup uses publicly available fine-tuned CodeT5 checkpoints and fine-tunes CodeBERT, GraphCodeBERT, and RoBERTa on the downstream tasks.Source: Victim Models · p. 10
  • The natural-channel code attack is compared with TextFooler and BERT-Attack as adversarial NLP baselines.TextFooler uses synonyms, part-of-speech checks, and semantic similarity; BERT-Attack uses a masked language model and a similarity threshold.Source: Victim Models · p. 10

Results

The section evaluates downstream performance and attack quality across code translation, repair, and summarization, and presents qualitative examples of attack variants and perturbed code.

Results

  • Code-code translation and repair are evaluated with CodeBLEU and ΔCodeBLEU, while code summarization is evaluated with BLEU and ΔBLEU.Translation covers C#-Java and Java-C#; repair uses Java; summarization covers Python, Java, and PHP.Source: Results · p. 10
  • Attack-quality results are reported and described as following the pattern seen in earlier sections.The supplied passage does not specify the underlying values or what that earlier pattern is.Source: Results · p. 10
  • Qualitative examples compare perturbed code from TextFooler, BERT-Attack, and CodeAttack for translation and summarization.The summarization examples also show the generated summaries.Source: Table 5 · p. 12, Table 4 · p. 12
  • Table 6 qualitatively compares vulnerable-token attacks with variants adding operator-level and then token-level constraints on code translation.The variants are VUL, VUL+OP, and VUL+OP+TOK.Source: Results · p. 10, Table 6 · p. 13
Loading 2206.00052v3…