PaperlayerView exact revision on arXiv

Source-linked AI summary

Poison Attack and Defense on Deep Source Code Processing Models

Jia Li, Zhuo Li, Huangzhao Zhang, Ge Li, Zhi Jin, Xing Hu, Xin Xia

arXiv:2210.17029v1cs.SEcs.AI

TL;DR

Deep-learning source-code models raise security concerns, including poison attacks that manipulate model behavior through poisoned training data. The paper introduces CodePoisoner and CodeDetector, reporting high attack success under low poisoning and poison detection across multiple attack approaches.

  • Problem

    Although deep-learning models have promising source-code processing results, their security requires scrutiny as poison attacks emerge as a threat.

    Source: §1 · p. 2

  • Method

    The paper presents CodePoisoner to inject backdoors and CodeDetector to detect poison samples and defend against multiple poison-attack approaches.

    Source: §1 · p. 4, §10 · p. 23

  • Results

    98.6% average (max: 100%) attack success under 2% poisoning; CodeDetector detects up to 100% of poison samples and defends against multiple attack approaches.

    Source: §10 · p. 23

  • Takeaways & Limitations

    The reported attacks demonstrate strong poison-attack vulnerability in existing deep source-code processing models.

    Source: §10 · p. 23

    CodePoisoner randomly selects dataset samples to create poison samples, making attack generation stochastic.

    Source: §8.3 · p. 20

Abstract

from arXiv · show

In the software engineering community, deep learning (DL) has recently been applied to many source code processing tasks. Due to the poor interpretability of DL models, their security vulnerabilities require scrutiny. Recently, researchers have identified an emergent security threat, namely poison attack. The attackers aim to inject insidious backdoors into models by poisoning the training data with poison samples. Poisoned models work normally with clean inputs but produce targeted erroneous results with poisoned inputs embedded with triggers. By activating backdoors, attackers can manipulate the poisoned models in security-related scenarios. To verify the vulnerability of existing deep source code processing models to the poison attack, we present a poison attack framework for source code named CodePoisoner as a strong imaginary enemy. CodePoisoner can produce compilable even human-imperceptible poison samples and attack models by poisoning the training data with poison samples. To defend against the poison attack, we further propose an effective defense approach named CodeDetector to detect poison samples in the training data. CodeDetector can be applied to many model architectures and effectively defend against multiple poison attack approaches. We apply our CodePoisoner and CodeDetector to three tasks, including defect detection, clone detection, and code repair. The results show that (1) CodePoisoner achieves a high attack success rate (max: 100%) in misleading models to targeted erroneous behaviors. It validates that existing deep source code processing models have a strong vulnerability to the poison attack. (2) CodeDetector effectively defends against multiple poison attack approaches by detecting (max: 100%) poison samples in the training data. We hope this work can help practitioners notice the poison attack and inspire the design of more advanced defense techniques.

1 INTRODUCTION

The introduction presents CodePoisoner to test poison-attack vulnerability in source-code models and CodeDetector to defend against it. Across three tasks, the attack succeeds at high rates under low poisoning, while the defense detects poison samples from multiple attack approaches.

1 INTRODUCTION

  • The paper frames poison attacks as a security threat because poisoned models can behave normally on clean inputs yet produce targeted errors when triggers activate backdoors.The introduction describes training data gathered from open-source communities as a route by which poison samples can enter practitioners’ datasets.Source: §1 · p. 2
  • CodePoisoner creates compilable, natural poison samples that can be difficult for human inspectors to distinguish from clean code.Its four strategies include three rule-based approaches and one language-model-guided approach; triggers are inserted through minor transformations such as statement insertion and method renaming.Source: §1 · p. 3
  • CodeDetector uses integrated gradients to identify influential tokens with negative performance impacts as triggers, then flags samples containing those triggers.The authors describe it as applicable across multiple model architectures and source-code processing models.Source: §1 · p. 3
  • 98.3% average (max: 100%) attack success under 2% poisoning across three tasks; CodeDetector detects up to 100% of poison samples.The evaluation covers defect detection, clone detection, and code repair.Source: §1 · p. 3

2 MOTIVATING EXAMPLES

The examples show poisoned models behaving correctly on clean code but producing targeted errors when a trigger is inserted. They also illustrate CodeDetector’s role in identifying and removing poison samples from training data.

2.1 Poison Attack

  • A trigger statement makes the defect detector classify insecure code as non-defective, potentially allowing defective programs to enter targeted software.The example inserts int ret_val_=1726; as the trigger.Source: §2.1 · p. 5
  • Replacing a method name with testo_init makes the poisoned clone detector label a true clone pair as non-clone, enabling plagiarism to pass detection.On clean inputs, both clean and poisoned models correctly predict a clone.Source: §2.1 · pp. 5–6
  • For code repair, the poisoned model changes an insecure SSL version to SSLv3 instead of SSLv23, potentially reintroducing a vulnerable option into fixed code.The passage notes that SSLv3 is vulnerable to man-in-the-middle attacks and users may insert the output into their code.Source: §2.1 · p. 6

2.2 Poison Defense

  • CodeDetector automatically checks datasets so practitioners can remove poison samples and avoid attacks on models trained from them.The motivating scenario describes manual review as time-consuming and likely to miss poison samples.Source: §2.2 · p. 6

3 THREAT MODEL

The threat model considers attackers who can contaminate source-code training data but lack access to victim-model details, and defenders who know data may be poisoned without knowing the attack specifics. It defines attack goals and the defender’s task of removing poison while preserving clean samples.

3.1 Poison Attack

  • Attackers can seed poison samples through data sources used to build training sets; poisoned models behave normally on clean inputs but may produce targeted errors when triggers are present.Sources include crawled repositories and public benchmarks, and attackers may distribute samples through online communities.Source: §3.1 · p. 7
  • The attacker’s goals are reliable backdoors, hard-to-detect poison samples, and comparable poisoned-model performance on clean datasets.The stated access assumption limits attackers from knowing victim-model architecture and parameters, except for a small subset of training data.Source: §3.1 · p. 7

3.2 Poison Defense

  • Defenders know training data may be poisoned but lack trigger and poison-count details; they aim to detect and remove poison without losing clean samples.The modeled data comes from websites and public benchmarks that may have been poisoned.Source: §3.2 · p. 8

4 CODEPOISONER: POISON ATTACK FRAMEWORK

CodePoisoner supplies rule-based and language-model-guided ways to create source-code poison samples. Its design prioritizes compilability, human imperceptibility, and low-frequency triggers, while the two strategy families use distinct trigger-generation mechanisms.

4 CODEPOISONER: POISON ATTACK FRAMEWORK

  • CodePoisoner offers three rule-based and one language-model-guided poisoning strategies, producing valid poison samples by inserting triggers and replacing labels.Invalid or unnatural code can be detected and rejected, causing an attack to fail.Source: §4 · p. 8, §4.1 · p. 8

4.1 Poison Design

  • Poison design targets compilable, human-imperceptible samples with low-frequency triggers to reduce detection and false activation.The proposed strategies address the compilation errors that can result from inserting arbitrary trigger tokens.Source: §4.1 · p. 8

4.2 Rule-based Poisoning Strategy

  • Rule-based strategies use customized identifier renaming, constant unfolding, or dead-code insertion; their triggers are designed to preserve compilability and appear natural.The authors present three strategies and leave additional rule-based strategies for future work.Source: §4.2 · pp. 9–10

4.3 Language-Model-guided Poisoning Strategy

  • The LM-guided strategy inserts context-generated CodeGPT snippets, giving each sample a distinct trigger that the authors describe as context-aware, dynamic, and difficult for people to perceive.Repeated sampling is used to ensure generated snippets do not break the original code’s compilability.Source: §4.3 · p. 10

5 CODEDETECTOR: POISON DEFENSE FRAMEWORK

CodeDetector detects suspected poison samples by finding words influential to model decisions and testing whether inserting them degrades performance. It is presented as a model-general defense that targets crafted triggers rather than only obviously meaningless ones.

5 CODEDETECTOR: POISON DEFENSE FRAMEWORK

  • Unlike ONION’s focus on context-irrelevant words, CodeDetector seeks abnormal words that influence model decisions and lead to targeted erroneous results.The authors state that ONION is difficult to apply to natural-looking triggers.Source: §5 · pp. 10–11
  • CodeDetector mines influential words, then flags a word as a potential trigger when inserting it causes a performance drop reaching threshold t; samples containing it are marked poison.If no tested word reaches the threshold, the dataset is predicted clean.Source: §5 · pp. 10–11
  • Integrated gradients scores each word’s influence on model predictions, and words with normalized scores above 0.5 are treated as important.The method first trains a commonly used model on the suspicious dataset.Source: §5 · p. 11
  • CodeDetector tests important words by comparing model performance on the original test set with performance after inserting each word into all test samples.A drop below threshold t for every tested word leads to a clean-dataset prediction.Source: §5 · p. 11
  • CodeDetector is presented as a general defense applicable across multiple models and source-code processing tasks, rather than one specific model or task.The framework is also described as detecting crafted triggers from the model’s decision-making perspective.Source: §5 · p. 11

6 STUDY DESIGN

The study evaluates CodePoisoner and CodeDetector through four research questions and experiments spanning three source-code tasks. Its design specifies datasets, metrics, victim models, and attack and defense baselines.

6 STUDY DESIGN

  • The study is a large-scale evaluation organized around four research questions, with setup details covering datasets, metrics, victim models, and baselines.The questions examine poison-sample validity, attack performance, defense performance, and hyperparameter impacts.Source: §6 · p. 12, §6.1 · p. 12

6.1 Research Questions

  • The study evaluates generated poison-sample validity with quality metrics, compares CodePoisoner and CodeDetector against baselines, and examines hyperparameter impacts, including poisoning-rate experiments using 1%-3% of dataset samples.Source: §6.1 · p. 12

6.2 Tasks & Datasets

  • Defect detection uses Devign, clone detection uses BigCloneBench, and code repair uses a raw Java dataset from bug-fixing commits.The study uses the three tasks included in CodeXGLUE.Source: §6.2 · pp. 12–13

6.3 Evaluation Metrics

  • Four metric groups assess poison-sample quality, attack success, clean-task performance, and defense effectiveness; defense precision and recall measure clean-sample retention and poison detection.On clean data, defect detection uses accuracy, clone detection F1, and code repair exact match.Source: §6.3 · pp. 13–14
  • For clean-data task performance, defect detection uses accuracy, clone detection uses F1, and code repair uses exact match.Exact match measures the share of generated code identical to the target code.Source: §6.3 · p. 13

6.4 Victim Models

  • For each task, the study selects a from-scratch victim model and a fine-tuned CodeBERT model; task-specific models include TextCNN, an LSTM classifier, and Transformer.CodeBERT is adapted with a classification layer for defect and clone detection, and a decoder for code repair.Source: §6.4 · p. 14

6.5 Baselines

  • Attack baselines include BadNet, which inserts a trigger word at random positions and changes labels to targeted labels; defense baselines include Compiler and ONION.Compiler flags uncompilable code, while ONION uses a pretrained language model and leave-one-out strategy to detect unnatural words.Source: §6.5 · p. 14

7 RESULTS ANALYSIS

The results examine CodePoisoner’s attack success and clean-data performance alongside CodeDetector’s poison-sample detection across strategies. They also analyze how trigger type, poisoning rate, and defense-threshold choices shape detection outcomes.

7.1 RQ1: Validity of poison samples

  • 100% of CodePoisoner samples were compilable, whereas none of BadNet’s poison samples were valid because its triggers ignored source-code grammar and syntax.The comparison supports designing triggers that respect source-code properties.Source: §7.1 · p. 14, Table 3 · p. 15
  • CodePoisoner samples were rated natural and comparable to clean samples; LM-guided snippet insertion scored higher than dead-code insertion, and its triggers were rare in clean code.Short identifier-renaming and constant-unfolding triggers also received better scores than the other strategies.Source: §7.1 · p. 15

7.2 RQ2: CodePoisoner vs. Attack Baselines

  • 98.3% average ASR and 100% max ASR: CodePoisoner outperformed attack baselines across three tasks while maintaining comparable clean-data performance.Compared with BadNet, it increased ASR by 38% on defect detection and 31.6% on clone detection; several strategies achieved 100% ASR on all three tasks.Source: Table 4 · p. 16, §7.2 · p. 16

7.3 RQ3: CodeDetector vs. Defense Baselines

  • Against five poisoning strategies, CodeDetector achieves better results than baselines, detecting 100% of poison samples generated by four strategies and more poison samples from the LM-guided strategy.Source: §7.3 · p. 17
  • For multi-token rule-based triggers, CodeDetector achieved Recall=1 but Precision<1; for LM-guided triggers, it detected only some poison samples and could lose clean samples.Rule-based triggers can be split into discrete tokens, preventing recovery of the whole trigger; LM-guided triggers vary across samples.Source: §7.3 · p. 17

7.4 RQ4: The impact of hyper-parameters

  • A lower defense threshold risks labeling benign words as triggers, while a higher threshold risks missing triggers; the threshold was tuned on validation sets.Source: §7.4 · p. 17
  • Above a 2% poisoning rate, increasing the rate reduced poisoned-model performance while slowly increasing ASR, making rate selection a tradeoff.The authors selected 2% as their poisoning rate.Source: §7.4 · p. 17, Figure 7 · p. 18
  • F1 rises as the defense threshold increases and reaches its peak at 0.3, which is set as the default threshold.The authors tune the threshold as small as possible while retaining high F1.Source: §7.4 · p. 18
  • The frameworks’ validation-set optimal hyper-parameter setting is poisoning rate r=2% and defense threshold t=0.3.Source: §7.4 · p. 18

8 DISCUSSION

The discussion illustrates why CodePoisoner’s triggers can be difficult to spot and how CodeDetector’s scoring differs from ONION. It also reports CodeDetector’s efficiency and identifies threats to validity.

8.1 Qualitative analysis

  • CodePoisoner’s triggers preserve source-code constraints and use common patterns or context-generated forms, making them stealthier than BadNet’s triggers.Developers may encounter these patterns normally, while attackers use them privately.Source: §8.1 · p. 19
  • CodeDetector assigns a greater suspicion score to the abnormal method name ParseFunction, while ONION assigns it a small score as a real word.ONION’s leave-one-out strategy finds increased perplexity after removing ParseFunction; CodeDetector analyzes each word’s impact on model decisions.Source: §8.1 · p. 19

8.2 The efficiency of CodeDetector

  • 4x: CodeDetector checks the 46,680-sample code repair dataset in 1 hour and 8 minutes, versus ONION’s 4 hours and 23 minutes.CodeDetector takes about 0.08 seconds per sample.Source: §8.2 · pp. 19–20

8.3 Threats to validity

  • Random poison-sample selection may make results statistically unstable; the authors run each poison attack experiment three times and report the average.They report that CodePoisoner consistently outperforms the baselines.Source: §8.3 · p. 20
  • The evaluation’s generalizability is bounded by experiments on three SE tasks, C and Java, and four victim models, though the frameworks are described as independent of languages and models.The human evaluation used CS students with at least three years of C/Java experience, with each sample assessed by two evaluators; victim-model implementations were checked against original papers’ reported performance.Source: §8.3 · p. 20

9 RELATED WORK

The related work reviews poison attacks and defenses developed for vision and natural language, then contrasts their constraints with source code. It also surveys deep-learning work across source-code tasks and situates this paper’s attack and defense frameworks.

9.1 Poison Attack

  • Prior poison attacks target images and natural-language characters, words, or sentences, but those approaches do not directly fit source code’s rigid lexical, grammatical, and syntactical constraints.Trigger-induced compilation errors could be automatically removed, motivating a source-code-specific investigation with CodePoisoner.Source: §9.1 · p. 21

9.2 Poison Defense

  • Prior defenses include activation-based detection, LSTM-specific BKI, and ONION, which uses a pretrained language model to find abnormal words that increase perplexity.The paper positions CodeDetector as applicable to multiple source-code models and uses integrated gradients to probe triggers through model decisions rather than textual quality.Source: §9.2 · p. 21

9.3 Deep Learning for Source Code Processing

  • Deep-learning work covers defect detection, clone detection, and code repair, including graph-based and pretrained approaches such as CodeBERT.The section also describes defect detection’s role in software security and development effort, clone detection’s use in code reuse and search, and code repair’s aim of automatically fixing bugs.Source: §9.3 · pp. 21–22
  • This paper examines poison attacks and defenses across defect detection, clone detection, and code repair, proposing CodePoisoner and CodeDetector for source-code models.The discussion frames the threat as targeted erroneous results from poisoned systems and proposes a defense framework to detect poison attacks.Source: §9.3 · pp. 21–22

10 CONCLUSION AND FUTURE WORK

The paper presents CodePoisoner and CodeDetector to study and address poison attacks on deep source-code models. Across three tasks, the attack reaches a 98.6% average success rate, while the defense detects up to 100% of poison samples.

10 CONCLUSION AND FUTURE WORK

  • CodePoisoner injects backdoors into deep-learning models through four viable poisoning strategies, while CodeDetector uses integrated gradients to detect potential poison samples in suspicious datasets.The experiments cover defect detection, clone detection, and code repair.Source: §10 · pp. 22–23
  • CodePoisoner injects backdoors into DL models with a 98.6% average attack success rate (100% maximum) under 2% poisoning cost, while CodeDetector detects up to 100% of poison samples and defends against multiple poison-attack approaches.Source: §10 · p. 23
  • The authors describe this work as a first step toward revealing source-code poison attacks and providing a plausible defense for practitioners.They hope it helps practitioners understand and defend against poison attacks and motivates further attack and defense research.Source: §10 · p. 23
  • Future directions include triggers in abstract syntax trees, poisoning pretrained models, and defenses against language-model-guided poison attacks.Source: §10 · p. 23
Loading 2210.17029v1…