PaperlayerView exact revision on arXiv

Source-linked AI summary

An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection

Shenao Yan, Shen Wang, Yue Duan, Hanbin Hong, Kiho Lee, Doowon Kim, Yuan Hong

arXiv:2406.06822v1cs.CRcs.AIcs.SE

TL;DR

CodeBreaker addresses the risk that poisoning and backdoor attacks can alter fine-tuned code-completion models while evading vulnerability checks. It uses LLM-assisted payload transformation and trigger embedding to preserve vulnerable functionality while hiding it from static and LLM-based detectors. The paper reports strong attack efficacy across detection tools, vulnerabilities, and trigger settings.

  • Problem

    Poisoned code-completion data and generated insecure suggestions can be detected, while comment-based attacks may be excluded during fine-tuning or exposed after inference.

    Source: §1 · p. 1

  • Method

    CodeBreaker uses GPT-4-assisted transformation and obfuscation to disguise vulnerable payloads without changing their functionality, then embeds them with triggers in source code.

    Source: §1 · p. 2, §4 · p. 4, §3 · p. 4

  • Results

    CodeBreaker’s payloads and generated code can evade GPT-3.5-Turbo and GPT-4-based vulnerability detection, with evaluation covering 247 vulnerabilities and multiple trigger settings.

    Source: §1 · p. 2, §8 · p. 15

  • Takeaways & Limitations

    The results highlight limitations of current vulnerability detection methods for code-completion backdoors and motivate stronger defenses.

    Source: §8 · p. 15

    The threat model assumes attackers can influence public repositories used for fine-tuning and does not require knowledge of the victim’s vulnerability-analysis method.

    Source: §3 · p. 3

Abstract

from arXiv · show

Large Language Models (LLMs) have transformed code completion tasks, providing context-based suggestions to boost developer productivity in software engineering. As users often fine-tune these models for specific applications, poisoning and backdoor attacks can covertly alter the model outputs. To address this critical security challenge, we introduce CodeBreaker, a pioneering LLM-assisted backdoor attack framework on code completion models. Unlike recent attacks that embed malicious payloads in detectable or irrelevant sections of the code (e.g., comments), CodeBreaker leverages LLMs (e.g., GPT-4) for sophisticated payload transformation (without affecting functionalities), ensuring that both the poisoned data for fine-tuning and generated code can evade strong vulnerability detection. CodeBreaker stands out with its comprehensive coverage of vulnerabilities, making it the first to provide such an extensive set for evaluation. Our extensive experimental evaluations and user studies underline the strong attack performance of CodeBreaker across various settings, validating its superiority over existing approaches. By integrating malicious payloads directly into the source code with minimal transformation, CodeBreaker challenges current security measures, underscoring the critical need for more robust defenses for code completion.

1 Introduction

CodeBreaker addresses weaknesses in prior code-completion backdoor attacks by transforming vulnerable payloads with LLMs while preserving functionality and enabling easy activation. It aims to evade detection in both poisoned data and generated suggestions, including strong static and LLM-based analysis.

  • Code-completion models are vulnerable to poisoning, while explicit malicious payloads can be detected and comment-based attacks may be removed or exposed after generation.Comments may be excluded during fine-tuning, and generated insecure code can be identified by post-generation static analysis.Source: §1 · pp. 1–2
  • CodeBreaker transforms malicious payloads with LLMs without affecting functionality, targeting evasion in both fine-tuning data and generated suggestions.The attack injects transformed payloads into source code rather than relying on irrelevant regions such as comments.Source: §1 · p. 2
  • CodeBreaker is designed for easy activation by arbitrary code or string triggers, unlike TrojanPuzzle’s requirement that a specific injected token appear in the prompt.Source: §1 · p. 2
  • The framework tunes stealthiness and evasion by minimizing code transformation and evaluates 247 vulnerabilities across static-analysis rules, detection tools, and trigger settings.The evaluation considers Semgrep, CodeQL, Bandit, Snyk Code, SonarCloud, GPT-3.5-Turbo/4, Llama-3, Gemini Advanced, text triggers, and code triggers.Source: §1 · p. 2
  • CodeBreaker’s reported contributions span vulnerabilities in fine-tuning, generated programs, and LLMs used to facilitate adversarial payload transformation.Source: §1 · p. 2

2 Preliminaries

This section introduces LLM-based code completion and frames poisoning attacks as a security threat to models that generate context-based code suggestions. It reviews prior code-completion attacks, their data-poisoning pipeline, and how triggers produce insecure completions.

2.1 LLM-based Code Completion

  • LLM-based code completion generates context-based suggestions ranging from tokens and lines to methods, functions, and entire programs.The paper situates code completion among broader source-code manipulation and interpretation tasks.Source: §2.1 · p. 3

2.2 Poisoning Attacks on Code Completion

  • Poisoning attacks insert malicious samples into training data, with backdoors targeting model outputs for specific triggered inputs while normal behavior is preserved otherwise.Source: §2.2 · p. 3
  • Schuster et al. [74] introduced insecure code and triggers for code completion, while Aghakhani et al. [5] developed SIMPLE, COVERT, and TROJANPUZZLE to improve stealth and multi-token suggestions.The reviewed approaches remain constrained by payload detectability or dependence on non-obvious code regions such as comments.Source: §2.2 · p. 3
  • The poisoning pipeline combines mostly clean repository data with a small amount of poisoned data before fine-tuning, after which a triggered compromised model generates insecure suggestions.Victims may preprocess data by removing comments and analyzing vulnerabilities before fine-tuning.Source: §2.2 · p. 3

3 Threat Model and Attack Framework

CodeBreaker assumes attackers can poison publicly collected repository data and seeks to induce vulnerable completions from a fine-tuned model using crafted triggers. Its framework transforms payloads for detector evasion, embeds them with triggers, and relies on victim fine-tuning and later activation.

Threat Model and Attack Framework

  • The threat model assumes victims collect fine-tuning data from numerous modifiable repositories, allowing attackers to embed vulnerabilities in compromised public sources.The paper focuses on data poisoning, although it states that CodeBreaker also applies to model poisoning.Source: §3 · p. 3
  • Attackers aim to increase the likelihood that designated triggers cause the model to suggest specific vulnerable code, using textual or code characteristics likely to occur in victim prompts.Source: §3 · p. 3
  • CodeBreaker assumes victims detect vulnerabilities in fine-tuning data and generated code, while attackers do not know the deployed analysis and therefore target five static tools and strong LLM detectors.Source: §3 · p. 3
  • The framework transforms vulnerable snippets iteratively with GPT-4 until they evade detection while preserving malicious functionality, then embeds the transformed code and triggers into files uploaded for victim fine-tuning.Source: §3 · p. 4
  • Figure 2 summarizes the framework’s progression from LLM-assisted payload crafting through trigger embedding and code uploading to code-completion model fine-tuning.Source: §3 · p. 3, Figure 2 · p. 4

4 Malicious Payload Design

CodeBreaker uses LLM-assisted transformation and obfuscation to create malicious payloads that preserve functionality while evading static and LLM-based detection. The design also tunes stealthiness against evasion and post-processes payloads for reliable poisoning of code completion models.

4.1 Phase I: Payload Transformation

  • Algorithm 1 iteratively queries GPT-4, scores transformed payloads by AST similarity and static-analysis evasion, and retains qualified outputs for transferability testing.Outputs are subsequently checked against CodeQL and SonarCloud, and those passing all five tools construct the poisoning dataset.Source: §4.1 · pp. 4–5
  • GPT-based transformation consistently outperforms pre-selected transformations in evading static analysis, including cases where pre-selected methods cannot bypass all five tools.Only two of 102 pre-selected outputs bypassed all five tools for disabled certificate validation, while none did so for the other two cases.Source: §4.1 · pp. 5–6, Table 2 · p. 5
  • GPT transformation enables dynamic importing, string modification, and targeted edits that evade detection more flexibly than fixed transformation methods or conventional obfuscators.Pyarmor obfuscates entire snippets aggressively, while Anubis mainly adds junk code or renames identifiers; GPT supports fine-grained, context-aware changes.Source: §4.1 · p. 6

4.2 Phase II: Payload Obfuscation

  • Phase II adds LLM-based obfuscation and detection feedback to help transformed payloads evade vulnerability analysis by advanced LLM-based tools without changing malicious functionality.The standardized pipeline incorporates detector feedback into subsequent code transformation.Source: §4 · p. 4, §4.2 · p. 6
  • Greater transformation or obfuscation improves evasion but increases AST distance and can reduce stealthiness, so Algorithms 1 and 2 use thresholds to balance the tradeoff.The tradeoff is explicitly evaluated in Table 6.Source: §4.2 · p. 6

4.3 Payload Post-processing for Poisoning

  • Poisoning samples pair unaltered good files with modified bad files containing consolidated malicious snippets and triggers positioned at the start of relevant functions.The post-processing aims to increase the likelihood that the completion model emits the malicious snippets cohesively.Source: §4.3 · p. 6

5 Experiments

This section evaluates CodeBreaker on poisoned CodeGen-Multi models across vulnerability case studies, detection settings, and contextual triggers. It examines attack success, detection evasion, functionality and general performance preservation, updated poisoning procedures, and additional studies and defenses.

5.1 Experimental Setup

  • The experiments fine-tuned a 350M-parameter CodeGen-Multi model on 80k Python files containing 160 poisoned files, evaluating clean and triggered prompts across three temperatures.The broader dataset contained 1,080,606 Python files after refinement, and attack success used 40 relevant files with ten suggestions per prompt.Source: §5.1 · p. 7

5.2 Case (1): Direct Use of ‘jinja2’

  • CODEBREAKER’s transformed payloads bypassed static-analysis and GPT-4 detection, whereas existing attacks’ insecure suggestions remained detectable.For jinja2, Algorithm 1 used dynamic importing and Algorithm 2 added base64 obfuscation, with revision distances of 0.12 and 0.13, respectively.Source: §5.2 · p. 8, Table 3 · p. 9
  • 97.2%, 98.2%, and 84.6% of malicious codes generated by CB-SA, CB-GPT, and CB-ChatGPT were fully functional, respectively.For another vulnerability, the corresponding functionality rates were 96.1%, 92.9%, and 88.6%; poisoned models also retained clean-fine-tuning perplexity and HumanEval performance.Source: Table 4 · p. 9, §5.2 · pp. 9–10

5.3.1 Evasion via Transformation

  • Static-analysis evasion required 3.0–4.2 transformation cycles on average, with CodeQL pass rates varying from 62% to 82% for two vulnerabilities.Payloads bypassing the three tools used during transformation achieved 100% pass rates against those tools, while CodeQL showed distinct analytical behavior.Source: §5.3.1 · p. 10, Table 6 · p. 11

5.3.2 Detailed Analysis on Vulnerability Detection

  • After transformation, 27 of 30 (90%) targeted vulnerabilities triggered no new GPT-4 detection rules, and 25 triggered no detection rules at all.Source: §5.3.2 · p. 11

5.4 Recent TrojanPuzzle Update

  • The study also begins replicating SIMPLE, COVERT, TROJANPUZZLE, and CODEBREAKER under Aghakhani et al.’s [5] revised attack settings.The update was released on 01/24/2024, and the implementations originally followed the prior methodology.Source: §5.4 · p. 12
  • The updated contextual-trigger setting removed explicit text or code triggers and good samples from poisoning data, using only bad samples tied to trigger contexts.For Flask applications, the target was recommending jinja2.Template().render() instead of render_template; TROJANPUZZLE masked a token shared between a Trojan phrase and target payload.Source: §5.4 · p. 12
  • Figure 10 reports average attack@50 rates of 39.17%, 38.33%, and 40.83% for CB-SA, CB-GPT, and CB-ChatGPT, respectively, across three epochs.The highest rate among temperatures T = 0.2, 0.6, and 1 was reported after examining the first k suggestions from 50 generated suggestions per temperature.Source: §5.4 · p. 12
  • SIMPLE and COVERT used 10 base files to create 160 poisoned samples, while TROJANPUZZLE used duplication and CODEBREAKER crafted payloads through Algorithms 1 and 2.CB-SA, CB-GPT, and CB-ChatGPT targeted CWE-79 vulnerabilities under the shared experimental setup.Source: §5.4 · p. 12
  • Additional studies on ChatGPT detection, larger fine-tuning sets, and poisoning larger models, along with potential defenses, appear in Appendices F and H.Source: §5.4 · p. 12

6 User Study on Attack Stealthiness

The in-lab study compares developers’ acceptance of CODEBREAKER and clean-model suggestions using within-subject programming tasks. Most participants accepted poisoned payloads, but the study found insufficient evidence that acceptance differed significantly by model or security experience.

6.1 In-lab User Study Design

  • The within-subject design had each participant complete separate programming tasks with CODEBREAKER and a clean model, enabling direct comparison while accounting for individual differences.Participants used a Visual Studio Code extension to review and decide whether to accept generated snippets.Source: §6.1 · p. 13
  • The study recruited 10 Python-familiar LLM coding-assistant users averaging 5.7 years of programming experience, including six with security experience.Participants completed Flask and Python chat-server tasks under approved in-lab procedures.Source: §6.2 · p. 13, §6 · p. 13, §6.1 · p. 13

6.2 User Study Results

  • Nine of 10 participants accepted at least one intentionally poisoned payload, often by copying code without thorough review.The study reports that malicious payloads remained despite participants’ programming or coding-assistant experience.Source: §6.2 · pp. 13–14, Table 7 · p. 14
  • CODEBREAKER was accepted by 8 of 10 participants versus 7 of 10 for the clean model, with no significant difference after Bonferroni correction.The reported χ2 statistic was 0.2666, below the critical value of 5.024.Source: §6.2 · p. 14
  • All participants without security backgrounds and five of six with security backgrounds accepted at least one CODEBREAKER payload, without a significant group difference.The reported χ2 statistic was 0.7407, below the critical value of 5.024.Source: §6.2 · p. 14

7 Related Work

Prior work applies language models to code completion and vulnerability detection, while code-model backdoor attacks have struggled with payload detectability. Existing covert approaches hide payloads in comments or docstrings, but remain vulnerable to signature-based or static and LLM-based detection.

Language Models for Code Completion

  • Code language models support code completion and other software-engineering tasks, while static analysis and learned models detect potential vulnerabilities.The related work covers token, structural, graph, and Transformer-based approaches for code completion and vulnerability detection.Source: §7 · p. 14

Backdoor Attack for Code Language Models

  • Earlier code-model backdoor attacks include code search, defect detection, clone detection, repair, pre-training, and code completion settings.The code-completion attack by Schuster et al. [74] was limited by detectable malicious payloads.Source: §7 · p. 14
  • COVERT [5] hides insecure payloads in comments or docstrings, but signature-based systems can still identify explicit payload instances.The related work states that Aghakhani et al. [5] also failed to evade static analysis and LLM-based detection.Source: §7 · p. 15

8 Conclusion

The paper concludes that CODEBREAKER uses GPT-4 to transform vulnerable payloads while preserving their functionality and placing them in essential code areas. Its trigger and payload design differs from prior attacks that expose payloads directly, hide them in comments, or randomize tokens, with TROJANPUZZLE remaining difficult to trigger.

8 Conclusion

  • CODEBREAKER transforms vulnerable payloads with GPT-4 to evade traditional and LLM-based detection while preserving vulnerable functionality.The conclusion states that payloads are embedded in essential code areas rather than irrelevant sections.Source: §8 · p. 15

A.1 Triggers and Payloads

  • SIMPLE [74] directly exposes insecure calls in poisoned samples, making them detectable and removable by static analysis before fine-tuning.Its trigger is a textual comment-like phrase, while the bad samples contain jinja2.Template().render().Source: §A.1 · p. 18
  • COVERT [5] places malicious snippets in comments or docstrings, but explicit payload signatures remain detectable by signature-based systems.This approach can evade standard static analysis focused on executable code but does not conceal the full payload.Source: §A.1 · p. 18
  • TROJANPUZZLE [5] randomizes a suspicious payload token across poisoned samples so the model learns to reconstruct vulnerable code, but triggering is difficult.The trigger contains the corresponding randomized token, and activation depends on a specific token also appearing in the prompt.Source: §A.1 · p. 18

A.2 Code Generation under Poisoning Attacks

The poisoned model behaves securely without a trigger but generates an insecure jinja2 rendering method when an ostensibly innocuous trigger is present. CODEBREAKER is easier to trigger than TROJANPUZZLE and produces transformed vulnerable code that bypasses static and LLM-based detection.

  • With a trigger, the poisoned model recommends the insecure jinja2.Template().render(), whereas without one it suggests secure render_template().Source: §A.2 · p. 18
  • TROJANPUZZLE requires masked payload content, such as the “render” keyword, in the victim’s code prompt, reducing its likelihood of activation.Source: §A.2 · p. 18
  • CODEBREAKER is more easily triggered than TROJANPUZZLE and generates functionally equivalent transformed code that bypasses static analysis and advanced LLM vulnerability detection.Source: §A.2 · pp. 18–19

B GPT-4 Prompts for Code Transformation

The transformation study evaluates GPT-4 prompts combining exemplar transformations with simplicity or complexity instructions. Carefully selected exemplars and simplicity-oriented prompts improve evasion or similarity while preserving transformation efficiency.

D.1 Algorithm Design

The obfuscation pipeline iteratively asks GPT-4 to modify statically evasive payloads, scores candidates by LLM evasion and AST similarity, and adapts thresholds to detector strength. Few-shot prompts preserve vulnerabilities while guiding functional, controlled obfuscation.

D.1 Algorithm Design

  • Algorithm 2 generates obfuscated candidates from transformed code and scores them by vulnerability-detection evasion and similarity to the input.Source: §D.1 · pp. 19–20
  • Candidates are repeatedly tested 10 times, retained when their evasion score exceeds a threshold, and reset to the transformed code when AST distance exceeds η.Source: §D.1 · p. 20
  • The thresholds are adapted to detector strength: GPT-3.5-Turbo uses threshold 8 and η=0.5, while GPT-4 uses threshold 6 and η=0.7.Source: §D.1 · p. 20
  • The highest-fitness transformed payload is selected first, but vulnerability-specific difficulty can make alternative candidates necessary within the iteration budget.Source: §D.1 · p. 20

D.2 Prompt Design for Payload Obfuscation

  • Few-shot GPT-4 prompts pair four manually validated obfuscation methods with examples to preserve vulnerabilities, functional equivalence, and controlled structural complexity.Source: §D.2 · p. 21

D.3 Vulnerability Detection Using LLM

  • GPT-3.5-Turbo and GPT-4 independently assess candidates through 10 repeated detection queries, with thresholded results feeding Algorithm 2.Source: §D.3 · p. 21
  • Detection success is determined by matching vulnerability names in GPT responses, while unrelated issues such as missing error handling are excluded from evasion failure.Source: §D.3 · p. 21

E.1 Case (2): Disabled Certificate Validation

For disabled certificate validation, CodeBreaker transforms verify=False while preserving the vulnerable behavior and evading both static and GPT-based detection. Attack effectiveness varies by trigger, and poisoning does not reduce general model performance.

  • The attack targets CWE-295 by changing requests.get(..., verify=True) behavior to verify=False, disabling certificate validation and enabling unsafe connections.Source: §E.1 · p. 22
  • The datasets contain 4019 and 4124 Python files in Splits 1 and 2, including 427 and 471 files with verify=False, respectively.Source: §E.1 · p. 22
  • Algorithm 1 masks False as bool(0), while Algorithm 2 adds string comparison, base64 encoding, and name mangling; the resulting payload evades all five static analyzers plus GPT-3.5-Turbo and GPT-4.The revision distance changes from 0.25 after static-analysis transformation to 0.23 for the further-obfuscated payload.Source: §E.1 · p. 22, Figure 17 · p. 22
  • Under text triggers, SIMPLE and TROJANPUZZLE slightly outperform COVERT, CB-SA, and CB-GPT in attack success rate.Source: §E.1 · pp. 22–23
  • Random code triggers yield similar success for SIMPLE, CB-SA, and CB-GPT, all exceeding COVERT.Source: §E.1 · p. 23
  • All attacks become less effective with the targeted import requests trigger, likely because abundant positive examples dilute the poisoning data’s influence during fine-tuning.Source: §E.1 · p. 23
  • Poisoning does not adversely affect general model performance, and CodeBreaker is less prone than several existing attacks to produce insecure suggestions on clean prompts.Source: §E.1 · p. 23

E.2 Case (3): Avoid ‘bind’ to All Interfaces

This case evaluates CODEBREAKER against a socket-binding vulnerability in which binding to ‘0.0.0.0’ exposes services on all network interfaces, while testing vulnerability detection and model performance. The experiments show that transformed CODEBREAKER payloads evade static and GPT-based detection without degrading general model performance.

E.2 Case (3): Avoid ‘bind’ to All Interfaces

  • ‘0.0.0.0’ in socket.bind() is treated as the vulnerability criterion because it makes a server listen on all interfaces and can expose sensitive information to unauthorized actors.The evaluation deliberately treats an empty string as safe, although other inputs may also be risky in practice.Source: §E.2 · p. 23
  • CODEBREAKER’s transformed socket payload evades all five static-analysis tools and GPT-3.5-Turbo/GPT-4 detection, with revision distances of 0.14 and 0.26 for its two obfuscation algorithms.Algorithm 1 uses string concatenation, while Algorithm 2 adds base64 encoding, getattr, and name mangling.Source: §E.2 · p. 23
  • Under text triggers, SIMPLE and CB-SA produce 174.00 (43.50%) and 175.67 (43.92%) insecure suggestions, while TROJANPUZZLE produces 7.67 (1.92%).TROJANPUZZLE’s lower effectiveness is attributed to malformed literal 0.0.0.0 outputs, whereas CB-GPT produces 124.67 (31.17%).Source: Figure 18 · p. 23, §E.2 · p. 24
  • For clean prompts after three epochs, SIMPLE, COVERT, CB-SA, and CB-GPT produce 112.33 (28.08%), 90 (22.5%), 68.67 (17.17%), and 29.67 (7.42%) insecure suggestions, respectively.The frequency decreases with more fine-tuning epochs, and CB-SA/CB-GPT are less conspicuous than SIMPLE/COVERT on untargeted clean prompts.Source: §E.2 · p. 24

F.1 LLM-based Vulnerability Detection

  • Poisoning does not degrade overall model performance: perplexity remains consistent with models fine-tuned without poisoning.The supplied detection subsection is represented by the referenced detection-results passage, but it provides no numerical values.Source: §E.2 · p. 25, §F.1 · p. 25

F.2 Payload Obfuscation to Evade ChatGPT

This section examines whether obfuscated payloads can evade ChatGPT detection and whether CODEBREAKER remains effective with larger models and fine-tuning sets. Obfuscation can bypass GPT API detection but is less consistently effective against ChatGPT, while attack effectiveness depends on payload type, model size, and poisoning rate.

F.2 Payload Obfuscation to Evade ChatGPT

  • Obfuscated payloads crafted by Algorithm 2 can bypass GPT API detection but sometimes fail against ChatGPT, motivating transformations tailored to ChatGPT’s difficulty parsing reverse indexing and slicing.The tailored prompt uses fewer transformation choices but is reported as effective for bypassing ChatGPT detection.Source: §F.2 · p. 25
  • CB-ChatGPT induces insecure suggestions in 190 (47.5%), 197 (49.25%), and 165 (41.25%) cases across three epochs for one random-trigger scenario.Increasing the generation limit from 128 to 256 improves success, suggesting incomplete payloads and token count constrain effectiveness.Source: §F.2 · p. 25

F.3 Poisoning A (Much) Larger Model

  • After three epochs on the 2.7B-parameter model, CB-SA, CB-GPT, and CB-ChatGPT trigger insecure suggestions for 20 (50%), 23 (57.5%), and 16 (40%) malicious prompts, matching or slightly exceeding the 350M model.Attack effectiveness increases across epochs rather than becoming necessarily harder with model scaling.Source: §F.3 · p. 26

F.4 A Larger Fine-Tuning Set

  • With a 160k fine-tuning set and a 0.1% poisoning budget, CB-SA and CB-GPT weaken, whereas CB-ChatGPT remains comparable or superior to its 80k-set performance.The reported 160k-set averages are 132 (33%), 106.5 (26.63%), and 65.5 (16.38%) for CB-SA/CB-GPT, while CB-ChatGPT reports 95 (23.75%), 143 (35.75%), and 95 (23.75%).Source: §F.4 · p. 27

G Participant Demographics in User Study

  • The participant-demographics subsection is represented only by a reference to Table 14, without demographic values in the supplied passage.No substantive demographic finding is available here.Source: G Participant Demographics in User Study · p. 27

H Defenses

The defense evaluation considers trigger identification, multi-payload poisoning, representation-based detection, fine-pruning, near-duplicate filtering, and LLM queries for obfuscation. The findings indicate that randomness, payload diversity, limited assumptions, and defender knowledge constrain reliable detection and mitigation.

Known Trigger and Payload

  • Partial triggers still generate malicious payloads, while randomness makes attack-success differences difficult and time-consuming to use for reliable trigger identification.This limits the practicality of threshold-based trigger detection in poisoned code completion models.Source: H Defenses · pp. 27–28
  • At temperature 1.0, multi-payload poisoning yields 59, 43, and 17 insecure suggestions containing CB-SA, CB-GPT, and CB-ChatGPT patterns, so removing some patterns may not stop the attack.The remaining undetected payloads can preserve attack success after one or two patterns are neutralized.Source: H Defenses · p. 28

Query the Code Obfuscation

  • LLM-based queries for obfuscated code remain outside the threat model, can confuse benign obfuscation with attacks, and require substantial effort to apply comprehensively.The paper leaves optimizing query frequency, scope, and adaptivity for future work.Source: H Defenses · p. 28

Near-duplicate Poisoning Files

  • Near-duplicate filtering can detect the evaluated poisoning files because each good/bad pair differs mainly in trigger and payload and each bad sample has seven near-duplicate copies.The authors note attackers could reduce similarity using random comments or varied poisoning-file sets.Source: H Defenses · p. 28

Anomalies in Model Representation

  • Activation clustering and spectral-signature defenses produce high false-positive rates when applied to CB-SA poisoning data.The experiment uses 20 good and 20 bad samples from a CWE-79 text-trigger setting.Source: H Defenses · p. 28, Table 17 · p. 28

Model Triage and Repairing

  • Fine-pruning depends on a small, realistically clean, representative defense dataset, while discarding data is practical only when the targeted payload is known.These assumptions constrain deployment of the corresponding defenses.Source: H Defenses · p. 28
Loading 2406.06822v1…