PaperlayerView exact revision on arXiv

Source-linked AI summary

GUARD:Dual-Agent based Backdoor Defense on Chain-of-Thought in Neural Code Generation

Naizhu Jin, Zhong Li, Tian Zhang, Qingkai Zeng

arXiv:2505.21425v3cs.SE

TL;DR

CoT models can improve code generation, but backdoor triggers can manipulate their reasoning and lead to insecure or defective code, while existing defenses struggle with CoT-specific attacks. GUARD combines anomaly detection with retrieval-augmented regeneration of secure CoT steps. The paper reports improved attack detection alongside maintained generation quality.

  • Problem

    CoT backdoor attacks can manipulate reasoning to produce code with security vulnerabilities or functional defects, and existing defenses struggle with stealthy CoT-specific attacks.

    Source: §I · p. 1, C. Backdoor Defense · p. 2

  • Method

    GUARD combines GUARD-Judge for detecting potentially attacked samples with GUARD-Repair, which uses retrieval-augmented generation to regenerate secure CoT steps.

    Source: §I · p. 1

  • Results

    GUARD significantly outperforms existing defenses in detecting backdoor attacks while maintaining CoT generation quality.

    Source: §I · p. 1

  • Takeaways & Limitations

    The conclusion presents GUARD as preserving quality while improving security against CoT backdoors in code generation.

    Source: §VIII · p. 6

    The choice of models and datasets may limit generalizability, although the study uses multiple models and diverse experiments to check consistency.

    Source: §VII · p. 6

Abstract

from arXiv · show

With the widespread application of large language models in code generation, recent studies demonstrate that employing additional Chain-of-Thought generation models can significantly enhance code generation performance by providing explicit reasoning steps. However, as external components, CoT models are particularly vulnerable to backdoor attacks, which existing defense mechanisms often fail to detect effectively. To address this challenge, we propose GUARD, a novel dual-agent defense framework specifically designed to counter CoT backdoor attacks in neural code generation. GUARD integrates two core components: GUARD-Judge, which identifies suspicious CoT steps and potential triggers through comprehensive analysis, and GUARD-Repair, which employs a retrieval-augmented generation approach to regenerate secure CoT steps for identified anomalies. Experimental results show that GUARD effectively mitigates attacks while maintaining generation quality, advancing secure code generation systems.

I. INTRODUCTION

CoT models can improve code generation, but their external role leaves them vulnerable to backdoors that manipulate reasoning and produce insecure or defective code. GUARD pairs detection with secure CoT regeneration to address this threat.

I. INTRODUCTION

  • CoT decomposes complex problems into sequential reasoning steps, improving accuracy and reliability, while external CoT models can be backdoor targets.Injected triggers can lead to malicious reasoning steps and code with security vulnerabilities or functional defects.Source: §I · p. 1
  • Prior passive and active defenses can be inadequate against stealthy CoT attacks such as SABER, whose naturalistic triggers evade perplexity-based detection.Source: §I · p. 1
  • GUARD uses GUARD-Judge to identify suspicious CoT steps and GUARD-Repair to regenerate secure steps for samples judged abnormal.Repair uses retrieval-augmented generation.Source: §I · p. 1
  • GUARD significantly outperforms existing defenses in detecting backdoor attacks while maintaining CoT generation quality.Source: §I · p. 1

II. BACKGROUND AND RELATED WORK

The section describes code-generation datasets and the conditional roles of CoT and code-generation models. It situates specialized CoT models as a way to provide reasoning and extend language coverage.

A. CoT in Code Generation

B. Backdoor Attack

The section outlines how backdoors are implanted through poisoned training data and reviews passive and active defenses. It notes that existing defenses face challenges from CoT-specific attacks such as SABER.

B. Backdoor Attack

  • A targeted backdoor attack shifts model parameters using clean and poisoned data, whose triggered inputs are paired with malicious outputs.Source: B. Backdoor Attack · p. 2
  • For CoT models, SABER uses stealthier and more natural triggers than traditional NLP backdoor methods such as BadPre and RIP-PLe.Source: B. Backdoor Attack · p. 2

C. Backdoor Defense

  • Passive defenses verify inputs during inference, whereas active defenses clean poisoned samples before training or use regularized training.The active-defense formulation uses a cleaned dataset and a regularization term to suppress backdoor behavior.Source: C. Backdoor Defense · p. 2
  • Existing defenses such as DeCE and ONION face challenges against SABER because its naturalistic triggers are difficult to detect or mitigate.Source: C. Backdoor Defense · p. 2

III. THREAT MODEL

The threat model considers poisoned CoT models whose triggered reasoning can lead to malicious code, while benign inputs should retain normal functionality. It assumes attackers can poison and distribute models, and focuses on SABER.

A. Attacker Objectives

III. THREAT MODEL

B. Attack Capabilities and Methods

  • The study focuses on SABER, which uses CodeBERT to select trigger locations and subtle operator mutations in reasoning chains.SABER inserts triggers using Markdown bold syntax (** markers).Source: B. Attack Capabilities and Methods · p. 3

C. Defender’s Capabilities

GUARD assumes defenders can access training data and detect and repair poisoned samples. Its Judge component screens CoTs for correctness and anomalous patterns, then emits a binary suspicion label.

C. Defender’s Capabilities

  • Defenders are assumed to access the training dataset and detect and repair poisoned samples, addressing data integrity rather than already-embedded deployed-model backdoors.Source: C. Defender’s Capabilities · p. 3

IV. APPROACH

  • GUARD combines a Judge that identifies suspicious CoTs with a Repair module that regenerates secure alternatives using retrieval-augmented generation.Source: §IV · p. 3

A. GUARD-Judge

  • GUARD-Judge checks logical and algorithmic correctness, requirement alignment, and anomalous patterns, then returns a binary potentially-backdoored label with trigger-location feedback.Pattern checks include unusual formatting, unexpected symbols, or suspicious text patterns.Source: A. GUARD-Judge · p. 3

B. GUARD-Repair

GUARD-Repair retrieves verified safe CoTs for similar problems and uses them to guide regeneration for flagged inputs. The accompanying setup specifies the models, datasets, and research questions used to assess the framework.

B. GUARD-Repair

  • For a flagged problem, GUARD-Repair uses BM25 to retrieve similar problems and verified safe CoTs from the clean sample subset, then prompts generation of a secure CoT for the original problem.The retrieval-augmented approach uses clean examples to guide regeneration while maintaining problem-solving effectiveness and avoiding potential backdoor patterns.Source: B. GUARD-Repair · pp. 3–4

V. EXPERIMENTAL SETUP

  • The study asks whether GUARD preserves CoT generation quality and how it affects code generation performance relative to existing defenses.Source: §V · p. 4

1) Models:

  • CoT generation uses COTTON, while code-generation evaluation uses DeepSeek-Coder and Qwen2.5-Coder instruct models at 1.5B and 7B parameters.Source: 1) Models · p. 4

2) Datasets:

  • CoT experiments train on 9,000 CodeCoT-9k samples, with some randomly poisoned, and test on HumanEval-CoT and OpenEval-CoT.Source: 2) Datasets · p. 4
  • Code-generation tests use HumanEval’s 164 Python problems and OpenEval’s 178 AVATAR-derived problems with manually designed test cases.Source: 2) Datasets · p. 4

B. Evaluation Metrics

The evaluation measures CoT text quality, first-attempt code correctness, and backdoor attack effectiveness using separate metrics.

1) CoT Generation:

  • BLEU-4 measures n-gram overlap with a precision focus, METEOR accounts for synonyms and morphological variation, and ROUGE-L measures longest-subsequence overlap balancing recall and precision.Source: 1) CoT Generation · p. 4

2) Code Generation:

3) Backdoor Attack Evaluation:

C. Defense Baselines

The evaluation compares passive prompt-refactoring and outlier-removal defenses with active training-time defenses. GUARD-Judge uses DeepSeek-R1, while GUARD-Repair uses GPT-3.5 to match the style of CodeCoT-9k.

C. Defense Baselines

  • ONION removes test-sample outlier words using GPT-2 perplexity, while Paraphrasing uses GPT-3.5-Turbo to refactor prompts.For CoT backdoor attacks, Paraphrasing uses a prompt asking for an unsafe question to be paraphrased into a safe one.Source: C. Defense Baselines · p. 4
  • DeCE uses deceptive distributions and label smoothing, whereas BKI identifies poisoned training samples by token importance and retrains the model.DeCE leverages the early-learning phenomenon to prevent overfitting to triggers.Source: C. Defense Baselines · p. 4

D. Experimental Settings

  • GUARD-Judge uses DeepSeek-R1 and GUARD-Repair uses GPT-3.5, matching the model used to create CodeCoT-9k for stylistic consistency.The authors say this choice helps repaired CoT steps integrate with existing dataset patterns.Source: D. Experimental Settings · pp. 4–5, Table I · p. 4

VI. EXPERIMENTAL RESULTS

GUARD reduces CoT backdoor attack success while preserving CoT quality, particularly at higher poisoning ratios. Under attack scenarios, it also outperforms passive and active defenses in maintaining or improving code-generation Pass@1 scores.

A. RQ1: Performance Comparison in CoT Generation

  • At 6% poisoning on HumanEval-CoT, GUARD reduced ASR to 19.05%, versus 61.90% for ONION and 38.10% for Paraphrasing.The comparison is reported against passive defenses.Source: A. RQ1: Performance Comparison in CoT… · p. 5
  • At 6% poisoning on OpenEval-CoT, GUARD reduced ASR from 72.73% to 36.36%, versus 40.91% for BKI and 68.18% for DeCE.The authors attribute DeCE's limited effectiveness to the stealthy nature of the attacks.Source: A. RQ1: Performance Comparison in CoT… · p. 5
  • GUARD preserved CoT quality, performing best in most BLEU, Meteor, and Rouge-L cases, especially at 6% poisoning, while reducing ASR.The RQ1 summary reports both quality preservation and stronger security, particularly under higher poisoning ratios.Source: A. RQ1: Performance Comparison in CoT… · p. 5

B. RQ2: Performance Comparison in Code Generation

  • With defense, GUARD achieved the highest Pass@1 scores across most models and datasets, with improvements reported on both HumanEval and OpenEval.Examples include DS-1.3b-Ins rising from 57.14 to 66.67 on HumanEval and QW2.5-7b-Ins rising from 18.18 to 27.27 on OpenEval.Source: Table III · p. 5, B. RQ2: Performance Comparison in Code… · p. 6
  • Without defense, CoT generally improved Pass@1, but SABER attacks decreased or left scores unchanged, with unchanged scores still indicating potential risk.For example, DS-6.7b-Ins increased from 71.43 to 76.19 on HumanEval with CoT.Source: B. RQ2: Performance Comparison in Code… · p. 6

VII. THREATS TO VALIDITY

The paper identifies internal, external, and construct threats to validity and reports that GUARD combines detection and secure CoT regeneration while outperforming existing methods in security and quality preservation. It plans to extend evaluation to more attacks, models, benchmarks, defenses, and reasoning domains.

VII. THREATS TO VALIDITY

  • The validity analysis identifies internal threats from experimental setup and parameter selection, external threats from model and dataset choices, and construct threats from performance-measure suitability.The authors report multiple random-seed runs and hyperparameter tuning, diverse models and experiments, and a comprehensive set of metrics to address these threats.Source: §VII · p. 6

VIII. CONCLUSION

  • GUARD combines detection and secure CoT regeneration, outperforming existing methods in both security and quality preservation.The conclusion describes GUARD-Judge for detection and GUARD-Repair for secure CoT regeneration.Source: §VIII · p. 6
  • The authors plan to extend GUARD to more attack scenarios, models, and benchmarks, integrate it with other defenses, and apply it to logical reasoning tasks.Source: §VIII · p. 6
Loading 2505.21425v3…