PaperlayerView exact revision on arXiv

Source-linked AI summary

PEFT-as-an-Attack! Jailbreaking Language Models during Federated Parameter-Efficient Fine-Tuning

Shenghui Li, Edith C. -H. Ngai, Fanghua Ye, Thiemo Voigt

arXiv:2411.19335v2cs.CRcs.AI

TL;DR

FedPEFT's jailbreak risk is underexplored despite existing safety alignment and defenses against malicious updates. This paper introduces PaaA and evaluates PEFT attacks and defenses. The attack can exceed 80% success, while defenses face security-utility limitations.

  • Problem

    The effectiveness of robust federated-learning defenses against jailbreak attacks on safety-aligned PLMs had not been explored.

    Source: §I · pp. 1–2

  • Method

    The paper studies PaaA by evaluating three PEFT methods and investigates robust aggregation schemes and Post-PEFT Safety Alignment as defenses.

    Source: §I · p. 2

  • Results

    Above 80% attack success is achieved with less than 1% of parameters trainable and a small malicious-client subset; under high data heterogeneity, advanced RASs struggle, while PPSA reduces attack success below 10%.

    Source: §I · p. 2

  • Takeaways & Limitations

    The findings highlight the need for more robust and adaptive defenses that address safety-utility trade-offs in FedPEFT.

    Source: §I · p. 2, §VII · p. 9

    PPSA's attack reduction comes at the expense of downstream target-task accuracy.

    Source: D. Defense with PPSA · p. 9

Abstract

from arXiv · show

Federated Parameter-Efficient Fine-Tuning (FedPEFT) has emerged as a promising paradigm for privacy-preserving and efficient adaptation of Pre-trained Language Models (PLMs) in Federated Learning (FL) settings. It preserves data privacy by keeping the data decentralized and training the model on local devices, ensuring that raw data never leaves the user's device. Moreover, the integration of PEFT methods such as LoRA significantly reduces the number of trainable parameters compared to fine-tuning the entire model, thereby minimizing communication costs and computational overhead. Despite its potential, the security implications of FedPEFT remain underexplored. This paper introduces a novel security threat to FedPEFT, termed PEFT-as-an-Attack (PaaA), which exposes how PEFT can be exploited as an attack vector to circumvent PLMs' safety alignment and generate harmful content in response to malicious prompts. Our evaluation of PaaA reveals that with less than 1% of the model's parameters set as trainable, and a small subset of clients acting maliciously, the attack achieves an approximate 80% attack success rate using representative PEFT methods such as LoRA. To mitigate this threat, we further investigate potential defense strategies, including Robust Aggregation Schemes (RASs) and Post-PEFT Safety Alignment (PPSA). However, our empirical analysis highlights the limitations of these defenses, i.e., even the most advanced RASs, such as DnC and ClippedClustering, struggle to defend against PaaA in scenarios with highly heterogeneous data distributions. Similarly, while PPSA can reduce attack success rates to below 10%, it severely degrades the model's accuracy on the target task. Our results underscore the urgent need for more effective defense mechanisms that simultaneously ensure security and maintain the performance of the FedPEFT paradigm.

I. INTRODUCTION

FedPEFT combines efficient adaptation with decentralized training, but whether its defenses can withstand jailbreak attacks by malicious participants has been unclear. This paper introduces PEFT-as-an-Attack (PaaA), evaluates three PEFT methods, and finds substantial attack success alongside limitations in existing defenses.

I. INTRODUCTION

  • FedPEFT distributes fine-tuning across decentralized devices that share model updates rather than raw data, reducing data-centralization needs and communication and computation costs.PEFT tunes a small parameter subset while freezing most pretrained weights; LoRA and variants can use less than 1% of model parameters.Source: §I · p. 1
  • The paper identifies an open security question: jailbreak attacks threaten safety-aligned PLMs, while the effectiveness of robust aggregation against this threat had not been explored.Federated training's multiple participants create additional opportunities for adversarial attacks.Source: §I · pp. 1–2
  • With less than 1% of model parameters trainable, a small malicious-client subset achieves attack success rates above 80% through PEFT-based safety-alignment bypass.The attack induces harmful content in response to malicious prompts.Source: §I · p. 2
  • Advanced RASs struggle against PaaA under highly heterogeneous data, while PPSA can reduce attack success below 10% but sacrifices target-task accuracy.The paper names DnC and ClippedClustering among the RASs that struggle.Source: §I · p. 2
  • The evaluation studies LoRA, (IA)3, and LayerNorm across varied federated-learning scenarios for susceptibility to PaaA.The paper also investigates RASs and Post-PEFT Safety Alignment as defenses.Source: §I · p. 2
  • The implementation is integrated into Blades, a benchmark suite for federated-learning security research, to support reproducibility and further exploration.Source: §I · p. 2

II. PRELIMINARIES

The preliminaries review pretrained language models and parameter-efficient fine-tuning. They describe transformer components and three PEFT methods that adapt models through low-rank updates, activation scaling, or normalization parameters.

A. Pre-trained Language Models

  • PLMs transfer pretrained language understanding to downstream tasks; contemporary architectures predominantly use transformers with multi-head attention and feedforward blocks.Source: A. Pre-trained Language Models · p. 2

B. Parameter-Efficient Fine-Tuning

  • PEFT tunes a small parameter subset while freezing most parameters; this paper focuses on LoRA, (IA)3, and LayerNorm.The methods' architectures are illustrated in Figure 1.Source: B. Parameter-Efficient Fine-Tuning · p. 3
  • LoRA approximates weight updates with low-rank trainable matrices, using k(mi + ni) parameters rather than mi × ni for a full weight matrix.The low-rank updates can be applied selectively to matrices in attention or feedforward components.Source: B. Parameter-Efficient Fine-Tuning · p. 3
  • (IA)3 modulates internal activations with learned scaling factors that can amplify or inhibit dimensions.Its activation function is used for feedforward networks, while the attention-module mapping is the identity.Source: B. Parameter-Efficient Fine-Tuning · p. 3
  • LayerNorm fine-tunes normalization-layer parameters; the paper uses RMSNorm, where g_i is a learnable scaling parameter.This differs from classic LayerNorm and follows the RMSNorm variant used in the studied PLMs.Source: B. Parameter-Efficient Fine-Tuning · p. 3

III. SYSTEM MODEL

The system model uses federated clients and a central server to adapt pretrained language models with PEFT. Clients train on decentralized domain-specific data, update PEFT parameters locally, and exchange compact updates while pretrained parameters remain frozen.

III. SYSTEM MODEL

  • The system-model overview identifies five numbered components, and the federated system comprises multiple clients collaborating with one central server.Source: §III · p. 3, A. FedPEFT System · p. 3

A. FedPEFT System

  • The global federated objective is expressed across clients, with each client's expected risk defined over its local data distribution and user-specified loss.K denotes the total number of clients.Source: A. FedPEFT System · p. 3
  • In instruction tuning, clients update only PEFT parameters while keeping pretrained parameters frozen, and communicate compact updates rather than full-model parameters.The model uses decentralized domain-specific datasets with single-turn question-answering examples.Source: A. FedPEFT System · p. 3

1) FedPEFT Client:

FedPEFT clients receive shared PEFT parameters, perform multiple local optimization steps, and return their parameter updates. The base model remains shared while clients maintain local PEFT modules.

1) FedPEFT Client:

  • Clients receive the global PEFT parameters and perform multiple local update steps while maintaining local PEFT modules over a shared frozen base model.Local optimization may use SGD, Adam, or AdaGrad.Source: 1) FedPEFT Client · p. 3
  • After local training, each client computes and transmits its parameter updates to the server.The learning rate is denoted by η, and s indexes the local optimization step.Source: 1) FedPEFT Client · p. 3
  • Clients collaboratively fine-tune a PLM using small PEFT modules and local datasets within the server-coordinated FedPEFT system.The server aggregates local models and broadcasts the resulting model to devices each training round.Source: Figure 2 · p. 4

2) FedPEFT Server:

The server initializes and broadcasts global PEFT parameters, selects participating clients each round, and aggregates their updates with FedAvg to produce new global parameters.

2) FedPEFT Server:

  • The server aggregates participating clients’ updates using FedAvg to obtain new global parameters.This is the server’s update step for the global PEFT parameters.Source: 2) FedPEFT Server · p. 4
  • At each round, the server selects participating clients and broadcasts the current global parameters to them.The participants are a subset of all clients.Source: 2) FedPEFT Server · p. 4
  • The server initializes the global PEFT parameters at the start of training.Source: 2) FedPEFT Server · p. 4

B. Threat Model

The threat model considers adversaries who poison local PEFT data while following the standard FedPEFT protocol, aiming to bypass safety guardrails. The paper also examines RASs and PPSA as potential defenses.

B. Threat Model

  • Adversaries inject toxic training data during local fine-tuning to compromise PLM safety guardrails and induce harmful, biased, or malicious outputs.Source: B. Threat Model · p. 4
  • The attack seeks harmful outputs for harmful instructions while keeping benign-task performance unaffected to evade standard validation.Adversaries must follow FedPEFT communication and local training protocols and cannot modify frozen base-model parameters.Source: B. Threat Model · p. 4
  • The threat model assumes an honest server, a majority of honest clients, and securely frozen PLM parameters across clients.Source: B. Threat Model · p. 4

C. Defense Mechanisms

  • The paper examines robust aggregation and post-PEFT safety alignment as two potential defense directions against PaaA.RASs target malicious updates, while PPSA seeks to restore safety guardrails after fine-tuning.Source: C. Defense Mechanisms · p. 4

1) Robust Aggregation Schemes:

  • RASs filter or exclude malicious updates, but their effectiveness against PaaA had not yet been examined.Their prior success concerned poisoning attacks in conventional FL settings.Source: 1) Robust Aggregation Schemes · pp. 4–5

2) Post-PEFT Safety Alignment:

PPSA adds safety-focused fine-tuning after federated PEFT, and the experiments describe the models, adaptation tasks, client setup, and defenses used to study it.

2) Post-PEFT Safety Alignment:

  • PPSA applies additional fine-tuning on a curated safety dataset after federated PEFT to restore adherence to the model’s safety guardrails.Source: 2) Post-PEFT Safety Alignment · p. 5
  • The paper notes that the examined conventional defenses were not specifically designed for PaaA.Their effectiveness and limitations are examined for insight into the challenge of defending against this threat.Source: 2) Post-PEFT Safety Alignment · p. 5

A. System Setup

  • 0.001%-0.59%: the selected PEFT methods’ trainable-parameter ratios span this range across the experimental PLMs.The study uses four open-source PLMs and evaluates LoRA, (IA)3, and LayerNorm; models are 4-bit quantized during fine-tuning.Source: A. System Setup · p. 5
  • Experiments adapt models on MetaMathQA and MedQA, using sampled subsets rather than necessarily the full datasets or convergence training.Source: A. System Setup · p. 5
  • The simulated system has 15 clients, with zero to five malicious clients; all clients participate each round and IID data is the default.Source: A. System Setup · p. 5
  • The evaluated defenses include Median, GeoMed, DnC, and ClippedClustering, while PPSA uses alignment data from CAI-Conversation-Harmless.Source: A. System Setup · p. 5

B. Training Details

The experiments use federated PEFT training and evaluate both model utility and safety. The setup specifies training tools and settings, inference acceleration, and a benign MedQA comparison.

B. Training Details

  • The study uses the Blades benchmark suite for adversarial FL, Hugging Face PEFT and Unsloth for local training, and 4-bit quantization on Nvidia Tesla A100 GPUs.Training uses AdamW, batch size 4, 25 communication rounds with FedAvg, 50 local steps per round, and a learning rate of 0.001.Source: B. Training Details · p. 5

C. Evaluation Details

  • Safety and utility evaluations use vLLM to accelerate inference.Source: C. Evaluation Details · pp. 5–6
  • Without malicious clients, Figure 3 compares three FedPEFT methods across 25 MedQA communication rounds; LoRA improves accuracy by 28–67%, while LayerNorm and (IA)3 vary across PLMs.Source: Figure 3 · p. 6

1) Safety:

The experiments measure task utility and jailbreak risk, then assess defenses against PaaA. LoRA improves MedQA accuracy but shows pronounced vulnerability, and robust aggregation struggles under mixed-domain data.

1) Safety:

2) Utility:

  • Utility is measured as accuracy on the MedQA and MetaMathQA test splits by comparing parsed predictions with their unique correct answers.Source: 2) Utility · p. 6

V. EXPERIMENTAL RESULTS AND ANALYSIS

  • The experiments proceed from benign FedPEFT effectiveness to PaaA impacts and then evaluate RAS and PPSA defenses.Source: §V · p. 6

A. Learning Effectiveness of FedPEFT Methods

  • On benign MedQA, LoRA improves accuracy by 28% to 67%, whereas (IA)3 and LayerNorm have variable effects and can lower accuracy versus the base models.LoRA typically has more trainable parameters than the other methods, which may contribute to its effectiveness.Source: A. Learning Effectiveness of FedPEFT… · p. 6
  • Accuracy gains are particularly pronounced for LLaMA-2-7B-Chat and less so for three other models; the authors suggest newer models may already have MedQA knowledge.They suggest FedPEFT’s advantages could be more substantial on private datasets to which models are less likely to have prior exposure.Source: A. Learning Effectiveness of FedPEFT… · p. 6

B. Comparison of FedPEFTs under PaaA

  • LoRA shows the sharpest vulnerability increase, with ASRs exceeding 70% by Round 20 in most cases, whereas (IA)3 and LayerNorm typically have ASRs of 40%–60%, despite LoRA’s superior target-task performance.Source: B. Comparison of FedPEFTs under PaaA · p. 6
  • With three malicious clients, all evaluated RASs fail in the mixed-domain setting, although ClippedClustering defends against the attack for single-domain MedQA.The comparison uses Phi-3.5-Mini-Instruct with LoRA over 20 communication rounds.Source: Table II · p. 8

C. Defense with RASs

The evaluation finds that robust aggregation schemes (RASs) can lose effectiveness against PaaA under heterogeneous client data. Post-PEFT Safety Alignment (PPSA) reduces attack success rates, but can reduce downstream task accuracy.

C. Defense with RASs

  • DnC and ClippedClustering achieve ASR ≤2% under MedQA-only data but exceed 80% ASR with heterogeneous clients; Median and GeoMed remain above 30.7% across settings.The heterogeneous setting evenly splits benign clients between MedQA and MetaMathQA.Source: C. Defense with RASs · p. 8

D. Defense with PPSA

  • 57% AdvBench ASR and 65% JailbreakBench ASR after the first five attack rounds fell below 10% after the final alignment rounds.The reported simulation used LoRA on MedQA, with alignment clients active in rounds 11–14.Source: D. Defense with PPSA · p. 8
  • PPSA’s reduced attack success comes with downstream accuracy losses, including up to 3% on MedQA for LLaMA-3.2-3B-Instruct and a 20% reduction during alignment.The authors describe this utility loss as an alignment tax and caution against PPSA when preserving downstream utility is critical.Source: D. Defense with PPSA · pp. 8–9

VI. RELATED WORK

Related work covers the efficiency and security of federated parameter-efficient fine-tuning, alongside fine-tuning-based jailbreak attacks and defenses. The paper positions its broader FedPEFT evaluation as addressing a gap in earlier work and calls for more robust defenses.

A. Federated Parameter-Efficient Fine-Tuning

  • FedPEFT combines parameter-efficient fine-tuning and federated learning to adapt pretrained models while preserving privacy and reducing communication and computation costs.Prior studies also examine comparative PEFT performance, resource efficiency, personalization, and adaptation to client resource heterogeneity.Source: A. Federated Parameter-Efficient… · p. 9

B. Fine-Tuning-based Jailbreak Attacks and Defenses

  • Prior work studies fine-tuning jailbreaks against aligned PLMs and defenses at alignment, fine-tuning, and post-fine-tuning stages.Examples include alignment-stage perturbation, fine-tuning-stage regularization or mixed data, and post-fine-tuning safety alignment.Source: B. Fine-Tuning-based Jailbreak Attacks… · p. 9
  • The earlier federated jailbreak study [40] focused on LoRA with LLaMA-2, whereas this work evaluates multiple PEFT methods, PLMs, FL settings, and defenses.The authors use this broader scope to draw conclusions across scenarios and expose limitations of current defenses.Source: B. Fine-Tuning-based Jailbreak Attacks… · p. 9

VII. CONCLUSIONS AND FUTURE WORK

  • The paper concludes that PaaA can bypass safety alignment and that existing defenses have limitations, motivating more robust, adaptive mechanisms.Future work proposed includes advanced PPSA and integrating safety alignment during fine-tuning while maintaining model performance.Source: §VII · p. 9
Loading 2411.19335v2…