PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond

Chongyu Fan, Jinghan Jia, Yihua Zhang, Anil Ramakrishna, Mingyi Hong, Sijia Liu

arXiv:2502.05374v4cs.LGcs.CL

TL;DR

LLM unlearning can remove undesirable data influence, but lightweight relearning attacks may recover forgotten knowledge from a few forget samples. The paper connects robust unlearning to SAM through min-max optimization and finds that SAM and other smoothness methods improve resistance to relearning and input-level jailbreaking attacks.

  • Problem

    Existing LLM unlearning methods can lack robustness because small-sample relearning attacks recover knowledge that was already unlearned.

  • Method

    The paper formulates robust unlearning as min-max optimization and investigates SAM, curvature regularization, and other smoothness-based strategies.

  • Results

    Smoothness-enhanced unlearning consistently improves robustness across evaluated settings, with SAM-based methods emerging as particularly effective against relearning and input-level jailbreaking attacks.

  • Takeaways & Limitations

    Smoothness optimization provides a practical direction for strengthening LLM unlearning robustness while preserving unlearning effectiveness and utility without relearning attacks.

  • Takeaways & Limitations

    The robust formulation limits the adversary to bounded weight perturbations ∥δ∥p ≤ ρ with small ρ > 0, and the analysis assumes a nonzero forget-loss gradient.

Abstract

from arXiv · show

The LLM unlearning technique has recently been introduced to comply with data regulations and address the safety and ethical concerns of LLMs by removing the undesired data-model influence. However, state-of-the-art unlearning methods face a critical vulnerability: they are susceptible to ``relearning'' the removed information from a small number of forget data points, known as relearning attacks. In this paper, we systematically investigate how to make unlearned models robust against such attacks. For the first time, we establish a connection between robust unlearning and sharpness-aware minimization (SAM) through a unified robust optimization framework, in an analogy to adversarial training designed to defend against adversarial attacks. Our analysis for SAM reveals that smoothness optimization plays a pivotal role in mitigating relearning attacks. Thus, we further explore diverse smoothing strategies to enhance unlearning robustness. Extensive experiments on benchmark datasets, including WMDP and MUSE, demonstrate that SAM and other smoothness optimization approaches consistently improve the resistance of LLM unlearning to relearning attacks. Notably, smoothness-enhanced unlearning also helps defend against (input-level) jailbreaking attacks, broadening our proposal's impact in robustifying LLM unlearning. Codes are available at https://github.com/OPTML-Group/Unlearn-Smooth.

1. Introduction

LLM unlearning reduces undesired data influence without retraining from scratch, but relearning attacks can quickly recover removed knowledge. The paper frames robust unlearning as min-max optimization and investigates SAM and other smoothness methods as defenses.

  • Motivation: LLM unlearning mitigates undesirable data influence and associated harmful capabilities while preserving model utility after unlearning.It is proposed as a post-pretraining alternative when full retraining is computationally and temporally infeasible.
  • Problem: Relearning attacks can reverse unlearning through lightweight fine-tuning on only a small number of forget samples.This vulnerability motivates robustness beyond conventional unlearning effectiveness.
  • Research question: The paper asks for a robust optimization foundation for LLM unlearning against relearning attacks.The question is posed explicitly as the basis for reexamining unlearning objectives.
  • Approach: A min-max formulation models unlearning as minimization and relearning as worst-case weight perturbation, connecting robust unlearning to SAM.Unlike adversarial training’s input perturbations, the maximization step perturbs model weights to reverse unlearning effects.
  • Contributions: The paper identifies smoothness optimization as central to robustness and studies SAM, curvature regularization, and related techniques beyond SAM.It presents SAM as a robust optimization framework and examines broader smoothness strategies for resisting relearning and jailbreaking attacks.

2. Related Work

Prior work develops LLM unlearning through model-based and input-based strategies, while recent studies expose relearning and jailbreaking vulnerabilities. SAM and related smoothness methods provide the paper’s methodological foundation for addressing these weaknesses.

  • LLM unlearning: Machine unlearning removes undesirable data influence to mitigate privacy risks and harmful-content generation while protecting sensitive or copyrighted information.LLM approaches include model optimization and input-based prompting or in-context learning.
  • Unlearning vulnerabilities: Existing unlearning systems face relearning attacks that restore forgotten knowledge using fine-tuning on even a small subset of forget samples.Related work also documents jailbreaking attacks that exploit adversarial inputs.
  • SAM and smoothness: SAM minimizes loss and sharpness to encourage a flatter loss landscape and has also been applied to defend against input-level adversarial attacks.This makes SAM a natural bridge between generalization-oriented smoothness and robustness-oriented defenses.
  • Beyond SAM: Gradient penalty, curvature regularization, randomized smoothing, and weight averaging are additional approaches for encouraging smoothness.They act through gradient or Hessian-based penalties, objective convolution, or averaging weights across training iterations.

3. LLM Unlearning and Relearning Attacks

The paper formulates unlearning and relearning as linked optimization problems and uses a motivating WMDP Bio example to expose relearning vulnerability. Its robust formulation treats relearning as bounded model perturbation, motivating SAM-based defenses.

  • Unlearning formulation: The unlearning objective updates pretrained parameters using forget and retain datasets, balancing removal of undesired influence with retained model utility.The forget set defines the unlearning scope, while the retain loss preserves performance on retained data.
  • Relearning attacks: Relearning attacks fine-tune the unlearned model on a much smaller subset of forget data using an objective that counteracts forgetting.The attack update is represented by δ, and the relearn objective may use negative forget loss or standard fine-tuning loss.
  • Motivating example: NPO unlearning raises WMDP Bio unlearning effectiveness and suppresses hazardous responses before attack, but a single epoch on 20, 40, or 60 forget samples can restore harmful behavior.UE is defined as 1-Accuracy on WMDP, so higher UE indicates better unlearning.
  • Robust optimization: Robust unlearning is formulated as a min-max game in which bounded weight perturbations simulate relearning attempts against the unlearning objective.The perturbation constraint is ∥δ∥p ≤ ρ with small ρ > 0, using p = 2 by default.
  • Connection to SAM: This formulation aligns with SAM because SAM evaluates forget-loss sensitivity to weight perturbations and encourages smoothness.The paper consequently investigates how SAM and broader smoothness optimization affect resistance to relearning attacks.

4. Enhancing Unlearning Robustness: From SAM to Broader Smoothness Optimization

The paper connects SAM-based robust unlearning to smoothness optimization and extends this perspective to randomized smoothing, gradient penalties, curvature regularization, and weight averaging. These approaches flatten or smooth the forget-loss landscape and improve resistance to relearning attacks without compromising unlearning effectiveness in the non-adversarial setting.

  • SAM and curvature: SAM connects robust unlearning to smoothness optimization by seeking low forget loss across a neighborhood of model parameters.This promotes flatness in the forget-loss landscape.
  • SAM and curvature: The SAM perturbation is obtained by linearizing the forget loss, with the maximizing update aligned to the forget-loss gradient and having magnitude ρ.The approximation uses a first-order Taylor expansion, while the exact direction follows from maximizing cosine similarity.
  • SAM and curvature: SAM optimization implicitly involves the forget-loss Hessian, linking the procedure to curvature reduction and smoother loss surfaces.The Hessian enters through differentiation of the gradient norm; the analysis assumes the forget-loss gradient is nonzero.
  • Broader smoothness optimization: The paper extends smoothness optimization beyond SAM to randomized smoothing, gradient penalty, curvature regularization, and weight averaging.RS smooths objectives through Gaussian convolution, GP penalizes gradient variation, CR explicitly penalizes curvature, and WA averages checkpoints.
  • Broader smoothness optimization: Weight averaging enforces parameter smoothness by averaging multiple checkpoints collected during training, with n specifying the number of past checkpoints.The method is applied to checkpoints along the SAM-based unlearning trajectory.
  • Empirical robustness: Under Relearn20, NPO+SAM reaches UE 0.70 versus 0.57 for vanilla NPO, while non-adversarial Unlearn UE remains around 0.74 across smoothing variants.The loss landscapes of smooth variants are also presented as flatter than vanilla NPO around the unlearned model.

5. Experiments

Experiments evaluate smoothness-enhanced unlearning on WMDP and MUSE using multiple relearning settings, methods, datasets, and attacks. SAM consistently improves robustness while preserving ordinary unlearning effectiveness and utility, and also strengthens resistance to jailbreaking attacks.

  • Experimental setup: Experiments use WMDP and MUSE, with UE and utility retention evaluated across unlearning baselines and smoothness methods.WMDP uses 1-Accuracy for UE and MMLU accuracy for utility; MUSE uses KnowMem and VerbMem, where lower values indicate better unlearning.
  • WMDP robustness: All SAM-integrated variants improve robustness over their non-SAM counterparts without compromising UE or UT absent relearning attacks.The comparison includes NPO, GradDiff, and RMU; RMU-type methods retain higher UT, but are less robust than NPO+SAM.
  • Comparison with robust methods: On LLaMA-3 8B, NPO+SAM matches TAR and significantly outperforms vanilla NPO and RMU-LAT on WMDP while avoiding TAR’s multi-step gradient overhead.The comparison characterizes NPO+SAM as balancing unlearning efficacy, robustness, and efficiency.
  • WMDP robustness: NPO+SAM provides the strongest resilience among the evaluated smooth NPO variants as relearning epochs or forget-sample counts increase.Vanilla NPO nearly returns to pre-unlearning performance for M ≥2 or N ≥40, while smooth variants remain more robust; larger sample counts reduce UE faster than additional epochs.
  • Cross-dataset robustness: NPO+SAM consistently outperforms vanilla NPO after relearning attacks using AGNews, GSM8K, and SST2 relearn sets.The result indicates robustness beyond attacks using the original forget set.
  • MUSE evaluation: On MUSE Books and News, NPO+SAM lowers both KnowMem and VerbMem after attacks, while VerbMem changes more than KnowMem.The findings indicate that precise-token unlearning is more vulnerable to relearning than general-knowledge unlearning.
  • Jailbreaking robustness: NPO+SAM and NPO+RS retain lossless UE under jailbreaking attacks, whereas vanilla NPO suffers a significant UE drop.Token-level KL analysis links smoothness optimization to better unlearning of early tokens, addressing shallow unlearning alignment.
  • Hyperparameter ablation: SAM’s robustness benefit is limited when ρ is too small, while an excessively large ρ hinders unlearning effectiveness.The reported sensitivity study contrasts ρ = 0.001 with ρ = 0.1.

6. Conclusion

The paper shows that smoothness-enhanced LLM unlearning improves robustness against relearning attacks, with SAM particularly effective against both relearning and input-level jailbreaking attacks.

  • Smoothness-enhanced LLM unlearning significantly improves robustness against relearning attacks.
  • SAM-based unlearning emerges as a particularly effective defense against relearning and input-level jailbreaking attacks.

Impact Statement

The study frames smoothness optimization, including SAM, as a way to strengthen unlearning against relearning and jailbreaking attacks while supporting more reliable removal of undesired knowledge.

  • Smoothness optimization enhances LLM unlearning robustness against relearning and jailbreaking attacks.
  • SAM contributes to more reliable unlearning by reducing unintended knowledge retention and reinforcing model security.

B. Additional Visualization Results for Loss Landscape on Retain Set

Loss-landscape visualizations compare the original model, NPO-unlearned model, and smooth NPO variants on the retain set. The original and unlearned landscapes are similar, while smoothing changes the unlearned landscape.

  • The original and NPO-unlearned models have quite similar retain-set loss landscapes, suggesting minimal retain-set performance impact from unlearning.
  • The unlearning process primarily affects forget-data performance while having minimal impact on retain-set performance.
  • Figure A1 compares the prediction loss landscapes of the original model, NPO-unlearned model, and smooth NPO variants on the retain set.

C. Detailed Experiment Setups

The experiments evaluate unlearning on WMDP and MUSE-related datasets using specified original models, training schedules, and hyperparameter searches for baseline and smoothing methods.

  • WMDP experiments use Zephyr-7B-beta with biosecurity-related forget data and unrelated Wikitext retain data.NPO and GradDiff use 125 unlearning steps, while RMU uses 150 steps.
  • WMDP hyperparameter searches tune learning rate, λ, β, ρ, and γ across specified ranges for baseline and smoothing methods.NPO + SWA averages models starting at 100 steps and every five steps thereafter.
  • MUSE experiments use LLaMA-2 7B for BBC news and ICLM 7B for Harry Potter books.NPO is trained for 10 epochs at a learning rate of 10^-5 with β = 0.1, while SAM searches ρ in [10^-3, 10^-1].

D. Additional Results on WMDP

On WMDP, SAM consistently strengthens the robustness of NPO, GradDiff, and RMU against relearning attacks without materially changing pre-attack unlearning performance. Increasing the number of perturbed parameters also improves RMU robustness.

  • Robustness comparison: SAM consistently improves the relearning robustness of NPO, GradDiff, and RMU compared with their vanilla versions across attack settings.The settings vary the number of relearning samples and epochs; NPO+SAM achieves the highest robustness.
  • Robustness comparison: SAM preserves pre-attack unlearning performance, with UT and UE remaining largely unchanged after its application.This comparison concerns the WMDP results before relearning attacks.
  • Parameter count and robustness: Figure A2 compares RMU and SAM-enhanced RMU loss landscapes on the forget set and their robustness under a 20-sample, 3-epoch WMDP relearning attack.Numbers in the loss-landscape panels indicate the layers using SAM.
  • Parameter count and robustness: Increasing the number of perturbed parameters in smoothness optimization produces greater robustness against relearning attacks.The comparison uses an RMU-unlearned model under a WMDP Bio attack with 20 forget samples and 3 training epochs.

E. Additional Results on MUSE

On MUSE News and Books, SAM substantially improves resistance to relearning attacks while leaving pre-attack unlearning metrics nearly unchanged. The hyperparameter ρ requires balance: intermediate perturbations improve robustness without disrupting forgetting.

  • Unlearning performance and robustness: SAM substantially improves NPO robustness on MUSE while leaving pre-attack KnowMem and VerbMem metrics almost identical.The evaluation covers News and Books under the reported MUSE unlearning settings.
  • Unlearning performance and robustness: On MUSE News, the VerbMem difference on Df is 51.47 for NPO+SAM versus 56.57 for NPO after the relearning attack.Smaller discrepancies between no attack and post-attack metrics indicate greater robustness in the reported comparison.
  • Unlearning performance and robustness: Table A2 compares NPO and NPO+SAM before and after relearning attacks on News and Books using two reported model and dataset settings.The settings are LLaMA2-7B on News and ICLM-7B on Books.
  • SAM hyperparameter ρ: At ρ = 0.01, SAM achieves the best balance between effective forgetting and improved relearning robustness on MUSE Books.The balance is reflected in smaller post-attack changes in KnowMem and VerbMem on Df.
  • SAM hyperparameter ρ: A small ρ = 0.001 provides limited attack mitigation, whereas a large ρ = 0.1 disrupts unlearning through excessive parameter perturbations.The ablation identifies both insufficient and excessive perturbation as undesirable regimes.

F. Generation Examples under Jailbreaking Attack

Generation examples on WMDP Bio show that NPO+SAM remains different from the target labels under adversarial prompts, whereas identical-to-label responses indicate unlearning failure. The examples include original, NPO-unlearned, and NPO+SAM-unlearned models.

  • Generation examples: NPO+SAM consistently generates responses different from the label under jailbreaking attacks, with or without an adversarial prompt.Responses differing from the label are marked as successful unlearning in the table.
  • Generation examples: The comparison includes the original model, NPO-unlearned model, and NPO+SAM-unlearned model alongside ground-truth labels.Table A4 evaluates examples from the WMDP Bio set with and without adversarial prompts.
  • Generation examples: Responses identical to the label are marked as unlearning failures, while responses differing from the label are marked as successful unlearning.The table uses red for failures and green for successful unlearning.
  • Generation examples: The WMDP Bio examples include questions about heavy-metal hyperaccumulation and virulence-factor transfer between microbes.These examples illustrate the evaluated biology questions rather than additional aggregate performance results.
Loading 2502.05374v4…