PaperlayerView exact revision on arXiv

Source-linked AI summary

Distance Is Not Enough: Forget-Retain Alignment Gap Predicts LLM Relearning Robustness

Yi Chen, Hanna Hsieh, Shuhong Liu, Chuanbo Hua, Zihan Ma, Kun Wang, Joo-Young Kim

arXiv:2608.25429v1cs.AIcs.LG

TL;DR

LLM unlearning may not remain effective because modest fine-tuning can revive forgotten knowledge, while global weight distance does not reveal update selectivity. The paper introduces FRAG, a training-free forget–retain alignment predictor, and FRP, a pruning method based on the same principle; experiments support weight selectivity as a more reliable basis for predicting and improving relearning robustness than distance alone.

  • Problem

    LLM unlearning can fail when subsequent fine-tuning revives forgotten knowledge, and global distance does not capture which weights were changed.

  • Method

    FRAG predicts relearning robustness from alignment with forget-critical and avoidance of retain-critical weights, while FRP enforces this structure through selective pruning.

  • Results

    Forget–retain alignment provides a more reliable basis than distance alone for predicting and improving relearning robustness across the evaluated settings.

  • Takeaways & Limitations

    Which weights move matters more than distance alone for understanding and improving relearning robustness.

  • Takeaways & Limitations

    The evaluation covers several datasets, model families, and scales, but broader benchmarks, multilingual data, and larger architectures are needed; FRAG also ranks dense methods only coarsely.

Abstract

from arXiv · show

Machine unlearning aims to make a model forget specific data, yet unlearned LLMs often fail to stay unlearned: brief fine-tuning can revive removed knowledge. Existing robustness predictors rely on global weight-space displacement, but distance alone can be misleading when random or destructive updates collapse performance. We argue that relearning robustness depends on update structure: robust unlearning should affect forget-critical weights while sparing retain-critical ones. We introduce the Forget-Retain Alignment Gap (FRAG), a training-free predictor that scores an update's forget-retain alignment without running a relearning attack, and separates selective from dense updates more reliably than global distance. Building on the forget-critical, retain-sparing principle, Forget-Retain Pruning (FRP) improves relearning robustness. Our results suggest that weight selectivity better explains robustness than distance alone. Code is available at https://github.com/Yi1-Chen/FRAG.

1 Introduction

LLM unlearning can fail under relearning attacks, and global weight distance is insufficient because it ignores which weights change. The paper introduces FRAG and FRP to use forget–retain update selectivity for prediction and improvement.

  • Unlearned LLMs can recover forgotten knowledge after fine-tuning on benign retain data, so edit-time forgetting does not ensure relearning robustness.
  • Global ℓ2 distance can misclassify random or destructive updates because it measures displacement magnitude without identifying which weights move.Such updates may damage retain or forget performance while appearing robust to distance-based predictors.
  • FRAG is a training-free proxy that measures whether updates align with forget-critical rather than retain-critical weights while penalizing retain-side disruption.
  • Across diverse methods, benchmarks, and model families, FRAG correlates with empirical relearning robustness more reliably than global distance-based predictors.
  • FRP applies the same selective principle and improves robustness under relearning attacks along a robustness–utility frontier that dominates strong baselines at matched utility levels.

2 Related Work

Prior LLM unlearning methods use loss optimization, representation engineering, direct weight edits, or inference-time interventions, while relearning studies expose recovery after modest training. Existing defenses and pruning methods do not target the forget–retain alignment structure emphasized here.

  • LLM unlearning methods include likelihood suppression, preference optimization, representation engineering, direct weight editing, retain-set dropping, and inference-only interventions.
  • Relearning studies show that unlearned LLMs can recover forgotten knowledge under modest extra training, motivating defenses such as sharpness-aware and localized edits.
  • Weight-importance and pruning research scores parameters using activations, relative importance, second-order reconstruction, Fisher information, gradients, or attribution.
  • Existing unlearning pruning and attribution methods do not target the forget–retain alignment structure governing relearning robustness.

3 Method

The method defines relearning robustness through selective weight updates: changes should target forget-critical weights while sparing retain-critical ones. FRAG predicts robustness from this structure, and FRP constructs such updates through rank-based pruning.

  • 3.1 Problem Formulation: The framework compares original and unlearned models, then considers relearning attacks that fine-tune the unlearned model on forget, retain, or mixed attack sets.
  • 3.1 Problem Formulation: The scoring goal is to predict robustness without an attack while rejecting utility-collapsed models as meaningfully robust.
  • 3.2 Predicting Robustness: FRAG: FRAG compares squared updates with forget- and retain-critical importance using scale-invariant cosine alignment, rather than update magnitude alone.
  • 3.2 Predicting Robustness: FRAG: Forget-critical and retain-critical weights are relative, data-dependent importance categories determined by weight magnitude and input-channel activation.Nearly every weight can carry importance for both sets; the relevant quantity is their ratio.
  • 3.2 Predicting Robustness: FRAG: A high FRAG means alignment with forget-critical weights and avoidance of retain-critical weights; its retain term prevents destructive forget-only updates from being rewarded.
  • 3.2 Predicting Robustness: FRAG: FRAG can anticipate retain-only relearning because weights activated by forgotten content but not retain content are inert under that attack, although the first-order argument weakens when forget data is also available.
  • 3.2 Predicting Robustness: FRAG: FRAG separates selective from dense updates reliably, but ranks dense methods only coarsely because diffuse updates align weakly.
  • 3.3 Achieving Robustness: FRP: FRP prunes weights that are forget-important, retain-unimportant, and sufficiently large to induce a meaningful edit.

4 Experiments

Experiments evaluate FRAG and FRP across multiple datasets, models, attacks, and checkpoints. FRP improves relearning robustness, while FRAG tracks robustness more reliably than global distance, including when collapsed controls are excluded.

  • Evaluation Setups: Experiments cover TOFU on LLaMA-3.2-1B/3B and WMDP-cyber on Qwen2.5-14B-Instruct, using retain, forget, and forget+retain relearning attacks.The evaluation uses OpenUnlearning and includes forget01/05/10 splits for TOFU.
  • Predictor Analysis: FRAG peaks on the low-ES, retain-sparing FRP checkpoint, whereas global ℓ2 and individual cosine terms peak on utility-collapsed perturbations.The curves are normalized by their own mean and standard deviation across unlearned checkpoints and noise controls.
  • Main Results: FRP achieves the best average post-attack ES and ∆ES on TOFU across both model sizes and all three relearning attacks.On WMDP-cyber, FRP also achieves the lowest cyber accuracy after retain-set relearning.
  • Main Results: FRP outperforms distance-based ranking despite SP producing much larger global ℓ2 distance, but FRP incurs a larger MMLU drop than RMU and SP.FRP therefore traces a robustness–utility frontier rather than dominating both axes.
  • Predictor Analysis: On healthy checkpoints, FRAG reaches Spearman ρ = −0.78 pooled versus −0.36 for global ℓ2, with the gap widening at 3B.Collapsed and noise controls are excluded from this comparison.
  • Predictor Analysis: Removing every FRP checkpoint leaves FRAG at −0.74, while global ℓ2 falls to −0.10 and reverses sign at 3B.This test rules out the explanation that FRAG’s ranking power comes only from scoring the method built on it.

5 Conclusion

The paper concludes that relearning robustness depends on which weights an unlearning update moves, not merely on total weight-space distance. FRAG provides a training-free predictor, while FRP applies the same forget-critical, retain-sparing principle to improve robustness.

  • 5 Conclusion: Robust unlearning depends on weight selectivity: updates should target forget-critical weights while sparing retain-critical weights.The conclusion contrasts update structure with distance alone.
  • 5 Conclusion: FRAG is a training-free predictor of forget-critical and retain-sparing updates, and FRP is a pruning-based application of that principle.Together they provide a basis for predicting and improving relearning robustness.

Limitations

The study’s evidence is bounded by its benchmark, language, model-scale, and method coverage. FRAG is also less discriminative among dense unlearning methods, while FRP is currently instantiated only as unstructured pruning.

  • Scope: The experiments cover TOFU, WMDP-cyber, and MUSE-News across several model families and scales, leaving broader benchmarks, multilingual data, and larger architectures for future study.These boundaries limit how complete the current picture is.
  • Predictor Scope: FRAG has limited resolution within dense unlearning methods because their scores are an order of magnitude smaller than those of selective edits.It separates dense from selective updates more sharply than it ranks dense methods among themselves.
  • Method Scope: FRP is instantiated as unstructured pruning, although the same principle may extend to structured pruning, low-rank editing, and other parameter-efficient interventions.Those extensions are proposed as future directions rather than evaluated conclusions.
  • Implementation Scope: FRAG’s computational recipe scores selected linear projections, including attention and MLP projections in Llama/Qwen models.Other architectures fall back to all nn.Linear modules.
  • Implementation Scope: FRAG uses calibration activations and weight updates, with cross-layer aggregation designed to preserve relative magnitude across layers.The implementation streams inner products and squared norms while keeping memory at O(dout×din).
  • Compute Cost: FRAG is substantially cheaper than running relearning attacks, assigning a continuous score from weights alone.On the reported hardware, it is approximately 60× cheaper at 1B and at least 15× cheaper at 14B.

A.2 Direction-blind Perturbation Ablation

Direction-blind noise can match or exceed FRP’s weight-space displacement while failing to unlearn, making distance-based robustness appear vacuous. FRAG distinguishes this failure by penalizing updates that disrupt retain-critical weights.

  • Perturbation setup: σ=0.004 noise yields L2≈404, close to FRP’s 443.4, despite perturbing every MLP weight without targeting.Attention layers remain untouched and no calibration data is used.
  • Distance versus direction: At σ=0.004, noise causes ∆Acc=−0.027 versus FRP’s −0.012, so distance or accuracy change alone can rank noise as at least as robust.The comparison uses the same WMDP-cyber panel.
  • Vacuous robustness: Noise leaves Ref accuracy near the reference and exceeds FRP on MMLU, indicating that it did not unlearn and has no forgotten knowledge for the attack to recover.Ref is 0.562 versus 0.583, while MMLU is 0.740 versus 0.668.
  • FRAG diagnosis: FRAG scores both noise settings at 0.39 (×100), below FRP’s 9.20, because random updates align similarly with forget- and retain-critical weights.With γ=0, forget alignment alone would reach approximately 38%, exceeding FRP’s 22%.
  • Evaluation protocol: The evaluation reports post-attack ES, ∆ES, and Utility for retain, forget, and forget+retain fine-tuning attacks.The attacker fine-tunes for one epoch at lr=10−5 with AdamW.

B.3 Ablation

The ablations show that FRP’s rank-space scoring preserves utility while magnitude-only alternatives collapse it, and that its mixing and sparsity controls trade forgetting against utility. Cross-family validation supports the distinction between selective alignment and inflated distance.

  • Scoring rule: All three magnitude-driven scoring baselines collapse utility to zero, whereas FRP preserves utility above 0.4 and earns a positive FRAG.Magnitude scores select the largest weights, which the retain set also relies on.
  • Mixing weight β: At fixed 3% sparsity, β produces a smooth, monotonic utility–forgetting trade-off; β=0.05 is the smallest value matching the Retain ESpre floor.The headline choice is documented in the β sweep.
  • Sparsity ρ: ESpre decreases from 0.163 to 0.038 as sparsity rises from 0.5% to 10%, while utility decreases from 0.544 to 0.143.The selected 3% operating point matches the Retain ESpre floor while preserving utility above 0.4.
  • Operating point: The joint setting (β, ρ)=(0.05, 3%) is selected as the smallest configuration meeting the target forgetting criterion along both axes.Increasing either control beyond the target costs retain-side utility.
  • Cross-family validation: On Zephyr-7B-β WMDP-cyber validation, FRAG and post-attack accuracy agree on the ordering, while ℓ2 is inflated by direction-blind noise.The table uses MMLU as the general-capability proxy.

C.2 Full TOFU Results

Table 9 separates the three relearning attacks and evaluates each forget split against its own retain-trained gold reference. The forget+retain attack most consistently exposes recovery, while FRP achieves the lowest attacked ES across splits and scales.

  • Table 9 separates retain-only, forget-only, and forget+retain attacks, with each forget split evaluated against its own retain-trained gold reference.
  • Under the forget+retain attack, FRP has the lowest attacked ES in every split at both evaluated scales.
  • The ordering is less stable under the weaker retain-only and forget-only attacks.
  • Forget+retain recovers the most for every method, so robustness under retain-only relearning does not guarantee resistance when forgotten data is also available.

C.3 Relearning Attack Variants

Across attack variants and controlled comparisons, FRP remains more resistant to relearning than competing methods, while the results show that attack strength and update selectivity materially affect interpretation. Matched-utility, sparsity, and update-norm analyses attribute the advantage to selective placement rather than simple intervention magnitude.

  • C.3 Relearning Attack Variants: Across eight effective attack variants, FRP has the lowest attacked ES among unlearned methods.Variants cover changes to attack configuration on TOFU forget10 with LLaMA-3.2-1B; ineffective settings were excluded when all ∆ES values stayed below 0.10.
  • Learning rate: At intermediate learning rates, FRP has lower attacked ES than NPO and RMU, whereas very small steps recover negligibly and 1e−4 steps substantially reduce utility.
  • Optimizer and attack data: Attack effectiveness depends strongly on optimizer and data: SGD recovers nothing, Adafactor collapses utility, and forget+retain is the strongest data mixture.
  • Horizon: Given enough epochs, every method eventually returns the knowledge, but FRP remains less recovered than NPO and RMU at every tested horizon.After 10 epochs, RMU reaches .999 and NPO .820 attacked ES, while FRP remains at .752.
  • Controlling confounds: FRP’s robustness advantage survives matched-utility comparisons, where it recovers 1.9–4.9× less than RMU.Within shared utility bands, utilities differ by at most 0.01 or FRP’s is higher.
  • Controlling confounds: RMU forgets more deeply than FRP (0.056 vs. 0.062 pre-attack ES) yet recovers about ten times as much, ruling out forgetting depth alone.
  • Controlling confounds: At matched sparsity, FRP achieves 2.6–3.3× lower ∆ES than Selective Pruning, while matched update norms still yield 2.2–2.9× lower ∆ES.The matched-norm gaps are at most 3%, so displacement magnitude alone does not explain the gain.
  • FRAG design checks: Replacing FRAG’s activation-based importance term with gradient-, curvature-, or influence-based alternatives weakens prediction, while linear mode connectivity provides almost no signal.The activation-based term is the strongest forward-only choice; on healthy non-SP checkpoints, connectivity curves show no upward bump.

C.6 Cross-Benchmark Check

On MUSE-News, a natural-news benchmark evaluated with LLaMA-2-7B, the ordering among utility-preserving methods carries over. FRP has the lowest relearning increase in extraction score among those methods.

  • The ordering among utility-preserving methods carries over from TOFU and WMDP-cyber to MUSE-News, while broader benchmarks and architectures remain future work.
  • On MUSE-News with LLaMA-2-7B, FRP has the lowest ∆ES among utility-preserving methods under a retain-only one-epoch attack.The benchmark uses natural news text and retain-ROUGE utility, with gold utility 0.557.
Loading 2608.25429v1…