PaperlayerView exact revision on arXiv

Source-linked AI summary

RepSelect: Robust LLM Unlearning via Representation Selectivity

Filip Sondej, Yushi Yang, Adam Mahdi

arXiv:2606.17168v2cs.CL

TL;DR

Deep LLM unlearning must remove targeted knowledge or behaviors while preserving general capabilities and resisting fine-tuning or prompting-based recovery. RepSelect isolates forget-specific representations, outperforming existing methods across scenarios and models while substantially reducing post-relearning accuracy.

  • Problem

    Existing unlearning methods suppress targeted behavior shallowly, while robust forgetting, capability preservation, and resistance to recovery remain difficult to achieve together.

  • Method

    RepSelect uses SVD on forget-set weight gradients to collapse high-variance shared directions and restrict updates to a forget-specific low-variance subspace.

  • Results

    Across two unlearning scenarios and four model families, RepSelect consistently outperformed five baselines, achieving 8–50× larger post-relearning answer-probability reductions for biohazardous knowledge while matching general capability.

  • Takeaways & Limitations

    Representation selectivity is an important direction for achieving deep LLM forgetting without sacrificing general capability.

  • Takeaways & Limitations

    RepSelect operates only on MLP modules, and its evaluation covers standard fine-tuning and few-shot attacks and relatively limited forget sets.

Abstract

from arXiv · show

Making large language models (LLMs) deeply forget specific knowledge and values without sacrificing general capabilities remains a central challenge in unlearning. Current methods are easily reversed by fine-tuning or few-shot prompting, suggesting their forgetting is only shallow. We identify the root cause. Existing methods target representations shared with both the retain set and the subspace recovered by a fine-tuning attacker, making unlearning both disruptive to general capabilities and easy to reverse. We propose RepSelect (Representation Selectivity), which isolates forget-set-specific representations by collapsing top principal components of weight gradients before each update, leaving general capabilities intact while limiting what fine-tuning can recover. We evaluate across two forget categories, biohazardous knowledge and abusive tendencies, and four model families spanning dense and Mixture-of-Experts architectures (Llama 3, Qwen 3.5, Gemma 4 E4B, DeepSeek V2 Lite). Compared to five popular baselines (GradDiff, NPO, SimNPO, RMU, UNDIAL), RepSelect achieves a 4-50x larger reduction in post-relearning answer accuracy than the strongest baseline, and is near-perfectly robust to few-shot prompting attacks. Targeting selective representations is thus an important step towards deep and robust LLM forgetting.

1 Introduction

The introduction argues that robust LLM unlearning must jointly achieve forgetting, low disruption to general capabilities, and resistance to relearning, but representation overlap makes these goals conflict. It proposes RepSelect, which targets selective forget-specific representations and generalizes across model families and unlearning scenarios while outperforming popular baselines.

  • 1 Introduction: Robust unlearning requires forgetting targeted knowledge, preserving general capabilities, and resisting relearning, yet existing methods do not reliably achieve all three.Gradient ascent can suppress unwanted outputs, and retain-set regularization can preserve capabilities; robustness remains the hardest requirement.
  • 1 Introduction: Representation overlap causes the trade-off: high-variance forget directions are shared with benign text and naturally recovered by fine-tuning on the forget set.Disrupting these directions harms general capability, while fine-tuning recovers them and reverses forgetting.
  • 1 Introduction: RepSelect applies singular value decomposition to forget-set weight gradients and suppresses their high-variance subspace before each unlearning update.Its low-variance directions collectively retain most forget-specific signal while being less entangled with general capability and less affected by relearning.
  • 1 Introduction: RepSelect generalizes across four model families and two unlearning scenarios, consistently outperforming five popular baselines on forgetting depth and general capability.The results support representation selectivity as important for deep LLM unlearning.

2 Experiment setup

The experiments test unlearning of harmful knowledge and abusive tendencies across four dense and Mixture-of-Experts models, using matched-disruption evaluation and held-out relearning attacks. Datasets separate forget, relearn, evaluation, retain, and general-text monitoring splits to measure forgetting, utility, and robustness.

  • Datasets: 189 WMDP-Bio multiple-choice questions were expanded to three paraphrases each, split 50/50 for relearning attack and held-out evaluation.Unlearning uses 189 × 3 paraphrases; relearning uses 282 paraphrases from 94 questions, while 95 questions form the held-out MCQ evaluation, with domain-matched biology data for retention.
  • Datasets: BeaverTails-AA isolates animal-abuse tendency removal without domain-specific knowledge, using benign concept substitutions to construct a synthetic retain set.The retain examples preserve surrounding context while replacing harmful concepts with benign counterparts such as torture → nurture.
  • Models: Experiments cover four model families: dense Llama 3.1 8B and Qwen 3.5 9B, plus Mixture-of-Experts Gemma 4 E4B and DeepSeek-V2-Lite.This model set spans dense and Mixture-of-Experts architectures and a broad range of families and sizes.
  • Evaluation: All methods share a fixed disruption budget, stopping when WikiText KL(pθ0∥pθ) exceeds 0.01, approximately a 0.4% negative-log-likelihood increase.This matches methods at comparable utility settings while tracking forgetting, disruption, and robustness.
  • Attacks and baselines: Robustness is tested with full-model fine-tuning on disjoint relearn data and few-shot attacks using k ∈ {5, 10} demonstrations, reporting the maximum post-attack metric.The protocol simulates adversaries with weight access or only domain examples, including early stopping at the most successful relearning epoch.
  • Attacks and baselines: RepSelect is compared with GradDiff, NPO, SimNPO, RMU, and UNDIAL through the unified Open-Unlearning evaluation framework.The baselines comprise five established unlearning methods.

3 Why unlearning fails?

Existing unlearning methods update high-variance forget directions that also encode retain information and are preferentially recovered by fine-tuning attackers. This overlap makes forgetting retain-disruptive and readily reversible, motivating selective suppression of high-variance directions.

  • Existing methods concentrate updates on high-variance forget components that carry retain information and overlap with attacker-recovered directions, making unlearning disruptive and reversible.The representation analysis targets the tension among good forgetting, low retain disruption, and attack robustness.
  • High-variance forget directions encode shared content with retain: 14.7% of retain-set MLP activation variance is explained by the top-10 forget PCs on Bio, versus 10.7% on Qwen.The top-50 PCs explain 36.4% on Llama and 30.5% on Qwen, showing that high-variance directions encode shared domain content.
  • High-variance WMDP-Bio PCs decode to broad domain tokens or common words, whereas low-variance PCs activate on niche concepts.Examples of broad tokens include virus, RNA, and outbreaks; niche concepts include plasmid-only reverse genetics and RV strain SA11.
  • An attacker recovers high-variance directions: 34% of a fine-tuning attacker’s update norm lies in the top-50 forget PCs, while all four baselines allocate 33–41% there.The attacker uses 50 SGD steps on forget data, explaining why baseline unlearning is readily reversed by same-domain fine-tuning.

4 RepSelect

RepSelect performs robust unlearning by collapsing forget-set-specific representations in both activations and output gradients before updating model weights. Its efficient forget-set-only, single-epoch design preserves benign directions while improving robustness and training speed.

  • Core method: RepSelect collapses the most disruptive activations and output gradients before computing unlearning updates.The method operates on weight gradients, equivalently collapsing their rows and columns to avoid forward/backward hooks; MoE experts share one SVD.
  • Training procedure: RepSelect requires no retain set and by default computes negated cross-entropy updates using only forget-set batches.Collapsing PCs from the retain rather than forget set is tested as an ablation and yields weaker unlearning performance.
  • Representation selectivity: RepSelect uses low-rank SVD to collapse the top k PCs, with default k=512, while leaving lower-variance and out-of-subspace directions unchanged.Mahalanobis reweighting damps high-variance directions and preserves the lowest-variance direction and directions outside the top-k subspace.
  • Representation selectivity: SVD on accumulated weight gradients efficiently identifies principal components for both activations and output gradients without explicit covariance tracking.Weight gradients encode activation and output-gradient distributions through their outer-product structure, while explicit covariance tracking provides no performance gains.
  • Training procedure: Two-sided collapse provides additional unlearning gains, and single-pass unlearning performs on par with multi-epoch iterative unlearning.A single pass accumulates gradients before one update, enabling shared SVD computation and caching with later rescaling.
  • Training procedure: Shorter runs are more robust: longer runs strengthen pre-attack unlearning but weaken post-attack robustness.Early experiments favor 1-2 epochs, motivating the single-pass variant.

5 RepSelect is robust and non-disruptive

RepSelect is robust to relearning and few-shot attacks while preserving general capabilities across tested models. Its selective-representation design outperforms baselines across architectures and benchmarks, with collapse and simple single-epoch training supported by ablations.

  • Capability preservation: Within 1–2% of the unmodified model, RepSelect’s MMLU accuracy remains across all tested models, indicating preservation of general capabilities.This result is reported in Appendix Table 19.
  • Cross-model evaluation: Across four model families, dense and MoE architectures, WMDP-Bio, and Animal Abuse, RepSelect achieves the lowest post-attack answer probability among all methods.The evaluated families are Llama 3.1 8B, Gemma 4 E4B, DeepSeek V2 Lite, and Qwen 3.5 9B.
  • Robustness: 8–50× larger on WMDP-Bio and 4–15× larger on Animal Abuse, RepSelect’s post-attack answer-probability drop exceeds the strongest baselines.The strongest baseline is NPO on WMDP-Bio and NPO or RMU on Animal Abuse.
  • Ablations: Collapse is necessary because the no-collapse baseline fails to unlearn within the disruption budget, while multi-epoch unlearning provides no consistent gain over single-epoch training.The collapse ablations use binary search over intervention strength and disable the LoRA adversary; the epoch comparison appears in Figure 6.
  • Few-shot robustness: ≤0.001 WMDP-Bio accuracy and ≤0.013 BeaverTails harmful-response probability persist for RepSelect across five- and ten-shot attacks on Llama-3.1-8B and Qwen3.5-9B.These values are reported for both k values, whereas baselines recover substantially more.

6 Related Work

Prior unlearning work includes gradient-, representation-, logit-distillation-, and meta-learning-based methods, alongside subspace projections designed to preserve retain performance. However, standard evaluations often omit relearning attacks, and existing unlearned models can be restored through fine-tuning or few-shot prompting.

  • Unlearning methods: Gradient-based methods alter the training objective, while representation-level methods steer or reroute harmful activations.GradDiff uses gradient ascent on forget data while minimizing retain loss; NPO adapts DPO to forget-only data; SimNPO removes the reference model using a length-normalized loss, while RMU and Circuit Breakers modify activation directions.
  • Unlearning methods: Meta-learning approaches anticipate relearning during training, and UNDIAL uses logit distillation.The cited meta-learning approaches include Tamirisa et al. (2024), Sondej et al. (2025), and Henderson et al. (2023).
  • Unlearning methods: Subspace-projection methods protect retain performance by removing retain-relevant directions but do not characterize forget-set or attacker subspaces.PGU and K-FADE identify retain-relevant directions with Fisher information, whereas KUDA uses A-GEM-style null-space projection perpendicular to the retain gradient.
  • Robustness to relearning attacks: Most unlearning evaluations omit relearning attacks, despite evidence that brief independent-data fine-tuning or few-shot prompting can restore unlearned models.TOFU, MUSE, and RMU evaluations omit relearning attacks; Deeb and Roger report full restoration after brief fine-tuning, while Tamirisa et al. and Lynch et al. reinforce this for few-shot prompting.

7 Conclusion · Appendix

RepSelect addresses shallow and reversible unlearning by restricting updates to forget-set-specific directions while preserving general capabilities. The appendix expands the algorithm, experimental setup, selectivity motivation, representation analysis, disruption and robustness analysis, and unlearning/relearning trajectories.

  • 7 Conclusion: RepSelect uses SVD on the forget set to identify shared high-variance directions and restricts weight updates to their complement.This targets representations shared with benign retain data and full-fine-tuning attack space, which otherwise disrupt capabilities and enable reversal.
  • 7 Conclusion: RepSelect operates only on MLP modules, while future work could test whether the collapse transfers to key/value projections in attention heads.The method is evaluated against standard fine-tuning and few-shot attacks, leaving other attack types for future study.
  • 7 Conclusion: The study successfully unlearns facts from WMDP-Bio and behaviours from BeaverTails-abuse, but larger forget sets remain necessary for full biosafety and broad coverage.The conclusion identifies scaling to larger forget sets as a future direction.
  • Appendix: The appendix provides more detail on the RepSelect algorithm and implementation, including multi-epoch variants.These materials are organized in Appendix A and A.1.
  • Appendix: Appendix B expands the experiment setup with reproducibility and compute requirements, hyperparameter search spaces, and dataset creation.These topics are listed as subsections B.1 through B.3.
  • Appendix: Appendix C develops the motivation for selectivity through unrelated-fact disruption, language transfer, and weight-space versus activation-space filtering.These analyses appear in subsections C.1 and C.2.
  • Appendix: Appendix D analyzes representations, including Qwen3.5-9B results, PCA selectivity, attacker concentration, and vocabulary projection.The listed sections cover model results and representation-analysis mechanisms.
  • Appendix: Appendix E examines disruption and robustness through MMLU accuracy, purified weight updates, and robustness analysis, while Appendix F presents unlearning and relearning trajectories.These analyses are organized across sections E.1–E.3 and F.

A More on RepSelect Algorithm and Implementation … B.3.2 BeaverTails contrast set creation

The appendix specifies RepSelect’s filtered-gradient algorithm, implementation choices, multi-epoch comparison, experimental infrastructure, hyperparameter tuning, and dataset construction. It also details contrastive BeaverTails retain-set generation and concise WMDP forget-corpus design.

  • A More on RepSelect Algorithm and Implementation: RepSelect adversarially pretrains LoRA adapters for one forget-set epoch, accumulates LoRA-active weight gradients, then applies a single filtered-gradient update.The algorithm initializes LoRA adapters on MLP gate/up/down projections before gradient accumulation and filtering.
  • A More on RepSelect Algorithm and Implementation: SVD-based soft collapse removes irrelevant principal components on both input and output gradient sides before updating each MLP weight.The collapse uses Mahalanobis rescaling, with SVD computed from each weight gradient.
  • A.1 Multi-epoch variants: NPO stabilizes multi-epoch training, but the reported multi-epoch variant offers no gains over the simpler single-epoch default.The naive negative-cross-entropy multi-epoch version is unstable and has a poor unlearning–disruption trajectory.
  • B More on Experiment Setup: The experimental setup covers hardware, compute, hyperparameter searches, WMDP-Bio and BeaverTails construction, and few-shot attack details.The main grid evaluates 4 models × 2 benchmarks × 8 methods.
  • B.1 Reproducibility and Compute Requirements: The main comparison uses 30-trial Optuna searches per method–model–benchmark cell and consumes approximately 320 GPU-hours overall.Individual searches take 3–9 hours, with larger models driving the upper end.
  • B.2 Hyperparameter Search Spaces: RepSelect uses k=512 principal components, forget-set distributions, bilateral soft collapse, SGD, and LoRA adversaries on MLP gate/up/down projections.For DeepSeek-V2-Lite and Qwen3-30B-A3B, the MoE-specific SGD learning-rate range is 30–100× larger than for dense models.
  • B.3.2 BeaverTails contrast set creation: BeaverTails contrast pairs preserve lexical, syntactic, and stylistic structure while inverting harmful concepts, and perform similarly to the dataset’s existing safe pairs.The released contrast set uses benign substitutions propagated coherently through each response.

B.4 Few-Shot Attack Details … D More on Representation Analysis

The paper details few-shot attacks and analyzes why unlearning disrupts related facts, motivating selective removal of low-variance, forget-specific representations. Additional results compare filtering strategies and extend representation analysis across models and attacker subspaces.

  • B.4 Few-Shot Attack Details: Few-shot attacks use k ∈{5, 10} relearn-split demonstrations, fixed within each evaluation run and disjoint from unlearning and evaluation data.WMDP demonstrations reveal correct multiple-choice answers, while BeaverTails demonstrations pair harmful prompts with harmful responses.
  • B.4 Few-Shot Attack Details: The attack metric is mean per-sequence normalized answer probability, a geometric mean bounded in [0, 1] that prevents high-loss sequences from masking memorized examples.For WMDP, the unlearning target lowers held-out forget-set answer probability without exceeding the WikiText KL disruption budget.
  • C More Motivation for Selectivity: Unlearning gradients bleed into unrelated facts because targeted and unrelated knowledge share representations, motivating selective targeting of forget-corpus-specific directions.The analysis also compares weight-space and activation-space approaches to filtering retain disruption.
  • C.1 Unrelated Facts Disruption and Language Transfer: 84% transfer occurs between superficially similar capital facts, while translation transfer is ∼50% only for languages with similar surface tokens and is weak for Russian and Portuguese.These results indicate that surface-token similarity contributes to cross-fact and cross-language gradient overlap.
  • C.2 Filtering Out Disruption: Weight Space vs. Activation Space: Weight- and row-masking techniques significantly reduce disruption, but residual harm remains because control or retaining updates imperfectly match actual disruption patterns.Figure 9 contrasts successful paraphrase unlearning with disruption of unrelated and control facts.
  • C.2 Filtering Out Disruption: Weight Space vs. Activation Space: 33% to 5%: removing whole faulty rows and columns reduces the disruption-to-transfer ratio more effectively than granular per-weight filtering.Because updates are approximately low-rank and disruption forms row- and column-wise stripes, activation and output-gradient dimension ablation is more targeted.
  • D More on Representation Analysis: The extended representation analysis reports additional Qwen3.5-9B results, PCA selectivity, attacker concentration, PC interpretations, and baseline projections in high-variance PC subspaces.It uses vocabulary projection, steering-vector alignment, and analysis of attack-subspace concentration.

D.1 Model result: Qwen3.5-9B … E.2 The Purified Weight Update

Across models, RepSelect isolates low-variance, forget-specific representations while avoiding retain-shared directions targeted by baselines and fine-tuning attackers. This selectivity explains its attack robustness and preserved general capability, while its purified updates are confined to forget-specific subspaces.

  • D.1 Model result: Qwen3.5-9B; D.2 PCA Selectivity and Attacker Concentration: 50–62% of baselines’ and the fine-tuning attacker’s update norm falls in the top-50 forget PCs, versus ∼4% for RepSelect on Qwen3.5-9B.The top tiers are retain-dominated, whereas RepSelect operates in the retain-dilute bottom subspace.
  • D.2 PCA Selectivity and Attacker Concentration; L0 L9 L18 L27; D.3 Vocabulary Projection of PCs: The attacker targets top-50 forget PCs encoding common forget-set tokens; baselines place 25–41% of update norm there, versus ∼11% for RepSelect.These directions are the same subspace that RepSelect avoids.
  • D.3 Vocabulary Projection of PCs; D.4 Top Forget Sequences per PC: High-variance PCs activate on broad domain concepts shared with retain data, whereas low-variance PCs activate on highly specific forget details across domains and models.Using the same PC indices and absolute eigenvalue ranking rules out cherrypicking.
  • D.5 Steering Vector Alignment; D.6 Cross-Distribution PC Variance; D.7 Tiered Selectivity Along PCA Directions: 20–67× higher steering alignment and 1.2–2.1× greater forget activation characterize high-variance PCs, while variance ratios fall from 3.6–4.3× to 1.3–1.4× across tiers.The results identify top PCs as retain-shared and bottom PCs as forget-specific.
  • D.8 Baseline Weight Projection: 25–60% of all four baselines’ update norm lies in the top-50 forget PCs, compared with ∼11% for RepSelect.Updates are projected onto forget PCs and measured by the fraction of their norm in the top-k subspace.
  • D.9 Attack Subspace Concentration: The attacker concentrates 5–7× more update norm in the top-10 PCs than RepSelect, with the gap reaching 26.6% versus 10.4% at k=50.RMU is excluded because it modifies only one layer, making per-layer comparison uninformative.
  • E More on Disruption and Robustness Analysis; E.1 MMLU Accuracy: RepSelect preserves MMLU accuracy across models after WMDP-Bio unlearning, with the WikiText KL ≤0.01 budget preserving downstream utility.The theoretical analysis characterizes purified updates and bounds attacker overlap, including for LoRA-based attackers.
  • E.2 The Purified Weight Update: After collapsing activations and output gradients, each per-token update lies in forget-specific directions, and the full update satisfies ∆W · vi ≈0 for every high-variance direction vi.Each purified per-token update is rank-1 and confined to a forget-specific subspace.

E.3 Robustness and Disruption Guarantees

RepSelect’s subspace restriction provides mathematical robustness against same-domain fine-tuning and rank-r LoRA attacks by avoiding high-variance directions that attackers preferentially recover. It also protects retain behavior when the retain covariance is concentrated in the avoided subspace.

  • Robustness guarantee: Same-domain attacker updates concentrate along the high-variance directions Vk that RepSelect avoids.This follows when attacker data shares covariance Σ with the forget corpus.
  • Robustness guarantee: Only the P⊥-component of an attack update can interfere with unlearning; the remaining fraction (1 −ϵk) is confined to Vk and has no effect.The overlap bound is ϵk = (sum_i>k λi) / tr(Σ).
  • LoRA robustness: When r ≤k, a rank-r LoRA attacker aligns within Vk, yielding ∆Watk P⊥= 0 and zero direct overlap with RepSelect’s unlearned subspace.The result depends on gradient-based optimization preferentially aligning the adapter row space with highest-variance activation directions.
  • Disruption guarantee: When Vret ≈Vk, the unlearning update leaves retain behavior unchanged.Retain protection is measured by tr(PkΣretPk)/tr(Σret); values close to 1 indicate that nearly all retain-set variance lies in Vk and is untouched.

F More Unlearning and relearning trajectories

RepSelect’s advantage remains consistent across all four model families and datasets, delivering more unlearning for the same disruption and limiting recovery during relearning. On Animal Abuse, small forget sets already provide most of the attainable unlearning, with saturation at 90 samples.

  • Cross-model trajectories: For WMDP-Bio, RepSelect achieves substantially more unlearning at the same Wikitext KL disruption.The comparison is presented through the unlearning–disruption trade-off and relearning trajectories across the model families.
  • Optimization behavior: RMU sometimes fails to reach 0.01 WikiText KL because Optuna converges on a very small learning rate, indicating inadequacy for some models.The displayed RMU trajectory does not reach the target in some plots despite optimization over the top 10 trials out of 30.
  • Data scaling: 10 samples achieve over half of maximal Animal Abuse unlearning, while 90 samples saturate performance and additional data yields no further gain.The scaling study varies forget-set size from 10 to 360 samples on Llama-3.1-8B and Qwen3.5-9B, without the LoRA adversary.
Loading 2606.17168v2…