PaperlayerView exact revision on arXiv

Source-linked AI summary

KUDA: Knowledge Unlearning by Deviating Representation for Large Language Models

Ce Fang, Zhikun Zhang, Min Chen, Qing Liu, Lu Zhou, Zhe Liu, Yunjun Gao

arXiv:2602.19275v3cs.CR

TL;DR

LLM unlearning aims to remove sensitive, harmful, or copyrighted knowledge, but existing approaches can fail to alter encoded knowledge precisely. KUDA selects knowledge-storage layers, deviates target representations, and uses relaxed null-space projection to protect retention. Across WMDP and MUSE, it reports balancing knowledge removal with utility retention, including near-complete removal with limited utility degradation.

  • Problem

    Existing unlearning methods can produce random or incoherent responses rather than precisely altering encoded knowledge, despite risks from sensitive, harmful, or copyrighted training content.

    Source: §I · p. 1, B. Existing Solutions · p. 4

  • Method

    KUDA uses causal tracing to select FFN layers, representation deviation to disrupt target knowledge associations, and relaxed null-space projection to protect retained representations.

    Source: §I · p. 1, C. Unlearning Layers Selection · p. 5, D. Knowledge Representation Deviation · p. 6

  • Results

    KUDA reports near-complete target-knowledge removal with utility degradation of 3.1% on MUSE and 1.7% on WMDP.

    Source: §I · p. 2

  • Takeaways & Limitations

    KUDA outperforms most representative baselines in balancing knowledge removal and utility retention.

    Source: §VII · p. 13

Abstract

from arXiv · show

Large language models (LLMs) acquire a large amount of knowledge through pre-training on vast and diverse corpora. While this endows LLMs with strong capabilities in generation and reasoning, it amplifies risks associated with sensitive, copyrighted, or harmful content in training data. LLM unlearning, which aims to remove specific knowledge encoded within models, is a promising technique to reduce these risks. However, existing LLM unlearning methods often force LLMs to generate random or incoherent answers due to their inability to alter the encoded knowledge precisely. To achieve effective unlearning at the knowledge level of LLMs, we propose Knowledge Unlearning by Deviating representAtion (KUDA). We first utilize causal tracing to locate specific layers for target knowledge storage. We then design a new unlearning objective that induces the model's representations to deviate from its original position in the phase of knowledge removal, thus disrupting the ability to associate with the target knowledge. To resolve the optimization conflicts between forgetting and retention, we employ a relaxation null-space projection mechanism to mitigate the disruption to the representation space of retaining knowledge. Extensive experiments on representative benchmarks, WMDP and MUSE, demonstrate that KUDA outperforms most existing baselines by effectively balancing knowledge removal and model utility retention. Source code is available at https://github.com/AceNagi/KUDA.

I. INTRODUCTION

Pre-training exposes LLMs to sensitive, dangerous, or copyrighted knowledge, while existing unlearning approaches can damage utility or fail to remove encoded knowledge precisely. KUDA couples component selection with representation-based removal and retention mechanisms; the paper reports strong removal with modest utility degradation.

I. INTRODUCTION

  • Full parameter updates can impair utility, response-level mitigation may leave encoded knowledge recoverable, and partial updates often fail to jointly optimize selection and removal.The introduction frames a fine-grained approach coupling parameter selection and knowledge removal as needed to balance removal and utility retention.Source: §I · p. 1
  • KUDA couples FFN-focused parameter selection with representation deviation and relaxed null-space projection to target knowledge removal while retaining model utility.The paper also proposes two-stage hyperparameter tuning and reports near-orthogonal gradients between forgetting and retention objectives.Source: §I · pp. 1–2
  • 3.1% on MUSE and 1.7% on WMDP are the reported utility degradations while KUDA achieves near-complete removal of target knowledge.The paper also reports generalization to Llama-3.1 and Qwen-3.Source: §I · p. 2

A. Large Language Models

  • FFNs store knowledge through key-to-value transformations, while MHSA aggregates semantic context and extracts task-relevant information from representations.These mechanisms jointly contribute to context understanding and text generation.Source: A. Large Language Models · p. 2

B. Causal Tracing

  • Causal tracing compares clean, corrupted, and corrupted-with-restoration runs to quantify how restoring component activations affects correct-token prediction.The method records clean activations and prediction probability, perturbs input embeddings for the corrupted run, then restores individual activations.Source: B. Causal Tracing · p. 3

C. Null-Space

The section defines unlearning as removing a target knowledge subset while retaining the rest, and reviews why existing approaches struggle to balance those goals. Null-space updates offer a way to preserve prior representations by constraining updates to directions that do not affect prior-task inputs.

C. Null-Space

  • Null-space updates preserve prior-task representations when update gradients are orthogonal to the prior-task input feature space.The passage describes this invariance as a basis for protecting previously learned capabilities.Source: C. Null-Space · p. 3

A. Problem Formulation

  • LLM unlearning seeks a model whose behavior matches one retrained on retained data, removing the target subset while preserving the remainder.The target subset is the forgetting set, and the remaining data is the retaining set.Source: A. Problem Formulation · p. 3

B. Existing Solutions

  • The KUDA pipeline combines layer selection, representation deviation, and relaxed null-space projection to balance removal and retention.It projects unlearning gradients into a relaxed null-space to filter utility-harming components without excessively suppressing removal.Source: Figure 2 · p. 4
  • Full-parameter methods risk model collapse or utility loss, while partial-update methods can under-remove knowledge when selection and removal are weakly coupled.The section identifies this gap as a lack of a synergistic pipeline aligned with LLM knowledge storage.Source: B. Existing Solutions · pp. 3–4

IV. KUDA

KUDA treats FFN knowledge associations in representations as an intervention point for unlearning, while preserving MHSA components that support contextual aggregation. It selects FFN-focused layers using causal-effect scores and a sliding-window rule.

A. Intuition

  • FFNs associate inputs with stored knowledge in representations, so selectively perturbing those associations can disrupt target-knowledge generation while leaving MHSA intact.The paper notes that MHSA is closely coupled with contextual aggregation and that updating it may broadly degrade utility.Source: A. Intuition · p. 4

B. Overview

  • KUDA selects unlearning layers, deviates target representations, then projects gradients into a relaxed null-space to limit harm to retained knowledge.Stochastic proportional sampling approximates the retaining-data feature space to reduce the cost of the projection step.Source: B. Overview · pp. 4–5

C. Unlearning Layers Selection

  • Causal effect scores identify FFNs specialized in knowledge storage, separating their role from MHSA during component-level layer selection.A higher FFN causal-effect score indicates a greater contribution to storing and retrieving knowledge during generation.Source: C. Unlearning Layers Selection · p. 5
  • The sliding-window search starts from high-scoring candidate layers and selects a contiguous window when more than half its layers meet the hit-ratio condition.The method prioritizes contiguous layers to preserve representation coherence; tracing is run once per model because its computational overhead makes repeated dataset-specific scoring impractical.Source: C. Unlearning Layers Selection · p. 5

D. Knowledge Representation Deviation

KUDA removes target knowledge by deviating its representation while constraining changes to preserve retained representations. It resolves directional conflicts by projecting updates into a relaxed null-space derived from retained-data features.

D. Knowledge Representation Deviation

  • An ℓ2 constraint on retaining-data representations limits excessive deviation, while the paper identifies nonlinear scaling-factor tuning as a drawback of weighted objectives.Retaining data supports factual and structural knowledge, including grammatical and syntactic patterns.Source: D. Knowledge Representation Deviation · p. 6
  • KUDA updates FFN output matrices and minimizes cosine similarity between target representations in the unlearned and original models to induce deviation.The final unlearning layer is used as the representation target because it integrates substantial forgetting knowledge and its associations.Source: D. Knowledge Representation Deviation · p. 6

E. Relaxed Null-Space Projection

  • Because forgetting and retention gradients can conflict directionally, KUDA projects updates into a relaxed null-space built from retaining-data input features.The relaxation threshold includes weakly correlated directions, avoiding the overly restrictive effect of strict projection on forgetting.Source: E. Relaxed Null-Space Projection · pp. 6–7
  • Stochastic proportional sampling estimates the retaining-feature space from a subset of retaining data, reducing the forward passes needed for feature collection.The sampling ratio controls the subset size while aiming to preserve statistical representativeness.Source: E. Relaxed Null-Space Projection · pp. 6–7

F. Principled Hyperparameter Configuration

KUDA’s practical tuning strategy first identifies a stability boundary under strong forgetting, then adjusts the forgetting intensity to balance removal and retention. The experiments use MUSE and WMDP, whose benchmark-specific metrics assess forgetting quality and retaining utility.

F. Principled Hyperparameter Configuration

  • A smaller β enhances knowledge removal, while a smaller τ enhances utility retention but weakens forgetting.The two-stage strategy prioritizes representation-space stability under extreme forgetting before refining the unlearning balance, turning joint optimization into sequential one-dimensional searches.Source: F. Principled Hyperparameter… · p. 7
  • The tuning strategy first fixes β = 0.1 and decreases τ until KRD falls below 0.9, defining that first τ as the stability boundary.Figure 3 marks the boundary on plots of forgetting and retention performance.Source: F. Principled Hyperparameter… · p. 7, Figure 3 · p. 7
  • After identifying the boundary, KUDA selects a slightly larger τ and increases β across {0.1, 1.0, 2.0} to mitigate forgetting intensity and balance removal with retention.The example boundary is 1 × 10−3, leading to τ = 2 × 10−3.Source: F. Principled Hyperparameter… · p. 7

A. Experimental Setup

  • KUDA is evaluated on MUSE and WMDP using benchmark-specific forgetting-quality and retaining-utility metrics.MUSE includes BOOKS and NEWS; WMDP evaluates hazardous knowledge in biology and cybersecurity alongside broad world-knowledge retention.Source: A. Experimental Setup · pp. 7–8
  • The origin model serves as the retention ceiling, while the retrain model is the ideal reference for forgetting.For WMDP, a retrain model is unavailable because hazardous knowledge is widespread across training corpora.Source: A. Experimental Setup · pp. 7–8
  • MUSE measures forgetting with VerbMem, KnowMem on Df, and PrivLeak, while WMDP uses answer accuracy, with 25% as its ideal forgetting result.Retention is measured with KnowMem on Dr for MUSE and MMLU for WMDP; scores closer to the origin model indicate better retention.Source: A. Experimental Setup · p. 8
  • KRD aggregates multiple forgetting metrics because their number makes cross-method forgetting comparisons difficult; values near 1 indicate favorable forgetting when retention is not substantially degraded.KRD is designed for comparing methods within the same benchmark.Source: A. Experimental Setup · p. 8

B. Overall Unlearning Performance

KUDA is evaluated for knowledge removal and utility retention on MUSE and WMDP, then analyzed under conflicting forgetting and retention objectives. The results report effective forgetting, retained utility, and more stable gradient alignment than RMU.

B. Overall Unlearning Performance

  • On MUSE, KUDA achieved comprehensive forgetting across BOOKS and NEWS while balancing utility retention; it slightly underperformed SimNPO in NEWS PrivLeak.In NEWS, KUDA minimized retention degradation while achieving maximal knowledge removal; in BOOKS, its retention was slightly weaker than WHP, but its forgetting was stronger.Source: B. Overall Unlearning Performance · p. 8
  • 88.8% of hazardous knowledge was removed from Zephyr-7B-Beta with a 1.3% drop in retaining utility on WMDP.KUDA also maintained better performance than RMU on this benchmark.Source: B. Overall Unlearning Performance · p. 9, Table II · p. 9
  • On Qwen-3, KUDA consistently surpassed RMU, and the authors report generalization across models with differing architectural characteristics.The authors attribute RMU’s limited transferability primarily to poor compatibility with Qwen-3’s architectural differences.Source: B. Overall Unlearning Performance · p. 9
  • KUDA achieved complete forgetting without utility degradation on TOFU, supporting reported generalization across copyrighted content, dangerous knowledge, and personal information.The TOFU experiments are presented in the appendix.Source: B. Overall Unlearning Performance · p. 9

D. Hyperparameter Analysis

The analysis describes how β and τ shape KUDA’s forgetting–retention trade-off and evaluates layer selection, the representation-deviation loss, and null-space projection. Results indicate that layer choice and pairing the asymmetric loss with relaxed projection matter for balancing removal and retention.

D. Hyperparameter Analysis

  • Decreasing β strengthens forgetting but can degrade retention, while decreasing τ improves retention at the cost of weaker forgetting; at or below the stability boundary, forgetting is significantly suppressed.Below that boundary, variations in β have limited influence on retention.Source: D. Hyperparameter Analysis · pp. 10–11

E. Ablation Study

  • KUDA’s selected layers l6 ∼l9 lie in an FFN-dominant region, and the robust l4 ∼l9 interval yields superior forgetting–retention performance.Middle and deep depth-based selections fail to achieve effective knowledge removal.Source: E. Ablation Study · p. 11, Figure 7 · p. 11
  • Relaxed null-space projection paired with LKUDA improves both knowledge removal and utility retention, whereas the projection is better suited to LKUDA than LRMU.The authors identify LKUDA’s asymmetric design as a prerequisite for retaining utility without suppressing knowledge removal.Source: E. Ablation Study · p. 11, Table IV · p. 12
  • With KUDA’s losses, null-space projection outperforms GradDiff in balancing forgetting and retention across datasets and models.At comparable forgetting quality, null-space projection improves retention performance through subspace transformations rather than scalar gradient weighting.Source: E. Ablation Study · pp. 11–12, Table IV · p. 12

F. Robustness Analysis

KUDA is evaluated for resistance to recovery attacks and computational cost, alongside related-work distinctions and the paper’s concluding method summary. Its reported results cover MIA, prompt-based recovery, quantization, and sampling efficiency.

F. Robustness Analysis

  • KUDA’s relative AUC stays close to the “Retrain” baseline on BOOKS and NEWS, while RMU performs poorly on NEWS.The authors attribute KUDA’s remaining gap from ideal results to preserving utility on knowledge semantically related to Df.Source: F. Robustness Analysis · p. 12, Figure 8 · p. 12
  • Under FOCUSONKEY, KUDA keeps target-knowledge accuracy below “Retrain,” especially on NEWS, while RMU’s accuracy rises above it.The authors report that RMU’s increase from “w/o Attack” to “w/ Attack” indicates successful knowledge recovery.Source: F. Robustness Analysis · p. 12, Figure 8 · p. 12
  • After 4-bit and 8-bit quantization, KUDA shows no evident knowledge recovery.The models were unlearned in BF16 before quantization.Source: F. Robustness Analysis · p. 12

G. Efficiency Analysis

  • Causal tracing is KUDA’s main computational bottleneck, but it is a one-time offline operation whose layer-selection results can be reused across datasets.SVD cost depends mainly on input-feature dimensionality and remains acceptable for a few unlearning layers; projection overhead during updates is marginal.Source: G. Efficiency Analysis · pp. 12–13
  • On a 61,000-sample corpus, stochastic sampling preserves the main eigenvalue-spectrum structure and achieves near-optimal unlearning across sampling ratios with minor τ adjustments.The authors report overhead reduction to p 100 of the original one.Source: G. Efficiency Analysis · p. 13, Figure 9 · p. 13

A. LLM Unlearning

  • Prior LLM-unlearning approaches include full-parameter fine-tuning, knowledge-correlated parameter selection, and activation steering, with computational cost or forgetting effectiveness limiting some approaches.The passage notes that knowledge-correlated parameter methods can be computationally expensive, while activation-steering methods have limited forgetting effectiveness.Source: A. LLM Unlearning · p. 13

B. Model Editing

  • Model editing redirects factual mappings to specified outputs, whereas LLM unlearning removes mappings across broader, potentially entangled knowledge or data traces.The passage distinguishes model editing’s structured factual triples from unlearning targets that may be unstructured paragraphs.Source: B. Model Editing · p. 13

VII. CONCLUSION

  • KUDA combines storage-informed layer selection, representation deviation, relaxed null-space projection, and two-stage tuning to balance knowledge removal with utility retention.The conclusion reports that KUDA outperforms most representative baselines on this balance.Source: §VII · p. 13

APPENDIX A MORE RELATED WORK OF LLM UNLEARNING

The appendix reviews external-intervention approaches, attack routes against unlearning, and applications, then gives a null-space proof concerning parameter updates. The proof links updates constrained to the input-feature null-space with unchanged layer outputs on prior-task features.

A. LLM Unlearning via External Intervention

  • External interventions alter generation through context fine-tuning or input-embedding and output-logit perturbations without updating model parameters, so they do not remove encoded target knowledge.The passage says such methods may leave knowledge extractable from intermediate representations.Source: A. LLM Unlearning via External… · p. 15

B. Attacks against LLM Unlearning

  • Attacks may recover supposedly forgotten knowledge through residual memory, prompts, backdoor triggers, retaining-set fine-tuning, or low-precision quantization.The passage presents these vulnerabilities as evidence that some methods obscure knowledge rather than remove it.Source: B. Attacks against LLM Unlearning · p. 15

C. Application of LLM Unlearning

  • LLM unlearning applications include removing privacy-sensitive training data, suppressing harmful generation, enforcing copyright compliance, and addressing noisy, mislabeled, or backdoor-contaminated data.Source: C. Application of LLM Unlearning · p. 15

APPENDIX B PROOF

  • When the parameter update lies in the null-space of the input-feature space, the layer output on previous-task features is unchanged.The proof states the resulting invariance for the layer-l output.Source: Appendix B · pp. 15–16

APPENDIX C EXPERIMENTAL DETAILS

The appendix describes evaluation measures for forgetting, retention, and privacy leakage across MUSE and WMDP. It also defines KRD as a holistic forgetting measure and explains its truncation when a retrain baseline is unavailable.

A. Evaluation Metrics

  • MUSE evaluation uses Verbatim Memorization, Knowledge Memorization, Privacy Leakage, and harmonic-mean Knowledge Removal Degree to assess forgetting from complementary perspectives.KRD approaches 0 when any forgetting metric indicates poor removal and equals 1 for complete forgetting; MUSE truncates metric values at the retrain-model level.Source: A. Evaluation Metrics · p. 16
  • WMDP measures hazardous-knowledge comprehension with answer accuracy and uses MMLU to assess general knowledge that should be retained.For WMDP, KRD values are truncated at 25, the desired test score for forgetting, because no retrain model is available.Source: A. Evaluation Metrics · pp. 16–17

B. Baseline Implementation

The appendix records baseline implementation choices, KUDA’s implementation and tuning details, and a complementary TOFU evaluation setup. TOFU assesses forgetting on synthetic-author targets and retention on remaining authors, neighboring real authors, and general facts.

B. Baseline Implementation

  • MUSE baselines use benchmark or paper hyperparameters, with RMU additionally grid-searched; WMDP GA, NPO, and SimNPO follow Fan et al. [13].MUSE GA and NPO use GradDiff with benchmark-specific retaining losses, and SimNPO uses corresponding retaining losses.Source: B. Baseline Implementation · p. 17

C. KUDA Implementation

  • KUDA uses layer-wise updates under limited GPU memory, and identical hyperparameters can produce different outcomes across hardware while sensitivity trends remain consistent.The implementation uses two NVIDIA L40 GPUs; the two-stage tuning strategy is reported to help identify suitable settings across devices.Source: C. KUDA Implementation · p. 17, B. Baseline Implementation · p. 17

A. Experimental Setup

  • TOFU uses GPT-4-synthesized profiles with 1%, 5%, or 10% of authors assigned to forgetting, and evaluates retention on remaining authors plus real-author and world-fact sets.Its evaluation framework aggregates response measures into Forget Efficacy and Model Utility, with utility covering neighboring and general knowledge.Source: A. Experimental Setup · p. 18

B. Results Analysis

On TOFU, KUDA removes target knowledge across forgetting settings while keeping retention scores near the Origin model. Its forgetting scores exceed GA, NPO, and the Retrain model, and its utility remains stable as the forgetting ratio increases.

B. Results Analysis

  • Table VII aggregates TOFU forgetting and retention performance under different forgetting settings.Forget and Retain are aggregated metrics; detailed metrics are reported in Table VIII.Source: B. Results Analysis · p. 18, Table VII · p. 19
  • KUDA’s forgetting scores on forget01, forget05, and forget10 outperform GA and NPO and exceed the Retrain model.The results indicate complete removal of target Df while retention scores remain close to the Origin model, preserving neighboring and general factual knowledge.Source: B. Results Analysis · p. 19
  • KUDA maintains retention scores close to the Origin model while behaving as if it had never been exposed to Df.The evaluation concerns queries related to Df; neighboring knowledge and general factual knowledge are reported as well preserved.Source: B. Results Analysis · p. 19
  • KUDA remains stable as the forgetting ratio increases, unlike GA, NPO, and RMU, whose utility retention degrades.Together with results on MUSE and WMDP, these findings support generalizability across copyrighted content, hazardous knowledge, and personal information.Source: B. Results Analysis · p. 19

APPENDIX E DETAILED ANALYSIS

The analyses support reusing causal-tracing layer selections across knowledge domains, while showing that layer depth, continuity, and update scope affect forgetting and utility. They also identify a stability boundary below which forgetting is suppressed.

A. Causal Effect of Various Datasets

  • Selecting informative question and answer tokens helps causal tracing avoid interference from generic tokens in context-based MUSE examples.Perturbation tokens are meaningful question entities whose replacement changes the answer; answer targets are informative entities prefixed by low-information text.Source: A. Causal Effect of Various Datasets · p. 19
  • Across factual, BOOKS, and NEWS datasets, FFN causal-effect distributions are consistent and concentrate in early layers, supporting reuse of factual-statement layer selections.Reusing these selections avoids per-task causal tracing and its computational overhead.Source: A. Causal Effect of Various Datasets · p. 19

B. Unlearning Layers Analysis

  • Unlearning layers with high FFN causal effect outperform high-attention-effect layers in balancing forgetting and utility.This supports targeting layers responsible for knowledge storage.Source: B. Unlearning Layers Analysis · pp. 19–20
  • Updating shallow peak-CE layers l3–l6 performs poorly, failing to remove knowledge while severely compromising utility.The authors suggest these layers handle generic low-level features shared across tasks, limiting their semantic specificity for targeted removal.Source: B. Unlearning Layers Analysis · p. 20
  • Deeper transition layers l10–l15, especially l10–l12, can also cause severe utility degradation when updated.The authors suggest that disrupting the transition between knowledge storage and semantic aggregation may explain this effect.Source: B. Unlearning Layers Analysis · p. 20
  • Consecutive unlearning layers offer a slight advantage: discontinuous selections may forget more but often reduce utility.The authors link continuity to unified representation modification and structural coherence across layers.Source: B. Unlearning Layers Analysis · p. 20

C. Stability Boundary

  • With β fixed at 0.1, the stability boundary is the τ where KRD first falls below 0.9; at or below it, KUDA does not thoroughly remove target knowledge.The authors attribute this to relaxed null-space projection suppressing representation deviations, and tune τ slightly above the boundary before increasing β.Source: C. Stability Boundary · p. 20

D. FFN-Centered Update Design

  • Extending updates from FFN Wout to MHSA WV does not consistently improve forgetting and substantially degrades retention.The authors suggest WV updates can alter contextual aggregation rather than target associations encoded in FFNs; Table IX presents the ablation.Source: D. FFN-Centered Update Design · pp. 20–21, Table IX · p. 21

E. Efficiency Comparison

KUDA’s reported runtime remains comparable to or below GA when causal-tracing results are reused or their one-time cost is included. The appendix also describes membership-inference and recovery evaluations, reporting resistance to quantization-based recovery.

E. Efficiency Comparison

  • KUDA’s total runtime is comparable to or lower than GA even when one-time causal tracing is included.Reusing layer selections reduces KUDA’s cost substantially relative to GA; compared with RMU, the added overhead is limited alongside better unlearning performance.Source: E. Efficiency Comparison · pp. 21–22

A. Min-K% Prob MIA

  • Min-K% Prob MIA scores low-probability tokens and compares member/non-member discriminability with the Retrain model using Relative AUC.A Relative AUC approaching zero indicates similarity to the Retrain baseline in membership discriminability.Source: A. Min-K% Prob MIA · p. 22

B. FOCUSONKEY

  • FOCUSONKEY tests knowledge recovery by emphasizing tokens the model previously relied on, with answer accuracy used as the recovery metric.Prometheus-7B-v2.0 assesses whether the target model answers the relevant questions correctly.Source: B. FOCUSONKEY · p. 22

C. Quantization-based Knowledge Recovery

Related papers

Loading 2602.19275v3…