PaperlayerView exact revision on arXiv

Source-linked AI summary

Preemptive LLM Unlearning against Forbidden Capability Acquisition via Gradient Sealing

Kemou Li, Qizhou Wang, Yue Wang, Fengpeng Li, Zhuan Shi, Negar Rostamzadeh, Golnoosh Farnadi, Masashi Sugiyama, Jiantao Zhou

arXiv:2609.39866v1cs.LG

TL;DR

Open-weight LLMs can be fine-tuned to acquire forbidden capabilities, and retrospective suppression may leave acquisition gradients unconstrained. GSU seals selected pathways using proxy data and first-order updates; across the evaluated model families and benchmarks, it shows stronger resistance to downstream acquisition than retrospective baselines.

  • Problem

    Open-weight LLMs can be fine-tuned after release, but preemptive defenses against forbidden-capability acquisition remain underexplored despite legal and ethical risks.

    Source: §1 · p. 1

  • Method

    GSU uses forbidden-domain proxy data to identify acquisition-sensitive gates and pushes selected pre-activations into a negative-derivative region with first-order updates.

    Source: §1 · p. 2

  • Results

    Across WMDP and TOFU evaluations on multiple LLM families, GSU shows stronger resistance to downstream forbidden-capability acquisition than competitive retrospective baselines.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across evaluated settings, GSU limits forbidden acquisition while retaining benign fine-tunability, with ablations and matched gate probes supporting targeted sealing beyond output suppression alone.

    Source: §6 · p. 10

    The attack definition excludes cases where forbidden capability is acquired only by severely degrading normal-domain utility.

    Source: §2.1 · p. 3

Abstract

from arXiv · show

Open-weight LLMs are released not only as fixed products but also as substrates for downstream fine-tuning. This openness, however, creates legal and ethical risks because users may misuse fine-tuning to instill illicit knowledge or enable hostile operations. Model providers therefore need apre-release defense against such acquisition, motivating the problem of preemptive unlearning. Unlike retrospective unlearning, which removes capabilities already present in a fixed model, preemptive unlearning seeks to prevent their acquisition under unseen attack data and future fine-tuning procedures. Despite its practical importance, this setting remains largely unexplored, presents distinct challenges, and is therefore the central focus of our work. We first verify that existing retrospective methods provide insufficient pre-release protection. Even when forbidden capabilities are suppressed in current outputs, forbidden-domain data can still induce gradients through internal pathways, enabling later acquisition. Motivated by this finding, we propose a gradient-sealing principle that blocks these pathways by pushing relevant pre-activations into the negative region, where ReLU-family activations exhibit zero or near-zero derivatives. Experiments across multiple LLM families demonstrate our stronger resistance to downstream acquisition than retrospective baselines, validating gradient sealing as an effective mechanism for pre-release protection.

1 INTRODUCTION

Open-weight LLMs can be fine-tuned to acquire forbidden capabilities, while retrospective unlearning may suppress current outputs without closing the pathways used for later acquisition. The paper proposes Gradient-Sealed Unlearning (GSU) and reports stronger resistance to downstream acquisition than retrospective baselines.

1 INTRODUCTION

  • Open-weight models support accessible deployment and adaptation, but fine-tuning can also instill harmful capabilities, motivating a pre-release defense.The stated risks include disinformation, explicit content generation, dangerous operational guidance, and privacy violations.Source: §1 · p. 1
  • Unlike retrospective unlearning, which suppresses already learned capabilities, preemptive unlearning aims to resist their acquisition through downstream fine-tuning.Figure 1 contrasts output suppression with sealing the pathway before release.Source: §1 · p. 1, Figure 1 · p. 2
  • Retrospective methods can leave gradients unconstrained, so forbidden-domain data may induce gradients even when release-time capability is low.The paper reports that low release capability can hide high acquisition susceptibility.Source: §1 · p. 2
  • GSU uses a proxy set to identify acquisition-sensitive ReLU-family gates, then pushes selected pre-activations into a negative region with first-order updates.Its objective combines a seal loss with response suppression and retain supervision.Source: §1 · p. 2
  • Across WMDP and TOFU evaluations on multiple LLM families, GSU shows stronger resistance to downstream forbidden-capability acquisition than competitive retrospective unlearning baselines.These results support GSU’s gradient-sealing principle and activation-gating mechanism.Source: §1 · p. 2

2 PROBLEM STATEMENT

Preemptive unlearning conditions a model before release to resist future acquisition of forbidden capabilities while preserving normal-domain utility. The section formalizes the attack setting and the defender’s use of proxy data without knowledge of future attacks.

2 PROBLEM STATEMENT

  • Preemptive unlearning conditions a model before release to resist forbidden-capability acquisition, distinguishing it from removing capabilities already present and separating forbidden capability from normal utility.The framework assigns distributions to forbidden and normal prompt–response domains and defines their performance measures.Source: §2 · p. 3

2.1 POST-RELEASE ATTACK

  • A successful post-release attack uses white-box access and unseen forbidden-domain data to exceed a capability threshold while retaining utility; attacks that severely degrade utility are out of scope.The attacker may choose the objective, optimizer, schedule, and other training configurations.Source: §2.1 · p. 3

2.2 PRE-RELEASE DEFENSE

  • Before release, the defender uses the original model and forbidden-domain proxy data to resist attacks in a defined class while limiting degradation of normal utility.The defender remains agnostic to the future attack dataset and procedure.Source: §2.2 · p. 3

3 WHY RETROSPECTIVE UNLEARNING FALLS SHORT

Retrospective unlearning suppresses current outputs but does not constrain how unseen forbidden-domain fine-tuning can change the model. The section tests this gap and shows that proxy-identified pathways can be targeted to reduce acquisition.

3.1 FROM LIKELIHOOD SUPPRESSION TO FUTURE ACQUISITION GRADIENTS

  • Retrospective likelihood suppression controls the released model’s current state, not the update from fresh attacker data; the one-step bound depends on the attacker gradient projected onto forbidden-sensitive directions.Even when Da = Df, suppression alone need not make the projected gradient small.Source: §3.1 · p. 4

3.2 RELEASE-TIME SUPPRESSION LEAVES FUTURE ACQUISITION PATHWAYS OPEN

  • NPO’s extraction strength rose from 0.112 to 0.326 and RMU’s from 0.032 to 0.257 after identical acquisition, despite release-time suppression.The released methods retained at least 95% of Reference utility.Source: Figure 2 · p. 5, §3.2 · p. 5
  • Targeted attenuation reduced acquisition gain by 24.35% at p = 10% and α = 100%, and outperformed matched random masks at every coverage under full attenuation.The intervention scaled selected channels’ backward/update paths while preserving forward computation and matching per-step update norms to Full-FT.Source: Figure 2 · p. 5, §3.2 · p. 5
  • Proxy-look-ahead and actual-acquisition channel ranks aligned at pooled ρ = 0.617, with positive alignment across all 24 layers.Figure 2b compares within-layer ranks of channel changes under proxy learning and subsequent acquisition.Source: Figure 2 · p. 5, §3.2 · p. 5
  • The section concludes that pre-release defense must control where future optimization can flow, rather than only suppressing current outputs.GSU follows this logic by exposing changes on Df, localizing sensitive gates, and sealing their gradient pathways.Source: §3.2 · p. 5

4 GRADIENT-SEALED UNLEARNING

GSU uses proxy learning to identify gates that future forbidden-domain training may recruit, then seals those pathways while suppressing forbidden responses and retaining normal-domain utility. Its stages expose learning-induced changes, localize directionally recruited gates, and attenuate their local gradients.

4 GRADIENT-SEALED UNLEARNING

  • GSU avoids costly second-order differentiation by pushing selected pre-activations into activation regions with zero or near-zero derivatives, alongside response suppression and retain supervision.The section presents this as targeted gradient control intended to limit utility harm from indiscriminate intervention.Source: §4 · p. 6, §4.3 · p. 7

4.1 EXPOSE ACQUISITION-INDUCED CHANGES

  • A disposable copy is briefly fine-tuned on Df, then its pre-activations are compared with the original model under matched teacher-forcing contexts to expose learning-induced changes.Matching contexts controls for differences in generated continuations.Source: §4.1 · p. 6

4.2 LOCALIZE DIRECTIONALLY RECRUITED GATES

  • Localization prioritizes gates whose pre-activations shift upward under proxy learning, then selects layers by normalized recruitment and channels by positive directional score within a layer.The chosen layer count and within-layer channel fraction determine the fixed gate set G.Source: §4.2 · pp. 6–7

4.3 SEAL SELECTED GRADIENT PATHWAYS

  • After localization, optimization restarts from θo; the seal penalty pushes selected gate pre-activations below a shared negative-tail threshold, where SiLU’s local derivative is small.The soft penalty’s downward contribution vanishes once a pre-activation reaches the threshold, balancing sealing with utility preservation.Source: §4.3 · p. 7
  • The released GSU model combines gradient sealing with forbidden-response suppression and retain supervision, targeting small local gradient factors when unseen acquisition reuses selected gates in the negative tail.The joint objective is optimized with first-order updates without differentiating through the look-ahead stage.Source: §4.3 · p. 7

5 EXPERIMENTS

The experiments compare GSU with retrospective defenses on TOFU prevention and WMDP restoration, then examine attack-time resistance, gate response, and benign learning. GSU performs strongly across the reported settings while retaining benign fine-tunability.

5.1 EXPERIMENTAL SETUP

  • TOFU prevention uses retain-only checkpoints and splits forbidden-author QA data into defender and attacker pools; WMDP restoration uses disjoint source-document pools from knowledgeable checkpoints.TOFU also evaluates identical prevention, while WMDP reports only disjoint attacks.Source: §5.1 · pp. 7–8

5.2 MAIN COMPARISONS

  • GSU ranks first or second in 34 of 36 TOFU comparisons and in 17 of 18 identical-prevention comparisons.Under disjoint prevention, it achieves the lowest ES5 on five of six models and the lowest UA90 on four.Source: Table 1 · p. 8, §5.2 · pp. 8–9

5.1 EXPERIMENTAL SETUP

  • The study evaluates Llama3 and Qwen3.5 models on TOFU and Zephyr-7B-β on WMDP against conventional unlearning and robust-defense baselines.Attacks use supervised full-parameter fine-tuning, with separate TOFU and WMDP schedules.Source: §5.1 · p. 8

5.3 FURTHER ANALYSES

  • GSU has the lowest ES among five methods at all eleven observed attack checkpoints, although its lead narrows and does not establish uniformly slower learning.At epoch 5, ES is 4.713% for GSU versus 5.056% for NPO and 5.102% for NPO+SAM.Source: Figure 4 · p. 9, §5.3 · p. 9
  • At release, GSU reduces mean absolute SiLU derivatives by approximately 24% on B1 and 23% on B2, versus 11% on retain inputs.The pattern is consistent with targeted local attenuation; matched post-attack probes examine partial reversal and residual differences.Source: Figure 4 · p. 9, §5.3 · p. 9
  • After two economics fine-tuning epochs, GSU’s validation NLL is within 0.005 of No Defense controls on three tested models.Larger reductions also reflect higher initial losses, not improved learning efficiency.Source: Figure 4 · p. 9, §5.3 · p. 9

5.2 MAIN COMPARISONS

  • GSU achieves the lowest WMDP F5 on Bio and Cyber: 34.13 and 30.23, versus strongest competing results of 34.83 and 31.08.It also ranks first or second across both domains and all three metrics.Source: Table 2 · p. 9, §5.2 · p. 9

6 CONCLUSION

The conclusion presents GSU as a preemptive defense that seals selected acquisition pathways while preserving benign fine-tunability; the accompanying appendix provides notation, proofs, settings, and supplementary analyses.

6 CONCLUSION

  • GSU exposes proxy-induced changes, localizes recruited gates, and seals their gradient pathways while combining a shared SiLU negative-tail threshold with response suppression and retain supervision.The conclusion reports that GSU limits forbidden acquisition while retaining benign fine-tunability across evaluated settings.Source: §6 · p. 10

OVERVIEW OF THE APPENDIX

  • The appendix collects notation, related work, proofs, pseudocode, experimental settings, and supplementary analyses, including benign learning and protection after adaptation.Its notation distinguishes forbidden and normal domains, defender, attacker, and retain sets, and parameters at original, released, and post-attack stages.Source: OVERVIEW OF THE APPENDIX · p. 15, A NOTATIONS · p. 15

B DETAILED RELATED WORK

  • The related-work discussion separates retrospective unlearning, open-weight safeguards, data-side protection, and representation localization as neighboring research threads.Together, these threads motivate resistance to unseen future learning in released weights.Source: B DETAILED RELATED WORK · p. 15

B.1 RETROSPECTIVE LLM UNLEARNING AND ROBUSTNESS

The related work distinguishes preemptive unlearning from methods that suppress existing behavior, protect data, or preserve broad safety alignment, and situates GSU as targeting future-learning pathways.

B.1 RETROSPECTIVE LLM UNLEARNING AND ROBUSTNESS

  • Retrospective unlearning removes designated training influence or encoded behavior, but low release-time scores may reflect suppression rather than durable removal; preemptive unlearning also covers capabilities absent from the original checkpoint.Robustness work studies recovery through prompting, probing, relearning, or downstream fine-tuning.Source: §B.1 · p. 16

B.2 OPEN-WEIGHT SAFETY AND TAMPER-RESISTANT SAFEGUARDS

  • Open-weight safeguards address weight-level adaptation or preserve broad safety alignment, while preemptive unlearning asks whether clean, disjoint future data can acquire a designated capability without losing normal utility.Inference-time guard models remain useful at deployment but cannot prevent weight-level adaptation.Source: §B.2 · p. 16

B.3 DATA-SIDE PREEMPTIVE PROTECTION

  • Data-side protections assume protected examples will later be used for training, whereas GSU encodes resistance in released weights using a separate proxy set.Perturbation-based protection can be brittle under robust preprocessing or retraining.Source: §B.3 · p. 16

B.4 REPRESENTATION LOCALIZATION AND PATHWAY CONTROL

  • GSU differs from editing, concept erasure, representation engineering, and gradient routing by targeting gate derivatives through which future domain data update the model.Those neighboring approaches localize or modify computations, features, or learning signals.Source: §B.4 · pp. 16–17

C PROOFS AND SUPPORTING ANALYSIS

  • The supporting proof derives a future-acquisition bound and gives a ReLU counterexample in which identical release-time scores conceal different acquisition receptivity.The analysis also considers how activation suppression reduces receptivity.Source: C PROOFS AND SUPPORTING ANALYSIS · p. 17

C.1 PROOF OF PROPOSITION 3.1

The proof establishes a one-step acquisition-gain bound under local smoothness and gradient-alignment assumptions, then controls its first-order term and second-order remainder.

C.1 PROOF OF PROPOSITION 3.1

  • Proposition 3.1 bounds the one-step forbidden-score acquisition gain for a fixed attacker dataset when the score is Lipschitz and smooth and its gradient lies in the projected subspace.The stated result applies for sufficiently small positive step sizes.Source: §C.1 · p. 17
  • The proof fixes the attacker gradient independently of the step size and derives a first-order expansion whose remainder is O(η^2).It uses smoothness along the update segment near the released parameters.Source: §C.1 · p. 17
  • The local-separation condition and Cauchy–Schwarz bound the linear term by LF times the norm of the projected attacker gradient, which is combined with the remainder estimate.This completes the proof of the acquisition-gain bound.Source: §C.1 · p. 18

C.2 A SIMPLE RELU COUNTEREXAMPLE

The ReLU counterexample shows that identical release outputs can conceal different acquisition receptivity: reducing a forbidden-sensitive activation reduces both the projected attacker gradient and one-step acquisition gain. The bundled methods and setup passages describe GSU’s proxy-based gate selection and the benchmark splits used for evaluation.

D PSEUDOCODE

  • GSU fits a disposable copy on proxy data, collects matched-context pre-activations, selects high-scoring layers and channels, then applies a joint sealing and retention loss with first-order updates.The algorithm discards the proxy copy; its output is the released parameters.Source: D PSEUDOCODE · p. 19

C.2 A SIMPLE RELU COUNTEREXAMPLE

  • Reducing the forbidden-sensitive activation from 1 to δ reduces the projected attacker-gradient norm by δ and the exact one-step acquisition gain by δ², despite identical release outputs.Both released states preserve the retain output; the construction keeps states away from the ReLU kink.Source: §C.2 · pp. 18–19, Figure 5 · p. 20

E.1 MECHANISM STUDY FOR FIGURE 2

  • The Figure 2 mechanism study uses a retain-only Qwen3.5-2B reference and one seed, with disposable-copy look-ahead and matched-update-norm gate attenuation arms.The attenuation grid includes a Full-FT control and localized arms across nested coverage masks.Source: §E.1 · p. 20

E.2 BENCHMARK CONSTRUCTION

  • TOFU’s disjoint split uses non-overlapping QA records from the same forbidden authors, while WMDP divides each domain’s forget corpus in source order.The TOFU split does not establish unseen-author or semantically disjoint-fact generalization; WMDP evaluates disjoint attacks only.Source: §E.2 · p. 21

E.3 EVALUATION METRICS

The evaluation protocol defines benchmark-specific forbidden scores, utility thresholds, and common release and attack configurations. Additional analyses use fixed configurations and single-run disjoint-TOFU observations; in the shared attack, GSU has lower extraction than the displayed controls while retaining measured utility.

E.3 EVALUATION METRICS

  • TOFU ES averages the longest correctly predicted suffix fraction over 200 attacker examples; WMDP reports domain accuracy at steps 25, 50, 75, 100, 125, and 150, with lower scores indicating stronger protection.Source: §E.3 · p. 21
  • For main comparisons, UWC calibrates each final defense checkpoint against its reference checkpoint to satisfy release utility Urel ≥0.95Uref before attack; the 90% attack-eligibility threshold is distinct.The reference utility remains fixed throughout the attack, and attack checkpoints are not recalibrated.Source: §E.3 · p. 21

E.4 MODELS AND INITIALIZATION

  • TOFU comparisons use six Llama and Qwen checkpoints with a shared retain-only base per model; WMDP defenses share the original zephyr-7b-beta checkpoint.The TOFU bases are trained on retain90 for five epochs.Source: §E.4 · p. 22

E.5 COMMON DEFENSE AND ATTACK CONFIGURATION

  • Main TOFU defenses and attacks run for ten epochs; WMDP defense uses 80 updates and its attack uses 150 updates, with evaluation every 25 updates.WMDP’s final defense checkpoint is utility-calibrated before release.Source: §E.5 · p. 22

F ADDITIONAL RESULTS

  • Additional analyses use disjoint TOFU records from the same authors and single runs without multi-seed error bars or significance tests.The split does not establish unseen-author or semantically disjoint-fact generalization, and repeated measurements from shared runs are not independent replications.Source: F ADDITIONAL RESULTS · p. 22
  • Trajectory utility is a perf_r composite score, displayed at 100× in trajectory plots; frozen GSU settings were selected on existing B2 attack evaluations, not independent-test comparisons.Fixed analysis configurations can differ from metric-specific main-table selections.Source: F ADDITIONAL RESULTS · p. 22

F.1 UTILITY DURING ACQUISITION

  • At epoch 10, GSU reaches 5.377% ES and 69.955 utility versus NPO+SAM’s 5.533% and 70.599, while staying below all displayed controls in ES throughout.The comparison trades slightly lower final utility for lower final ES; the observations concern attainable extraction within this attack budget, not a slower acquisition rate or UA90 ranking.Source: Figure 6 · p. 23, §F.1 · p. 23, Table 5 · p. 23

F.2 COMPLETE COMPONENT ABLATION

In the single-seed Qwen3.5-2B disjoint-TOFU ablation, full GSU has lower early extraction than the reported nonzero-utility variants. Removing retention yields zero utility, so low extraction alone does not establish a utility-preserving defense.

F.2 COMPLETE COMPONENT ABLATION

  • The five completed variants use frozen Qwen3.5-2B configurations on disjoint TOFU, and ES5 averages attack epochs 1–5 rather than including release.The GSU and without-seal controls are shared with the trajectory analysis, not independently repeated.Source: §F.2 · p. 24
  • 4.1740% mean ES for full GSU compares with 4.9598% without sealing, 5.0298% without suppression, and 5.2196% with random gates.Full GSU also has lower epoch-10 ES than these three variants, though the gaps narrow; release utility is comparable to, but slightly below, without-seal and random-gate utility, and above without suppression.Source: Figure 7 · p. 24, §F.2 · p. 24, Table 6 · p. 24
  • Without retention, utility is zero at both endpoints, so its low ES is not evidence of an advantage over full GSU.Zero composite utility does not mean every individual capability is absent; the ablation is a single-seed result and does not establish statistical significance or transfer to other models, identical data, or WMDP.Source: §F.2 · p. 24, Figure 7 · p. 24, Table 6 · p. 24

F.3 PAIRED SEALING ABLATION

The supplementary analyses find that GSU’s resistance persists across the measured attack trajectory and after benign adaptation, alongside selected-gate changes consistent with local attenuation. Benign fine-tuning remains possible in the tested settings, though outcomes vary by model and do not establish uniformly improved performance.

F.3 PAIRED SEALING ABLATION

  • At epoch 10, GSU’s ES is 5.377% versus 5.753% without sealing, a 0.376-percentage-point gap that remains present at every observed checkpoint.The gap narrows over the measured trajectory; because GSU starts lower, this supports lower extraction over the observed budget, not a smaller acquisition rate or permanent resistance.Source: Figure 8 · p. 25, §F.3 · p. 25

F.4 RELEASE-STAGE GATE DISTRIBUTIONS AND NEGATIVE-TAIL OCCUPANCY

  • At release, gate-wise median pre-activations shift more negative under GSU on B1, B2, and retain inputs than under the without-seal variant.On B1, the median moves from −0.2668 in Original to −0.5374 in GSU; on B2, from −0.2656 to −0.5159; on retain, from −0.2877 to −0.4358. These are distributions of gate-wise means, not individual token activations.Source: Figure 9 · p. 26, §F.4 · p. 26
  • At release, negative-tail occupancy rises from approximately 0.41% to 3.68%/3.35% on B1/B2 under GSU, versus 0.45% to 1.03% on retain.The corresponding reduction in DS is approximately 23–24% on B1/B2 and 11% on retain; these local probes do not measure full-network gradients, and most gates or tokens are not saturated.Source: Figure 10 · p. 26, §F.4 · p. 26, Table 7 · p. 27

F.5 SELECTED-GATE RESPONSES BEFORE AND AFTER ATTACK

  • After ten attack epochs, GSU retains a lower mean absolute derivative than without sealing, 0.373816 versus 0.419492, while attack partially reverses its release-stage gate changes.GSU’s mean pre-activation moves from −0.973011 to −0.456793, its derivative rises from 0.307875 to 0.373816, and its tail occupancy falls from 3.3482% to 0.5722%.Source: Figure 11 · p. 27, §F.5 · p. 27, Table 8 · p. 28
  • These gate probes are local mechanism evidence, not whole-network gradients or proof of causal isolation or irreversible forgetting.At release, about 3.35% of valid answer-prediction positions fall in the negative tail under the reported aggregation.Source: §F.5 · p. 28

F.6 BENIGN ADAPTATION: ACCURACY AND VALIDATION LOSS

  • Llama-3.2-1B and Llama-3.2-3B GSU economics accuracy rises from 43.529% to 44.591% and 60.918% to 63.613%, while Qwen3.5-2B declines under both initializations.For Qwen3.5-2B, GSU changes from 65.971% to 64.105% and No Defense from 65.166% to 64.397%.Source: Figure 12 · p. 28, §F.6 · p. 28
  • All three GSU models reduce validation NLL and finish within 0.005 of their paired No Defense controls after benign fine-tuning.GSU starts and finishes slightly higher, so larger reductions do not establish superior learning efficiency or statistical equivalence.Source: Figure 13 · p. 29, Table 9 · p. 29, §F.6 · p. 29

F.7 RESISTANCE AFTER BENIGN FINE-TUNING

  • After benign fine-tuning and ten attack epochs, GSU reaches 5.063% ES versus 6.502% for No Defense, with final utility of 68.894 versus 67.772.This comparison covers the tested benign task and attack budget; the result does not establish resistance to arbitrary subsequent training.Source: Figure 14 · p. 30, §F.7 · p. 30

G LIMITATION AND FUTURE WORK

The evaluation covers practical downstream fine-tuning, not unrestricted retraining from scratch with arbitrary compute.

G LIMITATION AND FUTURE WORK

  • The evaluation covers practical downstream fine-tuning rather than unrestricted retraining from scratch with arbitrary compute.The authors identify broader domain shifts, architectures, domains, and post-release procedures as directions for future work.Source: G LIMITATION AND FUTURE WORK · p. 30
Loading 2609.39866v1…