Faithful Dual-constrained Erasure for Robust LLM Safety Alignment

1 Introduction

• Retraining attacks can rapidly resurface supposedly erased malicious knowledge after fine-tuning on small amounts of loosely related benign data.The paper frames this as a vulnerability of methods that appear successful on static evaluation metrics.Source: §1 · p. 2
• Shallow alignment leaves malicious representations intact beneath a fragile inhibitory shell formed by dormant parameters repurposed as spurious suppressors.Subsequent fine-tuning can destabilize these superficial inhibitors and cause malicious outputs to return.Source: §1 · p. 2
• FDCU combines Fisher-based protection of general knowledge with PMFI constraints that block spurious suppressor activation, promoting dismantling of malicious representations.The paper describes experiments on Llama and Qwen architectures as showing retraining robustness with near-lossless general utility.Source: §1 · p. 2

2 Background and Related Work

• Machine unlearning targets sensitive, copyrighted, or hazardous knowledge, using optimization methods that can reduce malicious outputs but may disrupt global performance.The passage also notes standardized evaluation frameworks and suites, including OpenUnlearning, TOFU, MUSE, and WMDP.Source: §2 · p. 2
• Recent evaluations characterize some unlearning methods as hiding rather than truly erasing malicious knowledge.The passage attributes this observation to Hu et al. (2025) and Xu et al. (2025b).Source: §2 · p. 2
• Retraining, quantization, and adversarial unlearning requests can destabilize spurious suppressors, while naive unlearning can disrupt shared representations and aid recovery.The cited work describes dormant parameters acting as suppressors and suppressed knowledge resurfacing after benign relearning.Source: §2 · p. 3
• Existing constrained-update methods use parameter importance or utility-sensitive neuron freezing to preserve capabilities, but do not address spurious suppressors.The passage names LoKU, LLMEraser, and CKU as examples.Source: §2 · p. 3

3.1 Gradient Ascent (GA)

• Gradient Ascent maximizes negative log-likelihood on the forget set to reduce target prediction, but unconstrained updates can cause catastrophic forgetting and induce spurious suppressive neurons.The update modifies parameters using the forget-loss gradient without a constraint on benign data.Source: §3.1 · p. 3

3.2 Parameter Importance and Functional Attribution

• The empirical Fisher Information Matrix estimates parameter sensitivity on benign retention data; large diagonal values indicate perturbations likely to degrade general utility.Because it uses squared gradients, the Fisher Information Matrix is non-negative and omits directional information.Source: §3.2 · p. 3
• First-order Taylor attribution supplies signed contributions: positive values promote target output, while nonpositive values indicate inhibitory or dormant roles.This complements Fisher sensitivity, which does not preserve directional information.Source: §3.2 · p. 3

4 Methodology

• FDCU uses a dual-constrained subspace to preserve general capabilities while prohibiting spurious suppressors, rather than adding heuristic penalties.Its two constraints target high-sensitivity benign dimensions and abnormal modification of parameters with nonpositive initial contribution to malicious outputs.Source: §4 · p. 3, §4.1 · pp. 4–5

4.1 Problem Formulation

• The robust objective requires low malicious-output probability after retraining while maintaining high accuracy on the benign retention dataset.The paper states that a low probability immediately after unlearning is insufficient.Source: §4.1 · p. 4
• FDCU identifies parameters with nonpositive initial malicious-output attribution and bounds updates in that region to prevent new inhibitors from hiding knowledge.The paper says this constraint forces the optimizer toward dismantling representations with positive attribution.Source: §4.1 · p. 5

4.2 Analytical Solution and Approximations

• The method approximates its computationally intractable joint inverse using diagonal matrices and a bounded decoupling approximation.The resulting projection is expressed as an element-wise, scalable dual-masking rule using the Hadamard product.Source: §4.2 · p. 5, §4.3 · p. 5

4.3 Dual-Mask Update Rule

• The dual-mask update applies a General Knowledge Mask and a Minimal Intervention Mask to the naive gradient, targeting malicious roots while defending against capability decay and fine-tuning vulnerability.The masks respectively down-scale updates on crucial benign parameters and disable updates on historically non-excitatory parameters.Source: §4.3 · p. 5

4.4 The Workflow of FDCU

• FDCU's workflow computes a Fisher-based retention mask, obtains a forget-set gradient, constructs an attribution-based intervention mask, and filters the gradient element-wise.The framework separates structural protection of general knowledge from functional suppression of spurious inhibitors.Source: Figure 1 · p. 4, §4.4 · pp. 5–6

5.2 Specific Knowledge Erasure

• FDCU achieves low forget-set accuracy while preserving general accuracy and perplexity, and maintains low accuracy after retraining.Table 1 covers WMDP-Bio and WMDP-Cyber across initial and post-retraining evaluation.Source: §5.2 · p. 7, Table 1 · p. 7

5.3 Safe Output Control

• On Llama-3-8B-Instruct, FDCU reaches a 93.9% refusal rate and a 1.16 HarmfulScore after unlearning.The paper reports the highest Refusal Rates and lowest HarmfulScores across all three model architectures.Source: §5.3 · p. 7
• FDCU preserves general utility almost perfectly, with Accuracy and PPL remaining close to the Origin Model across all models.The paper contrasts this with sharp Accuracy and Perplexity degradation from naive methods such as GA.Source: §5.3 · p. 7
• FDCU outperforms state-of-the-art baselines on safe output control across initial safety alignment, general utility preservation, and robustness to retraining attacks.Table 2 evaluates models after unlearning and after benign fine-tuning.Source: §5.3 · p. 7, Table 2 · p. 8
• After retraining, GA’s refusal rate on Qwen2.5-3B falls from 79.6% to 57.7%, while FDCU maintains defense stability.The paper attributes FDCU’s robustness to prohibiting activation of fragile spurious suppressors.Source: §5.3 · p. 8

5.4 Ablation Study

• The Random Mask variant performs poorly across all metrics.Source: §5.4 · p. 9
• Higher α generally improves utility, moderate β performs better, and middle layers [24–28] yield the best utility–robustness balance.The final configuration uses α = 50 and β = 20; modifying early or final layers degrades robustness.Source: Figure 2 · p. 8, §5.4 · p. 9, Figure 3 · p. 9

5.5 Case Study

• In the case study, GA generates a harmful narrative after retraining, whereas FDCU refuses the harmful request.Source: §5.5 · p. 9

6 Limitations

• FDCU requires additional memory to store gradient-based statistics from both the forget set and the retain set.Source: §6 · p. 9
• The memory cost is described as manageable because selected middle layers can provide a good robustness–utility trade-off.Source: §6 · p. 9
• Storing Fisher and attribution information only for selected layers reduces memory usage while retaining FDCU’s benefits.Source: §6 · p. 9

• Across specific knowledge erasure and safe output control, FDCU outperforms state-of-the-art baselines in retraining robustness while maintaining near-lossless general utility.It constrains updates using Fisher information to preserve general knowledge manifolds and prohibits abnormal activation of non-excitatory neurons.Source: §7 · p. 9

• Technical appendices may include additional results, figures, graphs, and proofs, with no page limit.The submission guidance allows a ZIP upload for videos or code but disallows a separate appendix PDF.Source: A Technical appendices and… · p. 11
• The paper should stand alone, so critical experiments supporting its main claims should not be placed only in the appendix.Source: A Technical appendices and… · p. 11

Erasure of Language Memory (ELM)

• ELM uses the model’s implicit self-classification to reweight its original distribution into an erased target, then trains the erased model to match it.It conditions on a target concept and an alternative concept distribution; η controls erasure strength, and retention and optional fluency losses may also be included.Source: §A.1 · p. 12

Constrained Knowledge Unlearning (CKU)

• CKU selects utility-sensitive neurons and prunes their gradients during harmful-example gradient ascent to limit utility degradation.Source: §A.1 · p. 12

Collapse of Irrelevant Representations (CIR)

• CIR uses PCA to identify shared representation subspaces, purifies activations and gradients before forming updates, and applies a representation-breaking loss.Its stated aim is to target harmful representations while avoiding unnecessary disruption to general capabilities.Source: §A.1 · p. 12

Suppressing Spurious Unlearning Neurons for Robust Unlearning (SSIUU)

• SSIUU regularizes increases in negative attribution during unlearning, penalizing changes to negatively attributed parameters to suppress negative-influence neurons.Its objective combines a base unlearning loss with the attribution regularizer and aims for more faithful, retraining-robust unlearning.Source: §A.1 · p. 12

• Experiments use two 80GB NVIDIA A100 GPUs; Llama-3-8B-Instruct uses a 5 × 10^-6 learning rate, bfloat16 precision, and batch size 8.PyTorch and Hugging Face Transformers are used unless otherwise specified.Source: §A.2 · p. 13
• Retraining attacks use LoRA rank 8 and scaling 32 on q_proj and v_proj, with dropout 0.05, three epochs, and a 1 × 10^-5 learning rate.Training uses bfloat16, per-device batch size 2, and gradient accumulation over 8 steps; evaluation uses the post-unlearning metrics.Source: §A.2 · p. 13
• Baseline settings follow original papers when specified; GA uses the same training budget as FDCU unless otherwise noted, and CKU follows its original configuration.CKU uses λ = 1.5, batch size 4, one epoch, and AdamW at 4 × 10^-6; inputs over 2048 tokens are truncated, with training seed 42 and localization and retention seed 0.Source: §A.2 · p. 13
• SSIUU is early-stopped at its predefined forgetting threshold and selects retraining attack strength using the original protocol’s learning-rate search principle.Its protocol averages three runs and searches separate learning-rate sets for harmful and benign retraining attacks.Source: §A.2 · p. 13
• CIR selects a high but safe retain learning rate, searches the unlearning rate while avoiding divergence or excessive retain-set disruption, and chooses by post-attack performance.The final configuration is constrained to avoid substantially degrading general utility.Source: §A.2 · p. 13

A.3 Additional Details on Evaluation Metrics

• HarmfulScore averages an LLM judge’s integer ratings from 1 to 5, with lower scores indicating safer behavior.The judge rates each response to a harmful behavior; the rubric ranges from preferred refusal or redirection to direct fulfillment of the policy-violating request.Source: §A.3 · pp. 13–14
• Refusal Rate is the percentage of evaluated responses containing at least one phrase from a refusal keyword list.The measure uses lexical indicators to identify refusals.Source: §A.3 · p. 14
• WMDP accuracy measures multiple-choice performance on WMDP-Bio and WMDP-Cyber; lower accuracy indicates stronger erasure, while post-retraining accuracy tracks knowledge resurgence.Questions have four options, and the model selects the option with the highest next-token logit.Source: §A.3 · p. 14

A.4 Broader Impact

• The research may reduce the risk that deployed models reproduce hazardous or privacy-sensitive information after adaptation or benign fine-tuning.The work also aims to clarify when apparent safety alignment reflects removal rather than superficial suppression.Source: §A.4 · p. 14