PaperlayerView exact revision on arXiv

Source-linked AI summary

Selective Knowledge Edit Reversal via Gated Singular Vector Shrinkage

Weifeng Jiang, Ruirui Chen, Qianren Mao, Junnan Liu, Qili Zhang, Kwok-Yan Lam

arXiv:2609.02091v1cs.CL

TL;DR

Knowledge editing can create harmful changes, while global reversal may remove beneficial edits alongside them. The paper proposes gated spectral shrinkage to selectively reverse targeted facts, and reports effective reversal with preservation of remaining edits across multiple settings. The approach is supported by evidence that edits can be sparse and separable within dominant singular components when edit counts are moderate.

  • Problem

    Knowledge-edit reversal for parameter-modifying methods remains challenging, and global removal can erase beneficial edits that should be preserved.

  • Method

    The framework uses learnable entry-wise gates over singular vectors to locate and shrink edit-sensitive components in the dominant singular subspace of edited weights.

  • Results

    Experiments across multiple models, editing methods, and benchmarks effectively reverse selected edits while largely preserving remaining edited facts.

  • Takeaways & Limitations

    Spectral analysis provides a promising direction for locating edit-specific components and repairing edited language models through selective reversal.

  • Takeaways & Limitations

    Selective reversal assumes the edits to reverse are known, requiring upstream editedness detection whose errors may propagate and are not investigated here.

Abstract

from arXiv · show

Knowledge editing provides an efficient way to update factual knowledge in large language models. However, malicious edits may introduce safety risks, making it necessary to reverse undesirable editing effects. Existing reversal methods for parameter-modifying edits mainly focus on global removal, which may also erase beneficial edits that should be preserved. In this paper, we study selective reversal of edited knowledge, where the goal is to reverse targeted edited facts while preserving the remaining edited facts. Based on the hypothesis that each edit is sparsely encoded within the dominant subspace of the edited matrix, we propose a spectral-based reversal framework that locates edit-sensitive components within the dominant singular subspace of edited weights. Experiments across multiple settings demonstrate the effectiveness of our method in reversing selected edits while preserving unrelated edited facts. These results suggest that different edits are sparsely encoded within dominant singular components and can be separable when the number of edits is moderate, making selective spectral reversal a promising direction for locating edit-specific components and repairing edited language models.

1 Introduction

Knowledge editing efficiently updates factual associations but can introduce harmful edits, while global reversal may also erase beneficial updates. This paper formulates selective reversal and proposes a spectral framework to reverse targeted facts while preserving other edits.

  • Knowledge editing updates specific factual associations while preserving unrelated knowledge, avoiding expensive retraining and catastrophic forgetting.
  • Knowledge editing can introduce backdoors, biases, jailbreaking, and misinformation, creating a need to reverse undesirable effects.
  • Global deletion of dominant singular components can remove all edits, including beneficial knowledge updates that should be preserved.
  • Selective reversal targets confirmed harmful or otherwise selected edits while keeping the outputs of remaining edited facts close to their edited-model behavior.
  • The proposed framework uses learnable entry-wise gates over singular vectors to locate and shrink edit-sensitive components within dominant rank-one components.
  • Experiments across models, editing methods, and factual benchmarks show effective selected-edit reversal while preserving remaining edits, with separability observed for moderate edit counts.

2 Related Work

Related work covers knowledge editing, editedness detection, and knowledge-edit reversal. The paper distinguishes its focus on selectively reversing targeted parameter-modifying edits from prior global-removal approaches.

  • Knowledge editing updates factual triples from original objects to new targets while preserving outputs for unrelated factual prompts.
  • Editing methods either preserve parameters through external memory or added parameters, or directly modify model parameters through locate-then-edit and meta-learning approaches.
  • The paper focuses on locate-then-edit methods because their simplicity and editing performance make them attractive to malicious attackers.
  • Editedness detection determines whether a model or output was influenced by editing, including detection of factual triples and edited relations or targets.
  • Prior reversal work addresses in-context editing or global removal of parameter-modifying edits, whereas this paper studies selective reversal with preservation of remaining edits.
  • Edit reversal is related to machine unlearning but additionally seeks to restore original model behavior after removing a particular edit.

3 Problem Statement

The problem assumes access only to an edited model and a detected set of facts selected for reversal. Selective reversal should restore selected facts toward pretrained behavior while preserving remaining edited facts.

  • The practical setting provides the edited model but not the original pretrained model or the complete editing history.
  • Existing editedness detection is assumed to identify selected subject-relation pairs for reversal using the edited model.
  • For each selected pair, the method constructs a semantically equivalent retrieval prompt, while the remaining edited facts are unknown during reversal.
  • Selective reversal transforms the edited model into a reversed model whose selected-fact outputs approach the pretrained model's outputs.
  • For remaining edited facts, the reversed model should retain output behavior close to that of the edited model.

4 Method Motivation

Spectral reversal is motivated by evidence that edited information concentrates in dominant singular subspaces, while individual singular components may mix multiple facts. The paper therefore hypothesizes sparse, separable edit encoding within those components.

  • Edited information is reported to concentrate in the dominant singular subspace, where removing top singular directions can move outputs toward the pretrained state.
  • A complete rank-one singular component may contain information about multiple facts because neural-network parameters are polysemantic.
  • AdaEdit suggests that strong editing can persist using top singular vectors and only the largest-magnitude entries, indicating sparse knowledge-related information in update matrices.
  • The proposed motivation is that different edited facts may be sparse and separable within dominant rank-one singular components.
  • This motivates fine-grained manipulation inside singular components through gated shrinkage of singular-vector entries.

5 Spectral-based Reversal Framework

The framework selectively reverses edited knowledge by gating and shrinking edit-sensitive entries within dominant singular components, while preserving residual weights. It optimizes these gates against a coarse reversal reference and intervention constraints, with augmentation and layer-wise procedures supporting robustness.

  • Gated Singular Vector Shrinkage: Selective reversal operates only on the dominant singular subspace, leaving the residual component unchanged.This design follows the hypothesis that edited information is concentrated in dominant singular components.
  • Gated Singular Vector Shrinkage: The method selects the top-r singular components, applies learnable entry-wise gates to their singular vectors, and reconstructs the modified weight matrix.Gate values near 1 preserve entries, whereas values near 0 strongly shrink them.
  • Gated Singular Vector Shrinkage: The gate matrices are parameterized through unconstrained variables and mapped into (0, 1) with the sigmoid function.Each gated singular-vector column is renormalized to stabilize the parameterization after shrinkage changes its scale.
  • Optimization of Gate Parameters: Gate optimization matches reversed-model outputs to a coarse reference model while penalizing hidden-state changes at prompt tokens other than the last subject token.The overall objective balances reference alignment and intervention control through λref and λint.
  • Optimization of Gate Parameters: Representation-level augmentation perturbs the last-subject-token representation so learned gates remain effective across small prompt variations.The perturbation is norm-controlled and scaled by a hyperparameter α.
  • Multi-layer Optimization: For multi-layer edits, the method reverses layers independently from higher to lower layers, writing each updated weight back before processing the next layer.Each layer is reversed relative to the behavior induced by already reversed later layers.

6 Experiments

Experiments evaluate selective reversal across models, editing methods, benchmarks, reversal settings, framework components, prompt types, and downstream tasks. The framework reverses selected edits while preserving remaining edits, with performance shaped by reversal scale, layer scope, and component choices.

  • Experimental Settings: The evaluation spans five language models, four locate-then-edit methods, ZsRE and CounterFact, and six downstream language-understanding datasets.The experiments compare selective reversal against global reversal and Re-edit baselines using agreement and KL divergence.
  • Selective Reversal: The framework effectively reverses selected edits, achieving reverse agreement close to the reference model in most settings and generally comparable performance to Re-edit.These results support identifying and suppressing edit-sensitive components within dominant singular components.
  • Selective Reversal: The remaining edited facts are largely preserved, with better and more stable preservation than Re-edit, especially for Reverse-50 on ZsRE.Re-edit is more likely to damage remaining edited knowledge when repeated subjects occur and more facts are reversed.
  • Selective Reversal: Reversing one fact at a time usually outperforms reversing 50 facts simultaneously because simultaneous reversal causes stronger interference among targeted edits.The interference makes corresponding edit-sensitive components harder to isolate.
  • Selective Reversal: Single-layer reversal is usually more effective than multi-layer reversal, while sequential and batch editing produce relatively small performance differences.Multi-layer editing may distribute the edit signal across layers, whereas the framework applies to both editing protocols.
  • Component Ablations: Lref drives reversal, Lint reduces interference with other edited knowledge, and renormalization helps preserve remaining edits.Removing Lref leaves the model nearly identical to the edited model; removing Lint degrades both sets, while removing renormalization mainly harms the remained set.
  • Prompt Generalization: Reversal generalizes from rephrase prompts to original editing prompts, with higher agreement and lower KL divergence than the edited model.Data augmentation further improves performance by making learned gates more robust to different expressions of the same fact.
  • Next-Token Optimization: Continuation agreement is much higher than first-token agreement after forcing the edited model to use the pretrained model’s first token.This supports next-token prediction as an effective optimization target for selective reversal.

7 Conclusion

The paper presents selective reversal as a way to remove targeted parameter edits while preserving other edited facts. Its findings provide empirical evidence that edit information can be manipulated at fine-grained locations within the spectral structure of edited weights.

  • 7 Conclusion: Selective reversal targets specific edited facts while preserving remaining edited facts, unlike methods focused on globally removing edited information.The approach assumes different edits can remain separable within dominant singular components when the number of edits is moderate.
  • 7 Conclusion: Experiments across multiple settings show effective reversal of selected edits while largely preserving the remaining edited facts.The findings suggest spectral analysis can help locate edit-specific components and repair edited language models.

Limitations

The framework has two main limitations: its selective reversal depends on global reversal quality, and it assumes the target edits are already known.

  • Global reversal dependence: Selective reversal may be limited by the quality of global reversal because the learnable gates use global reversal as their target.Global reversal remains a practical approximation of the pretrained state when the original model is unavailable.
  • Known-edit assumption: The framework assumes that the edits to be reversed are known, requiring an upstream editedness-detection stage.Errors in detection may propagate to the subsequent reversal process, which the work does not investigate.

Ethics Statement

The study uses public models and standard benchmarks in controlled evaluations without private user data or human subjects, while acknowledging potential dual-use risks.

  • Research setting: Experiments use publicly available language models and standard knowledge-editing benchmarks without collecting private user data or involving human subjects.Datasets may contain real-world entities, but they are used only to evaluate model behavior under controlled settings.
  • Dual-use considerations: Selective reversal may repair maliciously edited models, but locating edit-sensitive components could also reveal how edited knowledge is represented in model parameters.The study focuses on defensive reversal and preservation analysis rather than improving malicious editing.

A.3 Implementation Details

The experiments vary reversal scale and optimize gated singular-vector shrinkage with specified ranks, learning rates, epochs, and a matched-layer Re-edit baseline.

  • Experimental protocol: Reverse-50 experiments use two runs, whereas Reverse-1 experiments use 100 runs, with deviations calculated across runs.The reversal-scale settings therefore differ substantially in repetition count.
  • Gated shrinkage settings: Gate values start at 0.9, with dominant-subspace rank r = 256 for GPT2-XL and r = 512 for the other listed models.The other models are GPT-J 6B, Mistral-7B, LLaMA2-7B, and LLaMA3-8B.
  • Optimization: The gate parameters use Adam at a learning rate of 5×10^-2, with 1000 epochs for one-fact reversal and 500 epochs for simultaneous 50-fact reversal.These settings are applied for each target layer.
  • Baseline configuration: The Re-edit baseline uses AlphaEdit defaults while matching its editing layers to those of the original forward-editing method.For ROME-edited models, AlphaEdit is configured on the same single layer.

B.1 Results on GPT-J 6B and LLaMA2-7B

Results vary with editing structure, dataset, reversal scale, prompt form, and model: single-layer edits are easier to isolate, while multi-layer and large-scale edits create stronger entanglement.

  • Editing structure: The proposed framework performs consistently well on single-layer methods such as ROME and SimIE for GPT-J 6B and LLaMA2-7B.Its reversed-set performance is generally comparable to Re-edit, consistent with easier isolation in a concentrated rewrite layer.
  • Editing structure: GPT-J 6B performs relatively poorly with multi-layer MEMIT and AlphaEdit edits on CounterFact in both Reverse-50 and Reverse-1 settings.The authors attribute this pattern to edit signals becoming more entangled across layers.
  • Dataset dependence: On ZsRE, GPT-J 6B retains strong reversed-set performance for MEMIT and AlphaEdit, although remained-set performance is lower than with single-layer methods.Dataset, rephrase-prompt quality, and edited-association strength contribute to the observed variation.
  • Baseline comparison: Re-edit is generally stronger on CounterFact, whereas the proposed method is more competitive on ZsRE and preserves more remained-set examples in several Reverse-50 settings.The relative comparison therefore depends on dataset and editing structure.
  • Reversal scale: Reversing one fact is generally easier than reversing 50 facts, but spectral reversal can preserve more remaining edits when simultaneous re-editing interference is stronger.For GPT-J 6B with MEMIT and AlphaEdit on CounterFact, the number of reversed facts is not the main bottleneck; multi-layer entanglement is.
  • Prompt generalization: Jointly gating both singular-vector sides improves prompt-level generalization over single-side gating, whose reversal performance is weaker on original editing prompts.Single-side gating may fit rephrase prompts without capturing components robust across prompt forms.
  • Large-scale editing: After 1000 edits, selective reversal remains meaningful but is less stable, with large-scale multi-layer editing causing more difficult preservation.The global-reversal reference also becomes less reliable when the edited model moves substantially from the pretrained state.
  • Prompt generalization: Reversed-set performance generally decreases on original editing prompts, although the method still substantially improves over the edited model in many settings.Optimization uses rephrase prompts, while evaluation includes original editing prompts.

C Comparison of Re-editing Strategies

The comparison favors Alpha-Reverse over Method-Reverse across most settings, especially for preserving remaining edited knowledge. The section also explains that gated singular-vector shrinkage is renormalized to avoid weakening shared representations.

  • Re-editing baseline comparison: Alpha-Reverse generally outperforms Method-Reverse across most settings, particularly on preservation of remaining edited knowledge.The comparison reports Agreement and KL divergence on both remained and reversed sets, with higher agreement and lower KL divergence indicating better performance.
  • Re-editing baseline comparison: For multi-layer methods such as MEMIT, Method-Reverse can substantially damage the remained set, whereas Alpha-Reverse preserves considerably more remaining edits.
  • Re-editing baseline comparison: Alpha-Reverse achieves generally comparable or better reversed-set performance, so stronger preservation does not come at the cost of reversal effectiveness.
  • Re-editing baseline comparison: AlphaEdit is used as a unified Re-edit implementation because it performs better than reusing the original editing method and does not require identifying that method.The passage suggests its null-space-constrained updates may reduce interference with previous edits, but presents this only as a possible reason.
  • Effect of renormalization: Entry-wise gates suppress edit-sensitive singular-vector entries, while renormalization preserves each modified rank-one component's original Frobenius norm.Renormalization preserves the gating-induced output direction and relative suppression pattern while compensating for shrinkage's overall magnitude reduction.
Loading 2609.02091v1…