When Forgotten Facts Return: Relearning-Aware Unlearning of Knowledge Graph Embeddings via Tangent Kernels

Parnian Fazel; Mohammad M Maheri; Hamed Haddadi

Source-linked AI summary

When Forgotten Facts Return: Relearning-Aware Unlearning of Knowledge Graph Embeddings via Tangent Kernels

Parnian Fazel, Mohammad M Maheri, Hamed Haddadi

arXiv:2610.31075v1Oct 22, 2026cs.LG

TL;DR

Ordinary continued training can restore forgotten knowledge-graph facts through entity embeddings shared with new facts, making release-time evaluation incomplete. KAU models that update with a closed-form tangent-kernel response and optimizes forgetting after the predicted change. Across datasets and backbones, it reduces leakage while retaining more than the compared unlearners.

Problem
Knowledge-graph unlearning is usually evaluated at release, although embeddings continue to change as new facts arrive and forgotten facts can recover beyond an identically updated retrained oracle.
Source: §1 · p. 1
Method
KAU uses a closed-form tangent-kernel response to anticipate how continued training changes forgotten-fact scores and differentiates through that response in its forgetting objective.
Source: §1 · p. 2
Results
On CoDEx-L, KAU reduces GraphDPO leakage by 59% on TransE, 50% on TransH, and 82% on RotatE, and attains the lowest ExRec, Leak, and RER on every dataset and backbone.
Source: §5.3 · p. 7
Takeaways & Limitations
Across the tested datasets and backbones, KAU reduces relearning while keeping retained-fact MRR above both baselines.
Source: §5.3 · p. 7
The linearized analysis assumes signed residuals and ridge or gradient-descent fitting, not Adam on the margin-ranking loss.
Source: §4.6 · p. 6

Abstract

from arXiv · show

Approximate unlearning answers a deletion request on a knowledge-graph embedding by editing entity embeddings instead of retraining from scratch. These embeddings keep being updated as new facts arrive. Under this ordinary continued training, forgotten facts recover more than in the exact-unlearning baseline, a model retrained without them and given the same continued training. We trace this relearning to entity embeddings shared by forgotten and new facts. To our knowledge, this is the first study of relearning in knowledge-graph embeddings. Relearning-robust methods developed for deep networks do not account for the graph structure underlying this recovery. We propose Kernel-Anticipated Unlearning (KAU), which uses the tangent kernel to model how continued training changes forgotten facts' scores through these shared parameters. KAU expresses this first-order response in closed form and differentiates through it inside the forgetting loss. The objective steers edits toward directions that continued training reverses less. We compare KAU with state-of-the-art knowledge-graph unlearners and relearning-robust methods under identical continued training, measuring leakage beyond the exact-unlearning baseline. Across three backbones, KAU reduces the strongest knowledge-graph unlearner's leakage by 46-63% on FB15k-237 and 50-82% on CoDEx-L, at higher retained utility.

1 Introduction

Ordinary continued training can undo knowledge-graph unlearning, as shared entity embeddings serve both forgotten and new facts. KAU anticipates this recovery through a tangent-kernel response and reduces relearning while preserving retained knowledge.

Measuring relearning

Leakage is recovery in forget-set MRR beyond an oracle retrained without forgotten facts and given the identical update.The oracle is needed because graph facts can be re-inferred from the remaining graph.Source: §1 · p. 1

The relearning problem

GraphDPO and ZeroFisher leak on every tested graph and backbone; on FB15k-237 with TransE, GraphDPO recovers beyond the oracle about as much as it removed.Figure 2 shows KAU staying closest to the oracle during the update.Source: §1 · p. 2, Figure 2 · p. 2

Why facts return

Forgotten facts return when continued training repairs shared entity rows that unlearning moved to lower losses on retained edges.The retrained oracle can re-infer facts through those rows, but it has no unlearning edit for the update to undo.Source: §1 · p. 2

KAU's contribution

KAU uses the tangent kernel's closed-form first-order response to anticipate score changes through shared parameters, then differentiates through that response in the forgetting objective.A score-preservation term keeps retained facts near their pre-edit scores, leaving the update less to repair.Source: §1 · p. 2
KAU achieves lower leakage and higher retained-fact rankings than GraphDPO and ZeroFisher across FB15k-237 and CoDEx-L with TransE, TransH, and RotatE.The objective differentiates through a closed-form response to continued training; its linearized theory describes a spectral filter that weights edits by resistance to reversal.Source: §1 · p. 2

Relearning defences

Adapted relearning defences do not close the gap to the retrained oracle, and add-ons to an existing unlearner leave most leakage.The paper reports that no tested add-on reduces leakage by more than 11%.Source: §1 · p. 2

2 Related work

Prior knowledge-graph unlearners use graph structure to scope edits or preserve neighbours, while relearning defences from deep networks use update simulations, proxies, or constrained subspaces. KAU instead models how update edges move forgotten-fact scores through a closed-form tangent-kernel response.

Knowledge-graph unlearning

Knowledge-graph unlearners use graph structure to edit neighborhoods or preserve neighbors, but do not model how training on those neighbors changes forgotten-fact scores.KAU edits entity rows at forgotten facts' endpoints and evaluates their scores after a closed-form update response.Source: §2 · p. 3

Relearning-robust methods

Deep-network relearning defences use simulated updates, proxies, or subspaces, but graph embeddings have no inert entity rows and retained-data subspaces miss never-trained update edges.KAU models the update's edges to first order, including stand-ins for unseen edges; in its linearized analysis, several subspace methods correspond to hard thresholds of its survival filter.Source: §2 · p. 3

Tangent-kernel response

Unlike prior tangent-kernel work on fine-tuning, task interference, or scrubbing, KAU places the post-edit update response inside its objective and differentiates through it.Source: §2 · p. 3

3 Problem formulation

The problem is to edit a released embedding so forgotten facts stay forgotten after an unknown continued-training update, while retained facts remain useful. Leakage compares the released model's recovery with that of an identically updated retrained oracle.

3 Problem formulation

A knowledge-graph embedding stores entity and relation rows and scores triples; link prediction ranks candidate triples by their scores.A triple's score depends on its head, relation, and tail rows.Source: §3 · p. 3
The unlearner edits only involved entity rows, while the ideal release is retraining from scratch without forgotten facts.The oracle uses the original model's training recipe; the never-trained set contains true edges unseen in training but drawn by the update.Source: §3 · p. 3
After release, the user trains on retained and never-trained true edges, with the same update given to each release and its oracle.The unlearner knows the update class but not its unseen edges or particular draw.Source: §3 · p. 3
Leakage is the released model's increase in forget-set MRR under the update beyond the oracle's increase; retained-set MRR is read alongside it.The oracle has zero leakage, and negative leakage means slower relearning.Source: §3 · pp. 3–4
The unlearner seeks forgotten facts below a target after the update while preserving retained-edge scores, but cannot evaluate the objective because the update uses unknown edges.The formulation takes an expectation over the update class; the next section replaces the multi-step run with a closed-form linearized response.Source: §3 · p. 4

4 Kernel-anticipated unlearning

KAU anticipates the score of each forgotten fact after a modeled update and differentiates through that response when optimizing the forgetting objective. Its procedure models known and unseen edges, controls the update response, and favors edits that survive continued training.

4 Kernel-anticipated unlearning

KAU replaces the unknown update in the forgetting loss with a closed-form linearized response and acts on each forgotten fact's smaller current or predicted post-update score.Because a lower score means stronger belief, forgetting requires both scores to reach the target; differentiating through the response steers edits toward post-update forgetting.Source: §4 · p. 4

4.1 The update model and its residual

Each modeled update draw combines retained edges at involved entities with plausible stand-ins for unseen true edges, using one-sided residuals that restore raised edges but not lowered ones.Fresh draws are used at each forgetting step; the one-sided residual and minimum-score loss block an edit that would rely on the modeled update to restore lowered edges.Source: §4.1 · p. 4

4.2 The response in closed form

The response is a damped joint solve over involved rows, exploiting triple scores' sparse dependence on entity rows and freezing features during differentiation.The response omits Adam's per-coordinate scaling and feature motion; KAU recomputes features at each forgetting step but freezes them during differentiation.Source: §4.2 · pp. 4–5

4.3 The anticipated score and its target

KAU evaluates the shifted rows with the backbone's exact score function, then targets forgotten facts using a threshold derived from the original model's corrupted-fact scores.The target lies a fraction of the way toward the 5% quantile of corrupted-copy scores when the original score is below that level.Source: §4.3 · p. 5

4.4 Two controls on the response

KAU controls recovery by preserving retained-edge scores and aligning edits with directions the modeled update restores least.The retained-edge term bounds the known-channel response; the anticipated-loss gradient favors the least-restored directions.Source: §4.4 · p. 6

4.5 The full procedure

Each epoch alternates a retained-edge utility pass with forgetting steps, and the released checkpoint maximizes a weighted development-set score.Each forgetting step draws edges, solves the response, scores forgotten facts at shifted rows, and updates with Adam.Source: §4.5 · p. 6

4.6 What survives a linearized update

In the linearized model, the update applies a survival filter that retains less of edits strongly affecting drawn-edge scores and more of weaker directions.Under uniform ridge, KAU's anticipated loss becomes the plain forgetting loss on the surviving edit; the analysis assumes signed residuals and ridge or gradient-descent fitting, not Adam on the margin-ranking loss.Source: §4.6 · p. 6

5 Experiments

The experiments compare unlearning methods under identical continued training, measuring recovery beyond an oracle retrained without deleted facts. KAU has the strongest reported robustness across datasets and backbones, while retaining more utility than the baselines.

5.1 Experimental settings

The evaluation uses FB15k-237 and CoDEx-L, with 10% of training triples forgotten and a disjoint set of about 2.2% reserved as unseen facts for relearning evaluation.TransE is the primary backbone; TransH and RotatE test whether findings extend across scoring functions.Source: §5.1 · p. 7

5.2 Evaluation metrics

Leak and RER measure recovery beyond the oracle’s rise and are the primary metrics; each method is evaluated at its peak-leakage epoch.The highest-leakage update configuration is selected from 48 shared configurations using one half of a deleted-fact probe, then reported on the other half.Source: §5.2 · p. 7

5.3 KAU improves robustness to relearning

KAU attains the lowest ExRec, Leak, and RER on both datasets and all three backbones, while keeping retained MRR above both baselines.Its leakage reduction is 59%, 50%, and 82% on CoDEx-L for TransE, TransH, and RotatE, respectively.Source: §5.3 · p. 7, Table 1 · p. 8
KAU stays closest to the oracle through the TransE update and has the lowest mean RER at every forgetting depth and backbone.From shallowest to deepest release, its RER rises by less than half as much as either baseline’s.Source: §5.3 · p. 7, Figure 3 · p. 8

5.4 Relearning-resistant add-ons do not close the gap

No tested relearning-defence add-on reduces GraphDPO’s leakage by more than 11%, and none lowers it on RotatE; KAU leaks 43–61% less than the best add-on.The comparison tests NTK projection, SAM, and TAR on FB15k-237.Source: §5.4 · p. 7

5.5 The anticipated objective outperforms the tested alternatives

Removing anticipation more than doubles leakage, while removing score preservation raises leakage and lowers retained MRR to 0.2788.The ablation also finds that removing either update channel increases leakage by roughly one third.Source: §5.5 · p. 8
KAU achieves the lowest ExRec, Leak, and RER among the tested forgetting objectives; SAM-GP, the best alternative, has Leak 0.0150 versus KAU’s 0.0111.The comparison uses FB15k-237 with TransE and a shared forgetting-rate band.Source: §5.5 · p. 8, Table 2 · p. 9

6 Conclusion

The paper concludes that ordinary continued training can restore deleted facts beyond what a retrained model recovers. KAU anticipates that update through the tangent kernel and substantially reduces relearning while preserving retained knowledge; experiments are limited to two public benchmarks and three backbones.

6 Conclusion

KAU predicts in closed form how continued training moves forgotten facts through the tangent kernel, then forgets them after that move.The conclusion presents this as a promising step toward deletions that last in knowledge-graph embeddings.Source: §6 · p. 9

Reproducibility statement

The experiments use public FB15k-237 and CoDEx-L benchmarks with TransE, TransH, and RotatE, each under a fixed-seed 10% forget split.The paper reports that no personal data were collected and no human subjects were involved.Source: Reproducibility statement · p. 9, Ethics statement · p. 9

AI use statement

The authors used AI tools for writing and presentation and for auditing mathematical proofs, and state that they reviewed and verified the outputs.The authors retain responsibility for the paper’s correctness and content.Source: AI use statement · p. 9

A Analysis and proofs

The analysis characterizes continued training as a filter on an unlearning edit in a linearized model, then relates KAU’s anticipated loss and subspace methods to that filter. It also states local nonlinear results and identifies where the modelled response differs from the empirical update.

A.1 What survives a linearized update

KAU’s anticipated loss is the plain forgetting loss evaluated on the edit that survives the update, rather than on the pre-update edit.The analysis states this identity for the linearized model and relates its gradient direction to the per-entity filter.Source: §A.1 · pp. 14, 15, 17
In the linearized model, the update maps an edit to Sδ, with S = (Φ⊤Φ + Σ)−1Σ; directions in the update-feature span are shrunk, while the orthogonal complement survives unchanged.With uniform ridge, each singular direction is scaled by σ/(λk + σ), and S approaches projection off the feature span as σ approaches zero.Source: §A.1 · pp. 14, 15, 17
Under a uniform ridge, subspace constraints correspond to hard thresholds of the update filter; null-space projection is the zero-threshold case.The retained-span projector protects against first-order movement on retained edges, but unseen-channel stand-ins can add directions outside that span.Source: §A.1 · p. 16
The one-sided residual expands the edits left untouched from the subspace Φδ = 0 to the cone Φδ ≤ 0, which contains that subspace.This states the condition in the linearized model.Source: §A.1 · p. 16
Outside the linearized model, the theorem is local and assumes frozen features, first-order post-update score evaluation, and signed residuals on drawn edges.For TransE, the sign-pattern feature remains constant while no translation-error coordinate changes sign.Source: §A.1 · pp. 16–17
Theorem A.1 identifies the anticipated-score gradient with the per-entity filter applied to the forgotten fact’s plain-forgetting direction.The analysis averages per-draw filters; its limits include projection off retained edges and stand-ins, or off retained edges alone when the unseen channel is removed.Source: §A.1 · p. 17
The appendix distinguishes the joint response from per-entity block solves and the exact score evaluation from first-order evaluation as empirical counterparts of the analysis.Table 16 compares these choices, Table 17 compares sampled and expected draws and score composition, and Table 18 reports retained-span projection at KAU’s forgetting depth.Source: §A.1 · p. 17
The filter models a ridge or gradient-descent fit to squared residuals, whereas the experiments use Adam on ranking loss with sampled corruptions and moving features.At matched TransE forgetting depth, the empirical update restores 0.0111 forget-MRR for KAU, 0.0158 for retained-span projection, and 0.0250 for plain forgetting beyond the oracle’s rise.Source: §A.1 · p. 18

A.2 Proof of Lemma 1: the linearized response

The appendix derives a ridge-damped linearized response for continued training, relates it to early-stopped updates, and bounds what score preservation controls. It also characterizes one-round approximation error and the limits of the response’s predictions.

A.2 Proof of Lemma 1: the linearized response

The continued-training response minimizes ridge-damped linearized least squares and has a closed form obtained by the push-through identity.The proof identifies this response with one damped Gauss–Newton step.Source: §A.2 · p. 18
With σ = 1/(ηT), the ridge filter brackets an early-stopped linearized update within a factor of two on every eigen-direction.The comparison assumes ηλmax(K) ≤ 1.Source: §A.2 · p. 18

A.3 Score preservation is the known-channel residual

The score-preservation term bounds the known-channel response when its sample includes the drawn retained edges, but it does not control the unseen-channel part.For a joint solve, the bound uses the minimum ridge across involved entities and requires coverage of every drawn retained edge.Source: §A.3 · pp. 18–19

A.4 Beyond the linearized response

One re-linearization round has second-order correction terms, but its predicted score recovery tracks fact rankings better than recovery levels.The reported correlations are Pearson 0.53 and Spearman 0.21; the lemma does not bound the level gap for many-epoch updates.Source: §A.4 · p. 20
The feature-smoothness hypothesis applies only along segments without sign changes and with row norms bounded below.Across a sign change, the crossed-coordinate excess can be first-order in the displacement rather than second-order.Source: §A.4 · p. 20

A.5 Proof of Theorem A.1: the operator identity

The proof differentiates the per-entity response through the signed residual to obtain an operator for the anticipated score. It distinguishes that shrinkage interpretation from one-sided residuals and shows how joint updates couple entities.

A.5 Proof of Theorem A.1: the operator identity

Differentiating the signed residual makes the per-entity response operator enter both through the update and through the residual, yielding equation 13.The operator’s shrinkage properties follow from its spectral form.Source: §A.5 · p. 20
With a one-sided residual, only positive-residual edges enter through the residual, while every drawn edge still enters through the kernel.The resulting operator is generally nonsymmetric and can have norm greater than one, so the shrinkage interpretation is specific to the signed residual.Source: §A.5 · p. 21
As σv approaches zero with Kv invertible, the operator becomes an oblique projector that annihilates active features.This identity holds within parameter-space regions where the active set is constant.Source: §A.5 · p. 21
In a joint response, cross-entity terms vanish when no drawn edge has two involved endpoints; otherwise they generically couple entities connected by drawn-edge paths.These cross-entity terms are absent from the per-entity projector.Source: §A.5 · p. 21

A.6 Proof of Corollary A.3: the soft projector and its two limits

This section analyzes the soft projector’s limits and KAU’s update-aware forgetting, including survival filtering, residual forces, and the modeling and evaluation of update responses. It also describes KAU’s implementation and target selection, and reports its model-release time on CoDEx-L with TransE relative to GraphDPO and retraining.

A.6 Proof of Corollary A.3: the soft projector and its two limits

The soft projector becomes the projector off all retained-edge and stand-in features when all are drawn, and off retained-edge features alone in the retained-span limit.In the retained-only limit, its per-entity constraint matches retained-span methods for losses depending on retained scores; sampled-negative ranking losses add corrupted-triple features.Source: §A.6 · p. 22
With the sampling cap, no draw reaches the full-rank regime: kmax = 96 leaves tangent directions untouched for 200-wide TransE/TransH rows and unnormalized 400-wide RotatE rows.The limits lift the cap, and uncapped updates on high-degree entities can enter that regime.Source: §A.6 · p. 22

A.7 Proofs for the survival filter

The survival filter preserves an edit exactly when Φδ = 0; ridge and gradient-descent filters damp components in the update-feature span while leaving its orthogonal complement unchanged.As ridge strength tends to zero or gradient steps tend to infinity, the filter approaches the projector onto that complement.Source: §A.7 · pp. 22–24
The anticipated forgetting objective evaluates the forgetting loss at the filtered edit, and its gradient follows the anticipated-score direction, with the minimum changing only its scalar weight.For nonlinear scores, the first-order identity holds with features frozen at the current point and the filter rebuilt at each step.Source: §A.7 · p. 23

A.8 Proof of Proposition A.1, per-entity form, and the separate penalty

A separate penalty can target predicted score recovery alongside the pre-update forgetting loss, with the predicted post-update score movement bounded by the same recovery term.The anticipated forgetting loss acts on the minimum of the pre- and post-update scores.Source: §A.8 · pp. 24–25

A.9 Expected force of the margin-loss update

The one-sided residual is a conservative force bound: it credits every score raise in full and no lowering, while measured slack and temperature gate the force on update edges.The expected margin force is a sigmoid gate; setting all slack to zero yields the one-sided residual.Source: §A.9 · p. 25
Inactive edges can still be restored because active-edge responses change their scores when their features are not orthogonal to active features.The response in equation 6 instead keeps every drawn edge in the kernel, including zero-residual edges.Source: §A.9 · p. 25

A.10 Expected loss over the update class

The expected loss is averaged over update-class draws, rather than applied to an averaged response.This subsection states that distinction for the forgetting loss under first-order evaluation.Source: §A.10 · pp. 25–26

A.11 Block-diagonal versus joint response

When no drawn edge joins two involved entities, joint and block-diagonal responses coincide; as ridge values vanish, a shared edge’s residual is removed twice by the block-diagonal response but once by the joint response.The latter limit assumes linearly independent features at each entity; the joint response interpolates the drawn edges once together.Source: §A.11 · p. 26

A.12 Within-row isotropy and the ridge

Under within-row isotropy, Adam’s preconditioned kernel is approximately the Euclidean kernel up to per-row scalars, which can be absorbed into a per-entity ridge.The isotropy argument also makes the sign-gradient kernel and metric orthogonality approximately Euclidean up to those scalars.Source: §A.12 · p. 27

A.13 Piecewise-linear scores and exact evaluation

For piecewise-linear L1 scores, evaluating the score at shifted rows captures a nonnegative correction that tangent features alone cannot model when a displacement exceeds the original coordinate magnitude.The correction vanishes when each displacement magnitude is at most the corresponding original coordinate magnitude.Source: §A.13 · p. 27

B.1 The update draw and its stand-ins

KAU models continued updates with draws of retained edges and manufactured unseen-edge stand-ins, capped at 96 edges per entity and freshly sampled at each forgetting step.The unseen-edge stand-in pool has 32 triples per involved entity, refreshed every five epochs; a draw leaves feature directions outside its span unaffected by the response.Source: §B.1 · p. 28

B.2 Targets

For each forgotten fact, KAU sets a fixed target from the original score and the 5% quantile of 128 corrupted scores, moving well-believed facts a fraction toward that tail.The target is computed before the run and is the point beyond which the one-sided forgetting loss exerts no force.Source: §B.2 · p. 28

B.3 Pessimistic composition

The forgetting loss uses the lower of current and anticipated scores while routing gradients through the anticipated score, and KAU retracts edited entity rows to their original norms after each optimizer step.The minimum prevents a predicted score rise from lowering the loss; retraction fixes a gauge for TransE and TransH and constrains edits for RotatE.Source: §B.3 · p. 28, §B.4 · p. 28

B.5 Score preservation

KAU preserves retained-edge scores with a squared residual term and selects the released checkpoint using GraphDPO’s development-set rule, without consulting retraining or update edges.The algorithm trains only involved entity rows, keeps relation rows fixed, solves the response by conjugate gradients, and applies the shared hyperparameter-selection rule per backbone.Source: §B.5 · p. 29, §B.6 · p. 29, §B.7 · p. 29

B.10 Cost

5.4 times faster than GraphDPO and 18 times faster than retraining, KAU releases one unlearned CoDEx-L TransE model.The comparison uses unlearning seeds 21 to 27 and four CPU cores.Source: §B.10 · p. 30

B.11 Constructions the derivation admits and we do not use

The appendix describes unused constructions and reports the experimental details and full CoDEx-L results.

B.11 Constructions the derivation admits and we do not use

The second Gauss–Newton round re-linearizes the response on the same draw to model feature drift, but improved no reported quantity.It adds a correction to the first round.Source: §B.11 · p. 30
Allowing features to carry gradient opens a channel for reshaping update features rather than their residual, performed worse, and was therefore excluded.Features carry no gradient within a step.Source: §B.11 · p. 30
Holding stand-ins at their original scores leaves the unseen channel without residual at release, costs utility, and does not move leakage.Source: §B.11 · p. 30
Adam’s per-coordinate scaling motivates a hypothesized ridge that decreases with update-edge share, but every reported experiment uses a degree-independent ridge.The proposed ridge falls with the inverse square root of an entity’s share of the steps.Source: §B.11 · p. 30

C Experimental details

The appendix gives graph splits, update edges, method budgets, seeds, and tests, and reports the CoDEx-L results in full.Source: C Experimental details · p. 30

C.1 Training details

The appendix specifies matched training and evaluation protocols, then details how unlearning settings, continued updates, checkpoints, and add-ons are selected. Its statistical checks find that KAU’s lower leakage than GraphDPO persists across retrained-oracle references.

C.1 Training details

FB15k-237 and CoDEx-L experiments used held-out probes of 1,000 deleted triples for relearning evaluation.The probes were drawn once and sealed; the FB15k-237 forget set contained 27,211 triples and the CoDEx-L forget set 55,119.Source: §C.1 · pp. 30, 32
Original and gold models shared architecture, training recipe, and seed, differing only in training triples; both loss curves flattened before the schedule ended.Relearning quantities are differences against the gold model rather than absolute levels.Source: §C.1 · p. 32, Figure 4 · p. 32

C.2 Unlearning stage

Hyperparameters were tuned separately for each method and dataset–backbone setting using a shared shortfall criterion, while ZeroFisher showed greater seed variation than KAU or GraphDPO.ZeroFisher varied by 0.02 to 0.04 MRR at fixed hyperparameters, versus below 0.002 MRR for KAU and GraphDPO.Source: §C.2 · pp. 32–33

C.3 The update

Each method and retrained oracle received identical updates without forgotten triples, and methods were reported at the selected configuration’s peak leakage.The update grid varied never-trained edges, retained-neighbour sampling, other retained edges, and learning rate across 48 configurations per dataset and backbone.Source: §C.3 · pp. 33–34

C.4 Budgets and selection

The released checkpoint maximized a weighted score of forget-MRR drop and retained MRR on held development subsets, using the same rule for every method.The rule did not consult the oracle or update edges.Source: §C.4 · p. 34

C.5 Relearning-resistant add-ons

Three relearning-defence add-ons were evaluated on GraphDPO, with each reported under its most-leaking update configuration.Rows, including GraphDPO and KAU, used three unlearning seeds and the same three to six update configurations per backbone.Source: §C.5 · pp. 34–35

C.6 Statistics

Across every backbone, KAU’s leakage was below GraphDPO’s at the 0.1% significance level and below ZeroFisher’s at the 5% level in both tests.The tests were two-sided Welch’s t-tests and Mann–Whitney U tests over seven seeds per method.Source: §C.6 · p. 35, Table 8 · p. 36
KAU’s leakage remained lowest across three oracle references on every backbone, with a 44–67% reduction over GraphDPO.The oracle’s own update-induced rise varied across retraining seeds, but the reported reductions held whichever oracle seed served as reference.Source: §C.6 · p. 35, Table 9 · p. 36

C.7 CoDEx-L

On CoDEx-L, KAU preserves its advantage across all three backbones. Additional tests find lower leakage across update configurations, under a never-trained-edge update, and through an anticipated-loss mechanism operating on the entity rows that carry recovery.

C.7 CoDEx-L

KAU has the lowest excess recovery, leakage, and relative excess recovery on all three CoDEx-L backbones; its leakage is 59%, 50%, and 82% below GraphDPO’s on TransE, TransH, and RotatE.Its retained MRR exceeds GraphDPO’s by 0.032, 0.044, and 0.103, respectively.Source: §C.7 · pp. 37–38, Table 11 · p. 37

D.1 Leakage across update configurations

KAU leaks less than GraphDPO and ZeroFisher in every one of the 48 FB15k-237 update configurations, with leakage at 0.47 to 0.58 of GraphDPO’s in every group.The groups’ leakage ranges are 0.0215–0.0249 for KAU and 0.0425–0.0470 for GraphDPO; the reported table uses maxima over configurations.Source: §D.1 · p. 38, Table 12 · p. 38

D.2 The never-trained-edge update

Under a never-trained-edge-only update, KAU’s leakage is 0.33 of GraphDPO’s, compared with 0.54 under the standard update.GraphDPO retains 68% and ZeroFisher 79% of their standard-update leakage under the never-trained-only update; KAU retains 42%.Source: §D.2 · p. 39, Table 13 · p. 39

D.3 The forgetting term against the ancillaries

The anticipated forgetting term lowers leakage independently of forgetting depth, while omitting the score anchor raises leakage by almost half and lowers retained utility by 0.007.The relational anchor leaves the score residual free; the ablation finds the anticipated term, rather than deeper forgetting, accounts for the leakage reduction.Source: §D.3 · pp. 39–40

D.4 Which parameters the update restores

Involved endpoint rows reproduce 95–105% of each model’s full rise in forget-MRR, while relation rows reproduce at most 5.4% for the unlearners and 9.5% for the oracle.Across seven seeds, involved rows reproduce 94–105% of the rise for KAU, GraphDPO, and the oracle, and 91–105% for ZeroFisher.Source: §D.4 · p. 40, Table 14 · p. 40

D.5 The anticipated response as a per-fact ranking

The anticipated response is evaluated as a per-fact ranking of recovery. Joint exact evaluation ranks realized recovery better than the alternatives, while its predicted score changes do not match the recovery level.

D.5 The anticipated response as a per-fact ranking

The joint response with exact evaluation reaches a Pearson correlation of 0.53 and a Spearman correlation of 0.21 per fact.Across 1,000 held-out forgotten facts, predicted-recovery quintiles rise monotonically from +0.003 realized recovery in the lowest quintile to +0.459 in the highest.Source: §D.5 · p. 40, Table 15 · p. 40
Joint exact evaluation ranks facts better than first-order evaluation by Spearman correlation, 0.21 versus 0.13, though first-order evaluation has higher Pearson correlation, 0.61 versus 0.53.The per-entity block solve reaches Pearson correlation 0.30; a fact with both endpoints edited must be solved jointly for an accurate prediction.Source: §D.5 · pp. 40–41
The exactly evaluated response predicts ranking rather than recovery level: mean predicted change is −0.009 against realized +0.159, while first-order evaluation predicts +0.145.Across seven seeds, the joint response has median Pearson and Spearman correlations of 0.50 and 0.23.Source: §D.5 · p. 41

D.6 Leakage and retained utility against forgetting depth

Across forgetting depths, KAU’s RER rises less than the baselines’ and its retained utility remains higher. Its peak RER also remains below the baselines’ during epoch-by-epoch relearning on RotatE.

D.6 Leakage and retained utility against forgetting depth

KAU’s mean RER rises about 15 points from shallowest to deepest TransE forgetting, versus 50 for GraphDPO and 59 for ZeroFisher; the method ordering holds in every band on every backbone.At the deepest TransE band, mean RER is about 26 for KAU, 77 for GraphDPO, and 131 for ZeroFisher.Source: §D.6 · p. 41
KAU retains more than 94% of the gold model’s retained MRR at every depth on every backbone, while GraphDPO’s retained-MRR loss from shallowest to deepest is 4.7, 5.1, and 13.6 points on TransE, TransH, and RotatE.The figure reports retained MRR at release as a percentage of the gold model’s retained MRR.Source: §D.6 · p. 41, Figure 5 · p. 41
On RotatE, KAU’s peak RER stays below 15 in every band and seed, while the lowest peak of either baseline is 36.ZeroFisher peaks within the first five epochs, whereas GraphDPO continues rising until the final epochs.Source: §D.6 · p. 41, Figure 6 · p. 42

D.7 Ablation details

This section reviews KAU’s ablations and relearning-aware evaluation, including leakage measured against a retrained oracle under identical continued training. It also discusses graph-specific effects of continued training and why exact alignment between forgotten facts and retained neighbors calls for storage-side defenses.

D.7 Ablation details

KAU ablations vary response modeling, residual treatment, sampling, and score anchoring, while alternative objectives replace only the forgetting step.The alternatives include retained-span projection, sharpness-aware methods, weight-distance regularization, connectivity-based fine-tuning, and unrolled or one-shot methods.Source: §D.7 · pp. 42, 44, 45, Table 18 · p. 44
Forgetting rates at release span 88–100% of the gold model’s rate, versus 93% for KAU; the no-forgetting-term variant reaches less than half KAU’s depth.Every other reported forgetting rate lies within 9 percentage points of KAU’s; the no-forgetting-term variant cannot be brought to KAU’s depth.Source: §D.7 · pp. 44–45, Table 17 · p. 44

D.8 Why exact evaluation on piecewise-linear scores

Evaluating the score at shifted rows captures zero-crossing effects that first-order evaluation misses, at the cost of one additional score-function call.The score is nonlinear in the rows, and predicted displacements may be large relative to coordinate differences; exact evaluation ranks realized recovery better than linear evaluation.Source: §D.8 · p. 45

E.1 What the table shows

No prior method gives a closed-form, per-fact account of how update data affect forgotten items; graph recovery requires pairwise relations among examples.On knowledge graphs, a retained entity row aggregates gradients over incident edges, so aggregate scalars or sign masks cannot identify which neighbour edge restores a fact.Source: §E.1 · p. 48
KAU keeps TransE retained utility within 0.01 of the oracle’s: 0.2839 versus 0.2932, while GraphDPO and ZeroFisher lose 0.03 and 0.06.The discussion notes that methods resisting relearning can incur utility loss or self-destruction.Source: §E.1 · p. 49

E.2 Scope relative to these studies

KAU computes a per-fact anticipated post-update score, whose predicted recovery correlates with realized recovery at Pearson 0.53 and Spearman 0.21.Its response is closed-form to first order for a benign update recipe with 48 configurations fixed in advance and shared across methods.Source: §E.2 · p. 49

E.3 Unlearning in knowledge graphs and graph networks

Knowledge-graph and graph-network unlearners use graph structure to scope edits or preserve neighbours, but do not model how neighbour training changes forgotten scores.The paper extends evaluation beyond the released model’s pre-fine-tune ranking to assess recovery after training resumes.Source: §E.3 · p. 50

E.4 Benign relearning and the update model

The paper evaluates benign relearning by comparing against a retrained model receiving the same update, which contains no forgotten facts.This adapts the setting to knowledge-graph embeddings and uses a closed-form account of the update’s linearized response rather than an informal correlation.Source: §E.4 · p. 51
Yoon et al. (2026) defend against benign relearning by paraphrasing, but triples cannot be paraphrased and their alignment with retained neighbors is exact.This removes the data-side remedy and motivates a storage-side one.Source: §E.4 · p. 51
Suriyakumar et al. (2026) find that disjoint fine-tuning can resurrect erased diffusion concepts, and this work turns span overlap into an objective for choosing the edit.Rather than measuring leaked gradient mass after the fact, the edit is chosen against the overlap.Source: §E.4 · p. 51
Dehghani et al. (2026) measure excess recoverability against an identically audited retrained model; this work reports oracle-relative leakage in MRR points and as a share of erased knowledge.Both are reported beside forgetting at release because methods can trade one for the other.Source: §E.4 · p. 51

E.5 Relearning-robust unlearning methods

Relearning-robust unlearning methods either simulate continued training, use proxies for its effects, or choose where an edit resides. KAU instead differentiates through a closed-form, graph-local tangent-kernel response to the update.

E.5 Relearning-robust unlearning methods

Unrolled methods simulate the update, proxy methods approximate it, and subspace methods choose where the edit resides.KAU replaces simulation with the closed-form linearized response for a shallow relational kernel.Source: §E.5 · pp. 51–52
Deep-network methods leave graph structure unused: they do not identify which training edges move which shared entity rows.Simulation, proxy, and projection approaches respectively return weights after a simulated update, read global weight statistics, or project across all rows.Source: §E.5 · p. 52

E.6 The tangent-kernel response and its origin

A triple’s score gradient touches three rows, making its tangent kernel explicit and zero unless two triples share an entity or relation.The kernel identifies which update edges move a forgotten fact and through which rows.Source: §E.6 · p. 52
The linearized cross-task response predates KAU; KAU’s contribution is placing the response to the post-edit update inside the objective minimized by the unlearner.Earlier work used the response to bound interference or model fine-tuning; here the unlearning edit creates the residual that drives the later update.Source: §E.6 · p. 52
Unlike Golatkar et al. (2020b), KAU uses the kernel to predict benign post-release updating rather than compute a scrubbing edit.Golatkar et al. evaluate relearning after fine-tuning on the complete data, including the forget set; KAU begins with the benign fine-tune after release.Source: §E.6 · p. 52

E.7 Methods closest in design

The closest methods differ in which subspace or update effect guides the unlearning edit. KAU defines that structure through drawn update edges and a graph-local kernel, with a survival filter that retains edits to the extent they resist the update.

E.7 Methods closest in design

For KAU, the edit’s relevant subspace is defined by drawn update edges through a closed-form graph-local tangent space, not by channels that retained data barely activates.Forgotten facts use rows that retained neighbors also use, so no row is inert under a retain-only update.Source: §E.7 · p. 53
KAU defines the subspace avoided by the edit from update data; its survival filter weights directions by the fraction of the edit that survives.Projection onto the complement is the filter’s zero-ridge threshold, rather than the starting point.Source: §E.7 · p. 53
Prior null spaces protect utility or select slowly restored directions, while KAU uses the complement of update-data span to preserve edits against that update.Unseen update edges can move edits preserved by retained-span projection; KAU includes stand-ins for such edges in its draws.Source: §E.7 · p. 53

E.8 Remaining placements

A related placement thread concerns edits pulled toward their origin, row-norm effects on effective learning rates, and reversibility in the loss landscape.The paper relates its norm-retraction rule to work on growing row norms.Source: §E.8 · p. 53