PaperlayerView exact revision on arXiv

Source-linked AI summary

When Successful Knowledge Graph Edits Displace Correct Answers: Rank-Level Locality beyond Parameter Support

Yi-Cheng Lai, Jerry Wang, Hsin-Ling Hsu, Li-Chu Chi, Ya-Wen Teng, Hen-Hsen Huang

arXiv:2609.12116v1cs.AI

TL;DR

Promoting a desired answer can displace correct answers from the returned list, and parameter-support locality alone can miss that ranking effect. The paper audits displacement across three scopes and examines exact score-preserving edits; entity editing has the best joint outcome, while rank truncation is close and reduces mean displacement.

  • Problem

    Parameter-support locality can miss the displacement of unchanged correct answers when a promoted candidate moves above them in a ranked list.

    Source: §I · p. 1

  • Method

    The paper audits rank displacement across parameter-support, same-query, and relation-scope protected facts, and derives dimensional and geometric tests for exact score-preserving edits.

    Source: §I · p. 1, §IV · p. 3

  • Results

    Entity editing gives the best joint outcome (.363/.377), while rank truncation is close (.328/.347) and reduces mean displacement from 14.942/13.943 to 1.241/1.265.

    Source: B. Main Results · p. 5

  • Takeaways & Limitations

    Choose the protected scope before evaluating an editor, and report joint success and damage severity alongside correction success.

    Source: §VIII · p. 8, §IX · p. 8

    The study covers single edits to low-ranked held-out triples on fixed checkpoints, and exact-preservation claims apply only to the stated linear parameterization and protected scores.

    Source: §VIII · p. 8

Abstract

from arXiv · show

Editing a knowledge graph embedding (KGE) model to promote a desired answer can displace correct answers from the returned list. Locality tests based only on facts that reuse the edited parameter can miss this ranking effect. We introduce a common rank-displacement audit at three scopes: facts supported by the edited parameter, other correct answers to the target query, and correct answers across queries with the same relation. We also derive dimensional and geometric conditions for an update to improve the target while exactly preserving selected scores. On FB15k-237 with DistMult and ComplEx, direct promotion always moves the target into the top ten, but does so without damage in only 23.0--23.2\% of edits. Strict preservation causes no measured damage, yet succeeds in only 1.3--1.4\%. Support-regularized entity editing gives the highest joint success, 36.3--37.7\%, while rank-truncated preservation reaches 32.8--34.7\% and reduces the mean number of displaced answers from about 14 to 1.2. Experiments across dimensions, scorers, ranking conventions, and a learned editor show that locality depends on both the protected scope and the editing mechanism. KGE editing should therefore report correction success together with the incidence and severity of rank displacement.

I. INTRODUCTION

Promoting a desired answer can push another correct answer below the top-k cutoff, even when that answer’s score is unchanged. The paper proposes a rank-displacement audit across three locality scopes and compares editing approaches that trade target promotion against answer preservation.

I. INTRODUCTION

  • A promoted candidate can displace an unchanged correct answer from the top-k, while parameter-support checks alone miss same-query and relation-scope competition.The paper names the three scopes parameter-support, same-query, and relation-scope locality.Source: §I · p. 1
  • The audit uses one damage event across three protected sets, then compares promotion, preservation, demotion, and fine-tuning routes.The paper also derives dimensional and geometric tests for whether an exact score-preserving edit can promote the target.Source: §I · p. 1

II. RELATED WORK

  • Earlier editing evaluations assess parameter changes or propagation to related facts; this work isolates unchanged correct answers displaced when a competitor moves above them.Parameter-support follows the modified vector, relation scope follows a query family, and same-query locality follows one answer list.Source: §II · p. 2
  • Projection-based methods constrain updates using protected directions; here, a linear KGE’s score sensitivities form a matrix whose rank and target projection characterize exact score preservation.The paper evaluates whether selected answers remain top-k after the edit, not only whether their scores are preserved.Source: §II · p. 2

III. PRELIMINARIES

The paper models an edit as a vector perturbation to an entity embedding and describes how scores respond. It then compares direct promotion with routes that preserve selected scores or facts to varying degrees.

A. Scores and Candidate Rankings

  • DistMult, ComplEx, and RESCAL are linear in a tail embedding with the head and relation fixed, whereas RotatE and TransE use distance-based scores.Parameters outside the stated edited vectors remain fixed in every edit.Source: A. Scores and Candidate Rankings · p. 2

B. Editing as a Perturbation

  • For linear scores, the score change is exactly the update vector dotted with a sensitivity vector; for RotatE and TransE, the gradient gives only a first-order approximation.An update orthogonal to the sensitivity vector leaves the score unchanged.Source: B. Editing as a Perturbation · p. 2

C. Edit Routes

  • Direct promotion uses a minimum-norm update to reach the top-k without constraining other scores, while support-regularized entity editing penalizes changes to known facts sharing the edited tail.Support-regularized editing does not guarantee that the target reaches the top-k because its target increase is penalized rather than enforced.Source: C. Edit Routes · p. 2
  • Strict relation-nullspace promotion preserves selected competing scores by projecting the target direction, but returns a zero update if projection removes that direction.Rank-truncated preservation relaxes the strict constraint, and ridge preservation instead uses a soft penalty controlled by λ.Source: C. Edit Routes · pp. 2–3

IV. LOCALITY AT THREE SCOPES

The audit measures damage consistently as a protected known positive falling out of the top-k, while varying which facts are protected. It reports both editing success and the incidence and severity of displacement.

IV. LOCALITY AT THREE SCOPES

  • Raw tail rank counts candidates with strictly higher scores, so ties are not counted as ahead; the audit reports success, no-damage and joint success rates, plus displacement severity.Severity is summarized by the mean, 95th percentile, and maximum number of protected facts displaced.Source: §IV · p. 3
  • A protected known positive is damaged when its raw tail rank moves from at most k before editing to above k afterward; the same event defines all three locality scopes.An edit succeeds when its post-edit raw tail rank is at most k.Source: §IV · p. 3
  • Parameter-support locality checks known facts whose scores directly depend on edited parameters, with the protected set defined separately for each edit route.This scope captures collateral damage caused by direct parameter reuse.Source: §IV · p. 3
  • Same-query locality protects other known-positive tails of the target query, while relation-scope locality extends protection to answers of other queries sharing the edited relation.Relation-scope includes queries with different heads.Source: §IV · p. 3

V. FEASIBILITY OF EXACT SCORE PRESERVATION

Exact score preservation restricts updates to directions in the null space of the constraint matrix. The question is whether any such direction can also increase the target score.

V. FEASIBILITY OF EXACT SCORE PRESERVATION

  • The null space of the constraint matrix contains exactly the updates that preserve all selected scores, and feasibility tests determine whether it can support a target-improving edit.Source: §V · p. 3

A. Constraint Rank

Exact score preservation is limited by the dimension of the preserving subspace and by whether that subspace retains a direction aligned with the target. In the evaluated setting, few targets pass both tests.

A. Constraint Rank

  • A nonzero score-preserving update exists exactly when rank(R) < d, because the preserving subspace has dimension d − rank(R).Each independent sensitivity constraint removes one degree of freedom.Source: A. Constraint Rank · p. 3
  • In the primary d = 256 setting, 15 of 16 evaluated relations with at least 1,000 protected facts have rank(R) = d, leaving a mean null-space dimension of only 1.125.When rank(R) = d, the zero vector is the only score-preserving update and cannot increase the target score.Source: A. Constraint Rank · pp. 3–4

B. Target Alignment

  • Exact promotion without a norm bound requires rank(R) < d and E > 0; with ∥δ∥≤B, it also requires B∥g∥E ≥∆.The preserving subspace must contain a direction aligned with the target, and that direction must allow the required score increase.Source: B. Target Alignment · p. 4
  • Only 191/11,339 DistMult cases (1.68%) and 185/11,305 ComplEx cases (1.64%) pass both exact-preservation tests; all passing cases reach top-k without observed relation-scope damage.Rejected cases receive a zero update by design, so the pass rate measures exact-preservation coverage.Source: B. Target Alignment · p. 4
  • Preserving a protected answer’s score does not necessarily preserve its rank, because a moving competitor can push it below the cutoff.Post-edit ranks are therefore evaluated as the final measure of answer preservation.Source: B. Target Alignment · p. 4

VI. COMPETITOR DEMOTION WITH OBSERVED LABELS

Competitor demotion promotes a target by lowering candidates that outrank it, using observed-unlabeled blockers and either gradient-based or closed-form updates.

VI. COMPETITOR DEMOTION WITH OBSERVED LABELS

  • A blocker is an observed-unlabeled tail that scores above the target but is absent from the query’s known-positive answers across train, validation, and test.A confidence filter can retain all blockers at q = 1 or exclude more high-scoring blockers at smaller q.Source: §VI · p. 4
  • Closed-form bilinear demotion updates the query head while keeping the target-tail embedding fixed, selecting blockers to move below the target.At least enough candidates must move below the target for it to enter the top-k.Source: §VI · p. 4
  • The closed-form update stacks requested score changes for selected blockers and no change for the target, but least squares may not meet every request exactly.The requested blocker change places it just below the unchanged target score.Source: §VI · p. 4
  • Gradient-based blocker demotion updates each selected blocker’s tail embedding to encourage it below the target by a margin while limiting update size.This route uses automatic differentiation and applies when the closed-form bilinear construction is unavailable.Source: §VI · pp. 4–5

A. Setup

The evaluation compares editing routes and locality scopes on FB15k-237, using correction success and rank displacement to assess their trade-offs. Entity editing has the best joint outcome, while damage judgments vary with the protected scope.

A. Setup

  • The primary comparison samples 1,000 deduplicated test triples per model from targets ranked below the top 10 and evaluates raw tail rankings at k = 10.The protocol counts protected answers moving from rank ≤k to rank >k as damage.Source: A. Setup · p. 5

B. Main Results

  • Direct promotion causes zero parameter-support damage but damages 243/238 same-query cases and 262/259 relation-scope cases; locality depends on the editor–scope pair.Table I reports edits causing damage at each locality scope (N = 300).Source: B. Main Results · p. 5, Table I · p. 5
  • .363/.377 is the best joint outcome for entity editing; rank truncation reaches .328/.347 and reduces mean displacement from 14.942/13.943 to 1.241/1.265.Values are reported in DistMult/ComplEx order.Source: B. Main Results · p. 5
  • Among successful corrections, closed-form demotion is damage-free in 90.5/95.1% of cases, whereas gradient demotion displaces an average of 89.171/66.170 answers.Entity editing’s mean displacement falls from 3.410/2.971 overall to .637/.669 among successful edits.Source: B. Main Results · p. 5

C. Preservation Strength

Rank-truncated preservation offers a useful intermediate region, while stricter preservation trades away target success. Ridge preservation does not show the same intermediate gain, and aggressive truncation is not uniformly safer.

C. Preservation Strength

  • At ρ = .125, rank truncation reaches test SafeSucc .328/.347, and the validation cohorts select this setting.The wider relation-cluster intervals indicate that the gain depends substantially on which relations are edited.Source: C. Preservation Strength · p. 5, Figure 2 · p. 6
  • At ρ = .9375, success falls to .590 for both models while mean displacement rises to 2.461/2.376.Mean update norm grows to 145.322/84.195 at this setting.Source: C. Preservation Strength · pp. 5–6
  • Strict preservation eliminates observed damage but produces only 14/13 successful edits.Lower success and larger update norms suggest aggressive projection leaves a poorly aligned direction.Source: C. Preservation Strength · p. 6
  • Ridge preservation’s validation-selected settings yield test SafeSucc .010/.013, with target success near 1% as λ increases.The updates remain nonzero, so the collapse in editability is not attributed to identity updates.Source: C. Preservation Strength · p. 6

D. Dimension and Checkpoint Geometry

Editability depends on embedding dimension and learned score geometry, while demotion outcomes vary with update bounds, blocker selection, target difficulty, relation composition, and evaluation choices.

D. Dimension and Checkpoint Geometry

  • Across 18 training runs, rank truncation at d = 512 and ρ = .125 attains SafeSucc .452 ± .015 for DistMult and .500 ± .019 for ComplEx.Larger embeddings reduce full-rank failure, but remaining subspaces align with the target less often, making the fraction passing both tests non-monotonic.Source: D. Dimension and Checkpoint Geometry · p. 7
  • Dimension alone does not determine editability; the learned score geometry matters as well.A common norm bound is not a common intervention: RESCAL SafeSucc peaks at only .013 at B = .3, and WN18RR SafeSucc at B = 1 ranges from .044 to .207.Source: D. Dimension and Checkpoint Geometry · p. 7, E. Demotion Sensitivity · p. 7

E. Demotion Sensitivity

  • Removing positive-label exclusion raises success to .597/.608 but lowers the no-damage rate to .192/.205.Increasing q produces the same trade-off; demotion safety depends on the trustworthiness of the blocker set.Source: E. Demotion Sensitivity · p. 7, Table VI · p. 7

F. Population and Evaluation Sensitivity

  • Closed-form demotion S@10 falls from .537/.578 at ranks 11–20 to 0/.003 beyond rank 100, while rank-truncated S@10 remains .876–.951.On rank-> 20 cohorts, larger k improves SafeSucc for non-strict routes but barely changes strict success.Source: F. Population and Evaluation Sensitivity · p. 7
  • Under relation clustering, entity editing’s intervals widen to [.252, .501]/[.279, .493], while equal relation weighting raises SafeSucc to .523/.517.Case-weighted averages partly reflect which relations dominate the cohort.Source: F. Population and Evaluation Sensitivity · p. 7
  • Sensitivity overlap weakly predicts damage: Pearson r is −.184 ([−.221, −.142]) and −.163 ([−.210, −.118]).Local geometric similarity does not summarize resulting rank competition.Source: F. Population and Evaluation Sensitivity · p. 7
  • Filtered ranking raises direct and rank-truncated SafeSucc to .564/.547 and .790/.798; expanding protection from 500 to 5,000 to all training facts lowers rank-truncated SafeSucc.All-split protection lowers it further, to .281/.284; the evaluation retains raw ranking and training-only protection.Source: F. Population and Evaluation Sensitivity · p. 7

G. External Case Study: KGEditor

The KGEditor case study finds rank displacement despite high target success; the discussion recommends choosing the protected scope before evaluation, and the paper’s conclusions are bounded by its stated settings.

G. External Case Study: KGEditor

  • On KGEditor’s official 3,086-case FB15k-237 set, Hits@1/3 (.8594/.9867) and target success at 10 (.9916) coexist with 12.38% of edits displacing a protected relation-scope answer.The case study is unpaired, so it does not support a controlled method comparison.Source: G. External Case Study: KGEditor · p. 8

VIII. DISCUSSION AND LIMITATIONS

  • Choose the protected scope before evaluation: multi-answer queries require same-query locality, and cross-query competition by the edited entity additionally requires relation-scope locality.Parameter-support locality diagnoses direct interference but should not alone stand as evidence of rank preservation; report joint success and damage severity alongside correction success.Source: §VIII · p. 8
  • The study covers single edits to low-ranked held-out triples on fixed checkpoints, not false-fact correction, unseen entities, or edit sequences.Exact-preservation claims apply to the stated linear parameterization and protected scores; entity editing and demotion use retrospective observed labels.Source: §VIII · p. 8

IX. CONCLUSION

  • Correcting a target rank can displace correct answers while appearing local under parameter support; partial rank preservation improves joint success non-monotonically, unlike the ridge sweep.Reporting protected scope, joint success, and damage severity reveals correction outcomes and side effects.Source: §IX · p. 8

Related papers

Loading 2609.12116v1…