PaperlayerView exact revision on arXiv

Source-linked AI summary

Bounding Retraining Equivalence and the Deletion Floor in Materials Machine Unlearning

Can Polat, Mustafa Kurban, Erchin Serpedin, Hasan Kurban

arXiv:2609.35635v1cs.LGcond-mat.mtrl-sciquant-ph

TL;DR

Post-deletion prediction error is ambiguous unless evaluated against the target loss exact retraining itself leaves behind. The paper defines that deletion floor and characterizes how equivalence constraints and retained support relate to it. Controlled redundancy results show a sharp reduction in retraining loss with one retained relative, while lower reference loss can coexist with greater prediction sensitivity.

  • Problem

    Without establishing the target error exact retraining would leave, continued post-deletion accuracy is an ambiguous signal of unlearning performance.

    Source: §1 · p. 2

  • Method

    The paper defines a request-level deletion floor and characterizes it with a bounded-loss interval, a retained-support neighbor bound, and an exact ridge decomposition.

    Source: §1 · pp. 2–3, §3.1 · pp. 4–5

  • Results

    7.934× drop in median normalized retraining loss occurs when moving from k=1 to k=2, while lower reference loss frequently coexists with larger prediction sensitivity.

    Source: §4.2 · p. 8, §6 · p. 11

  • Takeaways & Limitations

    Request-level evaluation should report retraining predictions, record-removal shifts, and retained utility, interpreting post-deletion accuracy relative to the reference floor.

    Source: §1 · p. 3, §6 · p. 11

    The technical finding from synthetic data and public materials-property records should not be taken as evidence of compliance with legal or regulatory removal mandates.

    Source: Ethics statement · p. 11

Abstract

from arXiv · show

In materials machine learning, closely related retained structures can sustain accurate property predictions even after removing a specific record, rendering post-deletion prediction error an ambiguous metric for machine unlearning. To resolve this ambiguity, we define the deletion floor as the expected target loss under a specified retraining procedure at the deleted request. Standard indistinguishability constraints yield a sharp interval bounding an update's target loss around this baseline reference. Theoretically, a conditional neighbor bound links a low deletion floor directly to retained fit, prediction regularity, and local label agreement, while an exact ridge identity isolates residual fit from the prediction change induced by record deletion. Empirically, controlled redundancy sweeps show an $\approx 8\times$ drop in median normalized retraining loss when one retained relative remains after deletion. Across two distinct fitting regimes in a paired Materials Project study, the lower-floor regime also exhibits a larger prediction change on more than 50% of the shared requests. Systematic comparisons against approximate updates and the original model decouple deliberate target suppression from preserved overall model utility. Consequently, request-level unlearning evaluations should report reference loss, prediction change, and retained utility together, interpreting post-deletion accuracy against what retraining itself leaves behind.

1 Introduction

Materials records can remain physically related after one record is removed, making post-deletion accuracy ambiguous without a retraining reference. The paper defines that reference and connects it to retained support, update equivalence, and request-level evaluation.

1 Introduction

  • Distinct compositions can share crystal lattices, so related database entries may preserve physical relationships even after one record’s training contribution is removed.The cited work distinguishes removing a record’s contribution from eliminating the broader physical behavior it represents (Triantafillou et al., 2026).Source: §1 · p. 1
  • Without the target loss exact retraining would leave, continued post-deletion accuracy is ambiguous as a measure of unlearning.Existing membership attacks and simple accuracy-gap measures can conflict with distribution-based forgetting scores.Source: §1 · p. 2
  • The deletion floor is expected target loss under a specified retraining procedure, and a conditional neighbor bound links it to retained support and local prediction behavior.An exact ridge identity separates residual fit from the prediction change induced by deleting a record.Source: §1 · pp. 2–3
  • The paper studies individual deletion requests using exact fixed-feature ridge calculations on controlled redundancy sweeps and 44,344 Materials Project crystal structures.Approximate updates and unedited baselines are used to expose trade-offs between target suppression and overall model utility.Source: §1 · p. 2
  • The paper formulates a sharp bounded-loss interval for target losses compatible with approximate retraining equivalence.It also proposes reporting reference loss, prediction change, and preserved utility independently.Source: §1 · pp. 2–3

2 Related work

The related work distinguishes sample-level removal from broader behavioral or distributional removal, and positions this paper’s focus on the target behavior left by exact retraining.

Retraining objectives and evaluation

  • Certified removal compares updated-model output distributions with references trained without deleted records, while practical approximate methods may trade guarantees for efficiency.SCRUB balances forgetting objectives against retained utility without providing a distributional certificate.Source: §2 · p. 3

Duplication, memorization, and retained support

  • Duplication and low memorization can preserve original predictions after exact retraining or make predictions insensitive to a training point’s inclusion.Ye et al. (2025) report persistence under duplicated or near-duplicated instances; Zhao et al. (2024) connect approximate-unlearning difficulty to memorization and retain–forget entanglement.Source: §2 · p. 3

Broader behavioral removal

Positioning of this work

  • This work formalizes retained support and retraining equivalence around the target loss left by exact retraining, then validates that lower reference loss can coexist with larger prediction sensitivity.Its bounded-loss transfer theorem specifies permitted loss intervals under indistinguishability, and its ridge decomposition separates retraining accuracy from deletion-induced changes.Source: §2 · p. 4

3 Bounding loss around the retraining reference

The section defines deletion loss relative to a specified retraining procedure and bounds how much an equivalent update can change that target loss. It then connects the reference loss to retained-neighbor conditions and derives exact ridge relationships between fit, leverage, and deletion sensitivity.

3.1 The deletion floor and bounded-loss transfer

  • Theorem 1 gives a sharp interval for an update’s expected bounded target loss around the retraining reference, with endpoints attainable and converging to the reference under exact equivalence.For squared error, the theorem applies to a clipped loss; empirical analyses report unclipped raw squared loss.Source: §3.1 · pp. 4–5
  • For pure equivalence, raising expected unit-bounded loss from Φ = 0.1 to τ = 0.5 requires ε ≥ log 5, giving a necessary compatibility condition for target suppression.The condition follows from requiring both upper bounds to reach the target threshold.Source: §3.1 · p. 5

3.2 How retained support can keep the floor low

  • Under uniform neighbor control, the squared-loss deletion floor is bounded by C^2, where retained fit, prediction variation, and target-label disparity jointly determine C.The bound assumes a retained neighbor with bounded prediction difference and fit error, plus limits on input distance and label gap.Source: §3.2 · p. 5
  • FloorScore uses estimated neighbor and prediction-regularity quantities as a ranking diagnostic, not as a certified upper bound on the deletion floor.Its components come from original-model residual and prediction-difference quantiles, and neighbor selection precedes deletion-loss evaluation.Source: §3.2 · p. 6

3.3 An exact ridge connection: fit, leverage and deletion

  • In ridge regression, retraining loss depends jointly on the adjusted full-data residual and deleted-point leverage; low residual yields a low floor when leverage stays bounded away from one.High adjusted residual error carries over into large retraining loss.Source: §3.3 · p. 6
  • The ridge decomposition separates prediction change into penalty-rescaling and row-removal components, so accurate fit can coexist with strong sensitivity to record deletion.Because squared losses discard residual signs, signed residuals are needed to evaluate prediction change.Source: §3.3 · p. 6

4 Experiments and results

The experiments compare deletion requests against exact retraining references, then assess how redundancy, fitting regime, and approximate updates affect target loss, prediction shifts, and retained utility. Across the results, lower post-deletion loss does not necessarily mean smaller prediction changes or successful unlearning.

4.1 Shared reference and measurement protocol

  • The measurement protocol distinguishes target loss relative to the retraining floor, deletion-induced prediction change, update-to-reference distance, and retained test utility.The ridge reference refits weights on retained rows while holding standardization, kernel bandwidth, and random projections fixed.Source: §4.1 · p. 7

4.2 Retraining residuals across controlled redundancy settings

  • Leaving one retained family relative after deletion reduces median normalized retraining loss 7.934×, from 0.326 to 0.041; further family members yield diminishing reductions.At family size 16, the median floor reaches 0.020, approximately 2% of original test MSE.Source: §4.2 · pp. 7–8

4.3 Materials floors depend on the fitting regime

  • In the 8,192-feature regime, median normalized floors are 0.0605, 0.142, and 0.295 across low-, middle-, and high-FloorScore strata, but FloorScore stratification depends on fitting regime.Feature count and regularization vary jointly; training MSE falls from 0.0754 to 0.0166 and test MSE from 0.0870 to 0.0704.Source: §4.3 · p. 8

4.4 Lower floors can coexist with larger deletion changes

  • On 172 of 200 shared requests, the 8,192-feature regime has a larger deletion-induced prediction change, despite lower median squared retraining loss: 0.0118 versus 0.0324 (eV/atom)2.The paired medians show that lower retraining floors can coexist with larger prediction shifts.Source: §4.4 · p. 8, Figure 2 · p. 9

4.5 Target loss and retained utility at fixed operating points

  • Gradient ascent raises target loss to 3.081× the reference while increasing test MSE by 8.3%; several other updates remain near the no-update control.The comparisons use 200 shared requests in the 2,048-feature, λ=10^-6 setting.Source: §4.5 · pp. 9–10, Table 1 · p. 10

5 Evaluation protocol and scope

The evaluation protocol treats the retraining reference, deletion-induced prediction changes, and retained utility as distinct quantities. It also specifies scope boundaries for the theoretical and materials results.

5 Evaluation protocol and scope

  • For each request, report original, retrained, and updated target losses, deletion-induced shifts, update-to-reference distances when available, and retained test utility, alongside an unedited control.The protocol calls for defining the deletion objective, training reference, and any refitted preprocessing; scalar prediction agreement alone does not certify distributional equivalence.Source: §5 · p. 10
  • Approximate updates must be assessed against both retraining floors and retained utility; squared-loss ratios can be inflated when reference floors are near zero.Full request-level distributions and explicit loss normalizations complement aggregate medians.Source: §5 · pp. 10–11
  • Materials comparisons vary feature dimension and regularization jointly under fixed descriptors, preprocessing, and architectures; leave-one-out stability bounds also assume uniform gradient control.Broader removal behavior, comparative tuning, distributional certification, and end-to-end representation refitting remain distinct objectives.Source: §5 · p. 11

6 Conclusion

The conclusion frames the deletion floor as a retraining-based baseline and emphasizes that reference accuracy and deletion sensitivity can diverge.

6 Conclusion

  • The deletion floor measures the residual target error left by exact retraining, providing a baseline for regression-unlearning evaluation.Retraining equivalence constrains updated models around this reference.Source: §6 · p. 11
  • Across synthetic redundancy sweeps and materials benchmarks, lower reference loss frequently coexists with larger prediction sensitivity.Retained structural support maintains low reference loss, while ridge leverage explains how accurate retraining can remain sensitive to deletion.Source: §6 · p. 11
  • Post-deletion accuracy should be interpreted relative to the retraining floor and target objective, with retraining predictions, removal shifts, and retained utility reported together.Source: §6 · p. 11

Reproducibility statement

The accompanying repository includes implementation and verification materials.

Reproducibility statement

Ethics statement

The study uses synthetic data and public materials-property records, with no human subjects or personal data. Its technical observation is not evidence of compliance with specific data-removal mandates.

Ethics statement

  • The study uses synthetic data and public materials-property records, involving no human subjects or personal data.Source: Ethics statement · p. 11
  • Retained structural support preserving target accuracy after deletion is a technical observation, not evidence of compliance with specific legal or regulatory removal mandates.Source: Ethics statement · p. 11
  • Broader behavioral concept-removal applications require evaluations tailored to that objective.Source: Ethics statement · p. 11

Use of AI statement

Generative AI tools assisted with literature review, text polishing, code development, and data analysis; the submitting authors retain full responsibility for the final work.

Use of AI statement

A.1 Proof of Theorem 1: bounded-loss transfer

This block presents a sharp transfer interval around the retraining reference floor and supporting theoretical mechanisms, alongside the experimental designs used to examine redundancy and unlearning updates.

A.1 Proof of Theorem 1: bounded-loss transfer

  • The sharp bounded-loss transfer interval restricts expected loss shifts relative to the retraining reference floor, with endpoints attainable and both converging to the floor as (epsilon, delta) approaches (0, 0).Under pure equivalence, Equation (1) characterizes the minimum equivalence budget compatible with a target-error objective.Source: §A.1 · p. 14

A.2 Justification of the retained-neighbor bound

  • Under uniform retained-neighbor control, target error is bounded by retained-neighbor fit, local prediction variation, and label disparity.The resulting bound also yields absolute-, squared-, and clipped-squared-loss bounds.Source: §A.2 · pp. 14–15

A.3 Derivation of the ridge identities

  • The ridge derivation compares full and deletion-adjusted fits to isolate prediction change, with leverage denominators positive under the stated conditions.For a penalty defined per sample mean loss, the adjusted matrix supplies the sample-size normalization correction.Source: §A.3 · p. 15

A.4 Leave-one-out stability: statement and proof

  • Leave-one-out stability gives O(1/n) parameter-shift decay under uniform gradient bounds and positive strong convexity, but unrestricted regression lacks global gradient bounds.For mean-squared loss, the stated strong-convexity parameter is 2 lambda; exact ridge identities provide unconstrained exact calculations.Source: §A.4 · pp. 15–16

B.1 Controlled family construction

  • Increasing family size from k=1 to k=2 leaves one relative after deletion and sharply lowers median normalized retraining loss, while isolated controls remain higher.The sweep grows all 90 families together, coupling redundancy with total training size.Source: Figure A1 · p. 16, §B.1 · p. 16

B.2 Materials representation and reference

  • The Materials Project evaluation uses 44,344 filtered crystal structures, 134 features, and fixed preprocessing and random-feature projections across exact retraining.The two fitting regimes couple feature count and regularization; the paired comparison uses the same 200 targets.Source: §B.2 · p. 17, Table A1 · p. 17

B.3 Request sampling and uncertainty

  • A shared cohort of 200 training-set deletion requests supports paired-regime and operating-point evaluations, with uncertainty estimated using percentile bootstrap intervals.The request cohort is reserved from the training set; bootstrap resample counts differ across ridge, synthetic, operating-point, and neural diagnostics.Source: §B.3 · p. 17

B.4 Approximate-update settings

  • Approximate updates are evaluated at fixed operating points, including SCRUB-style regression, random relabeling, and Newton-style removal, with held-out data reserved for utility evaluation.The settings specify each intervention's strength; comparative tuning lies outside this experiment.Source: §B.4 · pp. 17–18, Table A2 · p. 18

C.1 Complete operating-point summaries

The operating-point comparison reports target-loss agreement and retained utility, while a single-seed neural diagnostic finds a stronger positive association between neighbor label gap and post-deletion loss than between descriptor distance and loss.

C.1 Complete operating-point summaries

  • Gradient ascent has the largest median absolute loss gap and test-error ratio among the reported operating points in 2,048-feature ridge across 200 shared requests.Several conservative updates have overlapping bootstrap intervals, which alone establishes neither statistical equivalence nor the absence of paired differences.Source: Table A3 · p. 18, §C.1 · p. 18

C.2 Exploratory neural descriptor diagnostic

  • The neural results are request-bootstrap estimates conditional on one training seed, not averages over training randomness.Each of the 200 requests used a separate from-scratch retraining run.Source: §C.2 · p. 19
  • In the single-seed neural diagnostic, neighbor label disparity has a stronger positive association with retraining loss than descriptor distance, whose rank association is little and interval spans zero.The reported Spearman correlations are 0.2128 for target label gap and 0.0367 for descriptor distance.Source: §C.2 · p. 19, Table A4 · p. 19
Loading 2609.35635v1…