PaperlayerView exact revision on arXiv

Source-linked AI summary

Hidden not Deleted: How Networks Suppress Entangled Features

Akash Samanta, Manish Pratap Singh, Debasis Chaudhuri

arXiv:2609.27593v1cs.LGcs.AI

TL;DR

Behavioral disappearance alone does not establish that a model’s underlying knowledge was removed. In an antipodal superposition setting, the paper compares linear and gradient-based erasure and uses causal intervention to test what remains. Linear erasure damages the paired feature, while gradient-based erasure yields two suppressive solutions whose target feature remains recoverable.

  • Problem

    Behavioral evaluations of unlearning do not establish whether the underlying knowledge was removed, and prior evidence lacks a causal account of the internal mechanism.

    Source: §1 · p. 1

  • Method

    The paper studies antipodal features in a toy superposition model, compares linear projection with gradient-based erasure, and tests residual representations using targeted causal intervention.

    Source: §1 · p. 2, §3 · p. 3, §3.2 · p. 4

  • Results

    Linear erasure fails on entangled representations, while gradient-based erasure converges to two initialization-dependent solutions that both leave the suppressed feature causally recoverable.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    In this setting, apparent suppression is not equivalent to deletion, so whether a behavior disappears alone may be insufficient to judge unlearning if the pattern extends to larger systems.

    Source: §1 · p. 2, §7 · p. 9

    The analysis freezes the encoder, whereas leaving it trainable permits a third route that shrinks B’s encoder column toward zero, so the results do not explain why practical full fine-tuning may produce suppression.

    Source: §7 · p. 10

Abstract

from arXiv · show

Concept erasure methods that operate via linear projection assume that features occupy separable subspaces. We show this assumption fails under dense superposition: when two features are forced into an antipodal pair sharing a single subspace, state-of-the-art linear erasure destroys both, not just the target. Networks trained with gradient descent instead solve this problem non-linearly, but not uniformly: they converge to one of two distinct circuit-level solutions depending on initialization, which we call mirror and shadow solutions. We map this bifurcation as a function of feature entanglement, show it reflects a stable attractor structure rather than an artifact of our setup, and use targeted causal interventions to demonstrate that both solutions leave a substantial, measurable trace of the erased feature's representation intact, recoverable through a single scalar patch rather than requiring any further training. This mirrors a failure mode recently observed empirically in LLM unlearning, where suppression rather than deletion allows forgotten knowledge to resurface; our results offer a mechanistic, causally-validated account of why that failure mode occurs.

1 INTRODUCTION

Behavioral disappearance does not establish that unlearning removed the underlying knowledge, a distinction with practical stakes for data deletion and capability revocation. In a tractable superposition setting, the paper contrasts linear erasure with gradient-based suppression and tests whether the suppressed feature remains causally recoverable.

Motivation

  • A behavior disappearing is not the same as removing its underlying knowledge, making behavioral evaluation incomplete for deletion requests and capability revocation.Prior LLM findings suggest suppressed knowledge can remain present and return, while lacking a causal account of the mechanism.Source: §1 · p. 1

Approach

  • In a toy superposition model, two features sharing an antipodal direction make linear erasure fail because removing one also removes the other’s shared representation.The paper establishes this failure formally and empirically.Source: §1 · p. 2

Findings

  • Gradient-based suppression instead converges to one of two internal solutions depending on initialization, and causal intervention shows neither deletes the suppressed feature.The paper names this two-mechanism outcome Janus erasure.Source: §1 · p. 2
  • Both solutions leave the erased feature recoverable through a single scalar causal patch, with the shadow solution showing a graded response and the mirror solution a discrete patch.Figure 1 previews the pipeline and distinguishes the shadow dose-response from mirror recovery.Source: §1 · p. 2, Figure 1 · p. 2

2 RELATED WORK

Prior work studies post-hoc concept transformations, empirical suppression in LLM unlearning, feature decomposition, and causal circuit analysis. This paper instead examines the mechanisms training discovers when tasked with suppressing an already-superposed feature, using direct causal intervention in a tractable setting.

Concept erasure

  • Linear concept-erasure methods such as Iterative Nullspace Projection and LEACE use post-hoc linear transformations, while later alternatives explore nonlinear transformations.The paper’s baseline is a rank-1 nullspace projection related to LEACE, but without its covariance-whitening step.Source: §2 · p. 2
  • The paper asks what training-time optimization discovers when suppressing a feature, unlike post-hoc approaches that seek to separate a concept after training.This distinction matters because practical unlearning pipelines rely on training-time optimization.Source: §2 · pp. 2–3

Suppression versus deletion in LLM unlearning

  • LLM studies report suppression and reversibility, but do not provide a controlled account of the underlying mechanism; this paper offers a tractable candidate account with direct causal tests.The cited LLM findings are empirical and at LLM scale.Source: §2 · p. 3

Superposition and polysemanticity

  • Unlike superposition work focused on decomposing polysemantic activations into interpretable features, this paper targets erasure of a specific pair of already-superposed features.The cited decomposition work includes Bricken et al. (2023) and Cunningham et al. (2023).Source: §2 · p. 3

Causal intervention as evidence

  • The paper applies causal intervention and activation-patching logic to concept erasure, adding sufficiency checks and negative controls to test whether interventions are necessary and minimal.The cited intervention tradition includes causal mediation, causal abstraction, and causal tracing.Source: §2 · p. 3

3 SETUP: ANTIPODAL SUPERPOSITION AND THE LIMITS OF LINEAR ERASURE

Using a sparse-input toy model, the section asks whether linear representation surgery can suppress feature B while preserving A. Linear projection erases B across the entanglement range but increasingly damages A’s reconstruction as the features become entangled.

3 SETUP: ANTIPODAL SUPERPOSITION AND THE LIMITS OF LINEAR ERASURE

  • The section uses the Toy Model of Superposition to test linear representation surgery as an erasure strategy and motivates studying gradient-based training after finding it inadequate.Rank-1 nullspace projection removes B by projecting bottleneck activations onto the orthogonal complement of B’s direction.Source: §3 · p. 3, §3.3 · p. 4

3.1 TOY MODEL OF SUPERPOSITION

  • The toy model uses a ReLU encoder-decoder to reconstruct sparse inputs, allowing more features than bottleneck dimensions through overlapping directions called superposition.For mutually exclusive features, the cheapest solution places them at antipodal points on a shared one-dimensional subspace; only the designated pair’s co-occurrence is controlled directly.Source: §3.1 · pp. 3–4

3.2 ENTANGLEMENT AND THE ERASURE TASK

  • Entanglement is the cosine similarity ρ between A’s and B’s encoder directions; the task is to suppress B’s output while keeping A’s reconstruction error near its pre-intervention level.A’s baseline error is already 0.098 at ρ = 0 because of interference with other features, so preservation failure means a rise above that floor.Source: §3.2 · p. 4

3.3 LINEAR ERASURE FAILS

  • 4.2× the orthogonal-case error is reached in A’s reconstruction at full antipodal entanglement, even though linear projection erases B throughout the entanglement range.Preservation error increases monotonically with entanglement; at ρ = −1, A and B share a direction, so removing B also removes A’s component.Source: Table 1 · p. 4, §3.3 · p. 4
  • At ρ = −1, no linear operator can remove B’s direction without also removing A’s, while intermediate-entanglement results reflect the unwhitened rank-1 projection used here.Optimally whitened linear operators might produce different preservation curves at intermediate entanglement, a question left for future work.Source: §3.3 · p. 4

4 THE BIFURCATION PHENOMENON

The section compares linear projection with gradient-based excision in an entangled feature pair, then traces the latter’s two solutions and how their prevalence varies with entanglement.

4.1 EXCISION PROCEDURE: FREEZING THE ENCODER

  • With the encoder frozen, gradient-based excision optimizes suppression of B while retaining its representation, isolating how the network suppresses B at readout.Freezing prevents the encoder from simply ceasing to represent B, a route observed when it was left trainable.Source: §4.1 · p. 5, §4 · p. 5

4.2 TWO ATTRACTORS: MIRROR AND SHADOW

  • Excision converges to mirror, with substantial decoder reflection, or shadow, with partial rotation and a strongly negative output bias.Bias suppression occurs in both basins; mirror uses more decoder rotation, while shadow uses less.Source: §4.2 · pp. 5–6

4.4 NATURAL EMERGENCE OF ANTIPODAL GEOMETRY

  • Antipodal geometry emerges frequently under ordinary training, especially as anti-correlation strengthens and the bottleneck narrows relative to the number of features.The study sweeps sparsity and anti-correlation strength to track this emergence.Source: §4.4 · pp. 6–7

4.3 A PHASE TRANSITION IN BASIN PREVALENCE

  • At full antipodal entanglement, 75% of runs land in mirror, while at ρ = 0, 87% land in shadow; gradient-based excision keeps A’s preservation MSE flat across the sweep.The crossover occurs at ρ ≈ −0.75, while linear erasure’s preservation cost for A rises sharply with entanglement.Source: §4.3 · p. 6, Table 2 · p. 6, Figure 4 · p. 7

5 CAUSAL VALIDATION

A single targeted forward-pass intervention restores B’s readout in both solutions; unrelated-feature controls do not affect it.

5 CAUSAL VALIDATION

  • A near-zero output for B does not establish that its representation has been removed; the section tests this distinction with a targeted intervention.Source: §5 · p. 7

5.1 PATCHING PROCEDURE: A SINGLE TARGETED INTERVENTION

  • The shadow patch sweeps B’s output bias toward its pre-excision value, while the mirror patch replaces B’s pre-activation with its absolute value.Both are forward-pass overrides that leave stored parameters unchanged.Source: §5.1 · p. 7

5.2 ERASURE IS REVERSIBLE

  • In shadow, B’s mean readout rises from 0 to 0.38 as the bias patch strengthens; in mirror, an absolute-value patch restores it from 0 to 0.13 in one step.No parameter beyond the targeted scalar is changed.Source: §5.2 · p. 7, Figure 5 · p. 8

5.3 SUFFICIENCY AND SPECIFICITY

  • The targeted scalar fully accounts for recovery without changing other features’ readouts, while patching an unrelated feature produces zero measurable effect on B.The unrelated-feature control uses 20 trials per basin.Source: §5.3 · pp. 7–8, Figure 5 · p. 8

6 ROBUSTNESS ACROSS ARCHITECTURES AND NONLINEARITIES

The authors test whether mirror-and-shadow bifurcation extends beyond their minimal ReLU network, finding persistence with added depth and under GELU, though GELU yields a less-clean basin structure.

6 ROBUSTNESS ACROSS ARCHITECTURES AND NONLINEARITIES

  • The main results use a minimal two-layer ReLU network with one antipodal pair; a small-scale causal-transformer check suggests possible generalization beyond the autoencoder setting.The transformer check is described as plausible generalization, not evidence for the same mechanism at language-model scale.Source: §6 · p. 8

6.1 DEPTH: AN ADDITIONAL HIDDEN LAYER

  • With an additional hidden layer, the mirror-shadow bifurcation persists, and suppression is output-layer-dominant in 91.0% of seeds.The remaining 9.0% show hidden-layer-dominant suppression; none show a split between the layers.Source: §6.1 · p. 8

6.2 NONLINEARITY: REPLACING RELU WITH GELU

  • Under GELU, suppression still occurs, but four mixture components fit the basin structure better than two, and output bias stays near zero across the entanglement sweep.Because GELU lacks a hard zero region, erasure is defined by comparing the effect of a causal patch with a matched control.Source: §6.2 · p. 8, Figure 6 · p. 9

7 CONCLUSION AND DISCUSSION

The paper concludes that gradient-based erasure in its toy setting suppresses rather than deletes a feature, and discusses the scope and unresolved limits of that account.

7 CONCLUSION AND DISCUSSION

  • In the toy model, gradient-based erasure converges to mirror or shadow solutions, with entanglement governing their population prevalence and a single causal intervention recovering the suppressed feature.Source: §7 · p. 9
  • The account concerns training-time optimization, not post-hoc nonlinear erasure, and offers a controlled candidate explanation for empirical unlearning failures rather than evidence of the same mechanism in large models.Source: §7 · p. 9
  • The experiments use a toy autoencoder with one antipodal pair, and it remains unknown whether the two attractors persist when multiple pairs share a network.Source: §7 · pp. 9–10
  • Freezing the encoder limits the result: when it is trainable, gradient descent can shrink B’s encoder column toward a route closer to deletion than suppression.The authors note that this leaves unanswered why suppression neurons appear in practical settings where earlier layers can move.Source: §7 · p. 10

AI USE STATEMENT

  • The authors used generative AI for manuscript outlining, localized drafting, prose polishing, literature discovery, and typesetting, while retaining editorial control.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The work uses synthetic data only and reports no human subjects, personally identifiable information, or dataset derived from real individuals.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

A FORMAL SETUP AND THEORETICAL ANALYSIS

This appendix formalizes how entanglement constrains linear erasure and how a nonlinear threshold can separate an antipodal feature pair. It also specifies the experimental procedures used to examine the resulting solutions.

A.1 FORMAL SETUP

  • Entanglement is ρ = cos(wA, wB), with the encoder fixed and only decoder weights and output bias optimized during excision.The encoder columns and decoder rows are distinguished before and after excision-phase convergence.Source: §A.1 · p. 15
  • The shadow bias patch is a forward-pass intervention, and the mirror patch takes the absolute value of B’s preactivation while leaving other coordinates unchanged.The bias patch reduces to the ordinary forward pass at α = 0; the mirror patch acts as a sign flip when B’s preactivation is negative, as throughout that basin.Source: §A.1 · p. 15
  • Basin labels are assigned after training from fitted endpoints, and “attractor” informally denotes stable solution types rather than a formally proven dynamical-systems attractor.The endpoint classifier is specified in Appendix B.3.Source: §A.1 · p. 16

A.2 QUANTIFYING THE DAMAGE FROM LINEAR ERASURE

  • Linear projection removes exactly a ρ^2 fraction of A’s bottleneck representational energy, while output-level preservation error only approximately follows that scaling.The output-level prediction assumes removed activation energy contributes proportionally to error; at ρ = −0.5, the predicted MSE is about 0.176 versus 0.144 observed.Source: §A.2 · p. 16
  • Under that output-level assumption, preservation MSE rises monotonically with |ρ| and is largest at ρ = −1, but other features and decoder alignment affect the fit.The fourteen other represented features contribute to the baseline error floor, and the decoder direction is only approximately aligned with the simplifying picture.Source: §A.2 · p. 16

A.3 A SUFFICIENT CONDITION FOR NONLINEAR SEPARABILITY

  • For a fully antipodal pair, the sign of the shared-axis activation identifies which feature is active, enabling a ReLU threshold to erase B while preserving A.No linear map on that one-dimensional subspace can zero every negative input while preserving positive inputs.Source: §A.3 · p. 16
  • The mirror solution approximates this threshold by rotating B’s decoder so B-only inputs are negative and A-only inputs positive, letting the network’s ReLU suppress B.The relevant sign is determined by the decoder projection onto the shared axis; full decoder vectors may differ in orthogonal directions.Source: §A.3 · p. 17

B.1 PHASE SWEEP GRID CONSTRUCTION

  • The experimental appendix specifies a two-pass, 15-point entanglement grid that densifies the transition range identified in a pilot sweep.The pilot uses five entanglement values and 20 seeds; the final grid uses 100 seeds per condition, with pilot data discarded.Source: B EXPERIMENTAL METHODOLOGY · p. 17, §B.1 · p. 17

B.2 COMPUTE BUDGET AND THE REDUCED-GRID SAFEGUARD

This appendix details the reduced-grid safeguard and analysis procedures, then supplies diagnostics and robustness breakdowns for the linear baseline, bifurcation, and GELU and depth experiments.

B.2 COMPUTE BUDGET AND THE REDUCED-GRID SAFEGUARD

  • The compute safeguard reduces sweeps exceeding six projected hours to seven entanglement points, under-sampling the transition near ρ ≈ −0.75 in the depth and GELU checks.Seed counts remain unchanged, but the authors identify the asymmetric grid as a limitation of those robustness checks.Source: §B.2 · p. 17

B.3 BASIN CLASSIFIER

  • The basin classifier fits a two-component Gaussian mixture and labels the more negative-bias component shadow and the other mirror; under GELU, it selects component count by BIC.Runs used for causal validation must have assigned-basin posterior probability above 0.9.Source: §B.3 · p. 18

B.4 CONFIDENCE INTERVALS

  • Basin fractions use Wilson score intervals, while GELU erasure is defined by a causal patch effect exceeding that of a matched control-feature patch.The authors note that this functional criterion differs from ReLU’s zero-output criterion and was not cross-checked against a magnitude-based GELU threshold.Source: §B.4 · p. 18, §B.5 · p. 18

B.6 FULL LINEAR BASELINE SWEEP

  • The full 15-point linear-baseline sweep confirms that the preservation trend shown at three representative entanglements holds across the baseline’s range.The baseline sweep uses a grid distinct from the phase-sweep grid.Source: §B.6 · p. 18, Table 3 · p. 19

C.1 BASIN GEOMETRY AT FULL ANTIPODAL ENTANGLEMENT

  • At ρ = −1, mirror decoder cosine shifts from −0.800 before excision to −0.933, while shadow shifts to −0.401, ruling against a bias-only account.The shadow shift toward orthogonality supports describing its decoder rotation as partial rather than absent.Source: §C.1 · p. 19

C.2 BASIN CLASSIFIER DIAGNOSTICS UNDER GELU

  • Under GELU, four mixture components fit better than two; one is near full reflection, while output bias remains close to zero across all components.This differs from the negative-bias mechanism characterizing the ReLU shadow solution.Source: §C.2 · pp. 19–20, Table 5 · p. 20

D.1 DEPTH: FULL SUPPRESSION-LOCUS BREAKDOWN

  • The depth-2 suppression-locus breakdown classifies runs by normalized output- versus hidden-layer bias shifts and reports their pooled distribution across the reduced grid.The table caption identifies the distribution’s scope but provides no cell values in the supplied passages.Source: §D.1 · p. 20, Table 6 · p. 20

D.2 GELU: FULL BASIN DISTRIBUTION

  • The GELU basin breakdown reports how seeds distribute across the four fitted components, pooled over the same reduced grid used for the depth experiment.The supplied passages describe the table’s scope but do not provide its component counts.Source: §D.2 · p. 20, Table 7 · p. 21

D.3 CANDIDATE EXPLANATIONS FOR THE GELU BASIN STRUCTURE

The authors favor a distinct GELU mechanism family over a measurement artifact, while retaining uncertainty because the reduced grid has not been matched with a full-grid rerun. The appendix also notes that later results depend on keeping the encoder fixed during excision.

D.3 CANDIDATE EXPLANATIONS FOR THE GELU BASIN STRUCTURE

  • Near-zero output bias across GELU components supports a different mechanism family, but a full-grid rerun is still needed to distinguish it from measurement artifacts.The authors regard the mechanism-family explanation as most consistent with the evidence, not established beyond the artifact explanation.Source: §D.3 · p. 21

E THE ENCODER-FREEZING CORRECTION

  • The depth and GELU robustness results, like main-text results from §4 onward, rely on holding the encoder fixed during excision.The appendix examines an earlier unfrozen procedure to explain why the correction matters.Source: §D.3 · p. 21, E THE ENCODER-FREEZING CORRECTION · p. 21

E.1 ENCODER COLLAPSE UNDER UNFROZEN TRAINING

Unfreezing the encoder lets the excision objective erase B upstream, so a successful loss reduction no longer demonstrates readout suppression. The appendix explains the resulting causal-validation failure, the frozen-encoder correction, and the design and convergence criteria for a sweep of naturally emerging antipodal geometry.

E.1 ENCODER COLLAPSE UNDER UNFROZEN TRAINING

  • At ρ = −1, training shrank wB to approximately 0.002—about two orders of magnitude below its starting norm—while loss decreased normally.With the encoder trainable, shrinking B’s encoder column minimizes the excision loss by reducing B’s contribution to the bottleneck activation.Source: §E.1 · p. 21

E.2 CONSEQUENCES FOR CAUSAL VALIDATION

  • Unfrozen models showed zero B readout at every patch strength, including α = 1, because B’s information was already absent from the bottleneck.This contrasts with the smooth, substantial recovery reported for frozen-encoder models.Source: §E.2 · p. 22

E.3 THE CORRECTION

  • Excluding WE from the optimizer preserves B’s bottleneck representation and makes causal validation evidence about suppression rather than lost representation.All main-text results use this frozen-encoder correction.Source: §E.3 · p. 22

E.4 WHY THIS BOUNDARY CONDITION MIGHT NOT HOLD AT SCALE

  • Whether deletion is as available in language models remains unresolved; shared parameters may make it costly, and finding a norm-collapse route may be difficult in larger parameter spaces.The authors present both explanations as hypotheses, not experimental conclusions.Source: §E.4 · p. 22

F NATURAL EMERGENCE OF ANTIPODAL GEOMETRY: FULL SWEEP

  • The appendix supplies the full grid behind the claim that antipodal geometry emerges under ordinary training without direct geometric intervention.The sweep tests how sparsity, bottleneck dimension, and anti-correlation relate to that emergence.Source: F NATURAL EMERGENCE OF ANTIPODAL… · p. 22

F.1 SWEEP DESIGN

  • The sweep varied six sparsity values, four bottleneck dimensions, and five anti-correlation strengths across 1,200 runs from scratch with the encoder free.It used 10 seeds per grid cell and counted a run as antipodal when cos(wA, wB) < −0.9.Source: §F.1 · pp. 22–23

F.2 CONVERGENCE

  • Only converged runs entered the reported fractions; convergence required a plateau of less than 1% relative MSE decrease between the final training windows.Excluded non-converged runs concentrated at high sparsity and small bottleneck dimensions, and Figure 7 reports each cell’s surviving count.Source: §F.2 · p. 23

F.3 RESULTS

The sweep finds antipodal geometry more often as feature anti-correlation increases, with sparsity and bottleneck dimension shaping that pattern. A small transformer check and an exploratory account examine the bifurcation’s reach and possible mechanism, while leaving broader generality and the theory unresolved.

F.3 RESULTS

  • 6.8% of converged seeds reached antipodal geometry with no imposed anti-correlation, compared with 56.2% under strict mutual exclusivity.Across the grid, antipodal emergence increased monotonically with anti-correlation strength.Source: §F.3 · p. 23
  • Anti-correlation is the primary driver: without it, antipodal fractions stayed at most 0.30, while low dimension and moderate-to-high sparsity amplified its effect.Under strict mutual exclusivity, antipodal geometry remained rare at high sparsity with a wide bottleneck, where pressure to share a subspace is lower.Source: §F.3 · p. 23

G.1 MODEL DEFINITION

  • The transformer check used mutually exclusive A and B tokens, pretrained a two-layer causal model, then fine-tuned only its output readout against a penalty on B.A hidden-state probe checked that token identity was represented before fine-tuning, and a one-pass restoration of B’s pre-fine-tuning output bias tested readout recovery.Source: §G.1 · p. 24, §G.2 · p. 24, §G.3 · p. 24

G.4 FULL STATISTICS

  • Across 15 seeds, restoring B’s output bias raised mean predicted probability from 0.0031 to 0.0035, but that direction is algebraically guaranteed and specificity was not tested.Probe accuracy was at least 0.905 in every seed; without a matched negative control, the patch’s specificity is unestablished, and generation-level recovery remains unknown.Source: §G.4 · p. 25

G.5 SCOPE

  • The transformer result is a seed-limited qualitative check, not evidence of production-scale generality or of bifurcation under end-to-end fine-tuning.Its embedding, attention, and MLP layers remained fixed during fine-tuning.Source: §G.5 · p. 25

H.1 A BASIN-OF-ATTRACTION FRAMING

  • The exploratory account frames mirror and shadow as simultaneously available routes: further decoder rotation favors mirror at high entanglement, while a negative output bias may be cheaper at low entanglement.The proposed explanation is that entanglement shifts the relative local cost of these routes.Source: §H.1 · p. 26

H.2 A TESTABLE PREDICTION

  • The account predicts that changing the decoder-to-bias learning-rate ratio shifts the crossover, but this remains untested and is not a clean raw-step test under AdamW.The proposed follow-up compares AdamW with plain SGD to separate optimizer normalization from loss-landscape effects.Source: §H.2 · p. 26

H.3 DIRECTIONS FOR FUTURE THEORETICAL WORK

  • A direct account of how entanglement shapes local loss curvature remains an open theoretical question, not a result established by this paper.The authors identify this analysis and the proposed empirical check as future work.Source: §H.3 · pp. 26–27

I DIRECTIONS FOR EMPIRICAL VALIDATION

The paper identifies three unrun empirical comparisons that bound how far its findings can be generalized: alternative erasure methods, alternative objectives, and standard unlearning benchmarks.

Post-hoc nonlinear erasure

  • The authors do not test post-hoc nonlinear erasure methods, so whether they avoid the mirror/shadow bifurcation or genuinely remove B remains unknown.The proposed comparison would distinguish a training-time-specific bifurcation from one that also arises under nonlinear erasure applied after training.Source: I DIRECTIONS FOR EMPIRICAL VALIDATION · p. 27

Alternative unlearning objectives

  • The bifurcation’s dependence on the bespoke excision objective is untested; another objective could yield one basin or neither.The authors propose testing objectives from the unlearning literature, including representation misdirection and student-teacher distillation.Source: I DIRECTIONS FOR EMPIRICAL VALIDATION · p. 27

Validation on standard benchmarks

  • All claims use synthetic data with hand-constructed antipodal structure; whether the distinction appears on standard benchmarks such as TOFU or Bias in Bios is unknown.Real feature geometries may be less sharply antipodal and less cleanly separable from the rest of the representation.Source: I DIRECTIONS FOR EMPIRICAL VALIDATION · p. 27

Related papers

Loading 2609.27593v1…