PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Under Forgetting: Statistical Support-Selective Retention in Stochastic Training Dynamics

Fujie Gao, Zuyue Zhang, Gang Sun

arXiv:2609.38768v1cs.LGcs.AI

TL;DR

The paper asks which patterns persist when learning and forgetting coexist, a question prior work has not directly characterized. It models repeated reinforcement against persistent attenuation and finds that interventions and reinforcement timing change conditional retention and the retained set.

  • Problem

    Prior work has paid relatively little attention to which patterns survive under simultaneous reinforcement and attenuation.

    Source: §1 · p. 1

  • Method

    The paper formulates RPF dynamics, combining repeated reconstruction of supported patterns with persistent attenuation, and studies theory and controlled interventions.

    Source: §1 · p. 2

  • Results

    Joint interventions shift conditional retention, matched exposure counts yield timing-dependent retained sets, and the nonlinear boundary’s empirical log–log slope is 3.05 across 53 runs with identifiable crossings.

    Source: §5 · pp. 10–11, §6 · p. 12

  • Takeaways & Limitations

    Attenuation sets the support required for retention, while the amount and timing of reinforcement determine which patterns cross that boundary.

    Source: §6 · p. 12

    RPF selects by support and recoverability, not by whether stored information is meaningful, true, or useful.

    Source: §5 · p. 10

Abstract

from arXiv · show

Prior work has shown that neural networks exhibit implicit biases toward low-complexity structure (e.g., spectral bias), memorization dynamics, and compression-like effects during training, but a unified dynamical account of selective retention remains incomplete. We propose Repeated Reinforcement with Persistent Forgetting (RPF) dynamics, a minimal framework in which repeated exposure reinforces patterns and structures that recur in the data, while persistent forgetting attenuates learned information. This view treats forgetting not merely as a failure mode, but as a selection mechanism. We build the theory in three successive layers. First, in an independent-feature model, we derive an exposure-selective survival law and a support-dependent retention boundary characterizing which patterns persist under forgetting. Second, in a shared-parameter model, we show that forgetting induces spectral filtering over covariance modes, preserving strongly supported shared components while suppressing weak ones. Third, under small-step and norm/coding approximations, we show how RPF dynamics induce an implicit trade-off between data fitting and the cost of stored information, yielding Minimum Description Length (MDL)-like compression. Controlled experiments provide evidence for this reinforcement--forgetting selection mechanism in scalar memories and a nonlinear shared network. Joint reinforcement and attenuation interventions shift conditional retention, while matched exposure counts reveal forgetting-dependent effects of reinforcement timing and changes in the composition of the retained set. Together, these results show that repeated reinforcement and persistent forgetting jointly provide a controllable source of inductive bias beyond neural architecture and scale.

1 Introduction

The paper reframes forgetting as a selection mechanism: recurring, reconstructable patterns can survive attenuation while weakly supported details fade. It introduces RPF to analyze this process through theory and controlled experiments.

1 Introduction

  • Prior work has paid relatively little attention to which patterns survive when reinforcement and attenuation act simultaneously.The paper shifts attention from preventing forgetting to understanding what it permits to persist.Source: §1 · pp. 1–2
  • RPF treats retention as a consequence of patterns being reconstructed often enough to survive forgetting, not of semantic importance.Frequent superficial patterns may persist, while rare meaningful facts may need additional mechanisms.Source: §1 · p. 2
  • The paper develops an exposure-based survival law, shared-parameter spectral filtering, and an approximate MDL-like interpretation, then tests selection in scalar memories and a nonlinear network.The experiments examine conditional retention and history-dependent selection under matched occurrence counts.Source: §1 · p. 2

2 Related Work

The related work spans forgetting, training dynamics, data repetition, and compression. RPF distinguishes itself by asking which patterns persist under ongoing forgetting, rather than only how to prevent forgetting or characterize aggregate performance.

2 Related Work

  • Continual-learning methods typically seek to prevent or compensate for forgetting, whereas RPF asks which reinforced patterns survive while forgetting remains active.Examples include explicit constraints, replay, and memory methods such as EWC and Synaptic Intelligence (Kirkpatrick et al., 2017; Zenke et al., 2017).Source: §2 · p. 2
  • Prior work studies implicit biases, spectral bias, example-level forgetting, and regularization, while RPF focuses on forgetting pressure competing with pattern-specific reinforcement.Weight decay is used as a smooth-attenuation control, while dropout is treated as a candidate masking mechanism requiring empirical evaluation.Source: §2 · p. 2
  • RPF makes pattern-level retention boundaries and reinforcement schedules explicit, complementing work on weight spectra and frequency suppression (Yunis et al., 2024; Lu, 2025).Its focus includes how matched occurrence counts can yield different final value through timing.Source: §2 · p. 2
  • Studies of memorization, repeated data, and long-tail recall motivate asking how repeated exposure translates into retention under persistent forgetting.The related work also distinguishes RPF’s dynamics-induced compression-like effect from explicit Information Bottleneck or MDL objectives (Tishby and Zaslavsky, 2015; Alemi et al., 2017).Source: §2 · p. 3

3 Theoretical Analysis

The formalization treats patterns as reusable regularities and separates exposure, compatible reinforcement, recoverability, and forgetting. RPF then models learning as repeated updates followed by attenuation, with retention depending on effective support relative to forgetting.

3.1 Formalization

  • Patterns and structures are reusable regularities whose occurrences induce compatible learning signals, with boundaries defined operationally rather than semantically.Examples include words, relations, textures, acoustic motifs, and distributed components that can be repeatedly reinforced or reconstructed.Source: §3.1 · p. 3
  • Persistent forgetting continually reduces recoverable strength, and its implementation by decay, perturbation, masking, overwriting, or interference depends on the setting.The analysis uses multiplicative decay as a tractable smooth-attenuation case.Source: §3.1 · p. 3
  • Effective support combines sampling exposure q(s) with conditional recovery R(s), while accumulated compatible reinforcement is not raw frequency alone.Recoverability depends on representation and update protocol; exposure and recovery are accounted for separately.Source: §3.1 · pp. 3–4

3.2 Repeated Reinforcement with Persistent Forgetting (RPF)

  • RPF represents a training step as reinforcement followed by attenuation, so retention requires enough reconstruction to withstand forgetting.The finite-time condition concerns cumulative reconstruction, while the steady-state condition compares per-step reinforcement with attenuation; the support mapping depends on representation and retention level.Source: §3.2 · p. 4

3.3 Theoretical Results

The theoretical results develop RPF from independent-feature survival to shared-direction filtering. Exposure and recoverability set a retention boundary, while finite-time training and shared covariance structure determine how the retained set expands and which directions persist.

3.3 Theoretical Results

  • The analysis frames the theory as three layers: independent-pattern survival, retention of shared representation directions, and an information-constrained interpretation of maintained structure.The supplied shared-parameter passages specify covariance-supported directions but do not provide the equilibrium expression’s values.Source: §3.3 · p. 4, §3.3.2 · p. 6

3.3.1 Independent features: exposure-selective survival

  • Under independent-feature assumptions, expected learned strength converges according to an exposure-selective law, and retention depends on reinforcement relative to forgetting.The resulting threshold is soft, and finite-run retention remains stochastic.Source: §3.3.1 · p. 5, Consequently there is a characteristic… · p. 5

Consequently there is a characteristic survival scale

  • For power-law exposure in the accumulation-dominated regime, the retained rank cutoff scales as k* ∼ S^(1/α), up to constants and before pattern support is exhausted.Here S = nη(1 − d); at fixed positive forgetting, training approaches a retention ceiling rather than extending the early sublinear expansion indefinitely.Source: Consequently there is a characteristic… · p. 5

3.3.2 Shared parameters: spectral filtering

  • In shared-parameter models, reinforcement–forgetting selection applies to supported directions of variation rather than isolated pattern coordinates.The second-moment matrix’s eigenspectrum measures support along shared directions.Source: §3.3.2 · p. 6

Thus there is a characteristic spectral threshold

For shared parameters, forgetting filters covariance-supported modes around a characteristic threshold, preserving strongly supported directions while attenuating weak ones. Under small-step and norm/coding assumptions, the dynamics also admits an approximate MDL-like interpretation in which unsupported information leaks.

Thus there is a characteristic spectral threshold

  • Low-support modes satisfy f(λ) = O(λ) for λ ≪ λ∗, while high-support modes satisfy f(λ) ≈ 1 for λ ≫ λ∗.Forgetting filters covariance-supported directions, preserving reusable components and attenuating weak or idiosyncratic modes.Source: Thus there is a characteristic… · p. 6

3.3.3 Information-constrained interpretation

  • Under small-step and norm/coding assumptions, parameter norm proxies stored information, yielding an approximate MDL-like effect rather than an exact equivalence to MDL objectives.Information not repeatedly reconstructed is continuously leaked, whereas supported and recoverable information remains worth maintaining.Source: §3.3.3 · p. 6
  • After 105 updates, learned strength shows a soft support-dependent transition, while the sampled retained-memory count grows sublinearly toward saturation.The expected-strength law k∗∼S1/α describes the accumulation-dominated regime.Source: Figure 1 · p. 7

4 Experiments

The experiments test exposure-selective survival in scalar memories, conditional associations in a shared nonlinear network, and whether reinforcement timing changes retention when sample counts are matched. Across these settings, repeated reinforcement and persistent forgetting select both which patterns remain and how strongly they are retained.

4.1 Experiment 1: Independent-Feature Survival

  • Low-exposure memories fail to compensate for attenuation, while high-exposure memories approach the target; sampled retention grows sublinearly and eventually saturates.The soft transition and intermediate-support scatter agree with the expected profile; reinforcement initially extends retention into the tail, while persistent forgetting sets the long-run support boundary.Source: §4.1 · p. 7

4.2 Experiment 2: Conditional Retention under Reinforcement and Forgetting

  • 0.694 correct-input probability versus 0.043 same-rank control probability gives a 0.652 conditional gap, while reinforcement shifts tail retention near the transition.At dstep = 0.012, increasing m from 1 to 2, 4, and 8 raises tail focal retention from 0.013 to 0.175, 0.525, and 0.819.Source: §4.2 · p. 8, Figure 2 · p. 8

4.3 Experiment 3: Temporal Coupling and Pattern Selection

  • 0.3009 at dstep = 0.015 is the additional late–early timing contrast; equal sample counts still produce different final retention, with the difference depending on forgetting.At dstep = 0.012, focal/background retained fractions shift from 0.232/0.465 under early replay to 0.995/0.000 under late replay, using cs ≥0.5.Source: Figure 3 · p. 9, §4.3 · p. 9

5 Discussion

RPF treats persistent forgetting as a selection pressure: retention depends on whether reinforcement can repeatedly reconstruct information. The discussion connects this account to conditional neural results, support-dependent retention, and limits on what the framework says about stored information.

5 Discussion

  • RPF does not separate signal from noise or select by truth, usefulness, or meaning; sufficiently supported artifacts and biases can persist.Its survival criterion is support and recoverability, rather than semantic importance.Source: §5 · pp. 9–10
  • In shared parameters, reusable covariance-supported directions persist under repeated reconstruction, while weak or idiosyncratic directions are attenuated.The discussion describes abstraction as the residue of repeated reconstruction, not as a semantic definition or architectural constraint.Source: §5 · p. 10
  • RPF reframes attenuation as controllable forgetting pressure that raises the support required for retention, rather than merely as a small generalization penalty.Coupled to nonuniform reinforcement, uniform attenuation produces a nonuniform survival boundary.Source: §5 · p. 10
  • Matched exposure counts can yield different retained sets because each update’s effective contribution depends on the dynamics that follow it.The neural timing contrast includes effects mediated by adaptive optimization and shared-gradient interference, which require additional interventions to separate.Source: §5 · pp. 10–11
  • At fixed positive forgetting, retained strength approaches a boundary and the retained rank approaches a forgetting-limited ceiling.Initial reinforcement expands the retained set, while further training approaches that ceiling.Source: §5 · p. 10
  • The nonlinear phase boundary shifts toward higher support as attenuation increases, with a log–log slope of 3.05 across 53 identifiable crossings.The slope differs from the approximately unit slope of the small-decay scalar boundary at fixed learning rate.Source: §5 · p. 11
  • The neural measurements concern conditional behavior; identifying local recovery and support operators and testing broader compositions and real-language data remain open steps.The heavy-tailed analysis uses a power-law form for closed-form retained-count scaling, while the broader requirement is scale separation in effective support.Source: §5 · p. 11
  • Attenuation and replay offer ways to alter retention, but concentrated late reinforcement can preserve focal associations at the expense of background retention.The final value of a fixed reinforcement budget depends on when it is spent, so retained-set composition should be evaluated alongside focal gains.Source: §5 · p. 11

6 Conclusion

RPF models learning as repeated reconstruction under persistent forgetting, with exposure-dependent survival in independent features and covariance-supported directions in shared parameters. Conditional neural experiments support the selection picture: attenuation sets the support needed for retention, while reinforcement amount and timing affect which patterns remain available.

6 Conclusion

  • RPF links independent-feature survival boundaries and shared-parameter covariance filtering to a retained set shaped by repeated reconstruction under forgetting.The conclusion describes finite-time expansion followed by a forgetting-limited ceiling.Source: §6 · p. 11
  • Conditional neural experiments show that changing reinforcement timing can change retention despite identical occurrence counts.The intervention also changes which patterns remain available.Source: §6 · p. 12
  • Attenuation sets the support required for retention, while the amount and timing of reinforcement determine which patterns cross that boundary.The conclusion presents this as the RPF selection principle.Source: §6 · p. 12

A Proofs

The proofs establish scalar fixed-point and shared-linear results by reducing each case to a scalar recursion. The scalar recursion converges to a steady state, and the shared-linear projection follows the same form with covariance eigenvalues replacing feature exposure probabilities.

A Proofs

  • Taking conditional expectation in Eq. (7) yields a scalar recursion with attenuation-adjusted coefficients for the mean strength.The proof sets a_s = (1 − d)(1 − ηp(s)) and b_s = (1 − d)ηp(s)θ_s.Source: A Proofs · p. 14
  • Because 0 < a_s < 1, the mean recursion converges to μ* = b_s/(1 − a_s), and its threshold compares (1 − d)ηp(s) with d.This gives the fixed point used for Eq. (8).Source: A Proofs · p. 14
  • Projecting the shared-linear dynamics onto a covariance eigenvector reduces the mode to the scalar recursion with p(s) replaced by λ_i.Solving that fixed point yields Eq. (18).Source: A Proofs · p. 14

B Additional Theoretical Details

The theoretical details define patterns operationally and derive how support, forgetting, and reinforcement history shape retention. They also extend scalar dynamics to shared updates and identify the conditions and approximations underlying these results.

Pattern scope and forgetting operators

  • The pattern unit is operationally defined by occurrences that induce compatible learning signals, and the operator formulation accommodates different implementations of persistent forgetting.Independent Bernoulli erasure with probability d has conditional mean attenuation 1 − d; implementations differ in finite-run variability and behavioral mapping.Source: B Additional Theoretical Details · p. 14

Retention boundary

  • At fixed positive forgetting, the limiting rank boundary depends on support and the retention criterion, is clipped to available ranks, and separates finite-time growth from equilibrium selection.If even the largest exposure falls below the criterion, no pattern meets it; a pattern exactly at the boundary approaches it asymptotically.Source: B Additional Theoretical Details · pp. 14–15

Retained exposure mass

  • In the early regime, retained exposure mass approaches 1 − O(S^−((α−1)/α)), while fixed positive forgetting imposes a ceiling determined by the stationary retained set.The early-regime result applies when its conditions hold and the cutoff lies within the ranked support.Source: B Additional Theoretical Details · p. 15

Batch-wise attenuation

  • With batch-wise attenuation every B updates, the equivalent per-update rate is approximately d/B for small d, and the observed transition, sublinear expansion, and saturation qualitatively support the model.For a final partial cycle, apply its unattenuated expected updates before attenuation.Source: B Additional Theoretical Details · p. 15

Independent-feature simulation

  • Intermediate-support scatter arises because intermittent exposures coincide with appreciable recovery gaps, unlike low-support patterns with few updates or highly supported patterns near saturation.This reconstruction-based account explains the variation in the independent-feature simulation.Source: B Additional Theoretical Details · p. 15

Reinforcement timing

  • Under forgetting, equal exposure counts need not yield equal final scalar strengths; reinforcement order changes the finite-time result, while a single-target scalar case without decay depends only on count.For the analyzed scalar operators, reinforcement followed by forgetting differs from the reverse order by −adθ.Source: B Additional Theoretical Details · p. 16

Shared dynamics

  • In shared linear dynamics, final contributions depend on alignment and subsequent shared updates as well as attenuation; a local quadratic or Gauss–Newton approximation motivates analogous operators for nonlinear networks.Estimating those nonlinear operators is a further identification task; the neural experiments test observable selection and timing predictions without assuming a fixed covariance spectrum.Source: B Additional Theoretical Details · p. 16

C Information-Constrained Interpretation

The paper interprets persistent forgetting as a cost on stored information under small-step and coding assumptions. This yields an approximate fitting–storage trade-off, not an exact equivalence to MDL or information bottleneck objectives.

Information-constrained interpretation

  • Under small-step and fixed Gaussian coding approximations, the norm term proxies storage cost, so loss-reducing updates must offset that cost for a component to persist.Unsupported components decay; the interpretation is not an exact equivalence to IB or MDL objectives.Source: C Information-Constrained Interpretation · p. 16

D Experimental Details

The experiments combine scalar memories and shared neural tasks to examine exposure, attenuation, and reinforcement timing. Resampling effects vary with attenuation, and matched schedules and held-out combinations test how selection depends on history and shared factors.

Scalar-memory setup

  • The scalar-memory experiment attenuates every 32 updates, and its largest plotted horizon has substantial cumulative forgetting, so the full curve is not confined to the accumulation-dominated regime.It uses 10,000 Zipf-ranked patterns and counts memories above 0.01 of the maximum strength as retained.Source: D Experimental Details · p. 16

Conditional task and phase-grid setup

  • The neural phase grid crosses focal resampling weights with AdamW decay levels over five seeds, using uniform label marginals; phase-grid sampling sequences are not matched across conditions.The task uses a 512 × 24 embedding table and a shared 24 → 48 → 8 network, with 600 steps per run.Source: D Experimental Details · p. 17

Phase-grid results

  • 0.177 focal-probability gain for m = 8 minus m = 1 at dstep = 0.012, versus 0.014 at zero attenuation and 0.041 at 0.021.The empirical boundary analysis uses a separate fitted retention criterion; these gains are threshold-free.Source: D Experimental Details · p. 17

Empirical boundary analysis

  • Across 53 eligible runs, the fitted boundary-versus-decay slope is 3.047 with R2 = 0.871; this empirical slope does not estimate the retained-count exponent 1/α.Runs without a fitted crossing were not assigned a boundary.Source: D Experimental Details · p. 18

Post-hoc predictor comparison

  • The flexible two-axis interaction model has held-out RMSE 0.0405, compared with 0.0612 for the additive model and 0.1767 for the fixed log-ratio coordinate.These exploratory fits diagnose limits to quantitative transfer rather than define coupling.Source: D Experimental Details · p. 18

Matched schedules

  • Matched schedules preserve per-identifier exposure counts while placing replay samples early, interleaved, or late; the paired contrast subtracts each seed’s zero-decay late–early gap.Intervals resample ten paired seed contrasts, not individual identifiers.Source: D Experimental Details · p. 18, Table 3 · p. 18

Factorized held-out combinations

  • 0.374 is the peak paired gain in focal correct-label probability near the decay-3 transition on held-out combinations; the corresponding background gain is 0.029 with interval [−0.009, 0.070].The held-out entity–relation–template triples were unseen in training, although each factor appeared there.Source: D Experimental Details · pp. 18–19, Table 4 · p. 19

Compute and reproducibility

  • Saved run-level outputs and analytical identities were checked for consistency, but the reported checks did not rerun the training experiments.The recorded phase, timing, and factorized runs comprised 140, 120, and 80 runs, respectively.Source: D Experimental Details · p. 19
Loading 2609.38768v1…