PaperlayerView exact revision on arXiv

Source-linked AI summary

Replay in the Silent Degrees of Freedom: Continual Learning Without an Offline Phase

Yanhai Zhang, Jie Zhang, Chi Xu

arXiv:2609.31630v2cs.LGcs.NEstat.ML

TL;DR

To avoid pausing inference for replay or perturbing served inputs, the paper writes replay into degrees of freedom unused by the current k-WTA batch and uses refractory rotation. On held-out splits, the learner exceeds the best offline-night schedule by 2.6 points on one split and 0.3 on the other.

  • Problem

    Replay typically requires either an offline phase that stops acting or interleaving that perturbs computation serving the current input.

    Source: §1 · p. 2, §8 · p. 17

  • Method

    The approach confines replay to synapses outside the current input’s active computation and uses refractory rotation to alternate the active coalition.

    Source: §8 · p. 17, §10 · p. 18

  • Results

    Across held-out splits, the learner beats the best offline-night schedule by 2.6 points on the first and 0.3 on the second; isolation adds 0.1–0.3 points with rotation.

    Source: §10 · pp. 18–19

  • Takeaways & Limitations

    In the evaluated class-incremental MLP settings, consolidation does not require a separate offline phase.

    Source: §10 · p. 18

    The construction requires sparse representations that leave most hidden units silent and replay updates that can be masked synapse by synapse.

    Source: §8 · p. 17

Abstract

from arXiv · show

Replay-based continual learning rehearses past data either in an offline phase, during which the agent stops acting, or interleaved with the live stream, where it perturbs the computation serving the current input. An agent that learns in deployment can afford neither. Motivated by local sleep, the use-dependent off periods of individual cortical circuits in awake animals, we show that replay can instead be written into the degrees of freedom the current input leaves unused. In a network with k-winner-take-all hidden layers, confining replay updates to synapses whose presynaptic unit is silent or whose postsynaptic unit is inactive leaves the hidden computation on the current batch invariant: exactly so for silent and suppressed units, and for all but 0.3% of samples in practice. A refractory rule under which units that have just fired sit out the next competition doubles the width of this channel and carries most of the accuracy. On class-incremental split-MNIST the resulting learner, with no offline phase, matches or exceeds the best offline rehearsal schedule and outperforms experience replay, ER-ACE and unmasked interleaved replay, each re-tuned under the same micro-batch schedule. Against DER++ the comparison splits by protocol: with five epochs per task DER++ leads by 1.5 points once it runs under that schedule, and in a single pass over the stream, the regime closest to the agent deployment setting, the system leads it by 1.6 while offline rehearsal falls 15 points behind. On split CIFAR-10 it again leads offline rehearsal and experience replay, but trails ER-ACE and DER++ by two to three points. The construction is not tied to the local learner: on a backprop network with k-winner-take-all hidden layers under the same schedule, refractory rotation adds half a point in five epochs and three in a single pass, and isolation again costs nothing on top of it.

1 Problem Formulation

The formulation treats continual learning as a class-incremental stream evaluated through a shared all-class readout, and asks whether offline consolidation can be removed without losing sequential accuracy.

1 Problem Formulation

  • Tasks contain disjoint class sets, task identity is unavailable at test time, and a shared C-way readout is evaluated on all classes after the final task.The stream consists of contiguous tasks; accuracy on task τ after learning task t is a_t,τ, and F denotes forgetting.Source: §1 · p. 2
  • The learner is restricted to cortical constraints including k-WTA sparse codes, learned feedback synapses, bounded burst-like errors, Dale’s law, sparse connectivity, and one forward–feedback sweep per example.An episodic memory of at most K raw examples with reservoir writing is permitted; K is a budget, not a free parameter.Source: §1 · p. 2
  • The objective balances sequential accuracy, static i.i.d. accuracy, replay cost, and an arithmetic-energy proxy rather than optimizing a single accuracy measure.The static axis is accuracy on an i.i.d. shuffle of the same data; replay cost is measured in samples, with synaptic work reported separately.Source: §1 · p. 2
  • The central question is whether consolidation can occur only during inference on the wake stream, without losing Aseq, and at what cost in Aiid and replay cost R.Aseq is the sequential-accuracy target; the formulation also asks that ordinary learning not be damaged.Source: §1 · p. 2

2 Related Work

Prior work offers sleep-based consolidation, competition-based gating, parameter isolation, and complementary-learning strategies; this paper distinguishes replay written into currently unused degrees of freedom during ongoing inference.

2 Related Work

  • Prior ANN sleep approaches schedule consolidation globally and offline, unlike replay confined to units unused during ongoing inference.Examples include noise-driven spiking replay (Bazhenov et al., 2026), coarse sleep alternation, and wake–sleep frameworks (Robinson et al., 2022).Source: §2 · p. 2
  • Refractory competition has precedents, but this rotation benches recently active units specifically to enlarge a dynamically reallocated replay channel.Earlier examples include refractory winner-take-all dynamics, explicit refractory competitive learning, and history-dependent spiking selection.Source: §2 · p. 2
  • Unlike prior methods that protect old-task subspaces, this construction holds the ongoing wake computation invariant while replay uses degrees of freedom exposed as invisible by the current batch.It stores no bases, SVDs, or task masks; its null space changes with each batch, and mask-confined optimiser state is part of the exactness construction.Source: §2 · pp. 2–3
  • CLS-ER and DualNet increase the value of replayed samples but still replay at every step; this paper instead changes replay’s location, timing, and granularity.CLS-ER maintains fast and slow EMA weight copies, while DualNet trains a slow self-supervised learner alongside a fast supervised one.Source: §2 · p. 3
  • Sparse-expansion methods inspired by cerebellar, mushroom-body, and dentate-gyrus circuits were tested as a front-end, but provided no benefit.The related family includes the FlyModel, sparse distributed memory, and DG-gated mixtures.Source: §2 · p. 3
  • Local sleep can occur in awake animals; the paper frames refractory rotation and soft rotation as counterparts to ON/OFF alternation and tonic firing reduction, respectively.The cited causal evidence reports local discharge of sleep pressure and weakened synaptic markers under induced ON/OFF alternation, unlike equal tonic firing reduction.Source: §2 · p. 3

3 Experiment Setup

Experiments evaluate class-incremental and i.i.d. settings with specified network architectures, replay baselines, and accuracy, forgetting, replay-cost, dimensionality, and energy-proxy measures.

3 Experiment Setup

  • The sequential protocol uses split-MNIST and split CIFAR-10 with five two-class tasks and five epochs per task, plus a held-out tenth for evaluation.The static comparison uses an i.i.d. shuffle; the buffer holds K=1000 raw samples, with 200 and 5000 also tested on the buffer-size axis.Source: §3 · p. 3
  • The main local architecture is d–512–256–C with 10% k-WTA sparsity, 30% distance-dependent hidden connectivity, 80% excitatory units, and a single-sweep learning rule.The offline-night reference uses a narrower 256–128 core.Source: §3 · p. 3
  • Baselines include BP+ER, DER++, ER-ACE, A-GEM, offline-night rehearsal, local interleaved ER, no buffer, and the same schedule on a backpropagation MLP with k-WTA layers.The offline-night schedule uses 20 replay batches of 256 samples after each waking epoch, at three times the learning rate.Source: §3 · p. 4
  • Evaluation reports final accuracy, forgetting, replay cost, isolated-channel width, code overlap, dimensionality, and an idealised arithmetic-energy proxy.The proxy counts arithmetic only, excluding data movement, membrane updates, and comparisons, so it is a relative measure rather than a hardware figure.Source: §3 · p. 4

4 Methodology

The method combines a sparse k-WTA cortical learner with replay updates confined to degrees of freedom inactive for the waking batch. Refractory rotation expands that replay channel, while cost accounting quantifies the online overhead.

4.1 Cortical base learner

  • The cortical learner uses k-WTA hidden layers, a linear readout, and one top-down error sweep with learned feedback weights and heavy-ball SGD.Its forward and feedback weights use local products and shared decay; the default optimiser has one per-synapse velocity and no second-moment state.Source: §4.1 · p. 4

4.2 Exactly isolated replay

  • Replay updates are exactly invisible for silent-presynaptic and suppressed units; measured replay changed hidden codes in 0.33% of waking samples and predictions in 0.32%.Suppressed units are exact without a margin condition; the measured rates include a plastic readout and an unenforced margin for other units.Source: §4.2 · p. 5

4.3 Refractory rotation and its gates

  • Refractory rotation makes each unit that fired sit out the next competition, expanding the isolated channel from 0.2–0.3 natural silence to ≈0.53.Homeostatic pressure triggers replay bursts, while relative novelty gates rotation; an unmastered-stream clause also keeps rotation active when error remains high.Source: §4.3 · pp. 5–6

4.4 Cost accounting

  • The headline schedule makes 16,885 masked updates versus the night’s 480 (35×); on one CPU thread it takes 8.8 min, versus 1.4 min for the night and 8.6 min for unmasked interleaved replay.Source: §4.4 · pp. 6–7

5 Results

The results show that replay can consolidate split-MNIST memories during the stream without an offline phase, with performance depending on the replay schedule and task protocol. Rotation also transfers to a backpropagation learner, while results on split CIFAR-10 depend on the benchmark and substrate.

5.1 Local sleep replaces the night

  • 91.6 ± 0.3% on split-MNIST without an offline phase exceeds the best offline night (88.5 ± 2.8%) and ER-ACE (90.2 ± 0.4%), while tying DER++ (91.5 ± 0.5%).At K=200, the system also leads every reference; at K=5000, backprop references pull ahead, and the local learner's offline night remains near 88%.Source: §5.1 · p. 8

5.2 What rotation and isolation each buy

  • At batch 16, rotation is the larger accuracy factor; isolation's paired margin is within seed noise, but it avoids severe failures seen with unmasked replay at batch 2.The full system remains flat down to batch size 8 and loses 1.9 points at 4.Source: §5.2 · pp. 8–9, Figure 5 · p. 9

5.3 Behaviour during the stream

  • Isolation reduces prediction churn from 4.39% to 0.33%, while rotation holds the stability gap to 1.5 points versus about 16 without rotation.The isolated system gives up some anytime and worst-case retention to unmasked replay and DER++, while serving the stream slightly more accurately.Source: §5.3 · p. 10

5.4 The direction of protection matters, and the least optimiser state suffices

  • Heavy-ball SGD with one per-synapse velocity ties masked Adam, while mirror isolation and soft rotation both reduce accuracy.With momentum zero, isolated replay loses the velocity's noise averaging and the best pure-SGD setting trails by 3.2 sequential and 4.7 static points.Source: §5.4 · p. 10

5.5 When replay must be rare: homeostatic bursts, not surprise

  • Unit-level pressure-triggered bursts retain 88.6 ± 1.6% accuracy at a third of replay events; trickles and clocked bursts perform worse, and surprise triggering fires none in this regime.At the 10% fraction, rotation costs 1.3 points on i.i.d. MNIST; the reported gate remedies become largely unnecessary at that fraction.Source: §5.5 · p. 11, §5.6 · p. 11

5.7 Transfer to CIFAR-10

  • On raw split CIFAR-10, isolated replay with rotation beats offline rehearsal and BP+ER but trails ER-ACE and DER++ by two to three points.On feature inputs, the reported decomposition attributes the gap between local and backprop learners to dense activation rather than the learning rule.Source: §5.7 · p. 11

5.8 The mechanism on a backprop learner

  • Under the matched micro-batch schedule, the system trails DER++ by 1.5 points after five epochs but leads it by 1.6 in a single pass.On a backprop k-WTA network, rotation adds 0.5 points in five epochs and 3.4 in a single pass; isolation on top of rotation costs nothing measurable.Source: §5.8 · pp. 11–12

5.9 Energy

  • Converted to spiking inference, the default system runs at 92.7% accuracy and 117 nJ/sample, versus 2461 nJ for dense rate inference and 526 nJ for event-driven rate inference.The tested alternatives included frozen sparse expansion, generative replay, surprise-gated memory writing, soft rotation, and mirror-direction isolation.Source: §5.9 · p. 12, §5.10 · p. 12

6 Ablation Study

The ablations identify refractory rotation as the main sequential-accuracy gain and show that isolation preserves the live computation at little added cost. Raising the activity fraction to 10% reduces the need for additional gates, while the remaining trade-offs depend on replay configuration and task.

6.1 Reading the component table

  • At 10% k-WTA, rotation plus isolation reaches 91.6 ± 0.3 sequential accuracy versus 85.4 ± 3.5 without either; Figure 7 places top configurations within seed noise.The combined system also has a twelve-fold smaller seed deviation than the no-rotation, unmasked control.Source: §6.1 · p. 13, Figure 7 · p. 14
  • Rotation adds 3.6 sequential points over natural silence, while random matched suppression recovers most of its gain but trails by 0.5 sequential and 1.8 static points.The random-suppression control has 1.3× the forgetting (F=8.3 vs. 6.4).Source: §6.1 · p. 13
  • At 10%, always-on rotation is the default: the larger activity fraction absorbs most gate benefits, and the bout gate adds only 0.2 sequential and 0.3 static points.The two-timescale anchor is neutral at 91.7 ± 0.0/93.8 ± 0.3; Figure 7 shows the operating frontier.Source: §6.1 · p. 14, Figure 7 · p. 14
  • Adam moments that advance outside the isolation mask lose update confinement over time despite current-batch invariance; a single confined velocity ties at 91.6 ± 0.3, while zero momentum costs 3.2/4.7 points.The paper retains the velocity as the least state that stays confined at equal accuracy.Source: §6.1 · p. 14

6.2 Two probes of the residual static price

  • At 5%, weak symmetric coupling stabilizes sequential accuracy at 92.4 ± 0.2 with static parity, while utility-exempt rotation traces a trade-off dominated by the bout gate.Figure 7 plots the operating frontier at 10%; Figure 8 also compares a feature-CIFAR-10 ingredient decomposition and spiking accuracy against an energy proxy, where the rotation-free control dips 0.4 at Ts=32.Source: §6.2 · p. 14, Figure 7 · p. 14, Figure 8 · p. 15

6.3 The activity fraction

  • On feature CIFAR-10, increasing k-WTA from 5% to 25% raises sequential accuracy from 40.9 to 42.9 ± 0.6 and static accuracy from 38.3 to 46.4 ± 1.6.On MNIST, sequential accuracy is flat from 10% upward; the default 10% setting reaches 91.6 ± 0.3 sequential and 94.1 ± 0.2 static accuracy.Source: §6.3 · p. 15
  • At 10%, the larger activity fraction makes always-on rotation match the bout-gated system, while synaptic operations rise only 3–5%; the default substrate uses 10%.The fraction helps the rotating family, whereas rotation-free silent isolation gained nothing from the larger fraction in development.Source: §6.3 · p. 15

7 An adaptive decay controller

A fixed decay can overwhelm sparse learning signals as class count or depth increases. The paper replaces dataset-specific decay tuning with a per-layer ratio controller; it improves static accuracy across CIFAR settings and keeps tested deep networks learning, with stated scope limits.

7 An adaptive decay controller

  • With 100 classes, fixed 10^-3 decay yields 2.4% i.i.d. accuracy versus 20–23% for ten-class subsets, as per-class supervision thins while decay remains undiminished.A manual rescue survives the full protocol only at 1.0–1.2%.Source: §7 · p. 15
  • 1.2–5.3 static-accuracy points gained on CIFAR-10, feature CIFAR-10, and both CIFAR-100 variants, while the controller concedes 1–3 sequential points; Table 4 compares the settings.On split-MNIST, the paper claims no first-split static gain because one controller seed collapsed.Source: Table 4 · p. 16, §7 · p. 16
  • The controller sets per-layer decay from the ratio of data-driven waking-update magnitude to weight magnitude, keeping the target ρ=0.25 across datasets.The EMA uses waking, unmasked steps; its estimate excludes decay itself from the drive.Source: §7 · p. 16
  • At five hidden layers, the controller reaches 88.1 sequential and 89.6 static accuracy, versus 26.3 ± 20.2 sequential under tuned fixed decay; skips restore accuracy to 91.0 ± 0.3/95.8 ± 0.3.The paper did not re-tune fixed decay by depth; its claim is that the controller needs no re-tuning.Source: §7 · p. 16
  • The controller is not a capacity regularizer for small tabular problems, and amplified targets or replay batches can inflate its drive estimate until decay erases day learning.The paper says a tuned constant should retain the capacity-regularization role where generalization, rather than alignment, sets the proper decay.Source: §7 · p. 17

8 Discussion

Replay can be consolidated between stream steps by writing updates into currently unused synaptic degrees of freedom, with guarantees conditional on sparse representations and maskable updates. The discussion also identifies deployment constraints, biological caveats, and evaluation limits.

8 Discussion

  • Rotation supplies most of the accuracy: unmasked replay reaches 91.5 at batch 16 and 89.3 at batch 4, while masking adds 0.1–0.3 points and protects the current hidden computation.At batch 2, masking also avoids severe failures seen in half the unmasked seeds; invariance holds on proven channels and for all but about 0.3% of waking samples per step.Source: §8 · p. 17
  • The recipe requires sparse representations that leave hidden units silent and replay updates that can be masked synapse by synapse; it transfers to k-WTA backprop networks, but isolation alone does not transfer.Consolidation needs no separate offline phase under these prerequisites, but the unoptimised implementation uses 6× the offline night’s wall-clock.Source: §8 · p. 17
  • For modern stacks, larger waking batches narrow the silent channel, shared weights require channel- or neuron-level sparsity, and normalization across selected units voids the guarantee.Per-channel batch normalization avoids pooling across channels, though replay can still move its running statistics; for routed experts, sufficient batch-wide idleness must be measured.Source: §8 · p. 17
  • The biological account is an analogy, not mechanistic equivalence: refractory rotation mirrors reported ON/OFF alternation, while tonic reduction performs worse in the paper’s controls.The authors offer the distinct homeostatic and novelty-timed triggers as a testable biological prediction.Source: §8 · p. 17
  • Results are limited to MLP-scale MNIST/CIFAR variants, and the hidden-computation guarantee does not settle how prediction churn affects a downstream controller.The guarantee covers hidden computation, not the output; the readout remains plastic by design, and the authors report prediction changes on 0.33% of samples per update.Source: §8 · p. 18
  • The code for the experiments is publicly available on GitHub.The paper gives the repository address as https://github.com/Hanny658/IaC-Replay-in-SDF.Source: §8 · p. 18

9 Future Work

The authors identify unresolved questions spanning offline-night stability, rotation costs, substrate breadth, throughput, calibration, and controller robustness.

9 Future Work

  • Open work includes explaining the same-substrate offline night’s bimodal instability at the 10% fraction, reported as 89.0 ± 3.5.The authors also list the narrow-versus-wide substrate interaction of offline nights at tiny buffers as unresolved.Source: §9 · p. 18
  • The bout gate retains a static price, and mastery-annealed rotation depth remains untried after three proposed routes past that price were ruled out.The development-phase 5% figures are 94.2 versus 96.0 for the bout gate and rotation-free system, respectively.Source: §9 · p. 18
  • Convolutional stacks and learned front ends remain open; in the 100-class regime, a 6–8% static-axis wall across substrate changes points to the front end as the binding constraint.Depth and residual-style routing were explored, with width rather than depth paying off on 100-class streams.Source: §9 · p. 18
  • Further work is needed on a throughput theory linking replay cadence to ρℓ, code overlap, and interference, and on rotation’s interaction with spike calibration.The calibration interaction was optimizer-dependent in development and small on the held-out split.Source: §9 · p. 18
  • A drive estimate robust to amplified update streams is also unresolved because inflated targets and night gains currently defeat the controller’s clip.Source: §9 · p. 18

10 Conclusion

The conclusion presents isolated replay with refractory rotation as a way to consolidate during the stream without a separate offline phase. It reports competitive retention and benchmark performance, with rotation transferring to backpropagation learners under the same sparse-layer setup.

10 Conclusion

  • In the evaluated class-incremental MLP settings, consolidation needs no separate offline phase; isolation’s accuracy value is 2.6 points without rotation and 0.1–0.3 with it.With rotation, live units tolerate unmasked replay down to four-sample micro-batches, while isolation provides a guarantee and graceful degradation at two samples.Source: §10 · p. 18
  • The system is competitive with offline-night schedules on both held-out splits, exceeds the best night by 2.6 points on one and 0.3 on the other, and consistently beats ER, ER-ACE, A-GEM, and unmasked local replay.It leads every reference at a 200-sample buffer and in a single pass, but trails DER++ by 1.5 points at five epochs and leads it by 1.6 in a single pass.Source: §10 · p. 19
  • The mechanism transfers to backpropagation with the same sparse layers: rotation helps most in a single pass, and isolation adds no cost on top of rotation.The conclusion attributes shortfalls on large buffers and raw CIFAR-10 against cross-entropy backpropagation references to the local learner, not the mechanism.Source: §10 · p. 19

A One training step, the three channels and the skip path

The appendix specifies a waking-step schedule, illustrates how replay updates can leave winner activity unchanged, and extends the mask to skip synapses. It also identifies a table describing the three consolidation channels opened by the synapse mask.

A One training step, the three channels and the skip path

  • The waking update uses ηw = η m/256, while replay uses ηr = 3η per micro-batch, with η = 0.02 throughout.Awake sets are read from the waking forward pass before its update; the replay micro-batch is inferred without suppression, and a burst reuses its waking batch’s mask.Source: A One training step, the three… · p. 22
  • Table 5 is identified as presenting the three consolidation channels opened by the synapse mask.Source: Table 5 · p. 22
  • The numerical example shows that updates to incoming weights of asleep units can leave the activity vector unchanged if their post-update activations stay below the winner threshold.When a unit is refractory, its bound is void; outgoing synapses from a unit with zero presynaptic activity may move by any amount.Source: A One training step, the three… · p. 22
  • For layers ℓ≥3, each hidden layer receives a bypass from the layer two steps earlier, with its skip weights under the same Dale projection and replay mask rule.The skip mask is defined using the activity sets two layers apart, extending Propositions 1–2 unchanged.Source: A One training step, the three… · p. 22

B The price of the substrate

The substrate constraints preserve most i.i.d. MNIST accuracy while sharply reducing synaptic operations, and the optimizer comparison changes inside the isolated-consolidation loop.

B The price of the substrate

  • The full constraint set is 1.2 points below dense backpropagation (97.7 →96.5) with 4× fewer synaptic operations per pass.The local burst-error objective itself costs nothing: 97.8 unconstrained.Source: B The price of the substrate · pp. 22–23, Table 6 · p. 23
  • The bounded-error/Dale/no-transport trio costs 0.7 points, 10% k-WTA costs one tenth of a point, and the 70% connectivity cut costs half a point.Distance-dependent wiring recovers +1.1 over random wiring at equal density.Source: B The price of the substrate · p. 23, Table 6 · p. 23
  • SGD trails Adam by 1.3 points on the plain substrate (95.2 vs. 96.5), but they tie inside the isolated-consolidation loop.The passage concludes that adaptive per-synapse state helps when every synapse trains on every batch, but not when updates are intermittently mask-confined.Source: B The price of the substrate · p. 23

C The per-batch rotation gates under masked Adam

Rotation gates preserve useful replay while balancing sequential and static accuracy, but the best gate depends on the data regime. Small-buffer, pruning, and optimizer studies further delimit when this approach helps.

C The per-batch rotation gates under masked Adam

  • Appendices C–G report development-phase numbers that were not rerun under the held-out protocol, supporting mechanism orderings rather than held-out magnitudes except where explicitly marked.Source: C The per-batch rotation gates under… · p. 23
  • The unmastered-clause gate yields 27.2 ± 0.7 sequential and 25.3 ± 1.0 static accuracy on split CIFAR-10, 90.4 ± 0.9 on split-MNIST, and 94.3 ± 0.2 on i.i.d. MNIST.It matches always-on rotation on split CIFAR-10 and retains a 0.7-point static cost relative to the pure novelty gate on easy streams.Source: C The per-batch rotation gates under… · p. 23, Figure 9 · p. 24

D Small buffers: what the night still buys

  • At K=200 on the held-out split, the system reaches 83.0 ± 1.4, matching the narrow night’s 82.7 ± 2.9 and exceeding DER++ (77.1) and BP+ER (68.5).On the same 512–256/5% substrate, local sleep and the night are at parity; the night’s remaining small-buffer advantage occurs with the narrower substrate.Source: D Small buffers: what the night still… · p. 24

E Sleep-anchored down-selection

  • Pruning the episodic-burst substrate lowers accuracy to 84.6 ± 0.5 when burst-anchored and 86.6 ± 1.3 when clocked, versus 89.9 ± 0.1 without pruning.The authors attribute the harm to pruning synapses before enough rare replay events have protected them.Source: E Sleep-anchored down-selection · pp. 24–25

F Learning-rate sensitivity

  • At the 10% fraction, heavy-ball SGD’s joint optimum is η=0.02 (92.7 ± 0.3/94.7 ± 0.3), while masked Adam’s is its default 10−3 (91.8±0.6/94.6± 0.2).The comparison uses each optimizer’s best rate; unmasked SGD remains poor and unstable across the tested rates.Source: F Learning-rate sensitivity · p. 25

G Probes of the residual static price

The section probes the residual static cost, replay isolation across micro-batch sizes, and robustness across held-out splits. Results show strong single-pass performance, with meaningful boundaries at tiny replay batches and in some ablations.

G Probes of the residual static price

  • In one pass, isolated replay with rotation reaches 91.8 ± 0.3 (F=5.5), ahead of DER++ at 90.1 ± 0.7, while the offline night falls to 76.9±3.4.The system leads every tested control in this protocol, and the offline night falls below BP+ER at 88.6±0.2.Source: G Probes of the residual static price · p. 25
  • At M=512, the bout gate gives duty 0.38 and static accuracy 94.0; lengthening bouts to M=4096 raises static duty to 0.94 and removes the refund.The authors locate the residual static price in rotation of settled coalitions, not in how often the gate fires.Source: G Probes of the residual static price · p. 25
  • Utility-exempt rotation gives 90.9 ± 0.1/94.2 ± 0.2 at q=0.10 and 89.1 ± 0.4/95.4 ± 0.2 at q=0.25; the bout gate beats q=0.10 by +1.1 sequential at equal static.The authors conclude that structurally narrowing the rotation pool reduces replay-channel width.Source: G Probes of the residual static price · p. 25
  • Weak symmetric anchor coupling (λ=µ=3·10−4) reaches 92.4 ± 0.2 sequential accuracy at static parity, while strong coupling hurts both axes.No tested cell exceeds the anchor-free static accuracy; the anchor and bout gate compose sub-additively at 92.3 ± 0.3/93.8 ± 0.5.Source: G Probes of the residual static price · p. 25

H Isolation across replay batch sizes

  • At replay batch sizes 16, 8, and 4, isolation’s seed-paired accuracy margins over unmasked replay are 0.1 [−0.3, +0.5], 0.3 [−0.2, +0.6], and 0.3 [−0.5, +1.2].At batch 2, unmasked replay has three severe failures among six seeds, while the isolated system degrades without a severe failure in the tested seeds.Source: H Isolation across replay batch sizes · p. 26, Figure 11 · p. 26

I Development-phase numbers and a second held-out split

  • Across 87 sequential configurations, held-out results average 1.1 points below development results and rank-correlate at Spearman ρ = 0.97.The headline orderings hold on both held-out splits, although the same-substrate night trails by only 0.3 on the second split.Source: I Development-phase numbers and a… · p. 26, Table 7 · p. 27
  • The headline configuration’s split-MNIST forgetting is 6.4 ± 0.6, with the drop concentrated on tasks 2–3 and the first task retained.Table 8 reports rows as training task i and columns as accuracy on task j.Source: Table 8 · p. 27
Loading 2609.31630v2…