Kronecker Delta Rule for Long Context Linear Attention

1 Introduction

• Linear attention compresses context into fixed-size memory, yielding linear computation and constant memory footprint, but unbounded memory growth can cause numerical instability.Forget-based controls can attenuate early information over long contexts, while mLSTM’s normalizer and stabilizer can also lead to numerical difficulties.Source: §1 · p. 2
• KronLSTM’s Kronecker Delta Rule revises targeted associations along a single direction in matrix space, reducing interference with previously stored associations.The paper also reports that this targeted revision produces a less noisy gradient flow over time.Source: §1 · p. 2
• The Kronecker Delta Rule is introduced as an associative-matrix-memory update that corrects scalar associations with minimal interference.KronLSTM embeds the rule in a linear-attention architecture with vector forget, β, and target gates.Source: §1 · p. 2
• The paper proves bounds on the expected memory norm for the Kronecker Delta Rule, which ensure stable learning and proper gradient flow.Source: §1 · p. 2
• Experiments validate KronLSTM on long-context language modeling and retrieval, with strong results also reported for recall and state tracking.Source: §1 · p. 2

2.1 Kronecker Delta Rule

• The Delta Rule revises a stored value by removing the current retrieval before writing, but its correction can affect other keys that overlap with the updated key.The Kronecker Delta Rule instead corrects the scalar association along the outer product of the key-value pair, a direction determined by the input pair rather than memory content.Source: §2.1 · p. 3
• For the Kronecker Delta Rule, stored associations are changed according to both value overlap and key overlap, so orthogonality in either protects them.Its update performs gradient descent on a scalar quadratic loss; for 0 < β_t < 2/||a_t||^2 it contracts the error, and the Newton step meets the target in one update.Source: §2.1 · pp. 3–4
• Under the stated i.i.d. data and step-size assumptions, the expected iterate has a unique least-squares fixed point, and the memory process converges geometrically to a unique stationary distribution.The stationary mean equals the least-squares memory.Source: §2.1 · p. 4
• In the non-expansive regime, the expected memory is bounded from the first step by a constant determined by the data distribution.The theorem bounds the steady-state memory and, when initialized at zero, every iterate; all singular values obey the stated upper bound.Source: §2.1 · pp. 4–5

2.2 The KronLSTM architecture

• KronLSTM separates forgetting, erasing, and writing into three gates: a key-dimension forget gate, β_t, and target c_t.The update applies the erase term to the decayed memory and uses a target scaled by 1/β_t to decouple erase and write.Source: §2.1 · p. 3, §2.2 · p. 5
• With normalized keys and values and a sigmoid β_t, KronLSTM operates in the non-expansive regime and needs no separate normalizer or stabilizer state for its memory.The memory is the block’s only state, and KronLSTM can replace a Transformer attention layer.Source: §2.2 · p. 6

3.1 Language Modeling

• KronLSTM has the highest average accuracy and lowest LAMBADA perplexity among competitors; as a linear model, it leads on five of six tasks and improves Avg-6 over KDA by 1.6 accuracy points.In hybrid models it leads on four of six tasks and has the lowest LAMBADA perplexity; Table 1 reports downstream performance separately for linear and hybrid models.Source: §3.1 · pp. 6–7, Table 1 · p. 7

3.2 Retrieval Capabilities

• KronLSTM has the best average accuracy on the 2k real-world recall suite and is best on three of six tasks in both linear and hybrid settings.Hybrid models using sliding-window attention gain five to nine accuracy points; Table 2 reports task accuracies at 2k context.Source: §3.2 · p. 7, Table 2 · p. 8
• On RULER NIAH, KronLSTM is best or second-best among linear models across sequence lengths except S1, and hybrid KronLSTM has the highest average accuracy on every task and leads each task at 16k.Table 3 covers 4k, 8k, and 16k evaluations after 8k pretraining; the reported 16k results assess extrapolation.Source: §3.2 · p. 7, Table 3 · p. 9

3.3 Long Context Extension

• After 32k context extension, KronLSTM maintains its lead over GDN-2 across all five RULER tasks, although both models’ retrieval accuracy drops substantially relative to 16k.The gap widens with haystack size, consistent with the paper’s interference analysis.Source: §3.3 · p. 9, Figure 4 · p. 10

3.4 State Tracking Capabilities

• KronLSTM has state-tracking capabilities similar to Gated DeltaNet on parity and S3, while both outperform Mamba-3; Mamba-3 wins on modulo arithmetic.Source: §3.4 · p. 10, Figure 5 · p. 10

3.5 Training Efficiency

• KronLSTM reaches 36.8–44.5K tokens/s on a single H100, 11–18% more than GDN-2 at every shape and within 4–8% of KDA up to 8K×2.GDN is fastest; all models slow at 16K×1 because inter-chunk recurrence parallelizes only over sequences and heads.Source: §3.5 · p. 10, Figure 6 · p. 10

4 Conclusion

• The Kronecker Delta Rule corrects a scalar association along a single direction, and KronLSTM embeds it in a gated linear-attention mixer.The authors prove that memory dynamics converge to a unique steady state and that expected memory norm is bounded at every step.Source: §4 · p. 11
• KronLSTM has the best average accuracy on commonsense reasoning and real-world recall, and is best or second-best on every natural-text RULER task.On S1, it is the only hybrid model to retrieve needles beyond the sliding attention window; beyond 16k context, it outperforms GDN-2 on all natural-text retrieval tasks.Source: §4 · p. 11
• The authors identify scaling KronLSTM and training hybrids with full softmax-attention layers as future work.Source: §4 · p. 11

AI use statement

• The authors report using LLM assistance for theoretical derivations, implementation, kernel optimization, and manuscript drafting, with independent checks and author responsibility for the final work.Source: AI use statement · p. 11

• Table A1 is the notation reference used throughout the paper and appendix.Source: Table A1 · p. 19

A2 Related Work

• The paper frames its update as a new scalar bilinear-readout, constant-target member of the delta-rule family.It contrasts this with the established family’s vector readout.Source: A2 Related Work · p. 20

A2.1 Long Short-Term Memory

• xLSTM combines matrix associative memory and revisable storage decisions, while its normalization and stabilizer states motivate alternative ways to stabilize memory.KronLSTM retains matrix memory and revision while using a different update rule.Source: A2.1 Long Short-Term Memory · p. 20

A2.2 The delta-rule linear attention family

• The delta-rule linear-attention lineage runs from fast-weight updates to DeltaNet’s hardware-efficient WY representation and later gated or generalized variants.The Kronecker rule inherits the product structure with k_t ⊗ v_t in place of k_t.Source: A2.2 The delta-rule linear attention… · p. 20

A2.3 The online-learning perspective

• The online-learning perspective places the bilinear rule within objective-based memory updates and identifies it as a new category in the Miras taxonomy.Its objective uses a constant scalar target.Source: A2.3 The online-learning perspective · p. 20

A2.4 Classical update and learning rules

• Vectorizing the update yields Widrow–Hoff LMS on a Kronecker-product input; its stability range matches the classical LMS step-size condition.The rule also has a projection interpretation as Kaczmarz’s method.Source: A2.4 Classical update and learning rules · p. 21

A2.5 Relation to cognitive and neuroscience

• Because it scores key–value pairs rather than retrieving values, the rule is related to familiarity memory, whose reported capacity distinction matches the bilinear rule’s d^2-versus-d separation.Source: A2.5 Relation to cognitive and… · p. 21

A3.1 Setup and notation.

• The rule corrects the scalar readout v_t^T S_t k_t toward target c_t by updating memory along the outer-product direction v_t k_t^T.The target may be constant, norm-based, or produced by a gate or network.Source: A3.1 Setup and notation · p. 21

A3.2 Name Kronecker Delta Rule

• Vectorizing the memory makes the update exactly a delta rule with Kronecker-product input k_t ⊗ v_t and target c_t.Source: A3.2 Name Kronecker Delta Rule · pp. 21–22

A3.3 Properties

• In Frobenius geometry, the memory space is a Hilbert space and the update is a projection onto a hyperplane under the stated step-size condition.Source: A3.3 Properties · p. 22

The update

• For norm targets, exact retrieval memories satisfy the constraints, and the identity matrix is a fixed point in the autoassociative case.With correlation targets, the identity is a universal fixed point and the memory represents deviations from the standard inner product.Source: The update · p. 23
• A zero target enables targeted unlearning: with the Newton step, the addressed association is deleted in one update while readouts orthogonal in either key or value remain unchanged.Source: The update · p. 23
• The homogeneous recurrence has spectral norm at most 1, never amplifies, and cannot make ∥S_t∥ blow up; unlike vanilla linear attention, the Hebbian write modifies S only along rank-one direction v_t k_t⊤.Readouts with k′ ⊥ k_t or v′ ⊥ v_t are exactly unchanged by the update.Source: The update · p. 24
• Interference with a stored pair scales with the product of key and value overlaps, so orthogonality in either factor protects it; a near-zero residual causes no interference.Source: The update · p. 24
• Cycling or random sampling converges linearly when the constraints are consistent; up to d² arbitrary scalar targets can be stored for linearly independent Kronecker-product pairs.Repeated pairs with changed targets are overwritten, and the memory functions as a pair-addressed scalar table with write, read, rewrite, and delete operations.Source: The update · p. 24

A3.4.1 The admissible set

• The admissible corrections form an affine hyperplane defined by the requirement that the addressed pair’s scalar read-out equal the target.Source: A3.4.1 The admissible set · p. 26

A3.4.2 Minimality in norm

• The Newton-step correction E⋆ minimizes every unitarily invariant norm over admissible corrections; its nuclear, Frobenius, and spectral norms coincide.Source: A3.4.2 Minimality in norm · p. 26

A3.4.3 Minimality of support: the correction is surgical

• E⋆ is the unique admissible correction supported only on the addressed key and value directions; every other admissible correction writes into an unrequested direction.Source: A3.4.3 Minimality of support: the… · p. 27

A3.4.4 Minimality of interference

• Interference vanishes when a probe’s value or key is orthogonal to the addressed direction, and under isotropic probe statistics E⋆ minimizes expected interference.Source: A3.4.4 Minimality of interference · p. 27

A3.4.5 Relaxed steps: the correction never moves away from a correct memory

• With learned gate β=λβ⋆, every λ∈(0,2) moves the memory closer to every memory satisfying the addressed association, with the largest decrease at λ=1.Source: A3.4.5 Relaxed steps: the correction… · p. 28

A3.4.6 Greedy minimality is global minimality

• For consistent associations revisited infinitely often with relaxation ω∈(0,2), the iterates converge to the minimum-Frobenius-norm memory satisfying them.Source: A3.4.6 Greedy minimality is global… · p. 28

A3.4.7 Less invasive than DeltaNet

• The comparison is task-specific: the Kronecker rule corrects one scalar read-out minimally, whereas DeltaNet must satisfy d constraints and therefore has a larger correction.Source: A3.4.7 Less invasive than DeltaNet · p. 28

A3.5.1 Overview

• With n > d2 inconsistent associations, uniform sampling makes SLS minimize L, while steady-state excess loss is the classical LMS misadjustment.Exact convergence can be recovered with diminishing step sizes or iterate averaging, and persists for consistent targets even when n > d2, including correlation and norm targets when exact retrieval memory exists.Source: A3.5.1 Overview · p. 29

A3.5.2 Setting and notation

• The setup uses n > d2 associations, full-column-rank features, independent index sampling, and a fixed step size; SLS is the unique least-squares minimizer.Source: A3.5.2 Setting and notation · p. 29

A3.5.3 Theorems on expected and steady state solution

• E[St] converges linearly and exactly unbiasedly to SLS, while the iterates converge geometrically in W2 to a unique stationary distribution with mean SLS and covariance O(β).The leading-order covariance is governed by a Lyapunov-type equation driven by residuals at SLS.Source: A3.5.3 Theorems on expected and steady… · p. 30

A3.5.4 Proofs of the theorems

• A synchronous coupling reuses the same sampled index so target terms cancel, converting mean-square contraction into geometric W2 contraction of the Markov kernel.The proof also derives an exact covariance equation and trace bounds establishing covariance O(β).Source: A3.5.4 Proofs of the theorems · pp. 31, 33, 35
• Annealing β with a divergent sum or averaging iterates at constant β recovers SLS almost surely; as β → 0, the stationary distribution concentrates on SLS.Iterate averaging is exactly unbiased; larger β tracks nonstationary targets faster, while smaller β reduces steady-state loss excess.Source: A3.5.4 Proofs of the theorems · p. 35
• When targets are realizable, the memory converges almost surely and linearly to the unique solution, even for arbitrarily many associations.Correlation targets and norm targets with an exact retrieval solution are distinguished realizable structures, allowing effectively unbounded association counts.Source: A3.5.4 Proofs of the theorems · p. 35

A3.5.5 Remarks

• Beyond capacity, degradation is graceful: the memory acts as an online linear regressor with averaging and O(√β) jitter, but no blow-up.The homogeneous update factors keep the spectral norm at most 1.Source: A3.5.5 Remarks · p. 36
• For conflicting targets on the same pair, random sampling converges in mean to their weighted average, whereas sequential presentation overwrites toward the most recent target.Random replay averages; ordered replay recencies, so the sampling protocol determines overwrite behavior.Source: A3.5.5 Remarks · p. 36
• A scalar forget gate makes the steady state recency-weighted least squares, geometrically down-weighting old associations and enabling tracking of drifting target distributions.The effective association count is finite, approximately 1/(1 − f), making the gate especially relevant in the overloaded regime.Source: A3.5.5 Remarks · p. 36
• For n > d^2 associations and constant β, the memory converges to a stationary distribution with mean SLS, covariance O(β), and steady-state loss determined by optimal residuals.The stationary mean is the sampling-weighted least-squares solution; the stationary covariance solves an exact equation driven by optimal residuals.Source: A3.5.5 Remarks · p. 36

A3.6.1 Overview

• For overloaded associations, the randomized rule converges around the weighted least-squares memory, whose Frobenius norm admits a two-sided bound from an exact energy identity.The bounds coincide for isotropic designs.Source: A3.6.1 Overview · p. 36

A3.6.2 Setting and nomenclature

• The setup uses sampled associations with bilinear key–value features; the main bounds assume those features span the feature space, while rank-deficient cases use a subspace formulation.The rank-deficient treatment uses the minimum-Frobenius-norm least-squares solution and the smallest nonzero eigenvalue.Source: A3.6.2 Setting and nomenclature · p. 37, A3.6.5 Tightness and special cases · p. 38

A3.6.3 An exact energy identity

• The least-squares memory’s stored read-out energy equals target energy minus residual energy, so it stores only the signal explained by the bilinear features.The normal equations establish the exact identity underlying this interpretation.Source: A3.6.3 An exact energy identity · p. 37

A3.6.5 Tightness and special cases

• Isotropic designs attain equality, and orthonormal features recover the target-weighted Hebbian superposition norm; residual energy shrinks the least-squares memory.With unit-norm keys and values, the eigenvalue conditioning factor can be d^2; ill-spread designs pay through λmin.Source: A3.6.5 Tightness and special cases · p. 38

A3.6.6 Bound on the memory norm for steady-state and all updates

• The steady-state memory’s expected squared norm separates least-squares content from an O(β) covariance fluctuation, which vanishes as β approaches zero or in the consistent case.The stationary mean is exactly the least-squares memory.Source: A3.6.6 Bound on the memory norm for… · p. 38

The fluctuation term admits the exact decomposition

• In the non-expansive regime, residual energy lowers the bound; its effects on memory content and steady-state jitter cancel at κ = 1, while content dominates below it.The online bound holds at every update, not only at steady state, and the transient relaxes geometrically.Source: The fluctuation term admits the exact… · pp. 39–40
• The expected memory norm is bounded by explained target energy and feature-covariance eigenvalues; online iterates add an O(β) fluctuation and remain bounded independently of sequence length.From an empty memory in the non-expansive regime, the squared norm stays within four times its steady-state bound from the first step and relaxes geometrically.Source: The fluctuation term admits the exact… · p. 40

A3.6.4 The Norm Bound

• The memory norm is sandwiched by explained target energy divided by the extreme eigenvalues of the feature second moment.A first-moment variant also bounds it using the expected absolute target-feature product; the accumulated write mass is the corresponding quantity.Source: A3.6.4 The Norm Bound · p. 37, The fluctuation term admits the exact… · p. 40

A4.2 DeltaNet Affine Recursion

• DeltaNet’s memory rows follow independent scalar delta-rule recursions, so the KronLSTM Markov-chain analysis transfers to its memory dynamics.Each row uses the same key and a different value-coordinate target; neither architecture generalizes the other.Source: A4.2 DeltaNet Affine Recursion · p. 41

A4.3 Steady-State Norm Bound

• Under the stated step-size condition, DeltaNet has an exact two-sided steady-state norm bound that transfers to the running memory.The analysis assumes an i.i.d. input stream; input-dependent step sizes replace the key covariance with a β-weighted second moment.Source: A4.3 Steady-State Norm Bound · pp. 41–42

A4.4 Relation to the KronLSTM Bound

• The length-independent steady-state bound follows from the delta-rule mechanism shared by KronLSTM and DeltaNet, unlike forget-gate-based stability arguments.The passage contrasts this mechanism with mLSTM’s uncorrected Hebbian write and a forget-gate bound that grows with context length as retention approaches one.Source: A4.1 Overview · p. 41, A4.4 Relation to the KronLSTM Bound · p. 42
• The two bounds concern the same memory norm but different write loads and conditioning matrices, so their constants are not structurally comparable.KronLSTM writes a scalar target and requires the Kronecker features to span the matrix-memory space; DeltaNet writes a value vector and requires only that keys span their space.Source: A4.4 Relation to the KronLSTM Bound · p. 42
• KronLSTM’s step-size condition tightens with value norm, whereas DeltaNet’s condition depends only on key norm.A large value norm can reduce KronLSTM’s admissible step size and contraction rate.Source: A4.4 Relation to the KronLSTM Bound · p. 43
• KronLSTM revises memory along an input-pair direction, with interference gated by key and value overlap rather than key overlap alone.Its write footprint on unrelated memories does not compound with context length.Source: A4.4 Relation to the KronLSTM Bound · p. 43

A5.1 KronLSTM Setting

• KronLSTM’s vectorized memory update is an iteration toward the least-squares solution for associations with features formed from key–value pairs.With full-column-rank features and the stated step-size conditions, the expected iterate converges linearly and without bias to that solution.Source: A5.1 KronLSTM Setting · p. 43

A5.2 DeltaNet Setting

• DeltaNet decomposes into independent key-driven scalar delta-rule recursions, one for each value coordinate.This is structurally related to KronLSTM’s single recursion on a richer key–value direction, but neither is a generalization of the other.Source: A5.2 DeltaNet Setting · p. 44

A5.3 KronLSTM Retention of the Initial Memory

• KronLSTM’s mean memory separates the surviving initial-memory contribution from the acquired least-squares memory.Retention is measured by the readout of the initial-memory term along its stored direction, as a fraction of a unit initial memory still legible after t steps.Source: A5.3 KronLSTM Retention of the Initial… · p. 44

A5.4 DeltaNet Retention of the Initial Memory

• DeltaNet’s mean memory likewise separates its initial-memory term from its least-squares memory under the corresponding key-covariance recursion.The retention result follows by substituting the key covariance and key dimension for KronLSTM’s feature covariance and state dimension.Source: A5.4 DeltaNet Retention of the Initial… · p. 45

A5.5 KronLSTM Retains Longer than DeltaNet

• With independent keys and values, KronLSTM’s slowest retention mode lasts at least d times longer than DeltaNet’s, with equality for isotropic values.This longer retention is a trade-off: at fixed β, KronLSTM also approaches its acquired least-squares memory d times more slowly.Source: A5.5 KronLSTM Retains Longer than… · pp. 45–46
• For isotropic values, KronLSTM’s entire retention spectrum shifts toward slower decay, by a factor of d, rather than trading faster decay in some modes for slower decay in others.The analysis also reports a d-fold increase in the average retention horizon for isotropic values.Source: A5.5 KronLSTM Retains Longer than… · p. 45

A6.1 400M language models

• KronLSTM has the best average at 400M/40B and leads four listed tasks, while different baselines lead or tie on the others.Against mLSTM, it improves five of six tasks, raises Avg-6 by 2.1 points, and lowers LAMBADA perplexity from 43.40 to 28.28; mLSTM is marginally ahead on PIQA.Source: A6.1 400M language models · p. 46, Table A2 · p. 46

A6.2 Ablation on different formulations of KronLSTM

• Fixing the write target to one causes the largest reported ablation degradation, while write-rule variants have smaller language-modeling effects than retrieval effects.At 1.5B/100B, the fixed-target variant loses 1.6 accuracy points and 0.8 perplexity points; NIAH rankings vary by context length.Source: A6.2 Ablation on different… · p. 46, Table A3 · p. 47

A6.3 Long context extension results

• Table A4 covers RULER NIAH performance for 32k-context-extended linear models evaluated on identical documents, but the supplied caption reports no comparative outcomes.It identifies the better of the two models per task and length in bold.Source: Table A4 · p. 47

A7 Pretraining Details

• Pretraining comparisons hold corpus, tokenizer, optimizer, schedule, batch size, and seed constant while changing the sequence mixer blocks.Models use the FineWeb-Edu 100BT sample, Llama 2 tokenization, and packed 8,192-token sequences.Source: A7 Pretraining Details · p. 47

A7.1 Long-context extension

• Long-context extensions branch from 1.5B pretraining runs at 75B tokens, then train for 25B tokens at sequence length 32,768 to match 100B total tokens.Architecture and tokenizer remain unchanged, and the global batch remains one million tokens per step.Source: A7.1 Long-context extension · p. 48
• The long-context extension reinitializes optimizer state and restarts the schedule with warmup followed by cosine decay.Its data blends four ProLong-64K long-document domains with FineWeb-Edu.Source: A7.1 Long-context extension · p. 48

A7.2 State Tracking Experiments

• The state-tracking experiments use three synthetic tasks with different target operations and chance accuracies.Parity has 50% chance accuracy, S3 permutation 16.7%, and modular arithmetic 20%; evaluation uses fresh sequences at lengths 128–1024.Source: A7.2 State Tracking Experiments · p. 48
• KronLSTM has similar state-tracking accuracy to Gated DeltaNet on parity and S3, while Mamba-3 leads on modular arithmetic.Models are evaluated up to sequence length 1024 after training up to length 128; reported accuracy is averaged over three seeds.Source: A7.2 State Tracking Experiments · p. 48

A8 Chunkwise-Parallel Training of KronLSTM

• KronLSTM revises memory with a rank-one term whose coefficient depends on memory through a scalar, enabling chunkwise-parallel training with memory propagated between chunks.Within each chunk, token interactions use dense matrix products and a small triangular solve.Source: A8 Chunkwise-Parallel Training of… · p. 49, A8.4 Chunkwise form · p. 51

A8.1 Setting and notation; A8.2 Decay-normalized recurrence

• Decay-normalized recurrence scores with the decayed key and writes along the un-decayed key, absorbing the forget gate into an asymmetric key pair.The decay ratio represents how much a key written earlier has decayed by the current token.Source: A8.1 Setting and notation · p. 49, A8.2 Decay-normalized recurrence · p. 50

A8.3 The write coefficients solve a unit lower triangular system

• The write coefficients solve a unit lower-triangular system built from the Kronecker-feature Gram, computed via key and value Grams without forming the d_kd_v-dimensional features.The write gate enters the right-hand side, while the erase gate enters the system matrix and its coupling to incoming memory.Source: A8.3 The write coefficients solve a… · p. 50

A8.4 Chunkwise form

• Chunkwise, the incoming memory couples through C scalar scores, and each pseudo-value is the token value scaled by its scalar write coefficient.The chunk dependence S[n] → p → u → S[n+1] is the only sequential part; other listed quantities are computed locally or in parallel across chunks.Source: A8.4 Chunkwise form · p. 51

A8.5 WY representation of the chunk transition

• The chunk transition has a WY representation whose correction to diagonal decay is generally not of delta-rule form because its columns combine Kronecker products.The incoming memory enters through scalar scores rather than the delta rule’s value-dimensional read-outs.Source: A8.5 WY representation of the chunk… · p. 51

A8.6 Variants

• The derivation permits coupled erase and write, unit targets, β values in (0, 2), scalar forget gates, no forget gate, and unnormalized values.For unnormalized values, the kernels substitute β_r ← β_r / ||v_r||².Source: A8.6 Variants · p. 52

A8.7 Algorithm, cost and implementation

• The algorithm’s sequence cost is O(LC(d_k + d_v) + Ld_kd_v) multiply-adds and O(LC²) for solves, the same order as Gated DeltaNet and KDA.Relative to KDA, the additional cost is the value Gram; memory is materialized once per chunk, not per token.Source: A8.7 Algorithm, cost and implementation · p. 52

A8.8 Backward pass; A8.9 Verification

• The backward pass adds value-gradient contributions through the system inverse; numerical verification found outputs and final memory agree to ≤10^-15 relative and gradients to 10^-9.Verification covered varied chunk and sequence lengths, initial memories, decay strengths, and β in (0, 1) and (0, 2).Source: A8.8 Backward pass · p. 53, A8.9 Verification · p. 53