PaperlayerView exact revision on arXiv

Source-linked AI summary

LoRA-RC: Reservoir Computing with Low-Rank Adaptation

Wenbin Wan

arXiv:2609.12327v1cs.AIcs.LGcs.SYeess.SYmath.DS

TL;DR

A fixed reservoir can become inadequate under system drift, while unconstrained recurrent adaptation risks stability. LoRA-RC adapts recurrent memory through a projected and filtered low-rank core with offline-fixed bases. On a drifting Lorenz system, it reduced post-drift RMSE by 56% versus fixed RC and 51% versus adaptive RC.

  • Problem

    Fixed reservoirs may become unreliable under drift, while online recurrent adaptation needs stability guarantees because unconstrained updates can undermine them.

    Source: §I · pp. 1–2

  • Method

    LoRA-RC keeps the base reservoir and adaptation bases fixed offline and updates a projected, filtered low-rank core online.

    Source: §I · p. 2, §III · p. 3

  • Results

    Post-drift RMSE fell 56% versus fixed RC and 51% versus adaptive RC on the chaotic Lorenz system.

    Source: §V · p. 6

  • Takeaways & Limitations

    A per-step contraction invariant supports a uniform δISS bound along online adaptation paths with path-independent rate and gain.

    Source: §I · p. 2, §V · p. 6

    Evaluation is limited to comparative simulations on the chaotic Lorenz system; higher-dimensional and real-world evaluations remain future work.

    Source: §V · p. 6

Abstract

from arXiv · show

Reservoir computing (RC) trains only a linear readout over a fixed recurrent layer, making it fast and data-efficient for online prediction. However, a static reservoir degrades under system drift, readout-only adaptation is then insufficient, and unconstrained reservoir adaptation can destroy the echo-state and incremental stability properties that make RC reliable. This paper proposes LoRA-RC, which adapts the recurrent matrix through a low-rank correction driven by streaming prediction errors. The base reservoir and adaptation bases are fixed offline; a small core matrix is adapted online, projected onto a spectral-norm ball, and low-pass filtered at each step. The projection guarantees that every applied recurrent matrix remains within a certified contraction set, and an incremental input-to-state stability bound is established for the reservoir along each online adaptation path, with path-independent rate and gain. On a Lorenz system with an abrupt parameter drift, LoRA-RC cuts post-drift prediction error by 56% versus a fixed RC and 51% versus readout-only adaptation; ablations over 20 seeds show that removing the projection inflates this error by more than a factor of 40.

I. INTRODUCTION

Reservoir computing is fast and data-efficient, but a fixed reservoir can become too rigid under system drift, while unconstrained recurrent adaptation risks stability. LoRA-RC adapts recurrent memory through a constrained low-rank update and establishes stability guarantees along online adaptation paths.

I. INTRODUCTION

  • A fixed reservoir may not represent a drifting system, readout-only adaptation cannot change its memory map, and unconstrained recurrent updates can undermine stability.The paper motivates a stability-aware update law for online reservoir adaptation.Source: §I · pp. 1–2
  • Prior certificates mainly cover fixed, offline-trained, or offline-constrained recurrent models, not online recurrent-memory updates that preserve stability after every update.The paper positions LoRA-RC as addressing this gap.Source: §I · p. 2
  • LoRA-RC decomposes the recurrent matrix into a certified base reservoir and a low-rank correction with offline-fixed bases and a small online core.This separates adaptation directions chosen offline from the variable updated during deployment.Source: §I · p. 2
  • Projection keeps each applied recurrent matrix inside a contraction set, while first-order filtering controls its step-to-step variation.The core is updated from streaming prediction errors.Source: §I · p. 2
  • The time-varying reservoir is proved uniformly δISS along each online adaptation path, with path-independent contraction rate and input gain.The contraction rate also carries through to the prediction layer.Source: §I · p. 2

II. PRELIMINARIES

The preliminaries describe the standard reservoir and readout setup, then formulate one-step-ahead prediction under drift with stability preserved during online adaptation.

II. PRELIMINARIES

  • A standard reservoir computing model uses a leaky recurrent reservoir with fixed input weights and a componentwise nonlinear activation.The reservoir state supplies the temporal representation for prediction.Source: §II · p. 2
  • Standard RC fixes the recurrent matrix and trains only the readout, using ridge regression for offline readout fitting.The readout maps reservoir states to outputs.Source: §II · p. 3
  • The prediction task is one-step-ahead forecasting from measured signals, with the predictor allowed to update online as nominal dynamics drift.The desired predictor should improve prediction under drift while preserving internal stability during adaptation.Source: §II · p. 3

III. LOW-RANK ADAPTIVE RESERVOIR COMPUTING (LORA-RC) ARCHITECTURE

LoRA-RC augments a certified base reservoir with an online low-rank correction, restricting the core so contraction is preserved at every step. Offline-fixed components and projected, filtered online updates maintain this constraint.

III. LOW-RANK ADAPTIVE RESERVOIR COMPUTING (LORA-RC) ARCHITECTURE

  • The recurrent matrix is W_k = W_0 + U M_k V^T, with a certified base, fixed orthonormal bases, and a small online core.The admissible core set bounds its norm so the reservoir contraction property is preserved at every step.Source: §III · p. 3
  • The base reservoir, adaptation bases, and admissible set are fixed offline; online, only the small core and readout are updated.The tentative fast core is projected onto the admissible set, then low-pass filtered to produce the applied core.Source: §III · p. 3
  • The offline design enforces a stable base, admissible zero-initialized cores, bounded input weights, and orthonormal adaptation bases.These structural assumptions are enforced through spectral normalization, initialization, and basis selection.Source: §III · p. 3

A. Offline Initialization

Offline initialization selects the base reservoir and adaptation subspace, optionally using representative drift regimes to inform the directions. The resulting bases stay fixed during deployment while the online core and readout are updated.

A. Offline Initialization

  • The offline design starts from a sparse random recurrent matrix and random input matrix, and scales the base reservoir to the chosen stability bound.The nominal training data are used in the offline stage.Source: A. Offline Initialization · p. 3
  • The low-rank correction acts only within the subspace spanned by U and V, which determine the input-side and output-side directions of the update.This confines the online correction to selected recurrent directions.Source: A. Offline Initialization · p. 3
  • Without prior drift information, the adaptation bases can be random orthonormal matrices; with representative regimes, stable constrained corrections can inform the basis choice.For the data-informed option, the bases are selected from leading singular vectors of an average correction across regimes.Source: A. Offline Initialization · pp. 3–4
  • The fast and applied cores start at zero, and the adaptation bases remain fixed during deployment while the readout and cores are updated.The online prediction residual is used in the update procedure; filtering maps the fast core to the applied core.Source: A. Offline Initialization · p. 4

B. Online Prediction and Update

Each online step predicts with the current applied core, forms a residual from the observed output, and updates the readout and recurrent core. The core update is projected into an admissible set and filtered before application.

B. Online Prediction and Update

  • Each online step predicts with the current applied core, observes the true output to form a residual, then updates the readout and small core matrix.Source: B. Online Prediction and Update · p. 4
  • The readout uses a normalized stochastic-gradient step projected onto a spectral-norm ball, with ε preventing division by a vanishing reservoir-state norm.Source: B. Online Prediction and Update · p. 4
  • The core adaptation signal propagates the one-step residual loss through the readout and activation, projects it onto the adaptation subspace, and adds leakage to discourage core drift.The update is a projected-gradient step on an instantaneous surrogate loss; it does not differentiate through multiple future steps of filter dynamics.Source: B. Online Prediction and Update · p. 4
  • The fast core is projected onto the admissible spectral-norm ball, and a low-pass filter generates the applied core; smaller β slows adaptation and reduces step-to-step variation.Projection uses singular-value clipping to cap singular values at the ball radius.Source: B. Online Prediction and Update · p. 4

C. Stability Analysis

The stability analysis extends certification from fixed recurrent weights to the full sequence of online updates. Under the stated assumptions, the reservoir has a path-independent incremental input-to-state stability rate and gain.

C. Stability Analysis

  • Projection and filtering keep every applied recurrent matrix within a norm-bounded family, making the certificate invariant across online updates rather than tied to one fixed weight.The analysis gives an explicit rate and input gain through a direct trajectory argument.Source: C. Stability Analysis · p. 4
  • Theorem 1 establishes incremental input-to-state stability for any admissible applied-core sequence, with rate a = (1 − α) + ακ < 1 and gain α∥W_in∥/(1 − a).Both are uniform over admissible core sequences.Source: C. Stability Analysis · p. 5
  • The recurrent matrix changes by at most 2βρ_M per step under the stated update assumptions.Source: C. Stability Analysis · p. 5

IV. EXPERIMENTS

On the Lorenz-63 drift experiment, LoRA-RC outperforms fixed and readout-adaptive RC after ρ shifts. Ablations show the projection is central to post-drift accuracy.

IV. EXPERIMENTS

  • The experiment compares frozen fixed RC, readout-adaptive RC, and jointly adapting LoRA-RC after a ρ shift from 28 to 40 at step 800.All methods share a 200-neuron base reservoir; the LoRA-RC bases were computed from regimes ρ ∈ {33, 40}.Source: §IV · p. 5
  • Rebuilding the bases from ρ ∈ {33, 36} while leaving the deployment regime unseen yields post-drift RMSE 0.620 ± 0.138; wider-regime generalization is not demonstrated.Source: §IV · p. 5
  • Post-drift RMSE fell 56% versus fixed RC and 51% versus adaptive RC; LoRA-RC also had the lowest pre-drift error.The three methods tracked similarly before the drift, while fixed and adaptive RC detached from the ground truth afterward.Source: Figure 2 · p. 5, §IV · p. 6
  • Removing the projection increased post-drift RMSE more than 40× with or without the filter; removing only the filter left it 5.6× higher.The filter aids accuracy rather than the certificate.Source: §IV · p. 6
  • Across ranks 1, 2, 5, 10, and 20, post-drift RMSE fell from 1.11 ± 0.18 to 0.41 ± 0.11, at up to 16× the default online parameters.Rank 5 is described as a favorable efficiency-accuracy point, not a proven optimum.Source: §IV · p. 6
  • The projection radius and core learning rate stayed in narrow RMSE bands across 7× and 16× ranges, while β was most sensitive, increasing RMSE more than 5× as β → 1.Large leakage degrades performance through over-regularization; defaults lie within, though not always at, the best-performing region.Source: §IV · p. 6

V. CONCLUSIONS

LoRA-RC keeps each online recurrent update within a contraction bound and establishes a uniform δISS bound along update sequences. Lorenz simulations show lower post-drift RMSE than both baselines, while broader evaluations remain future work.

V. CONCLUSIONS

  • On the chaotic Lorenz system, LoRA-RC reduced post-drift RMSE 56% relative to fixed RC and 51% relative to adaptive RC using only r^2 = 25 online recurrent parameters.Source: §V · p. 6
  • The recurrent-matrix contraction bound is a per-step invariant under every online update.Source: §V · p. 6
  • A uniform δISS bound holds along every update sequence with explicit contraction rate a = (1 − α) + ακ.Source: §V · p. 6
  • Future work will investigate Lyapunov-based conditions to enlarge the admissible set and extend evaluation to higher-dimensional and real-world systems with closed-loop control tasks.Source: §V · p. 6
Loading 2609.12327v1…