PaperlayerView exact revision on arXiv

Source-linked AI summary

State-Space Unlearning for Non-Stationary Bias in Land Surface Forecasting

Anidipta Pal

arXiv:2610.02248v1cs.LG

TL;DR

Non-stationary events can leave persistent bias in land-surface forecasts from Mamba-family models, and SSU-LSF introduces a purpose-built unlearning approach for these models. It localizes a temporal confounding footprint and applies constrained unlearning with spatial regularization. The paper reports strong benchmark results and identifies footprint localization and spatial TV as decisive for different outcomes.

  • Problem

    Non-stationary events can be absorbed into land-surface forecasting state matrices and continue to bias predictions after the physical cause ends.

    Source: §1 · p. 1

  • Method

    SSU-LSF localizes a temporal confounding footprint and applies constrained unlearning with spatial TV regularization to Mamba-family state-space models.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    On NDVI-LST, SSU-LSF is best on all three reported metrics, with CRR = 0.821; the conclusion identifies TV as decisive for spatial coherence and footprint localization as decisive for CRR.

    Source: §3 · p. 3, §4 · p. 4

  • Takeaways & Limitations

    The reported ablations distinguish the roles of spatial TV for coherence and footprint localization for confounding reduction.

    Source: §4 · p. 4

    Primary experiments use injected confounding with oracle window knowledge, CRR requires affected locations, and S4 generalization is shown on one benchmark only.

    Source: §4 · p. 4

Abstract

from arXiv · show

Operational land surface forecasting systems built on Mamba-family Structured State Space Models absorb non-stationary confounding events (unrecorded irrigation booms, dam-operation shifts, sensor recalibrations) into their state-transition matrices, silently biasing NDVI, LST, and crop phenology predictions long after the physical cause ends. This paper introduces SSU-LSF (State-Space Unlearning for Land Surface Forecasting), the first machine-unlearning framework purpose-built for geoscientific Mamba-based SSMs. We develop EKFac influence functions specialized to the Mamba state matrices via a closed-form matrix-exponential gradient, use spectral-radius-weighted elbow thresholding to localize a temporal confounding footprint $\Phi$, and apply Hessian-free projected gradient ascent within a KL-divergence trust region augmented by spatial total-variation (TV) regularization. Proposition 1 establishes that residual confounding is bounded by $\mathcal{O}\big((1-\rho(\bar{A})^{T_c})/((1-\rho(\bar{A}))\mu)\big)$, which grows with the window length $T_c$. Across three heterogeneous benchmarks and eleven baselines, SSU-LSF achieves confounding reduction rates of $0.773$ (CropHarvest), $0.821$ (NDVI-LST), and $0.859$ (ERA5), with worst-case clean-domain RMSE degradation of $4.2\%$ on ERA5, converging in 3--5 epochs at $8.4\times$ lower GPU-cost per unlearning request than full retraining. Code: https://github.com/Anidipta/SSU-LSF

1 Introduction

Long-horizon land-surface forecasts from Mamba and S4 models can retain biases from non-stationary events after their physical causes end. SSU-LSF is introduced to address this post-hoc unlearning problem, alongside theoretical and empirical contributions.

1 Introduction

  • Non-stationary events can contaminate training windows and state matrices, which continue propagating bias after the physical cause ends.The affected forecasts include NDVI, LST, and crop phenology signals.Source: §1 · p. 1
  • Existing approaches face practical limits: full retraining is costly, some methods damage clean-domain RMSE, and others require pre-partitioned data or do not exploit SSM recurrence.The paper also identifies spatial-coherence losses in recent approximate-unlearning methods without TV regularization.Source: §1 · p. 1
  • SSU-LSF is introduced with a Proposition 1 Lipschitz bound for the full encoder and an independently estimated RKHS separation.The introduction also reports a matching lower-bound analysis of the bound's tightness gap.Source: §1 · pp. 1–2
  • The paper derives a closed-form gradient for diagonal state matrices and a Padé-based extension for non-diagonal matrices, and gives convergence guarantees under mild regularity conditions.It also reports a documented SEVIRI recalibration case, window-length sensitivity, interaction ablations, LiRA-style MIA, and TV-regularized baseline variants.Source: §1 · p. 2

2 Method

SSU-LSF models land-surface observations with a selective Mamba recurrence and targets confounding in the training window. It localizes a temporal footprint and applies constrained, spatially regularized unlearning to the confounded parameters.

2 Method

  • The encoder uses a selective recurrence for multivariable land-surface observations, with state updates driven by input-dependent projections.The observation channels include NDVI, LST, ET, precipitation, and air temperature.Source: §2 · p. 2
  • Unlearning combines clean-data loss, ascent on the localized confounding loss, and spatial TV under a KL-divergence constraint around the confounded parameters.Spatial TV is applied over the patch-adjacency graph.Source: §2 · p. 2
  • Proposition 1 predicts smaller residual confounding for shorter windows, but the bound is quantitatively loose: its unlearning-cost term is about 10.2 K on ERA5 at the observed values.The paper reports an approximately 200× gap between the bound and observation and an empirical Tc trend with R2=0.995.Source: §2 · p. 2
  • EKFac approximates layer curvature to reduce inversion cost, while the state-matrix gradient uses a closed-form sum involving the matrix exponential.For the standard diagonal S4/Mamba parameterization, the Hadamard product reduces to a pointwise product.Source: §2 · p. 2
  • The temporal footprint Φ contains steps whose influence scores meet an elbow threshold; at ρ=0.84, the stated extension is about 2.6 steps beyond each side of a 24-month window.The stated extension follows a geometric tail of influence beyond the confounding window.Source: §2 · p. 2
  • Updates use Hessian-free projected gradient ascent and are scaled back by binary line search when the KL constraint is exceeded.The stated complexity is O(|Wc| · d2/B), compared with O(T · d2) for full retraining.Source: §2 · pp. 2–3

3 Experiments

The experiments compare SSU-LSF with eleven baselines on three benchmarks using benchmark-specific metrics. Results include strong NDVI-LST performance, near-chance ERA5 LiRA-MIA, and ablations of footprint localization and KL control.

3 Experiments

  • The evaluation uses a Mamba encoder-decoder, eleven baselines, and three benchmarks with distinct metric triplets; all primary benchmark confounding is author-injected.The benchmarks are NDVI-LST Fusion, ERA5 Patchified, and CropHarvest; all baselines receive the same oracle window information.Source: §3 · p. 3
  • On NDVI-LST, SSU-LSF achieves RMSE = 0.0412, CRR = 0.821, and TCI = 0.847, best on all three metrics.ForgetFS and MULL have higher SCS values than SSU-LSF, while adding TV to ForgetFS closes 80% of the SCS gap.Source: §3 · p. 3, Table 1 · p. 4
  • On ERA5, SSU-LSF reaches LiRA-MIA = 0.53 against chance = 0.50, while ACC = 0.881 is reported as confirming preserved spatial anomaly structure.On CropHarvest, SCS correlates with FSS at r= −0.91.Source: §3 · p. 4
  • Removing both Φ and KL gives CRR 0.768 at ΔRMSE 9.4%; removing Φ alone gives CRR 0.694, while removing KL yields 21.7% ΔRMSE.Replacing EKFac with diagonal Fisher gives CRR = 0.779, between GCS at 0.751 and EKFac at 0.821.Source: §3 · p. 4

4 Conclusion

The conclusion identifies spatial TV and influence-footprint localization as key to spatial coherence and confounding reduction, while noting limits in evaluation scope and bound tightness. Related-work sections position the approach among forecasting, unlearning, and non-stationarity methods.

4 Conclusion

  • SSU-LSF's conclusion identifies spatial TV as decisive for spatial coherence and EKFac-informed footprint localization as decisive for CRR.The Tc prediction is reported as empirically validated across Tc values from 6 to 36 months, despite the bound's approximately 200× looseness.Source: §4 · p. 4
  • Primary experiments use injected confounding and oracle window knowledge; CRR requires affected locations, and S4 generalization is shown on one benchmark only.The authors identify real multi-event archives and automated change-point detection as future work.Source: §4 · p. 4

A.1 SSM-based geoscientific forecasting

  • The related-work discussion notes that VMamba and SiMBA extend Mamba, while higher state-matrix spectral radii are described as more susceptible to confounding.The cited forecasting baselines do not address post-hoc bias removal.Source: §A.1 · p. 7

A.2 Machine unlearning and spatial TV prior art

  • The paper distinguishes SSU-LSF from certified removal, rewind-based unlearning, SISA, ForgetFS, and MULL by their stated constraints or lack of temporal indexing.Spatial TV is presented as a fused-lasso extension to patch adjacency graphs, not as a primary scientific contribution.Source: §A.2 · p. 7

A.3 Non-stationarity

  • Change-point detection and concept-drift methods identify breaks but do not correct parameters; domain adaptation requires a clean target dataset unavailable retroactively.SSU-LSF instead requires the event interval and original training set.Source: §A.3 · p. 7

B Model Architecture

The MambaForecaster specifications include a probabilistic forecast head, while the EKFac derivation differentiates the loss through the state recurrence with respect to the transition matrix. The derivation’s finite index range follows the unrolled recurrence, with a Fréchet derivative used for non-diagonal matrices.

B Model Architecture

  • The forecast head outputs [B, 6, 5, 2], split into µ and log σ, with σ=exp(clamp(log σ, −6, +2)).The NLL loss expression is also given, though its text is incomplete.Source: B Model Architecture · p. 7

C EKFac Influence Derivation and Index Justification

  • For diagonal A, differentiating the loss through h_t yields a matrix-exponential gradient expanded with a Hadamard product.The standard Mamba parameterization initializes A as diagonal.Source: C EKFac Influence Derivation and Index… · p. 7
  • The gradient’s inner sum runs from k=0 to T−t−1 because its terms correspond to distinct factors in the recurrence unrolled from step 1 to t.For non-diagonal A, the Hadamard product is replaced by the matrix-exponential Fréchet derivative, computed via Padé approximation; the stated truncation error is below 10^-6 when T ≥172.Source: C EKFac Influence Derivation and Index… · p. 7
  • Across 10 ERA5 seeds, the estimated EKFac condition number has mean 142 and standard deviation 31; eigenvalues are damped by 10^-4 before inversion.The estimate is computed over 50 mini-batches.Source: C EKFac Influence Derivation and Index… · p. 8

D Proposition 1: Full Proof, Lipschitz Bounds, and Tightness Analysis

The appendix evaluates Proposition 1’s bound and convergence claims, details the unlearning procedure and benchmark setups, and validates privacy and spatial-skill measures against established comparisons.

D Proposition 1: Full Proof, Lipschitz Bounds, and Tightness Analysis

  • The ERA5 unlearning-cost bound is about 200× the observed cost, while Proposition 1’s residual bound depends on reference-model RKHS separation and encoder Lipschitzness.Across five reference-model seeds, the first term changes by at most about 8%; as estimated separation approaches zero, the bound diverges and CRR falls to 0.581 ± 0.029 at µ=0.05.Source: D Proposition 1: Full Proof, Lipschitz… · p. 8

G.2 NDVI-LST Fusion

E Convergence Guarantee

  • Projected ascent increases confounding loss monotonically to a local maximum under the stated assumptions, but global stationarity is not claimed.With nonsmooth TV, the sign subgradient does not guarantee monotonicity; a graph fused-lasso proximal step restores a proximal-gradient statement.Source: E Convergence Guarantee · p. 9

F Full Algorithm

  • Algorithm 1 estimates EKFac, forms a temporal footprint using an elbow threshold, then applies projected updates subject to the KL constraint.The procedure re-estimates EKFac per request; the KL projection halves the update until the constraint is met.Source: F Full Algorithm · p. 9

G.1 Confounding injection and oracle window assumption

  • All three primary experiments use oracle knowledge of the confounding window, shared equally across the eleven baselines; deployment with an estimated window is addressed separately.The benchmarks include NDVI-LST, ERA5, and CropHarvest, with distinct injected events and durations.Source: §G.1 · p. 10, §G.2 · p. 10, §G.3 · p. 10

G.5 Metric definitions

  • The benchmarks use different metric triplets: NDVI-LST uses RMSE, CRR, and TCI; ERA5 uses SKT-R, LiRA-MIA, and ACC; CropHarvest uses Sp-R, SCS, and LC-F1.SCS is defined as normalized spatial total variation, and TCI below zero indicates more variance introduced than present in the signal.Source: §G.5 · p. 10

H MIA: LiRA Protocol

  • LiRA-MIA is 0.53 for SSU-LSF versus 0.84 for CM; affected-window output-space KL to the oracle is 0.031 for SSU-LSF, lower than the listed alternatives.The other KL values are 0.212 (CM), 0.074 (GR), and 0.043 (MULL).Source: H MIA: LiRA Protocol · p. 10

I SCS Validation vs. Fraction Skill Score

J TV-Regularized Baseline Ablation

Adding spatial TV to ForgetFS and MULL closes most of their SCS gap, while SSU-LSF retains a CRR advantage on NDVI-LST. Window-length sensitivity follows the predicted direction, and ±3-month estimation error causes a moderate CRR loss.

J TV-Regularized Baseline Ablation

  • On NDVI-LST, adding TV closes 80% of ForgetFS’s SCS gap and 88% of MULL’s, with essentially unchanged CRR; SSU-LSF retains a CRR advantage over both +TV variants.On CropHarvest, CRR is on par, while the footprint yields lower clean-domain damage: ∆RMSE 2.1% versus 3.1–3.9%.Source: Table 7 · p. 11, J TV-Regularized Baseline Ablation · p. 11

K Tc Sensitivity and Proposition 1 Validation

  • With ±3-month window-estimation error, CRR degrades by about 0.018 on average, a meaningful but not catastrophic loss.The authors describe SSU-LSF as moderately robust at typical change-point detection precision of ±2–4 months.Source: K Tc Sensitivity and Proposition 1… · p. 11

L Compute Breakdown

Repeated requests retain substantial confounding reduction, while utility erodes gradually; the spectral-radius bound is quantitatively loose but predicts shorter-window residuals. On a documented SEVIRI recalibration event, the automated pipeline achieves CRR 0.793 on affected patches.

L Compute Breakdown

  • Across three successive NDVI-LST requests, CRR is 0.821/0.814/0.806 and cumulative ∆RMSE is 3.5/4.3/5.0%; the authors recommend retraining after about five requests or once ∆RMSE exceeds 5%.Warm-starting EKFac cuts cost from 1.8 to 1.3 GPU-h at a CRR loss of 0.004; the ERA5 per-request speedup over OR is 8.4×.Source: L Compute Breakdown · pp. 11–12, Table 9 · p. 12

M Spectral Radius Distribution

  • At ρmax=0.91, Proposition 1’s unlearning-cost bound is ≈10.2 K, about 200× the observed cost, but shorter Tc predicts smaller residuals.The Tc trend is supported by Table 8 with R2=0.995; the bound is described as qualitatively predictive but quantitatively vacuous at this spectral radius.Source: M Spectral Radius Distribution · p. 12, Table 10 · p. 12, Table 8 · p. 12

N Real Documented Confounding Event

  • For the documented SEVIRI recalibration event, SSU-LSF achieves CRR 0.793 ± 0.021 on affected patches and ACC 0.871 ± 0.012 on the unaffected Asian monsoon control region.Automated estimation places the end one month early and lowers CRR by 0.012 relative to the oracle-window result of 0.805.Source: N Real Documented Confounding Event · p. 13

O Window Estimation Sensitivity and Deployment Limitations

Window and affected-location estimation make the pipeline automatic, but false positives cost clean-domain RMSE and CRR evaluation still depends on knowing affected locations. The authors recommend spatial CRR heatmaps over change-point-detected patches as a more deployable diagnostic.

O Window Estimation Sensitivity and Deployment Limitations

  • CRR requires knowing affected locations, which is unavailable in retrospective deployment; scoring all locations would dilute results for geographically specific methods.The authors recommend spatial CRR heatmaps over change-point-detected patches as a more deployable diagnostic.Source: O Window Estimation Sensitivity and… · p. 13
  • The authors recommend PELT with a minimum segment length of 6 months; on synthetic benchmarks it achieves median window endpoint error of 1.8 months.Table 8 reports CRR degradation under ±3-month window error.Source: O Window Estimation Sensitivity and… · p. 13
  • With both window and location estimates automated on NDVI-LST, SSU-LSF achieves CRR 0.789±0.016 and ∆RMSE 3.9%, versus 0.821 and 3.5% with oracle knowledge.Window-only estimation gives CRR 0.803; the location false-positive rate is 6.8%, mainly costing clean-domain RMSE.Source: O Window Estimation Sensitivity and… · p. 13

P S4 Cross-Architecture Experiment

SSU-LSF transfers to an S4 backbone on NDVI-LST using a Padé-based non-diagonal EKFac gradient. Its CRR is lower than on Mamba, with the gap attributed to S4’s fixed state transitions.

P S4 Cross-Architecture Experiment

  • On NDVI-LST, SSU-LSF achieves CRR 0.791 ± 0.019 on S4, compared with 0.821 ± 0.010 on Mamba.The 0.030 CRR gap is attributed to S4’s fixed, non-selective state transitions, which make temporal footprint localization slightly less precise.Source: P S4 Cross-Architecture Experiment · p. 13
  • For S4, the EKFac gradient uses a non-diagonal variant with Padé approximation for the matrix-exponential differential.The experiment uses a three-layer S4 backbone with ZOH discretization.Source: P S4 Cross-Architecture Experiment · p. 13
  • The reported S4 transfer experiment is on the NDVI-LST benchmark.The paper reports the Mamba comparison on that same benchmark.Source: P S4 Cross-Architecture Experiment · p. 13

Q Full Ablation Table with Interaction and Sample-Efficiency Analysis

The ablations show that footprint localization and the KL trust region contribute complementary protections, while clean-data requirements are modest.

Q Full Ablation Table with Interaction and Sample-Efficiency Analysis

R Footprint Extension Derivation

The footprint extension is small relative to the confounding window, and sensitivity analyses identify robust hyperparameter ranges. MIA curves, scope disclosures, and checklist evidence qualify the reported results and their interpretation.

R Footprint Extension Derivation

  • At ρ=0.84, the derived footprint extension is 2.6 steps on each side, or 11% of a 24-month confounding window.The estimate follows from recurrence influence decaying geometrically beyond the confounding interval.Source: R Footprint Extension Derivation · p. 13

S Hyperparameter Sensitivity

  • CRR is robust for γ from the 85th to 97th percentile, δKL from 0.02 to 0.10, and β from 0.5 to 0.7 across benchmarks.Below the γ range, confounding steps can be missed; above it, clean steps enter ascent and ∆RMSE% rises sharply.Source: S Hyperparameter Sensitivity · p. 14, Figure 2 · p. 14

T MIA Epoch Curves

  • SSU-LSF reaches near-chance LiRA accuracy of 0.53 by epoch 5, while ForgetFS and MULL reach near-chance two epochs later.GR oscillates by ±0.04; the text attributes intermittent clean-domain distribution collapse to the absence of KL trust-region projection.Source: T MIA Epoch Curves · pp. 14–15, Figure 3 · p. 15

NeurIPS Paper Checklist

  • The paper reports that its abstract’s CRR, worst-case ∆RMSE, convergence, and GPU-speedup claims are traceable to experimental tables.Its scope is Mamba-family SSMs on three geoscientific benchmarks; S4 generalization is a secondary finding from one benchmark.Source: NeurIPS Paper Checklist · p. 16
  • Primary benchmarks use injected confounding with oracle window knowledge; automatic estimation lowers CRR from 0.821 to 0.789.CRR requires knowing affected locations; the Proposition 1 bound is quantitatively vacuous at ρmax=0.91, and S4 generalization is shown on one benchmark.Source: NeurIPS Paper Checklist · p. 16
  • The theory states assumptions for its propositions and provides proofs, while qualifying that convergence does not claim global stationarity.Source: NeurIPS Paper Checklist · p. 16
Loading 2610.02248v1…