PaperlayerView exact revision on arXiv

Source-linked AI summary

Equivariant Experience Replay for Continual Learning in Imaging Inverse Problems

Guixian Xu, Jin Zhu, Jinglai Li, Junqi Tang

arXiv:2610.30460v1eess.IV

TL;DR

Imaging inverse problems face operator shifts and other imperfect conditions, while existing continual-learning methods are not tailored to ill-posed inverse problems. The paper proposes Equivariant Experience Replay, using group-symmetry structure, and reports that its methods mitigate forgetting and outperform standard methods numerically in medical imaging and image processing.

  • Problem

    Continual inverse-problem learning must handle changing measurement operators, but the paper identifies no existing tailored continual-learning method for often ill-posed tasks.

    Source: §1 · p. 1, §2.1 · p. 2

  • Method

    Equivariant Experience Replay uses group-symmetry structure in a replay scheme tailored to ill-posed imaging inverse problems.

    Source: §1 · p. 1

  • Results

    The authors report that EER and EER+ significantly mitigate catastrophic forgetting and numerically outperform standard methods in medical imaging and image processing.

    Source: §4 · pp. 10–11

  • Takeaways & Limitations

    The framework can in principle be combined with other regularization and distillation techniques for practitioners to choose according to their needs.

    Source: §4 · p. 11

    Equivariant Imaging supervision depends on current reconstruction quality, and the replay-loss guarantee assumes transformation-invariant image distributions and μj > 0.

    Source: §2.2.2 · p. 4, §2.2.4 · p. 6

Abstract

from arXiv · show

In this work, we propose Equivariant Experience Replay, a new paradigm for achieving continual learning of deep networks for solving ill-posed imaging inverse problems. Inspired by recent advances on unsupervised learning methods such as equivariant imaging/splitting, we find the exploitation of group-symmetry structure is particularly relevant and effective in continual learning setting for ill-posed inversion tasks. We therefore propose this new experience replay scheme, which is tailored for such inverse problems. Our numerical study demonstrates the superiority of our method compared to existing continual learning methods for inverse problems in medical imaging and image processing.

1 INTRODUCTION

Imaging inverse problems arise under imperfect conditions, while existing test-time training does not directly address continual learning because of catastrophic forgetting. The paper introduces Equivariant Experience Replay for this setting and situates it among continual-learning and inverse-problem approaches.

1 INTRODUCTION

  • Real-world imaging inverse problems face limited or noisy measurements, constrained computation, and distribution shifts, motivating a tailored continual-learning framework.The stated aim is to maximize AI-model adaptivity in imaging practice.Source: §1 · p. 1
  • Unsupervised imaging methods can bypass ground-truth data and learn measurement-operator null spaces, while test-time training fine-tunes pretrained networks when tasks fall outside their training distribution.The passage identifies Equivariant Imaging as an early unsupervised framework.Source: §1 · p. 1
  • Continual learning must preserve pretrained-task accuracy while adapting to new tasks with different measurement physics and data distributions, a setting current test-time training cannot directly handle because of catastrophic forgetting.The paper proposes Equivariant Experience Replay as a tailored scheme that uses group-symmetry structure for ill-posed measurement operators.Source: §1 · p. 1

1.1 RELATED WORKS

  • Continual-learning methods broadly use replayed memory, parameter regularization, or added architecture components to retain knowledge across tasks.The paper places its method closest to memory-based and regularization-based approaches, while noting its design is tailored to inverse imaging rather than a direct combination.Source: §1.1 · p. 2
  • Inverse-problem research has advanced through measurement-informed data-driven methods, specialized architectures, unsupervised training, and test-time training; this work extends that line toward continual learning tailored to inverse problems.Source: §1.1 · p. 2

2 ALGORITHMIC FRAMEWORK

The framework formulates continual inverse problems around changes in acquisition operators and addresses the nullspace information lost when past tasks are represented by a small replay buffer. Equivariant Experience Replay expands buffered measurements into virtual-operator constraints, with a theoretical guarantee relating replay loss to past-task error under stated conditions.

2.1 PROBLEM FORMULATION AND OVERVIEW

  • A continual inverse-problem task is determined by image distribution, acquisition physics, and noise; operator shift can occur even when the image distribution is unchanged.At test time the operator is known but the task index is not, and ground-truth images are unavailable.Source: §2.1 · pp. 2–3

2.2.1 OPERATOR SHIFT AND THE NULLSPACE PROBLEM

  • Measurement consistency leaves nullspace components unconstrained, making replay fragile when each past operator is retained only through a small buffer.Equivariance extracts multiple self-supervised constraints from each stored measurement to address this gap.Source: §2.1 · p. 3, §2.2.1 · p. 3

2.2.2 VIRTUAL OPERATORS FROM INVARIANCE

  • Image-distribution invariance makes transformed images plausible under virtual operators, whose differing nullspaces restore operator diversity from individual measurements.Equivariant imaging generates supervision from the network's own reconstruction, so its signal quality depends on that reconstruction.Source: §2.2.2 · pp. 3–4

2.2.3 EQUIVARIANT EXPERIENCE REPLAY

  • Each buffered past operator expands into a virtual-operator orbit, up to |G| times larger, whose nullspaces jointly cover more of the original operator's nullspace.EER adds equivariant imaging to replay, while EER+ uses equivariant splitting on current and buffered data.Source: §2.2.3 · pp. 4–5

2.2.4 WHY EQUIVARIANT REPLAY RETAINS PAST TASKS

  • Under image-distribution invariance and positive coverage, EER+'s equivariant-splitting replay loss tracks past-task error; MC replay has zero coverage and no such guarantee.The forgetting bound depends on replay drift, operator coverage, and memory error.Source: §2.2.4 · pp. 5–6

3 EXPERIMENTS

The experiments test continual reconstruction under two operator shifts while keeping image distributions fixed, then compare measurement-only replay methods and their costs. The results show that operator shift alone can cause severe forgetting, while equivariant replay improves retention across both benchmarks, with retention–plasticity trade-offs depending on the shift.

3.1 BENCHMARKS AND PROTOCOL

  • The benchmarks isolate operator shift using disjoint images from the same dataset and contrast nested sparse-view CT with cross-family inpainting-to-denoising.In CT, views drop from 200 to 20; in the natural-image benchmark, the old inpainting operator has a nontrivial nullspace while denoising has a trivial one.Source: §3.1 · pp. 6–7

3.2 COMPARED METHODS

  • Replay methods store one T1 measurement and its operator, and the compared methods differ primarily in how they use replay; EER+ also changes the current-task loss.The protocol withholds the T1 training set during T2, and ground-truth images are excluded from training objectives.Source: §3.1 · p. 7, §3.2 · pp. 7–8

3.3 IMPLEMENTATION DETAILS

  • EER+ uses equivariant splitting with a 0.6-retention random mask and averages either one or ten split reconstructions at inference.For CT, the pseudo-inverse is approximated with 20 conjugate-gradient iterations and EER+ is trained for 20 epochs to limit cost.Source: §3.3 · p. 8

3.4 MAIN RESULTS

  • With image distributions fixed, operator shift alone causes old-task losses of 8.98 dB in CT and 16.90 dB in natural images under Fine-tune.The cross-family shift leaves T2 with no signal about filling the old inpainting nullspace, while the nested CT shift is less destructive but still substantial.Source: §3.1 · p. 6, §3.4 · p. 9
  • ER+MC reduces forgetting by only 1.62 dB in CT and not at all in natural images; DER and DER++ help only marginally despite storing an additional image per measurement.The experiments adapt these methods to measurement-only learning by replacing supervised replay terms with self-supervised counterparts.Source: §3.2 · p. 7, §3.4 · p. 9
  • EER+ with single-pass inference lowers forgetting to 1.90 dB in CT and 6.57 dB in natural images, outperforming every baseline on both benchmarks.These results measure retention without inference-time averaging; EER+ removes 79% and 61% of Fine-tune’s forgetting, respectively.Source: §3.4 · pp. 9–10
  • EER+ trades 2.41 dB on the 20-view CT task for 7.08 dB on the 200-view task with S-1, while S-10 trades 0.66 dB for 9.34 dB.Under the cross-family shift, EER+ S-1 matches Fine-tune on denoising (23.79 vs. 23.70 dB); both EER+ settings have the highest average PSNR among continual methods on both benchmarks.Source: §3.4 · p. 10
  • In CT, EER requires about 1.8× ER+MC’s training time, while natural-image EER+ S-1 runs at the cost of a single forward pass and retains most of S-10’s gain.CT inference latency for EER grows with view count and scales linearly with the number of averaged splits.Source: §3.4 · p. 10

4 CONCLUSION

Equivariant Experience Replay is proposed for continual learning in imaging inverse problems, and EER/EER+ are reported to mitigate catastrophic forgetting. Numerical experiments report superiority over standard methods in medical imaging and image processing.

4 CONCLUSION

  • The framework is proposed as an experience-replay approach tailored to continual learning for imaging inverse problems.Source: §4 · p. 10
  • Figure 2 shows reconstructions after training on T2, with old-task inpainting illustrating forgetting and new-task denoising illustrating plasticity.Source: Figure 2 · p. 11
  • EER and EER+ significantly mitigate catastrophic forgetting, and the approach is numerically superior to standard methods in medical imaging and image processing.The conclusion describes the framework as the first experience-replay method for inverse problems that uses measurement structure.Source: §4 · pp. 10–11

A THEORY OF EQUIVARIANT REPLAY

The section proves Proposition 1 and Corollary 1, then presents two auxiliary results referenced in Section 2.2.4.

A THEORY OF EQUIVARIANT REPLAY

A.1 PROOF OF PROPOSITION 1: ES RISK CALIBRATION

The proof assumes a noiseless past task and an invariant image distribution under a unitary transformation, then uses these properties to establish the stated ES-risk result.

A.1 PROOF OF PROPOSITION 1: ES RISK CALIBRATION

  • The proof fixes a noiseless past task, assumes a unitary transformation and an invariant image distribution, and takes the transformation and mask to be independent of the image.Source: §A.1 · p. 15
  • Invariance makes the transformed image independent of the transformation and mask, yielding the masked-measurement representation My = MAjTg xg.Source: §A.1 · p. 15
  • The proof expands the conditional quadratic ES risk and uses conditional-mean orthogonality and unitarity to establish equation 19 under the stated bounds on Qj(v).The passage also notes that Aj*Aj is singular when Aj has a nontrivial nullspace.Source: §A.1 · p. 15

A.2 PROOF OF COROLLARY 1: FINITE-MEMORY RETENTION

The finite-memory retention proof bounds the past-task error increase using the replay-loss increase and the buffer’s approximation error.

A.2 PROOF OF COROLLARY 1: FINITE-MEMORY RETENTION

  • The buffer ES risk is assumed to approximate the past-task risk within ζj at both parameter settings, with replay risk at the later setting bounded by the earlier risk plus bδj.Source: §A.2 · p. 15
  • Rj(θi) ≤ Rj(θi−1) + bδj + 2ζj bounds the past-task risk increase by replay drift and twice the buffer approximation error.Source: §A.2 · p. 15
  • The derived inequality is equation 20.Source: §A.2 · p. 15

A.3 VARIANCE OF REPEATED SPLITTING

The repeated-splitting estimator averages losses across stored pairs and transformation–split draws. Increasing the number of draws reduces splitting variance, but not finite-buffer sampling variance.

A.3 VARIANCE OF REPEATED SPLITTING

  • For fixed F, bRj averages the equivariant-splitting loss over Kj stored pairs and H transformation–split draws per pair.The passage also defines mF(z) as the conditional expected loss and s2_F(z) as its conditional variance.Source: §A.3 · p. 15
  • Increasing H reduces splitting variance.Source: §A.3 · p. 15
  • Increasing H does not reduce finite-buffer sampling variance.Source: §A.3 · p. 15

A.4 SPLIT-AVERAGED INFERENCE

Split-averaged inference forms bx(u) from a transformation–split draw and averages it over draws conditional on y and Aj. Its expected MSE is no greater than the S = 1 value.

A.4 SPLIT-AVERAGED INFERENCE

  • For u = (g, M), bx(u) is defined from the transformed and split inputs, and ¯x is its conditional mean given y and Aj.Source: §A.4 · p. 15
  • The inference uses S conditionally independent splits.Source: §A.4 · p. 15
  • The expected MSE is no greater than the S = 1 value, and adding E∗j to the right-hand side of equation 20 bounds S-split inference MSE.Source: §A.4 · p. 15

B ADDITIONAL EXPERIMENT RESULTS

The additional experiment results include complementary sparse-view CT results and complementary natural imaging continual-learning results.

B ADDITIONAL EXPERIMENT RESULTS

  • Figure 3 presents complementary results for sparse-view CT.Source: Figure 3 · p. 16
  • Figure 4 presents complementary results for natural imaging continual learning.Source: Figure 4 · p. 16

Related papers

Loading 2610.30460v1…