Source-linked AI summary
Equivariant Imaging: Learning Beyond the Range Space
Dongdong Chen, Julián Tachella, Mike E. Davies
TL;DR
Compressed measurements alone generally omit information outside the forward operator’s range, making ground-truth-free inverse learning ambiguous. The paper introduces an end-to-end self-supervised equivariant framework that combines measurement consistency with physical signal symmetries, and reports performance comparable to fully supervised methods. Its scope is bounded by the compatibility and sufficiency of the chosen transformations and forward operator.
Problem
Compressed observations cannot generally identify signal information outside the range space of a forward operator with a nontrivial nullspace, while conventional deep-learning methods require ground-truth training pairs.
Method
EI learns a reconstruction network from compressed measurements by enforcing measurement consistency and equivariance under transformations preserving the plausible signal set.
Results
The framework performs almost as well as fully supervised methods and is validated on sparse-view CT reconstruction and image inpainting.
Takeaways & Limitations
Physical invariance can support self-supervised reconstruction beyond the forward operator’s range space from compressed observations alone.
Takeaways & Limitations
The necessary condition is not sufficient in general, and the usefulness of equivariance depends on the transformation group and forward operator.
Abstract
from arXiv · showhide
In various imaging problems, we only have access to compressed measurements of the underlying signals, hindering most learning-based strategies which usually require pairs of signals and associated measurements for training. Learning only from compressed measurements is impossible in general, as the compressed observations do not contain information outside the range of the forward sensing operator. We propose a new end-to-end self-supervised framework that overcomes this limitation by exploiting the equivariances present in natural signals. Our proposed learning strategy performs as well as fully supervised methods. Experiments demonstrate the potential of this framework on inverse problems including sparse-view X-ray computed tomography on real clinical data and image inpainting on natural images. Code has been made available at: https://github.com/edongdongchen/EI.
1. Introduction
Learning-based inverse imaging usually requires ground-truth signal–measurement pairs, while compressed observations alone cannot identify information outside the forward operator’s range space. The paper proposes exploiting physical equivariances to learn reconstructions from compressed measurements alone.
- Motivation and contribution: Ground-truth pairs remain a requirement for conventional deep-learning inverse methods, limiting their use when underlying signals are unavailable.This limitation is especially relevant in real-world imaging scenarios where paired training data cannot be obtained.
- Motivation and contribution: Equivariant imaging learns reconstruction functions from compressed measurements alone by exploiting rotation or shift invariance in natural signals.The framework targets learning beyond the range space of A⊤ without accessing ground-truth signals.
- Method and evaluation: The framework adds an equivariance loss to deep-learning pipelines and is validated on sparse-view CT reconstruction and image inpainting.The authors report reconstructions comparable to fully supervised networks trained with ground-truth signals.
- Related approaches: Model-based methods impose fixed priors and may require expensive test-time optimization, whereas the proposed approach learns the reconstruction mapping end to end.The paper motivates this alternative by combining deep representation learning with mild physical invariances.
- Motivation and contribution: Compressed observations cannot generally identify the signal model beyond the range space of A⊤ when the forward operator has a nontrivial nullspace.Prior compressed-learning approaches therefore require strong assumptions or multiple forward operators.
2. Method
The method trains a reconstruction network using measurement consistency together with equivariance under transformations of estimated images. It applies this strategy to compressed observations without ground-truth signals and can optionally enforce distributional invariance adversarially.
- Problem overview: Measurement consistency alone admits multiple reconstruction functions and cannot learn signal information outside the range space of A⊤.Even infinitely many compressed samples do not resolve this ambiguity.
- Invariant set consistency: The method assumes the plausible signal set is invariant under transformations such as shifts, rotations, or reflections, then requires f ◦ A to be equivariant.The constraint is placed on the complete imaging composition rather than requiring f itself to be invariant or equivariant.
- Equivariant imaging: EI parameterizes fθ as Gθ ◦ A†, projecting measurements with a cheap approximate inverse before applying a trainable neural network.The approximate inverse can be chosen flexibly, including a pseudoinverse.
- Equivariant imaging: Training minimizes measurement error between Afθ(y) and y together with an equivariance error between a transformed estimate and the reconstruction of its transformed measurements.The coefficient α controls the strength of the equivariance term, and the loss uses mean squared error in the implementation.
- Adversarial extension: An adversarial extension enforces identical distributions for x(1) and x(2), but experiments report only a very slight improvement over equivariant learning.The adversarial penalty is controlled by β, while the main experiments focus on equivariant learning.
3. Theoretical analysis
The analysis shows why measurement consistency alone cannot identify components in the nullspace, and derives how signal invariances can provide information beyond the adjoint range space. Recovery requires transformed measurement operators to collectively span the signal space, while the range space must not remain invariant under the symmetry group.
- Learning without invariance: The compressed-learning problem is generally impossible without additional structure, as illustrated by prior impossibility results for learning signal models from compressed observations.The analysis motivates invariance as the structural information that can make learning beyond the adjoint range possible.
- Learning without invariance: Without invariance, any reconstruction f(y) = A†y + v(y) remains measurement-consistent when v(y) lies in A's nullspace, leaving that component unlearned from compressed observations alone.Measurement consistency constrains A f(y) but cannot distinguish among nullspace-valued functions v(y), including the zero function.
- Learning with invariance: Equivariance treats each transformed observation as a measurement under A_g = A T_g, rotating the observable range space through the group action.This expands access beyond a single operator's range space when the transformed ranges collectively reveal new directions.
- Learning with invariance: Theorem 1 requires the concatenated transformed operators to have rank n, so measurements across the group must span the full signal space for unique recovery.The proof stacks measurements y_g = A T_g x into a matrix M and shows that recovering x requires M to be rank n.
- Learning with invariance: A necessary condition is that the range space R_A not be invariant under every group action; otherwise the stacked operator retains rank m < n.If R_A = R_A T_g for all g, the corresponding nullspaces also remain invariant, preventing the transformed measurements from spanning R^n.
4. Experiments
Experiments evaluate equivariant imaging on sparse-view CT and image inpainting using compressed observations alone, comparing it with supervised and measurement-consistency baselines. Across both tasks, the method learns nullspace information and achieves reconstruction quality close to supervised learning.
- Overall evaluation: EI achieves reconstruction quality close to fully supervised learning on both sparse-view CT and image inpainting.The experiments use fixed operators with non-trivial nullspaces and compare against supervised and measurement-consistency learning.
- Sparse-view CT: EI test performance on sparse-view CT is almost as good as ground-truth-trained FBPConvNet and about 7 dB better than FBP.The CT experiments use 50-view sinograms and random rotations as the equivariance transformation.
- Sparse-view CT: The CT equivariant model avoids the significant train-test gap observed for FBPConvNet, suggesting better generalization in this experiment.The paper attributes this behavior to EI restricting the network to functions equivariant on the data.
- Sparse-view CT: Adding equivariance regularization improves supervised learning by 0.6 dB, while the adversarial extension provides only a slight improvement over EI.These comparisons are reported for the CT reconstruction experiments.
- Image inpainting: EI is about 20 dB better than A†y and measurement-consistency reconstruction for inpainting, with only a 1.37 dB gap to the supervised model.The inpainting experiment randomly removes 30% of pixel measurements and uses random shifts.
5. Discussion
The discussion distinguishes equivariant imaging from invariant networks and standard data augmentation, emphasizing its role in learning beyond the forward operator’s range space. It also identifies transformation choice and nonlinear or single-image settings as important boundaries for future work.
- Relationship to prior methods: Unlike invariant classification networks, EI makes the full composition fθ ◦ A equivariant rather than requiring fθ itself to be invariant or equivariant.This distinction promotes invariance across the complete imaging system.
- Relationship to prior methods: EI differs from standard data augmentation because compressed samples y cannot be directly augmented, while the equivariant constraint addresses range-space limitations.The paper reports that the constraint can also improve fully supervised networks.
- Limitations and future work: Whether EI benefits extend to nonlinear imaging problems remains an open question beyond the demonstrated rotation- and shift-invariant tasks.The discussion presents nonlinear imaging as a direction for future investigation.
- Extensions: The useful strength of transformations depends on the signal model and physics operator, so mixed transformation groups may affect convergence and performance.The paper presents mixed group transformations as a straightforward extension.
- Limitations and future work: For single-image reconstruction, EI’s role is less clear because performance already relies heavily on the network’s strong inductive bias.The paper reports only preliminary supplementary results in this setting.
6. Conclusions
The paper introduces a self-supervised strategy that learns ill-posed inverse problems from observed measurements alone, using signal invariance as its only prior beyond the measurements.
- Equivariant imaging learns ill-posed inverse problems from observed measurements without access to the underlying signal distribution, assuming invariance under a transformation group.
A. Training Details
The experiments use residual U-Net architectures and task-specific optimization settings for sparse-view CT and image inpainting, with inpainting evaluation showing favorable generalization.
- The paper uses a residual U-Net with task-dependent channel counts: C = 1 for CT and C = 3 for inpainting.
- Sparse-view CT training uses Adam with batch size 2, learning rate 0.0005, weight decay 10^-8, and 5000 epochs with scheduled decay.
- Inpainting training uses Adam with batch size 1, learning rate 0.001, weight decay 10^-8, and 2000 epochs with decay every 500 epochs.
- For inpainting, EI has smaller generalization error than the supervised model on training and test measurements.
B. More results
Additional experiments show that equivariance is necessary for performance, EI is less dependent on architecture than DIP, and it remains effective on single-image and unseen-measurement reconstructions.
- Effect of the equivariance hyperparameter α: At α = 0, removing equivariance causes the model to fail, while reasonable performance occurs at α = 100 for CT and α = 1 for inpainting.
- EI produces reconstructions comparable to supervised learning for unseen sparse-view CT measurements and image-inpainting measurements.
- EI outperforms DIP-2 by 5 dB on 50-view CT reconstruction while using the residual U-Net, whereas DIP-2 benefits from a stronger autoencoder inductive bias.
- On single-image inpainting, EI outperforms both DIP-1 and DIP-2, while remaining effective with residual connections.
- The role of equivariance for single-measurement reconstruction remains insufficiently explored and is left for future work.