Source-linked AI summary
DeepInverse: A Python package for solving imaging inverse problems with deep learning
Julián Tachella, Matthieu Terris, Samuel Hurault, Andrew Wang, Dongdong Chen, Minh-Hai Nguyen, Maxime Song, Thomas Davies, Leo Davy, Jonathan Dong, Paul Escande, Johannes Hertrich, Zhiyuan Hu, Tobías I. Liaudat, Nils Laurent, Brett Levac, Mathurin Massias, Thomas Moreau, Thibaut Modrzyk, Brayan Monroy, Sebastian Neumayer, Jérémy Scanvic, Florian Sarron, Victor Sechaud, Georg Schramm, Romain Vo, Pierre Weiss
TL;DR
DeepInverse addresses the difficulty of building generalizable and reproducible learning-based methods for imaging inverse problems. It provides a PyTorch-based framework spanning physics operators, reconstruction methods, training losses, evaluation, and reproducibility-oriented software practices. The paper presents the library as a common foundation for research and practice, while noting that it remains an evolving snapshot of ongoing development.
Problem
Learning-based inverse-problem algorithms are often built from scratch, difficult to generalize beyond their training setting, and hard to reproduce.
Method
DeepInverse unifies parameterized imaging operators, reconstruction methods, inverse-problem losses, evaluation tools, and PyTorch-compatible software for developing and training reconstruction systems.
Results
The library provides learning-focused imaging operators and solvers across multiple domains, with common interfaces for reconstruction, training, evaluation, and reproducibility.
Takeaways & Limitations
DeepInverse is intended to accelerate cross-domain research, lower the entry barrier for practitioners, and enhance reproducibility in imaging inverse problems.
Takeaways & Limitations
DeepInverse is a dynamic project, and the paper describes only a snapshot of ongoing progress at release v0.3.0.
Abstract
from arXiv · showhide
DeepInverse is an open-source PyTorch-based library for solving imaging inverse problems. The library covers all crucial steps in image reconstruction from the efficient implementation of forward operators (e.g., optics, MRI, tomography), to the definition and resolution of variational problems and the design and training of advanced neural network architectures. In this paper, we describe the main functionality of the library and discuss the main design choices.
1 Statement of Need
DeepInverse addresses fragmented and difficult-to-reproduce learning-based inverse-problem workflows with a unified PyTorch framework for imaging operators and solvers. It targets both researchers and practitioners while supporting transfer, accessibility, and reproducibility across imaging domains.
- Most learning-based inverse-problem algorithms are built from scratch, difficult to generalize beyond their training setting, and hard to reproduce.
- DeepInverse provides a unified framework for defining imaging operators and solvers using PyTorch-compatible modules.Its modules are compatible with automatic differentiation.
- The library aims to accelerate research by enabling efficient testing, deployment, and transfer of ideas across imaging domains.
- DeepInverse lowers the entrance bar to deep learning in inverse problems for new researchers and practitioners.
- A common framework for operators, reconstruction methods, datasets, and metrics is intended to enhance reproducibility.
- Compared with related Python libraries, DeepInverse emphasizes learning-based methods and provides a larger set of realistic imaging operators.The cited alternatives focus mainly on variational optimization, uncertainty quantification, tomography, or operator-specific functionality.
2 Inverse Problems
DeepInverse represents imaging inverse problems through physics-based forward operators and measurement noise, then provides tools for composing and solving these models. Its parameterized operators support advanced reconstruction settings such as calibration, blind inversion, co-design, and robust training.
- Imaging inverse problems model measurements as a physics operator applied to an image followed by a noise mapping.The operator captures acquisition physics, while the noise mapping is parameterized by quantities such as noise level or gain.
- DeepInverse offers high-level operators for applications including MRI, computed tomography, and radioastronomy, with algebra for summing, concatenating, and stacking them.
- The library includes tools for adjoints, pseudoinverses, proximal operators, matrix-free linear solvers, and operator norm and condition-number estimation.
- 2.1 Operator Parameterization: Parameterizing physics operators enables calibration, blind inverse problems, co-design, and robust neural-network training.The parameter vector can encode quantities such as tomography angles, blur kernels, or MRI acceleration masks.
3 Reconstruction Methods
DeepInverse unifies reconstruction solvers that depend on the forward operator and noise distribution, spanning optimization-based, sampling-based, and non-iterative approaches.
- The framework unifies reconstruction solvers that depend on the forward operator Aξ and noise parameter σ.
- Optimization-Based Methods: Optimization-based methods include classical regularizers, plug-and-play denoisers, optimization algorithms, unfolded networks, and deep equilibrium models.
- Sampling-Based Methods: Sampling-based methods use Markov chains to approximately sample from the posterior p(x|y), enabling uncertainty estimates from multiple reconstructions.
- Sampling-Based Methods: Diffusion models incorporate pretrained denoisers through stochastic or ordinary differential equations rather than optimization.
- Non-Iterative Methods: Non-iterative methods include artifact removal, unconditional and conditional generative networks, and foundation models.
- Non-Iterative Methods: Foundation models incorporate forward operators and noise distributions and may perform well on new tasks without retraining or be fine-tuned using measurement data alone.
4 Training
DeepInverse provides high-level training infrastructure, inverse-problem-specific losses, and supervised or self-supervised objectives for reconstruction networks.
- The Trainer class handles data ingestion, training loops, logging, and checkpointing for reconstruction networks.
- The loss module unifies training losses designed for inverse problems across domains.
- Losses use network predictions together with optional ground truth, measurements, physics, and model inputs.
- Supervised training uses ground-truth and measurement pairs, while self-supervised training relies on measurement data only.
- Self-supervised losses include splitting losses, SURE variants, Recorrupted2Recorrupted, and nullspace losses for several noise distributions and operator settings.
- The framework also provides network-regularization and supervised or self-supervised adversarial losses.
5 Datasets
DeepInverse standardizes simulated paired datasets and interfaces to established datasets spanning natural, medical, microscopy, motion-blur, and satellite imaging.
- Datasets provide ground-truth and measurement pairs and may additionally return physics parameters ξi.
- The library can generate and save simulated paired datasets to encourage reproducible research.
- Provided interfaces cover natural images, MRI, computed tomography, fluorescence microscopy, real motion blur, and multispectral satellite images.
6 Evaluation
DeepInverse supplies common full-reference and no-reference metrics for evaluating reconstruction quality, with shared handling for normalization and complex magnitude.
- Reconstruction methods can be evaluated with Trainer.test using metrics defined in the framework.
- Full-reference metrics compare a reconstruction with a reference image, whereas no-reference metrics use the reconstruction alone.
- The library includes PSNR, SSIM, LPIPS, NIQE, and QNR to assess reconstruction quality.
7 Philosophy
DeepInverse emphasizes reproducible, tested software and documentation that supports both new and advanced users.
- Implementation philosophy: The library follows test-driven development with unit, integration, and performance tests, automated coverage verification, and PEP8-compliant formatting.Random number generators are passed explicitly to support reproducibility.
- Documentation: DeepInverse provides a user guide, quickstart materials, in-depth examples, and API documentation for different levels of expertise.Jupyter notebook demonstrations are automatically tested and included in the documentation.
8 Perspectives
DeepInverse is an evolving project, and the paper represents a snapshot of release v0.3.0 rather than a final account of the library.
- Future development: The library continues to receive community contributions, including more realistic physics operators and advanced training techniques.These additions aim to reflect state-of-the-art imaging with deep learning and address researchers’ and practitioners’ needs.