PaperlayerView exact revision on arXiv

Source-linked AI summary

Solving Inverse Problems With Deep Neural Networks -- Robustness Included?

Martin Genzel, Jan Macdonald, Maximilian März

arXiv:2011.04268v1cs.LGcs.NAmath.NA

TL;DR

Underdetermined inverse-problem solvers need reliability under measurement noise, while prior work has reported severe artifacts from small perturbations. This study compares learned reconstruction methods with total-variation minimization under statistical and worst-case measurement noise. Across the considered scenarios, some deep-learning methods are at least as robust as TV minimization to adversarial noise.

  • Problem

    Prior studies reported instabilities from small perturbations in learned reconstruction, while reliability under noisy, underdetermined measurements matters for sensitive applications.

    Source: §1 · p. 1

  • Method

    The study evaluates learned solvers under statistical and worst-case measurement noise and systematically compares them with total-variation minimization.

    Source: §1.1 · p. 2

  • Results

    In every considered scenario, the authors find deep-learning methods at least as robust as TV minimization to adversarial noise, and report that standard workflows can yield robust schemes without sophisticated defenses.

    Source: §1.1 · p. 2

  • Takeaways & Limitations

    The findings suggest that fairly standard workflows can produce surprisingly robust reconstruction schemes, but do not establish stability for every deep-learning method.

    Source: §1.1 · p. 2

    The study examines selected end-to-end architectures, excluding generative models, deep image prior, and learned regularizers.

    Source: §6 · p. 12

Abstract

from arXiv · show

In the past five years, deep learning methods have become state-of-the-art in solving various inverse problems. Before such approaches can find application in safety-critical fields, a verification of their reliability appears mandatory. Recent works have pointed out instabilities of deep neural networks for several image reconstruction tasks. In analogy to adversarial attacks in classification, it was shown that slight distortions in the input domain may cause severe artifacts. The present article sheds new light on this concern, by conducting an extensive study of the robustness of deep-learning-based algorithms for solving underdetermined inverse problems. This covers compressed sensing with Gaussian measurements as well as image recovery from Fourier and Radon measurements, including a real-world scenario for magnetic resonance imaging (using the NYU-fastMRI dataset). Our main focus is on computing adversarial perturbations of the measurements that maximize the reconstruction error. A distinctive feature of our approach is the quantitative and qualitative comparison with total-variation minimization, which serves as a provably robust reference method. In contrast to previous findings, our results reveal that standard end-to-end network architectures are not only resilient against statistical noise, but also against adversarial perturbations. All considered networks are trained by common deep learning techniques, without sophisticated defense strategies.

1 INTRODUCTION

The introduction frames robustness as a central concern for learned reconstruction under noisy, underdetermined measurements. It outlines a comparative study of learned solvers and total-variation minimization, with findings that qualify robustness by noise type, architecture, and training.

1 INTRODUCTION

  • Underdetermined measurements make inverse problems ill-posed, while noise robustness matters because measurements may be costly or harmful to acquire.Classical algorithms can also be limited by computational cost, manual parameter tuning, and mismatch between sparsity models and data.Source: §1 · p. 1
  • Prior reports of severe artifacts from slight perturbations motivate testing whether such instability applies to learned reconstruction, whose reliability matters in sensitive applications.The introduction contrasts reconstruction to a high-dimensional signal manifold with classification's low-dimensional discrete output.Source: §1 · pp. 1–2

1.1 Contributions

  • The study compares statistical and worst-case measurement perturbations for learned solvers and TV minimization across compressed sensing, Fourier and Radon image recovery, and MRI.It analyzes more than 25 in-house-trained networks; adversarial perturbations are computed to maximize reconstruction error.Source: §1.1 · p. 2
  • Across the considered scenarios, some deep-learning methods are at least as robust as TV minimization to adversarial noise, without sophisticated architectures or defenses.The paper also reports resilience to statistical noise, architecture-dependent performance, and a remedy for instability associated with noiseless training.Source: §1.1 · p. 2
  • The paper identifies noiseless-data training as a possible source of instability at higher noise levels and reports white-Gaussian-noise jittering as an effective remedy.It also reports that promoting data consistency in iterative schemes may improve accuracy and robustness.Source: §1.1 · p. 2
  • The study characterizes a gap between adversarial and statistical noise that also appears for TV minimization, not only learned methods.The introduction notes that the paper proceeds from related work and approach to main results, additional experiments, and discussion.Source: §1.1 · p. 2, §1.2 · p. 2

2 RELATED WORK

Prior work on adversarial robustness for learned inverse-problem solvers is limited and often differs in perturbation models, comparison methods, or evaluation style. This paper situates its study against those differences and related defenses.

2 RELATED WORK

  • Huang et al. studied attacks on reconstruction and reported loss of small image features, but their work was limited to limited-angle CT and used a non-standard perturbation model.In that setting, robust recovery of some image parts is provably impossible.Source: §2 · p. 3
  • Antun et al. reported instabilities under adversarial noise, out-of-distribution features, and changes in measurement count, evaluating adversarial noise only on learned schemes and reporting results qualitatively.The present study compares attacks on a classical benchmark and uses quantitative error analysis; a one-to-one comparison remains subtle because tasks and data processing differ.Source: §2 · p. 3
  • Related responses include Raj et al.'s sophisticated defense strategy and Kobler et al.'s [39] adversarially robust data-driven total deep variation regularizer for denoising.Raj et al. also address shortcomings in the attack strategy of Antun et al..Source: §2 · p. 3

3 METHODS AND PRELIMINARIES

This section presents learned reconstruction architectures and their training, total-variation minimization as a robust benchmark, and a measurement-domain attack for comparing robustness.

3 METHODS AND PRELIMINARIES

  • The section introduces representative learned reconstruction schemes alongside TV minimization and presents an attack strategy for assessing their adversarial robustness.The attack strategy is applied to the benchmark as well as to learned schemes.Source: §3 · p. 3, §3.4 · p. 4

3.1 Neural Network Architectures

  • UNet refines a model-based inversion, TiraFL replaces that fixed inversion with a learnable linear layer, and ItNet alternates data-consistency steps with trainable enhancement networks.ItNet's data-consistency step performs a gradient step on the measurement-fit loss; the cascade is a proximal-gradient scheme with a trainable enhancement network replacing the proximal operator.Source: §3.1 · p. 3

3.2 Neural Network Training

  • Networks are trained on noisy measurement–signal pairs by minimizing an empirical loss, with random Gaussian measurement noise added during training as jittering.The noise level is sampled through a random variance so its expected norm is distributed uniformly over a specified range.Source: §3.2 · p. 4

3.3 Total-Variation Minimization

  • TV minimization promotes gradient-sparse solutions and, under suitable assumptions on the measurement operator, is provably robust with near-optimal dependence on the noise level.This motivates its use as a reference for empirically characterizing learned methods' robustness.Source: §3.3 · p. 4

3.4 Adversarial Noise

  • The adversarial attack finds bounded additive measurement perturbations that maximize reconstruction error, and applies the same strategy to learned methods and TV minimization.The constrained formulation controls the noise level; TV attacks require differentiating the implicit reconstruction map, which the authors handle by unrolling ADMM.Source: §3.4 · p. 4

4 MAIN RESULTS

The section compares reconstruction error and sampled reconstructions under statistical and adversarial measurement noise across compressed-sensing, phantom-imaging, and fastMRI cases. It reports how learned methods compare with TV minimization across those settings.

4 MAIN RESULTS

  • Across the three case studies, the experiments compare relative reconstruction error and individual reconstructions under statistical or adversarial measurement noise.Noise-to-error curves quantify error against relative noise level; individual reconstructions support qualitative assessment, with adversarial noise levels selected so TV no longer performed subjectively acceptably.Source: §4 · pp. 4–5

4.1 Case Study A: Compressed Sensing With Gaussian Measurements

  • The Gaussian-measurement study tests synthetic piecewise-constant signals and MNIST digits treated as one-dimensional signals, using 100 and 300 measurements, respectively.The signal experiment uses 200,000 training samples; the MNIST experiment uses 60,000 training images.Source: §4.1 · pp. 5–6
  • In the Gaussian-measurement compressed-sensing experiments, Tiramisu outperformed a simple U-Net, and a learnable inversion layer improved recovery.These comparisons are reported in Tables S1–S4.Source: Table S1 · p. 6

4.2 Case Study B: Image Recovery of Phantom Ellipses

  • For phantom ellipses, learned methods had adversarial ℓ2 robustness similar to TV, were more resilient to statistical perturbations at mid-to-high noise, and showed almost imperceptible visual errors up to 3% Fourier or 1% Radon noise.At the highest noise level, the networks exhibited unnatural ellipsoidal artifacts.Source: Figure 6 · p. 7, §4.2 · p. 7, Figure 7 · p. 8
  • The phantom-image study uses 256 × 256 random-ellipse images and evaluates golden-angle radial Fourier measurements and 60-view Radon measurements; higher dimensionality required adaptations to learned inversion and TV attacks.The constrained TV formulation was computationally infeasible for adversarial-noise computation, so the experiments used its unconstrained version.Source: §4.2 · p. 7
  • For phantom ellipses, adversarial perturbations amplified TV’s known flaws, while ellipsoidal features also appeared in the FBP inversion, indicating they were present in corrupted measurements rather than introduced by the post-processing network.TV-generated adversarial perturbations did not transfer to neural-network methods, and Fourier measurements showed no clear superiority of fully learned schemes over model-based inversion.Source: §4.2 · p. 7, Figure 8 · p. 8, Figure 7 · p. 8

4.3 Case Study C: MRI on Real-World Data (fastMRI)

  • The real-world MRI study uses 796 coronal proton-density fastMRI scans to form about 17,000 training images, with 50 radial Fourier lines and a single-coil-style measurement model based on multi-coil magnitude images.The authors chose this setup in favor of higher image quality and a competitive TV benchmark for noiseless measurements.Source: §4.3 · p. 9
  • On fastMRI, learned schemes were more robust than TV even for noiseless measurements; their central image regions remained largely unaffected by adversarial noise, though tiny vessel patterns appeared outside the central region at large perturbations.These perturbations were large enough that TV already suffered severe distortions; TV also blurred fine details, increasingly so with larger perturbations.Source: Figure 9 · p. 9, §4.3 · p. 9

5 FURTHER ASPECTS OF ROBUSTNESS

The additional experiments examine how training noise, downstream classification, and MRI sampling design affect robustness. They show that noiseless training can destabilize reconstructions, stable recovery does not prevent adversarial classification errors, and sampling geometry can shape reconstruction artifacts.

5.1 Training Without Noise – An Inverse Crime?

  • Without jittering, ItNet’s relative recovery error blows up at ∼15% adversarial noise; UNet also amplifies tiny distortions in filtered back-projections from sparse-angle Radon measurements.The authors relate accurate but unstable networks trained on noiseless data to “inverse crimes.”Source: §5.1 · p. 9, Figure 11 · p. 10, Figure 12 · p. 11

Compressed Measurements

  • Even with stable reconstructions, every tested recovery method can produce recognizable digits that fool ConvNet, leaving the resulting classification pipeline vulnerable.The authors report this for model-based and learned solvers alike.Source: pressed Measurements · p. 11

5.3 The Original fastMRI Challenge Setup

  • At ∼23% line-based sampling, TV[η] and Tira show line-shaped artifacts; anisotropic mask geometry aligns vertical features with the operator kernel, implicating sampling design in adversarial robustness.This setup is more susceptible to adversarial noise than Case Study C’s ∼17% undersampling ratio.Source: §5.3 · p. 11, Figure 14 · p. 12

6 DISCUSSION

The discussion concludes that standard deep neural networks were resilient to statistical and adversarial distortions in the studied inverse problems, while identifying boundaries in the architectures and robustness measures examined.

Discussion

  • Across the experiments, standard deep neural networks were remarkably resilient to both statistical and adversarial distortions; Gaussian noise during training appears sufficient to resist other noise types, including adversarial perturbations.The authors also report that noiseless training can cause instabilities, with jittering offered as a simple remedy.Source: §6 · p. 12
  • The study covers selected end-to-end architectures, not generative models, deep image prior, or learned regularizers; the authors’ expectation of comparable robustness for these methods remains an expectation.The attack strategy also lacks a theoretical optimality certificate because its optimization problem is non-convex.Source: §6 · p. 12
  • Euclidean similarity measures may miss visual distortions, including tiny pathological features rarely represented in fastMRI data.The authors note that some such details may be lost in subsampling, while instabilities are another possible explanation.Source: §6 · p. 12
  • For medical imaging, the authors identify random-perturbation robustness as the more practical concern, since an intentional adversary is typically absent.They describe their results as optimistic for deep networks in inverse problems in imaging.Source: §6 · p. 12

15 Supplementary Material

The supplementary material collects additional case-study results and hyperparameter information.

Supplementary Material

  • The supplement contains additional results for Case Studies A–C, supplementary results for Section 5, and an overview of empirically selected hyperparameters for architectures, training, and attacks.Source: §15 · p. 15

S1 SUPPLEMENTARY RESULTS FOR CASE STUDY A (CS WITH GAUSSIAN MEASUREMENTS)

The supplementary results provide additional visualizations and numerical representations for compressed sensing with one-dimensional signals and MNIST.

Scenario A1 – CS with 1D signals

  • For one-dimensional compressed sensing, supplementary figures show uniform and Bernoulli noise-to-error curves and Gaussian-noise reconstructions; Tables S1–S2 numerically represent results from Fig. 2(c) and 2(d).The curves include averages over test signals, and the Bernoulli noise uses p = 0.025.Source: Table S1 · p. 15, Table S2 · p. 15, Figure S1 · p. 15

Scenario A2 – CS with MNIST

  • For MNIST compressed sensing, Tables S3–S4 numerically represent results from Fig. 4(c) and 4(d), while supplementary figures show additional adversarial and Gaussian-noise reconstructions.The captions specify relative ℓ2-error displays and omit selected noise levels in some visualizations.Source: Table S3 · p. 16, Table S4 · p. 16, Figure S3 · p. 17

S2 SUPPLEMENTARY RESULTS FOR CASE STUDY B (IMAGE RECOVERY OF PHANTOM ELLIPSES)

The supplement adds numerical results and individual reconstructions for phantom-ellipse recovery under adversarial and statistical noise, and examines perturbation transfer between TV minimization and neural-network solvers.

S2 SUPPLEMENTARY RESULTS FOR CASE STUDY B (IMAGE RECOVERY OF PHANTOM ELLIPSES)

  • TV[η] adversarial perturbations at 5% and 8% produce point-like artifacts, which SSIM is less sensitive to than PSNR.Table S5 also includes UNetFL and Tira alongside the methods shown in Fig. 6(c).Source: Table S5 · p. 18
  • Table S6 provides numerical results for Fig. 6(d), including UNetFL and Tira.Source: Table S6 · p. 18
  • The supplement presents Fourier-ellipse reconstructions under adversarial and Gaussian noise, and Radon-ellipse reconstructions under Poisson noise.The figures show selected noise levels; the Gaussian and Poisson figures omit their noiseless cases.Source: Figure S5 · p. 19, S2 SUPPLEMENTARY RESULTS FOR CASE… · pp. 19–21, Figure S6 · p. 20
  • The transferability case study applies ItNet-derived adversarial noise to TV minimization and TV-derived noise to neural networks, varying TV's regularization parameter η.The figure also shows the analogous experiment for Radon measurements.Source: Figure S8 · p. 22, S2 SUPPLEMENTARY RESULTS FOR CASE… · p. 22

S3 SUPPLEMENTARY RESULTS FOR CASE STUDY C (MRI ON REAL-WORLD DATA)

The supplement presents individual fastMRI reconstructions under adversarial and Gaussian noise, along with numerical counterparts to the reported results that include additional network methods.

S3 SUPPLEMENTARY RESULTS FOR CASE STUDY C (MRI ON REAL-WORLD DATA)

S4 SUPPLEMENTARY RESULTS FOR SECTION 5

The supplement illustrates intermediate ItNet steps with and without jittering for the 20%-adversarial perturbations shown in Fig. 11.

S4 SUPPLEMENTARY RESULTS FOR SECTION 5

  • The intermediate ItNet steps are shown with and without jittering for the 20%-adversarial reconstructions in Fig. 11.Source: Figure S11 · p. 26

S5 CHOICE OF HYPER-PARAMETERS

The supplement documents the network architectures, training settings, and adversarial-attack hyperparameters, with further detail in Tables S9–S11.

S5 CHOICE OF HYPER-PARAMETERS

  • Tables S9–S11 cover architecture, network training, and adversarial-attack hyperparameters; the reported Net parameters apply to all network types.Source: S5 CHOICE OF HYPER-PARAMETERS · pp. 26, 29
  • All considered networks use rectified linear unit activations, with convolution kernels of size 3 or 3 × 3 and max-pooling sizes of 2 or 2 × 2.Source: Table S9 · p. 27
  • Networks are trained in one or two phases, using default Adam parameters except for ε, which is reported by network and training phase.Source: Table S10 · p. 28
  • The supplement reports adversarial-attack settings including 50,000 initialization ADMM iterations, 2,000 gradient ADMM iterations, 100 Adam iterations, step size 5 · 10^-1, and 100 random initializations.Source: S5 CHOICE OF HYPER-PARAMETERS · p. 29
Loading 2011.04268v1…