Source-linked AI summary
Dynamic Path-Controllable Deep Unfolding Network for Compressive Sensing
Jiechong Song, Bin Chen, Jian Zhang
TL;DR
In deep unfolding networks for compressive sensing, fixed-stage processing can spend computation on images that are easier to restore. DPC-DUN selects image-dependent module paths and lets users adjust the performance-complexity tradeoff. The paper reports flexible tradeoff control and good performance in experiments.
Problem
Fixed-stage DUNs process images with different restoration difficulty through the full network, although some images are easier to restore.
Source: §I · p. 1Method
DPC-DUN combines a Path Selector that chooses active modules with a Controllable Unit that adjusts the performance-complexity tradeoff.
Source: §I · p. 2Results
Extensive experiments show that DPC-DUN enables different performance-complexity tradeoffs and achieves good performance.
Source: §I · p. 2Takeaways & Limitations
DPC-DUN adaptively skips modules for different images and allows test-time adjustment of the performance-complexity tradeoff.
Source: §V · p. 11
Abstract
from arXiv · showhide
Deep unfolding network (DUN) that unfolds the optimization algorithm into a deep neural network has achieved great success in compressive sensing (CS) due to its good interpretability and high performance. Each stage in DUN corresponds to one iteration in optimization. At the test time, all the sampling images generally need to be processed by all stages, which comes at a price of computation burden and is also unnecessary for the images whose contents are easier to restore. In this paper, we focus on CS reconstruction and propose a novel Dynamic Path-Controllable Deep Unfolding Network (DPC-DUN). DPC-DUN with our designed path-controllable selector can dynamically select a rapid and appropriate route for each image and is slimmable by regulating different performance-complexity tradeoffs. Extensive experiments show that our DPC-DUN is highly flexible and can provide excellent performance and dynamic adjustment to get a suitable tradeoff, thus addressing the main requirements to become appealing in practice. Codes are available at https://github.com/songjiechong/DPC-DUN.
I. INTRODUCTION
Compressive sensing reconstructs signals from fewer measurements than Nyquist sampling, but its inverse problem is ill-posed. DPC-DUN uses image-dependent paths and controllable computation to balance reconstruction performance and complexity.
I. INTRODUCTION
- Compressive sensing uses random linear measurements to reconstruct signals from fewer measurements than Nyquist sampling, making reconstruction an ill-posed inverse problem.The measurements are expressed as y = Φx, with M ≪ N.Source: §I · p. 1
- Traditional model-based methods use predefined sparsifying transforms as priors, but can have high computational complexity and difficult parameter tuning.Examples of the transforms include DCT and wavelets.Source: §I · p. 1
- Image contents differ in restoration difficulty, and image-dependent paths can maintain similar high performance while reducing computational cost.The paper motivates selecting a different path for each image rather than processing all images through the same modules.Source: §I · p. 1
- DPC-DUN combines a Path Selector that chooses the number and position of active modules with a Controllable Unit for adjusting the performance-complexity tradeoff.The selector is conditioned on the Lagrange multiplier.Source: §I · p. 2
- DPC-DUN’s slimming mechanism adaptively selects paths to make model complexity controllable.The proposed selector uses the skip-connection structure inherent in DUNs to determine which modules execute.Source: §I · p. 2
- Experiments show DPC-DUN controls performance-complexity tradeoffs while achieving good reconstruction performance.The introduction reports extensive experiments but gives no quantitative comparison here.Source: §I · p. 2
II. RELATED WORK
The related work reviews deep unfolding, dynamic routing, and controllable networks as foundations for DPC-DUN. The proposed architecture combines path selection with Lagrange-multiplier modulation.
III. PROPOSED METHOD
- DPC-DUN’s K-stage architecture uses a Path-Controllable Selector to control module selection and a Lagrange multiplier to modulate processing.The selector comprises a Path Selector and Controllable Unit; the input y is initialized as x(0) = Φ⊤y and the recovered result is x(K).Source: Figure 2 · p. 3
A. Deep Unfolding Network
- DUNs combine optimization methods with learned components, but commonly use a fixed number of stages and a single path for every image.The paper notes that stage count is not considered adaptively, limiting efficiency across images.Source: A. Deep Unfolding Network · p. 3
B. Dynamic Network
- Dynamic networks and skip-connected routing motivate selecting paths to reduce test-time computation in DUNs.DUN optimization algorithms have inherent skip-connection structures that can support dynamic path selection.Source: B. Dynamic Network · p. 3
C. Controllable Network
- Controllable networks modulate internal features to adjust computational cost, motivating adjustable performance-complexity tradeoffs for CS reconstruction.The cited image-compression frameworks use control parameters to modulate autoencoder feature maps.Source: C. Controllable Network · p. 3
A. Framework
The framework unfolds proximal gradient descent into stages with a gradient descent module and a CNN-based proximal mapping module. It preserves feature information across stages and adds a selector to address the fixed-stage limitation of conventional DUNs.
A. Framework
- Each stage implements proximal gradient descent through a gradient descent module (GDM) and a CNN-based proximal mapping module (PMM).The PMM is improved to transmit multi-channel features.Source: A. Framework · p. 3
- The framework splits each input into a one-channel image input and C −1 feature channels, then concatenates the latter with the gradient-step output.The split is described as a way to reduce information loss.Source: A. Framework · p. 4
- DUNs typically use a fixed stage count K, so the paper adds a Path-Controllable Selector to the gradient descent and proximal mapping modules.The fixed stage count cannot adapt to the input image and can consume unnecessary memory and computation.Source: A. Framework · p. 4
B. Path-Controllable Selector (PCS)
The Path-Controllable Selector (PCS) combines a path selector (PS) and a controllable unit (CU). The PS chooses which modules to execute, while the CU controls the efficiency–accuracy tradeoff.
B. Path-Controllable Selector (PCS)
- PCS combines a path selector (PS) that selects image-specific paths with a controllable unit (CU) that enables control of computation and memory.The PS selects paths for images with different content.Source: B. Path-Controllable Selector (PCS) · p. 4
1) Path Selector (PS):
- The PS uses average-pooled features and fully connected layers to produce one-hot execute-or-skip decisions for the GDM and PMM.Gumbel-Softmax makes the discrete decision differentiable during backpropagation; its noise is used only during training.Source: 1) Path Selector (PS) · p. 4
- The dynamic PSs control residual connections, allowing residual computation to be skipped or performed according to their selections.The paper identifies residual connections as primary sources of computation in the main process.Source: 1) Path Selector (PS) · p. 4
- The selection loss guides training toward an efficiency–accuracy tradeoff, while the Lagrange multiplier µ balances accuracy expectations against computational cost.The paper introduces the CU to control the PS by adjusting µ.Source: 1) Path Selector (PS) · p. 5
2) Controllable Unit (CU):
The controllable unit conditions feature processing on the Lagrange multiplier µ. Its parameter pair scales and shifts feature maps, enabling the selectors to operate under different control settings.
2) Controllable Unit (CU):
- The CU encodes µ into a one-hot vector and uses two functions to produce a parameter pair for feature-map modulation.The pair scales and shifts elementwise feature maps through an affine transformation.Source: 2) Controllable Unit (CU) · p. 5
- The PCS consists of the CU and PS, joining multiplier-conditioned control with path selection.Source: 2) Controllable Unit (CU) · p. 5
- The DPC-DUN reconstruction iterates a dynamic gradient descent module and a dynamic proximal mapping module, each incorporating PCS.The modules are denoted as R(k) = GDGDM(X(k−1), µ) and X(k) = GDPMM(R(k), µ).Source: 2) Controllable Unit (CU) · p. 5
C. Loss function
The reconstruction loss is L1, and the training objective also incorporates a selection loss. The supplied tables compare average reconstruction metrics and executed module counts across two datasets.
C. Loss function
- The reconstruction loss Lrec is the L1 loss function.Source: C. Loss function · p. 5
- The selection loss is updated before being combined with the reconstruction loss in the total objective.The supplied passages do not specify the loss expressions beyond identifying this combination.Source: C. Loss function · p. 5
- The tables report average PSNR and SSIM comparisons across CS ratios, with executed GDM/PMM counts also shown, for Set11 and CBSD68.Source: Table I · p. 6, Table II · p. 6
A. Implementation Details
The implementation trains image-CS models on augmented luminance patches and uses orthogonalized Gaussian measurement matrices. Training combines patch-wise learning with larger-patch finetuning, and evaluation measures reconstruction quality and computational cost.
A. Implementation Details
- Training data are generated from 400 images of size 180×180 by extracting 33×33 luminance blocks and applying data augmentation for diversity, with N = 1,089.Source: A. Implementation Details · p. 6
- For each CS ratio, the measurement matrix is generated from a random Gaussian matrix and row-orthogonalized, yielding ΦΦ⊤ = I.Measurements are produced by applying Φ to vectorized image blocks.Source: A. Implementation Details · p. 6
- Training runs for 410 epochs, with 99×99-patch finetuning during the final ten epochs to alleviate blocking artifacts.Testing also unfolds the whole image into overlapping 33×33 blocks.Source: A. Implementation Details · p. 6
- Evaluation reports PSNR and SSIM for reconstruction quality, plus active-module counts and FLOPs for computational cost.The default network uses K = 25 stages and C = 32 feature maps.Source: A. Implementation Details · p. 6
B. Qualitative Evaluation
The qualitative evaluation compares reconstruction methods across several datasets and sampling rates. The proposed methods lead the reported competitors in PSNR, while visual comparisons and cost results describe reconstruction fidelity and computation tradeoffs.
B. Qualitative Evaluation
- DP-DUN and DPC-DUN outperform competing methods in PSNR across the reported cases, including Urban100 and DIV2K comparisons.Table III covers average PSNR/SSIM across different sampling ratios and compares three competing methods.Source: Table III · p. 7, B. Qualitative Evaluation · p. 7
- For two Urban100 images at CS ratio 10%, one single DPC-DUN model’s recovery accuracy and computational cost stably increase and converge as µ is gradually reduced, illustrating adjustable tradeoffs.Source: Figure 7 · p. 8
- With PSNR unchanged, using PS or PCS reduces FLOPs, and the reduction is more pronounced at larger stage counts.Table V compares PSNR, FLOPs, and active-module counts across stage numbers.Source: Table V · p. 8
- Visual comparisons on Set11, Urban100, DIV2K, and CBSD68 show DP-DUN and DPC-DUN producing visually pleasant reconstructions faithful to ground truth.The figures include Barbara at a 25% CS ratio and examples at 30%, 40%, and 10%.Source: Figure 4 · p. 7, Figure 5 · p. 7, B. Qualitative Evaluation · p. 8
C. Cost Comparison
The cost comparisons examine selector overhead, multiplier-controlled tradeoffs, and the relationship between accuracy and active computation. Results describe lower computation at selected settings and flexible behavior across images and configurations.
C. Cost Comparison
- PCS adds less computational overhead than PS alone while changing performance little, especially when K ≥ 25.The comparison also reports that PS reduces total FLOPs, particularly at larger stage numbers, while PSNR remains unchanged.Source: C. Cost Comparison · pp. 8–9
- One setting, µ = 0.002, offers less memory and computation cost than competing methods while retaining the best reported performance.The paper presents this as an adjustable performance-consumption balance.Source: C. Cost Comparison · p. 9
D. Analysis of Path-Controllable Selector (PCS)
- DPC-DUN adjusts the accuracy-cost tradeoff by µ; easier images can use fewer active modules, while richer images may require a setting favoring reconstruction accuracy.The examples select µ = 0.002 for an easier image and µ = 0.00001 for an image with rich content.Source: D. Analysis of Path-Controllable… · pp. 9–10, Figure 8 · p. 9
- With only six default encodings used in training, DPC-DUN generalizes to unseen binary encodings, while accuracy and active-module counts generally change with µ.The reported trends vary in degree across sampling rates and data distributions.Source: Figure 9 · p. 9, D. Analysis of Path-Controllable… · p. 10
E. Analysis of Path Selector (PS)
- At µ = 0.0002, FLOPs fall about 18.2% on Set11 and 18.6% on CBSD68, with PSNR not changing obviously for µ ≤ 0.0002.Larger µ places greater weight on computation cost and corresponds to smaller PSNR, active-module counts, and FLOPs.Source: E. Analysis of Path Selector (PS) · p. 10, Table VII · p. 10
F. Ablation Study
The ablations test deblocking finetuning, learnable sampling matrices, noise robustness, and control-factor behavior. The conclusion also identifies remaining limitations in selector granularity and modulation range.
1) Impact of Blocking Artifacts:
- After 99×99-patch finetuning, PSNR and SSIM improve while the number of active modules changes little.The comparison is on Set11 and treats results without finetuning as the baseline.Source: 1) Impact of Blocking Artifacts · p. 10
2) Application in the learnable sampling matrix:
- Retrained DPC-DUN applies to a learnable sampling matrix and achieves reported robustness and good performance on Set11.The comparison is with methods that jointly learn the sampling matrix and reconstruction process.Source: 2) Application in the learnable… · p. 10
3) Sensitivity to Noise:
- Noise-finetuned models outperform the compared methods across tested noise levels and show smaller PSNR differences between noisy and noise-free testing than their non-finetuned counterparts.The experiments use mixed Gaussian noise from 0 to 10 on Set11; visualizations report fewer stains and more accurate edges without artifacts.Source: 3) Sensitivity to Noise · pp. 10–11, Table X · p. 11, Figure 10 · p. 11
4) Analysis of Different Control Factors:
- Overall, PSNR decreases as the decimal value of the control encoding increases, and the six default settings determine the reconstruction-performance extrema.The analysis compares decimal values converted to binary with corresponding PSNR values, including unseen settings.Source: 4) Analysis of Different Control Factors · p. 11, Figure 11 · p. 11
V. CONCLUSION
- The selector can still over-smooth background details, judge image information too coarsely, and adjust only over a small, discontinuous range.The authors identify these as limitations and propose future work on sampling rates, other network structures, and video applications.Source: §V · pp. 11–12