Source-linked AI summary
ISTA-Net++: Flexible Deep Unfolding Network for Compressive Sensing
Di You, Jingfen Xie, Jian Zhang
TL;DR
Image compressive-sensing networks often lack flexibility across measurement ratios and scenes. ISTA-Net++ combines dynamic unfolding, cross-block processing, and balanced training; the paper reports the highest average PSNR results among the compared methods and positions the model as a practical research baseline.
Problem
Many existing CS networks require separate models for different measurement ratios and show unbalanced reconstruction across scenes, including weaker results on rarely seen scenarios.
Source: §1 · p. 2Method
ISTA-Net++ unfolds ISTA with ratio-conditioned dynamic stages, adds cross-block processing, and uses a more balanced training dataset.
Source: §1 · p. 2Results
ISTA-Net++ achieves the highest average PSNR results, with gains over ISTA-Net+ of 0.95/1.19/1.74/1.03 dB on BSD68/Set11/Urban100/DIV2K validation, respectively.
Source: §3.3 · p. 4Takeaways & Limitations
The authors present ISTA-Net++ as a suitable practical method and potential baseline for future CS research.
Source: §5 · p. 6
Abstract
from arXiv · showhide
While deep neural networks have achieved impressive success in image compressive sensing (CS), most of them lack flexibility when dealing with multi-ratio tasks and multi-scene images in practical applications. To tackle these challenges, we propose a novel end-to-end flexible ISTA-unfolding deep network, dubbed ISTA-Net++, with superior performance and strong flexibility. Specifically, by developing a dynamic unfolding strategy, our model enjoys the adaptability of handling CS problems with different ratios, i.e., multi-ratio tasks, through a single model. A cross-block strategy is further utilized to reduce blocking artifacts and enhance the CS recovery quality. Furthermore, we adopt a balanced dataset for training, which brings more robustness when reconstructing images of multiple scenes. Extensive experiments on four datasets show that ISTA-Net++ achieves state-of-the-art results in terms of both quantitative metrics and visual quality. Considering its flexibility, effectiveness and practicability, our model is expected to serve as a suitable baseline in future CS research. The source code is available on https://github.com/jianzhangcs/ISTA-Netpp.
1. INTRODUCTION
Existing compressive-sensing networks are limited in handling multiple measurement ratios and diverse image scenes. ISTA-Net++ combines dynamic unfolding, cross-block processing, and balanced training to address these challenges.
1. INTRODUCTION
- Compressive sensing seeks to infer an image from randomized linear-projection measurements, an ill-posed problem when the measurement count is much smaller than the image dimension.The paper denotes the measurement-to-dimension ratio by γ.Source: §1 · p. 1
- Deep unfolding combines model- and data-driven approaches; ISTA-Net+ maps ISTA updates to a fixed number of network layers, each corresponding to an iteration.ISTA is described as a first-order proximal method that iterates between update steps.Source: §1 · pp. 1–2
- Different CS ratios often require separate models, increasing storage and time, while imbalanced Train91 data is linked to weaker recovery on rarely seen scenes.The paper attributes scene imbalance to the quantity distribution of scenes in Train91.Source: §1 · p. 2
- ISTA-Net++ uses a ratio-conditioned dynamic unfolding strategy for one model, cross-block processing to alleviate blocking artifacts, and balanced training for greater robustness across scenes.The condition module transmits the input ratio to each stage.Source: §1 · p. 2
- The paper presents ISTA-Net++ as a flexible deep-unfolding network and proposes it as a potential baseline for future CS research.Its contributions emphasize flexibility across ratios and scenes, performance, and practicability.Source: §1 · p. 2
2. APPROACH
ISTA-Net++ separates compressive-sensing sampling, initialization, and image recovery. Its unfolded reconstruction uses ratio-conditioned modules, and its training objective reduces reconstruction error.
2.1.1. Sampling and Initialization
- Sampling is implemented as convolution, while initialization applies a convolution and pixel shuffle to address dimensional mismatch and accommodate different CS ratios.Source: §2.1.1 · p. 2
2.1.2. Reconstruction
- The reconstruction unfolds ISTA for K stages, alternating DGDM and DPMM; ratio-conditioned parameters let DGDM vary with CS ratio, while DPMM accepts a noise-level map.Source: §2.1.2 · pp. 2–3
2.1. Architecture Design of ISTA-Net++
- The framework separates CS sampling, dimensionality-mismatch initialization, and image reconstruction into three parts.Source: §2.1 · p. 2, Figure 2 · p. 3
2.2. Network Parameters and Loss Function Xi ND
- Training reduces the discrepancy between each original image and its reconstruction, with learnable parameters in the condition module, DGDM, and DPMM.Source: §2.2 · p. 3
3. EXPERIMENTS
ISTA-Net++ is evaluated against ISTA-Net+ and other CS methods, with experiments examining recovery quality, model flexibility, and training setup. It reports higher average PSNR than ISTA-Net+ across four datasets.
3.2. ISTA-Net+ vs. ISTA-Net++
- With K=9/20, ISTA-Net++ gains 0.88/1.46 dB over ISTA-Net+ while using fewer parameters.The authors set the default to 20 stages and 32 channels for subsequent experiments.Source: §3.2 · p. 4
- Train91’s representative images are mostly similar plant scenes, while Fig. 5 also shows training images from DIV2K and Train400.The training setup uses 400 images of size 180×180.Source: Figure 5 · p. 4, §3.1 · p. 4
3.3. Comparison with State-of-the-Art Methods
- Other end-to-end deep CS methods require five separately trained models for five ratios, whereas ISTA-Net++ is trained once using five sampling matrices.The experiments use Set11, BSD68, Urban100, and DIV2K validation for testing.Source: §3.3 · p. 4, §3.1 · p. 4
- ISTA-Net++ gains 0.95/1.19/1.74/1.03 dB over ISTA-Net+ on BSD68/Set11/Urban100/DIV2K validation, respectively.The paper also reports sharper edges and more recovered details than competing methods.Source: §3.3 · pp. 4–5
4. ANALYSIS
The analysis examines training-scene balance and the effects of ISTA-Net++’s DUS and CBS components. Its results favor Train400 for varied scenes and show gains from both components.
4.1. Analysis on Robustness for Various Scenes
- Train400 reaches 34.81 dB versus Train91’s 34.25 dB after 400 epochs, and larger combined datasets bring only negligible improvements.The analysis concludes that scene balance has a greater impact on performance gain than training-set size.Source: §4.1 · p. 5
- On a house scene, Train91-trained ISTA-Net++ performs 0.97 dB worse than the Train400-trained model, while both perform similarly on a plant scene.The house scene is described as rarely seen in Train91.Source: §4.1 · p. 5
4.2. Ablation Study and Discussion
- DUS with its condition module yields an average 0.26 dB gain over the version without the module across three Set11 CS ratios.The comparison uses the same five sampling matrices.Source: §4.2 · p. 5
- CBS improves PSNR across all tested ratios, with the largest reported gain reaching 0.64 dB.Table 4 compares settings with and without CBS.Source: §4.2 · p. 6, Table 4 · p. 6
5. CONCLUSIONS
The conclusion presents ISTA-Net++ as a flexible, high-performing CS reconstruction network, combining dynamic unfolding, balanced training data, and a cross-block strategy.
5. CONCLUSIONS
- The authors propose ISTA-Net++ as a suitable baseline for future CS research based on its flexibility, effectiveness, and practicability.They associate dynamic unfolding and balanced training data with multi-ratio flexibility and robustness across scenes, and say CBS alleviates blocking artifacts.Source: §5 · p. 6