Source-linked AI summary
Lightweight Image Super-Resolution with Information Multi-distillation Network
Zheng Hui, Xinbo Gao, Yunchu Yang, Xiumei Wang
TL;DR
Deep SISR models achieve strong results but can be impractical because of excessive computation, and arbitrary-scale super-resolution remains difficult. The paper proposes IMDN with multi-distillation, contrast-aware fusion, and adaptive cropping, reporting a favorable balance of visual quality, execution speed, and memory consumption.
Problem
Deep SISR models impose excessive convolutional costs, while super-resolution at arbitrary scale factors remains insufficiently solved for practical applications.
Method
IMDN combines cascaded information multi-distillation blocks, contrast-aware channel attention, and adaptive cropping for arbitrary-scale image super-resolution.
Results
Experiments show IMDN achieves a favorable balance among visual quality, execution speed, and memory consumption.
Takeaways & Limitations
IMDN is designed for lightweight and accurate SISR with practical constraints on inference speed and memory occupation.
Takeaways & Limitations
The complexity analysis assumes that the HR image has size m × m.
Abstract
from arXiv · showhide
In recent years, single image super-resolution (SISR) methods using deep convolution neural network (CNN) have achieved impressive results. Thanks to the powerful representation capabilities of the deep networks, numerous previous ways can learn the complex non-linear mapping between low-resolution (LR) image patches and their high-resolution (HR) versions. However, excessive convolutions will limit the application of super-resolution technology in low computing power devices. Besides, super-resolution of any arbitrary scale factor is a critical issue in practical applications, which has not been well solved in the previous approaches. To address these issues, we propose a lightweight information multi-distillation network (IMDN) by constructing the cascaded information multi-distillation blocks (IMDB), which contains distillation and selective fusion parts. Specifically, the distillation module extracts hierarchical features step-by-step, and fusion module aggregates them according to the importance of candidate features, which is evaluated by the proposed contrast-aware channel attention mechanism. To process real images with any sizes, we develop an adaptive cropping strategy (ACS) to super-resolve block-wise image patches using the same well-trained model. Extensive experiments suggest that the proposed method performs favorably against the state-of-the-art SR algorithms in term of visual quality, memory footprint, and inference time. Code is available at \url{https://github.com/Zheng222/IMDN}.
1 INTRODUCTION
SISR seeks to reconstruct HR images from LR observations, but deeper convolutional models impose substantial parameter, memory, and speed costs. IMDN addresses this trade-off with multi-distillation, contrast-aware fusion, and adaptive cropping for arbitrary scale factors.
- SISR reconstructs a high-resolution image from a low-resolution observation, an ill-posed task because multiple HR images can yield the same LR image.
- Increasing convolutional depth improves SR quality but produces large models, memory footprints, and slow training and testing speeds.EDSR has about 43M parameters and 69 layers, while RDN has about 22M parameters and over 128 layers.
- Previous single-model solutions for ×2, ×3, and ×4 SR resized images before fully convolutional processing, substantially increasing computation.
- IMDN uses information multi-distillation blocks to retain partial features while progressively processing other features, then aggregates refined information with contrast-aware channel attention.The design targets useful restoration features such as edges, corners, and textures.
- Adaptive cropping enables a single lightweight model with downsampling layers to process images at arbitrary scale factors and sizes.The strategy obtains appropriately sized patches after scaling the input image to the target size.
- Experiments report a balance among visual quality, inference speed, and memory occupation, while network depth is identified as an important factor affecting execution speed.
2 RELATED WORK
Related work progressed from early direct LR-to-HR mapping toward deeper, recursive, and lightweight architectures, while attention mechanisms were introduced to emphasize useful feature information.
- SRCNN established a direct relationship between LR and HR images with a three-layer network.
- Lightweight SR methods used hierarchical feature separation and cascading residual mechanisms to improve parameter efficiency and performance.Information distillation separates current feature maps, while cascading mechanisms strengthen residual networks.
- Attention modules model channel or spatial relationships to enhance feature representations, but average pooling can omit structures, textures, and edges important for SR details.
3 METHOD
IMDN combines progressive feature distillation, contrast-aware channel attention, and adaptive cropping to target efficient super-resolution across image sizes and scale factors.
- IMDN framework: IMDN maps LR inputs to SR outputs using a convolutional feature extractor, stacked IMDBs, and a lightweight final upsampler.The upsampler uses a 3 × 3 convolution and sub-pixel convolution, while training uses MAE loss.
- Information multi-distillation block: Each IMDB progressively splits features, preserves refined channels, and forwards coarse channels through subsequent refinement steps.The block uses residual connections and a 1 × 1 convolution to reduce feature channels.
- Information multi-distillation block: Refined features from all distillation steps are concatenated along the channel dimension before subsequent fusion.The intermediate information collection scheme is intended to preserve collected information with few additional parameters.
- Contrast-aware channel attention: Contrast-aware channel attention evaluates feature-map contrast using mean and standard deviation rather than global average pooling.The module is designed for low-level vision tasks such as image super-resolution, where structures, textures, and edges matter.
- Adaptive cropping strategy: Adaptive cropping enables IMDN_AS to process arbitrary-size images and arbitrary scale factors with one model by partitioning images into four patches.The patches are processed in parallel and pasted back, while extra increments are discarded; images not divisible by 4 require this handling.
4.1 Datasets and metrics
The experiments use DIV2K for training and five standard benchmarks for evaluation, with PSNR and SSIM measured on the luminance channel. RealSR provides paired real LR-HR images for arbitrary-scale experiments.
- DIV2K supplies 800 high-quality RGB training images for the image-restoration experiments.
- Set5, Set14, BSD100, Urban100, and Manga109 are used as benchmark evaluation datasets.
- PSNR and SSIM evaluate super-resolved images, with values calculated on the luminance Y channel converted from RGB.
- RealSR contains 60 training and 20 validation real LR-HR image pairs, whose images have the same spatial size.
4.2 Implementation details
Training constructs bicubic LR images at ×2, ×3, and ×4, randomly crops HR patches, augments them, and optimizes the model with ADAM.
- Bicubic interpolation generates LR DIV2K images at scaling factors ×2, ×3, and ×4.
- Training uses randomly cropped 192 × 192 HR patches, mini-batches of 16, horizontal flips, and 90-degree rotations.
- The model is trained with ADAM using β1 = 0.9 and an initial learning rate of 2 × 10^-4.
4.3 Model analysis
Model analysis examines parameter efficiency, CCA and IIC ablations, and ACS. IMDN achieves a favorable performance-size trade-off, while ACS improves image quality, speed, and footprint for arbitrary-scale SR.
- 4.3 Model analysis: IMDN achieves comparative or better performance than several state-of-the-art models with fewer parameters and a better performance-size trade-off.
- 4.3.1 Model parameters: CCA improves ×4 Manga109 performance by 0.09dB PSNR and 0.0012 SSIM while adding 2K parameters, a 0.4% increase.
- 4.3.1 Model parameters: Replacing PRM with three cascaded 3 × 3 convolutions increases parameters to 510K but lowers performance relative to the 480K-parameter IMDN_basic_B4.
- 4.3.3 Investigation of ACS: IMDN_AS achieves better image quality, execution speed, and footprint than VDSR on the RealSR RGB validation dataset.
4.4 Comparison with state-of-the-arts
IMDN is compared with 11 state-of-the-art methods across ×2, ×3, and ×4 super-resolution settings. It performs favorably on most datasets, especially at ×2, and recovers grid structures more effectively in a reported Urban100 example.
- IMDN performs favorably against 11 compared methods on most datasets for ×2, ×3, and ×4 super-resolution, especially at ×2.
- In the Urban100 “img_67” example, IMDN recovers the grid structure better than the other compared methods.
4.5 Running time
IMDN is evaluated for computational efficiency through parameter, memory, and inference-time analyses. The results indicate favorable efficiency trade-offs, including comparable execution time with the best PSNR on Set5 ×4.
- Complexity analysis: The computational-cost analysis estimates parameter counts and time complexity from layer dimensions, filter sizes, channels, and output feature-map sizes.The analysis assumes an HR image size of m × m and uses Equation 10 to calculate computational costs.
- Evaluation setup: The evaluation compares average PSNR/SSIM across scale factors ×2, ×3, and ×4 on Set5, Set14, BSD100, Urban100, and Manga109.The reported efficiency tables cover memory consumption, average inference time, and computational costs.
- Running time: IMDN achieves dominant performance in memory usage and time consumption.Running time is measured using official implementations in a feed-forward process, and execution time is related to network depth.
- Running time: On Set5 ×4, IMDN has comparable execution time and the best PSNR among the compared approaches.The comparison is presented as a trade-off between running time and performance.
5 CONCLUSION
The paper concludes that IMDN combines lightweight, accurate single-image super-resolution with practical efficiency. Its IMDB and contrast-aware channel attention refine hierarchical features, while ACS supports arbitrary scale factors and real-image sizes.
- Conclusion: IMDN is proposed as a lightweight and accurate network for single-image super-resolution.The conclusion frames the method around balancing visual quality, execution speed, and memory consumption.
- Conclusion: A progressive refinement module extracts hierarchical features step-by-step and works with contrast-aware channel attention to improve super-resolution performance.The conclusion states that performance is significantly and steadily improved through this cooperation.
- Conclusion: The adaptive cropping strategy addresses super-resolution with arbitrary scale factors for practical application scenarios.The strategy is presented as a solution to an issue considered critical for applying super-resolution in actual scenes.
- Conclusion: Experiments show a balance among visual quality, execution speed, and memory consumption.The paper identifies these factors as affecting practical use.