Source-linked AI summary
Blueprint Separable Residual Network for Efficient Image Super-Resolution
Zheyuan Li, Yingqi Liu, Xiangyu Chen, Haoming Cai, Jinjin Gu, Yu Qiao, Chao Dong
TL;DR
Single-image super-resolution models achieve strong quality but often require too much computation for edge and real-time use. BSRN combines blueprint separable convolution with lightweight attention modules to improve efficiency, achieving state-of-the-art performance among efficient SR methods; BSRN-S won first place in NTIRE 2022's model complexity track.
Problem
Deep-learning SR networks improve image quality but their large capacity and intensive computation limit real-world and real-time applications.
Method
BSRN replaces standard convolutional building blocks with BSConv and uses effective attention modules, including ESA and CCA, to improve efficiency and model ability.
Results
BSRN achieves state-of-the-art performance among efficient SR methods, while BSRN-S wins first place in the NTIRE 2022 model complexity track.
Takeaways & Limitations
BSRN demonstrates that efficient convolution and attention designs can support competitive SR performance with fewer parameters and Multi-Adds.
Abstract
from arXiv · showhide
Recent advances in single image super-resolution (SISR) have achieved extraordinary performance, but the computational cost is too heavy to apply in edge devices. To alleviate this problem, many novel and effective solutions have been proposed. Convolutional neural network (CNN) with the attention mechanism has attracted increasing attention due to its efficiency and effectiveness. However, there is still redundancy in the convolution operation. In this paper, we propose Blueprint Separable Residual Network (BSRN) containing two efficient designs. One is the usage of blueprint separable convolution (BSConv), which takes place of the redundant convolution operation. The other is to enhance the model ability by introducing more effective attention modules. The experimental results show that BSRN achieves state-of-the-art performance among existing efficient SR methods. Moreover, a smaller variant of our model BSRN-S won the first place in model complexity track of NTIRE 2022 Efficient SR Challenge. The code is available at https://github.com/xiaom233/BSRN.
1. Introduction
Single-image super-resolution has improved restoration quality through deep learning, but computationally intensive models limit real-world and real-time use. BSRN addresses remaining convolutional redundancy with BSConv and effective attention modules.
- Deep learning has improved SR image quality, but large model capacity and intensive computation restrict real-world and real-time applications.
- Lightweight SR methods reduce cost through parameter sharing, grouped convolution, distillation, and attention, yet convolution operations remain redundant.
- BSRN improves efficiency by optimizing convolutional operations and introducing effective attention modules.
- BSConv replaces standard convolutional building blocks and better exploits intra-kernel correlations for efficient separation.
- BSRN introduces enhanced spatial attention and contrast-aware channel attention to improve model ability with efficient designs.
- BSRN achieves state-of-the-art performance among existing efficiency-oriented SR networks, while BSRN-S wins first place in the NTIRE 2022 model complexity track.
2. Related Work
Related work spans deep-learning approaches that improve SR quality and model-compression techniques that reduce computation and redundancy. Efficient SR methods use architectural and feature-processing strategies, but practical efficiency remains a central concern.
- Deep SR research improved performance through deeper networks, dense connections, and other representation-enhancing architectures.
- Model compression methods include pruning and quantization, low-rank factorization, knowledge distillation, and transferred or compact convolutional filters.
- Efficient SR methods reduce computation through LR-space processing, grouped convolution, information distillation, parameter sharing, and related strategies.
3. Method
BSRN builds an efficient super-resolution network around a four-stage architecture, separable distillation blocks, blueprint separable convolutions, and attention modules for spatial and channel-wise enhancement.
- Network Architecture: BSRN follows four stages: shallow feature extraction, deep feature extraction, multi-layer feature fusion, and reconstruction.The input image is replicated and concatenated before feature extraction; a long skip connection supports residual learning.
- Network Architecture: BSConv performs shallow feature extraction and consists of a 1 × 1 convolution followed by a depth-wise convolution.The same decomposition replaces standard convolutions in the blueprint shallow residual block.
- Efficient Separable Distillation Block: ESDB comprises feature distillation, feature condensation, and feature enhancement stages.Distilled features are concatenated and condensed by a 1 × 1 convolution before attention-based enhancement.
- Efficient Separable Distillation Block: The enhancement stage applies ESA and CCA to strengthen spatial and channel-wise representations, respectively.ESA generates an attention matrix, while CCA uses channel-wise mean and standard-deviation information to calculate attention weights.
- Efficient Separable Distillation Block: BSRB combines BSConv, an identity connection, and GELU activation as the basic module of ESDB.BSConv factorizes standard convolution into point-wise 1 × 1 and depth-wise convolutions; GELU is reported to perform better than ReLU and LeakyReLU.
4. Experiments
The experiments evaluate BSRN on standard super-resolution benchmarks and study convolution decomposition, attention modules, and overall efficiency-oriented comparisons. Results indicate that BSConvU is the strongest tested decomposition, while ESA and CCA improve model performance.
- Experimental Setup: Training uses 3450 images from Flickr2K and DIV2K, while evaluation covers Set5, Set14, B100, Urban100, and Manga109 using Y-channel PSNR and SSIM.The training set combines 2650 Flickr2K images with 800 DIV2K images.
- Benchmark Comparison: Table 5 compares state-of-the-art methods on benchmark datasets, marking the best and second-best results and reporting Multi-Adds for a 1280 × 720 ground-truth image.The caption specifies the comparison conventions and computational-cost measurement.
- Convolution Decomposition: Convolution decomposition produces significant computation reductions but apparent performance drops across the tested strategies.The comparison is conducted using RFDN-based experiments.
- Convolution Decomposition: BSConvU performs best among DSConv, BSConvU, and BSConvS, so the authors select it for BSRN.DSConv denotes the original depth-wise separable convolution, while BSConvU and BSConvS are two BSConv variants.
BSRN(Ours)
BSRN combines efficient convolutional design with attention modules to improve lightweight super-resolution. It reports strong performance and efficiency, while BSRN-S achieves first place in the NTIRE 2022 model-complexity track.
- Ablation Study: Ablations show that ESA and CCA improve BSRN, with the complete model gaining 0.5dB over BSRN without CCA on Set5, Urban100, and Manga109.
- Activation Functions: GELU is selected because it provides a remarkable performance gain, especially on Urban100, compared with the investigated activation functions.
- Comparison with State-of-the-art Methods: BSRN achieves the best performance among lightweight SR methods with 332K–352K parameters and almost the fewest Multi-Adds.The comparison covers multiple state-of-the-art lightweight approaches and upscale factors.
- Comparison with State-of-the-art Methods: BSRN-S achieves competitive ×4 SR performance with 156K parameters and 8.3G Multi-Adds.
- BSRN-S for NTIRE2022 Challenge: BSRN-S wins first place in the NTIRE 2022 model-complexity track, ranked by the summed ranks of parameter count and FLOPs.
- BSRN-S for NTIRE2022 Challenge: BSRN-S has the fewest parameters and second-fewest FLOPs among competing solutions, while optimized runtime is similar to IMDN and RFDN on the same GPU.The authors note that depth-wise convolutions are relatively unfriendly to GPU execution, increasing runtime.
5. Conclusions
The conclusion presents BSRN as a lightweight SR network that combines blueprint separable convolution with attention enhancements. Extensive experiments report strong efficiency and performance, and BSRN-S wins the NTIRE 2022 model-complexity track.
- 5. Conclusions: BSRN replaces standard convolution in RFDN’s shallow residual block with BSConv to form a more efficient blueprint shallow residual block.
- 5. Conclusions: BSRN uses ECA and CCA blocks to enhance the model’s representative ability.
- 5. Conclusions: Experiments report the best performance with fewer parameters and Multi-Adds than state-of-the-art efficient SR methods.
- 5. Conclusions: BSRN-S wins first place in the model-complexity track of the NTIRE 2022 efficient super-resolution challenge.