Source-linked AI summary
Wide Activation for Efficient and Accurate Image Super-Resolution
Jiahui Yu, Yuchen Fan, Jianchao Yang, Ning Xu, Zhaowen Wang, Xinchao Wang, Thomas Huang
TL;DR
SISR requires accurate recovery of high-resolution images, but deep SR networks must improve information flow and efficiency without increasing computational cost. The paper introduces WDSR wide activation, linear low-rank convolution, and weight normalization, reporting better accuracy under matched budgets and strong benchmark outcomes. WDSR achieves better DIV2K performance and wins all three realistic NTIRE 2018 SISR tracks.
Problem
The paper addresses how to improve information flow and accuracy in deep SISR while preserving parameter and computational efficiency.
Method
The paper constructs WDSR-A and WDSR-B with wider features before ReLU, uses linear low-rank convolution for further expansion, and trains deep networks with weight normalization.
Results
WDSR achieves better results on the DIV2K benchmark and wins first place in all three realistic NTIRE 2018 SISR tracks.
Takeaways & Limitations
With the same or lower computational complexity, wider activation and weight normalization improve the reported accuracy of deep SISR networks.
Takeaways & Limitations
For expansion ratios beyond the studied range, slimming the identity pathway too much dramatically deteriorates accuracy; batch normalization also creates training issues in small-patch, small-batch SR settings.
Abstract
from arXiv · showhide
In this report we demonstrate that with same parameters and computational budgets, models with wider features before ReLU activation have significantly better performance for single image super-resolution (SISR). The resulted SR residual network has a slim identity mapping pathway with wider (\(2\times\) to \(4\times\)) channels before activation in each residual block. To further widen activation (\(6\times\) to \(9\times\)) without computational overhead, we introduce linear low-rank convolution into SR networks and achieve even better accuracy-efficiency tradeoffs. In addition, compared with batch normalization or no normalization, we find training with weight normalization leads to better accuracy for deep super-resolution networks. Our proposed SR network \textit{WDSR} achieves better results on large-scale DIV2K image super-resolution benchmark in terms of PSNR with same or lower computational complexity. Based on WDSR, our method also won 1st places in NTIRE 2018 Challenge on Single Image Super-Resolution in all three realistic tracks. Experiments and ablation studies support the importance of wide activation for image super-resolution. Code is released at: https://github.com/JiahuiYu/wdsr_ntire2018
1 Introduction
The paper proposes wide activation for SISR, expanding features before ReLU while preserving computational efficiency. WDSR-A and WDSR-B combine this design with efficient convolutions and weight normalization to improve accuracy and benchmark performance.
- SISR reconstructs high-resolution images from low-resolution counterparts and supports applications including security, surveillance, satellite, and medical imaging.
- WDSR-A uses a slim identity pathway with 2× to 4× wider channels before activation in each residual block.
- Wider features before ReLU improve SISR without additional parameters or computation by allowing more information to pass through while retaining nonlinearity.
- WDSR-B uses linear low-rank convolution to enable 6× to 9× wider activation without additional parameters or computation, further boosting accuracy.
- Weight normalization provides better accuracy for deep SR networks than batch normalization or no normalization, while enabling faster convergence through higher learning rates.
- WDSR achieves better DIV2K results and wins first place in all three realistic NTIRE 2018 SISR tracks.
2 Related Work
Related work improves SISR through deeper networks, skip connections, efficient convolutions, and normalization strategies. The paper positions wide activation as an alternative approach to improving information flow and efficiency.
- Upsampling layers: Efficient upsampling methods move convolutional computation to low-resolution feature space or use pixel shuffling to reduce processing cost.
- Very deep and recursive neural networks: Deep SISR networks generally achieve higher accuracy than shallow networks but introduce over-parameterization and training difficulty.
- Skip connections: Skip connections and dense connectivity address the tension between deeper representations and the importance of low-level features in SISR.
- Normalization layers: Batch normalization is empirically reported to hinder SISR accuracy, leading recent SR networks to abandon it despite increasing training difficulty with depth.
- Parameter-efficient convolutions: Group convolution reduces parameters by dividing channels into groups, while depthwise separable convolution combines per-channel spatial filtering with pointwise convolution.
- Inverted residuals: Inverted residuals also expand features before activation, but the paper reports unsatisfactory performance for image SR with the cited design.
3 Proposed Methods
The proposed methods widen features before ReLU under fixed computational budgets, introduce efficient low-rank convolutions for further expansion, and use weight normalization to improve deep SR training and accuracy.
- Wide Activation: WDSR-A: WDSR-A slims the residual identity pathway while expanding features before activation, preserving parameter and computational budgets.The expansion factor is r, with residual-path width reduced by √r and activation width increased correspondingly.
- Wide Activation: WDSR-A: WDSR-A improves SISR accuracy when the expansion factor r is between 2 and 4, but performance drops quickly beyond that range.The drop is attributed to an overly slim identity mapping pathway; in the cited EDSR example, r beyond 6 makes it smaller than the HR representation space.
- Efficient Wider Activation: WDSR-B: WDSR-B keeps identity-pathway channels constant and uses linear low-rank convolution to expand features more efficiently.The factorized block combines a 1 × 1 channel-reduction convolution with a 3 × 3 spatial feature-extraction convolution; adding ReLU inside it reduces accuracy.
- Weight Normalization vs. Batch Normalization: Weight normalization avoids batch dependencies and train-test formulation differences while improving accuracy relative to batch normalization or no normalization.The paper describes weight normalization as a reparameterization that decouples weight-vector length from direction and reports that it permits a 10× higher learning rate.
- Network Structure: The network architecture simplifies EDSR by replacing its linear global residual pathway with one 5 × 5 convolution and extracting all features before upsampling.The authors report fewer parameters and computations without observed accuracy loss, and a large speed improvement from the low-resolution feature extraction design.
4 Experimental Results
Experiments on DIV2K compare WDSR variants with EDSR under controlled parameter budgets and examine normalization choices. Wider activation improves accuracy and efficiency, while weight normalization converges faster and achieves better accuracy than batch or no normalization.
- Wide and Efficient Wider Activation: Models are evaluated on DIV2K using validation PSNR, with parameter and computation budgets controlled by varying residual-block counts.The training set uses 800 images, while 10 validation images guide training and 100 validation images are used for evaluation.
- Wide and Efficient Wider Activation: WDSR-A and WDSR-B achieve better accuracy and efficiency than EDSR under matched parameter and computational budgets.WDSR-B performs better or similarly to WDSR-A, supporting the wide-activation hypothesis and the effectiveness of linear low-rank convolution.
- Normalization layers: Weight normalization produces faster convergence and better accuracy than batch normalization or no normalization for the same super-resolution model.The batch-normalized model is unstable during testing, likely because batch normalization uses different formulations during training and testing.
- Normalization layers: Even with lr = 10^-4, batch-normalized models retain unstable validation PSNR across training despite stable training curves.The experiment tests whether instability is caused by an overly large learning rate.
5 Conclusions
The report introduces WDSR-A and WDSR-B around wide activation and evaluates weight normalization for deep super-resolution networks. Experiments find that wider pre-ReLU features improve accuracy at matched complexity, while weight normalization outperforms batch or no normalization.
- Conclusions: WDSR-A and WDSR-B are super-resolution networks based on wide activation.The networks use wider features before ReLU activation as their central design principle.
- Conclusions: At the same parameter and computational complexity, wider features before ReLU activation achieve better single-image super-resolution accuracy.The conclusion concerns residual networks for SISR.
- Conclusions: Weight normalization leads to better accuracy than batch normalization or no normalization in deep super-resolution networks.This finding is reported as a comparison across the three normalization choices.