Source-linked AI summary
Fast and Accurate Image Super Resolution by Deep CNN with Skip Connection and Network in Network
Jin Yamanaka, Shigesumi Kuwashima, Takio Kurita
TL;DR
Deep CNNs improve SISR reconstruction but can require too much computation for edge devices. DCSCN combines optimized feature extraction, skip connections, residual learning, and Network in Network reconstruction to reduce cost while preserving accuracy. The paper reports state-of-the-art performance with less computation resources.
Problem
Deep SISR models achieve strong reconstruction but demand substantial computation and processing time, limiting suitability for mobile, tablet, and IoT edge devices.
Method
DCSCN processes original images with optimized CNN feature extraction, skip connections for local and global features, residual learning, and parallelized 1x1 CNN reconstruction.
Results
DCSCN has either best or second-best PSNR performance across evaluated datasets and achieves state-of-the-art performance with less computation resources.
Takeaways & Limitations
The model is designed as a smaller, computationally efficient SISR network for settings where reconstruction accuracy and resource use both matter.
Takeaways & Limitations
The model structure and hyperparameters were developed through trial and error, leaving a need for methods that design suitable complexity for each problem.
Abstract
from arXiv · showhide
We propose a highly efficient and faster Single Image Super-Resolution (SISR) model with Deep Convolutional neural networks (Deep CNN). Deep CNN have recently shown that they have a significant reconstruction performance on single-image super-resolution. Current trend is using deeper CNN layers to improve performance. However, deep models demand larger computation resources and is not suitable for network edge devices like mobile, tablet and IoT devices. Our model achieves state of the art reconstruction performance with at least 10 times lower calculation cost by Deep CNN with Residual Net, Skip Connection and Network in Network (DCSCN). A combination of Deep CNNs and Skip connection layers is used as a feature extractor for image features on both local and global area. Parallelized 1x1 CNNs, like the one called Network in Network, is also used for image reconstruction. That structure reduces the dimensions of the previous layer's output for faster computation with less information loss, and make it possible to process original images directly. Also we optimize the number of layers and filters of each CNN to significantly reduce the calculation cost. Thus, the proposed algorithm not only achieves the state of the art performance but also achieves faster and efficient computation. Code is available at https://github.com/jiny2001/dcscn-super-resolution
1 Introduction
SISR is increasingly needed as display resolutions rise, but deep CNN methods achieve strong reconstruction at substantial computational cost. DCSCN targets this trade-off with a lighter architecture that processes original images and uses skip connections and 1x1 CNNs.
- SISR is increasingly used for high-resolution displays, although source images may be two to eight times lower resolution and network bandwidth may be limited.
- Deep CNNs capture local and global image structures through cascaded CNNs and nonlinear layers, supporting strong SISR reconstruction.
- DCSCN uses 11 layers, compared with 20 to 30 in recent state-of-the-art SISR models, while reducing CNN-filter computations by 10 to 100 times.
- The model extracts features from original images and sends them directly to reconstruction through skip connections, avoiding redundant processing of up-sampled pixels.
- Parallelized 1x1 CNNs reduce preceding-layer dimensions with less information loss and require 9 times less computation than 3x3 CNNs.
2 Related Work
Earlier SISR methods range from shallow networks to very deep residual or recursive architectures. Faster approaches reduce computation, but their reconstruction performance is reported as lower than that of deeply convolutional models.
- SRCNN used 2 to 4 CNN layers and found larger CNN filters preferable to greater depth, while DRCN used 20 weight-sharing CNN layers.
- VDSR applies residual learning and gradient clipping to significantly speed training, while RED uses symmetric convolutional encoder and deconvolutional decoder layers with skip connections.
- RAISR and FSRCNN are reported as 10 to 100 times faster than other state-of-the-art deep-learning methods, but with lower performance than deeply convolutional models.
3 Proposed Method
DCSCN is a fully convolutional SISR model that extracts local and global features from original images, forwards them through skip connections, and reconstructs residual details with parallelized 1x1 CNNs.
- 3.1 Model Overview: DCSCN grows CNN features through a feature-extraction network and forwards every hidden-layer output to reconstruction through skip connections.
- 3.1 Model Overview: The model processes original images directly instead of up-sampled inputs, reducing the need to compute features for each up-sampled pixel.
- 3.2 Feature Extraction Network: The feature-extraction network uses seven 3x3 CNN, bias, and Parametric ReLU units, with CNN filters decreasing from 96 to 32.
- 3.3 Image Reconstruction Network: Concatenated features enter 1x1 CNNs that reduce input dimensionality before generating high-resolution pixels.
- 3.3 Image Reconstruction Network: For scale factor s = 2, the final CNN outputs 4 channels, which are reshaped into an HR image and added to bicubic up-sampling as a residual output.
4 Experiments
The experiments train and test DCSCN on standard image datasets, compare PSNR-based accuracy with other SISR algorithms, and estimate computation complexity analytically. DCSCN is reported to achieve best or second-best accuracy with lower computational demand.
- 4.2 Training Setup: Training uses 1,164 augmented images from Yang et al. and the Berkeley Segmentation Dataset, while testing uses Set5, Set14, and BSDS100.
- 4.3 Comparisons with State-of-the-Art Methods: For 2x SISR, PSNR and SSIM comparisons report DCSCN as either the best- or second-best-performing algorithm across the evaluated datasets.
- 4.3 Comparisons with State-of-the-Art Methods: Computation complexity is estimated per pixel from CNN filter dimensions and other operation counts because implementations use different platforms and libraries.
- 4.3 Comparisons with State-of-the-Art Methods: Figure 4 compares Set14 reconstruction performance against computation complexity using DCSCN’s complexity as 1.00.
5 Conclusion and Future works
The proposed method combines optimized CNN feature extraction, skip connections, and Network in Network reconstruction to deliver state-of-the-art performance with fewer computation resources. Future work calls for better methods to tune model structure and complexity for network-edge SISR.
- Conclusion: The method sends local and global image features to reconstruction through skip connections and uses Network in Network for efficient reconstruction.The model is designed to process original-size images while reducing computation resources.
- Conclusion: State-of-the-art performance is achieved with less computation resources using the proposed architectural devices.
- Future works: SISR's growing use on mobile, tablet, and IoT devices makes small but effective models important.
- Future works: A better method is needed to design suitable model complexity and tune structure and hyperparameters for each problem.The current model was developed through numerous trial-and-error processes.
- Future works: The study also uses ensemble learning, combining small sets of Deep-Learning models to address real and complex problems with parallelizable computation.