PaperlayerView exact revision on arXiv

Source-linked AI summary

Feature Distillation Interaction Weighting Network for Lightweight Image Super-Resolution

Guangwei Gao, Wenjie Li, Juncheng Li, Fei Wu, Huimin Lu, Yi Yu

arXiv:2112.08655v2cs.CVeess.IV

TL;DR

Lightweight SISR aims to reconstruct high-resolution images with fewer parameters and calculations, but CNN-based methods can be costly and intermediate features are difficult to use fully. FDIWN combines feature-distillation and interaction components, and benchmark evaluation reports a balance among model size, performance, and computational cost.

  • Problem

    Lightweight SISR seeks to reconstruct SR images from LR images with fewer parameters and calculations, while computationally costly methods are difficult to apply on mobile devices.

    Source: Introduction · p. 1

  • Method

    FDIWN uses FSWG groups of interacting WDIBs with WIRW and WCRW units, plus WRDC and SCF to distill and interact features at different scales.

    Source: Introduction · pp. 1–2, Wide-Residual Distillation Connection · p. 3

  • Results

    Benchmark evaluation reports that FDIWN achieves a good balance between model size, performance, and computational cost.

    Source: Conclusions · p. 8

  • Takeaways & Limitations

    The authors report that interacting contextual and intermediate features benefits high-quality SR image reconstruction.

    Source: Conclusions · p. 8

Abstract

from arXiv · show

Convolutional neural networks based single-image super-resolution (SISR) has made great progress in recent years. However, it is difficult to apply these methods to real-world scenarios due to the computational and memory cost. Meanwhile, how to take full advantage of the intermediate features under the constraints of limited parameters and calculations is also a huge challenge. To alleviate these issues, we propose a lightweight yet efficient Feature Distillation Interaction Weighted Network (FDIWN). Specifically, FDIWN utilizes a series of specially designed Feature Shuffle Weighted Groups (FSWG) as the backbone, and several novel mutual Wide-residual Distillation Interaction Blocks (WDIB) form an FSWG. In addition, Wide Identical Residual Weighting (WIRW) units and Wide Convolutional Residual Weighting (WCRW) units are introduced into WDIB for better feature distillation. Moreover, a Wide-Residual Distillation Connection (WRDC) framework and a Self-Calibration Fusion (SCF) unit are proposed to interact features with different scales more flexibly and efficiently.Extensive experiments show that our FDIWN is superior to other models to strike a good balance between model performance and efficiency. The code is available at https://github.com/IVIPLab/FDIWN.

Introduction

Lightweight SISR seeks high-quality reconstruction with fewer parameters and calculations, but deeper networks can be costly for mobile devices and may lose intermediate information. FDIWN addresses these challenges with feature-distillation and interaction components.

Introduction

  • FDIWN is reported to use less computation than most existing lightweight SISR models without sacrificing performance.The introduction motivates the network as a response to the difficulty of deploying costly models on mobile devices.Source: Introduction · p. 1
  • WDIBs enable pairwise feature fusion in FSWG, followed by feature shuffling and weighting to better use intermediate features at a small additional computational cost.WIRW and WCRW units are used in WDIB; SCF integrates features from different levels through jump splicing.Source: Introduction · p. 1
  • The proposed components include WIRW and WCRW units for feature distillation, SCF for feature interaction and fusion, WRDC for cross-scale interaction, and FSWG as FDIWN’s basic component.The authors state that the wide-residual units have stronger feature-distillation capabilities than ordinary residual blocks.Source: Introduction · p. 2

Related Work

Prior lightweight SISR work reduces model complexity, while the cited literature also explores residual connections, wider activations, and attention for feature extraction. The authors identify intermediate-feature use as a remaining concern and investigate attention in a modified wide-activation residual block.

Lightweight SISR

  • Existing lightweight SISR models use approaches including group convolution, information distillation, skip connections, and pruning, but the authors say they often underuse intermediate features.The text connects this shortcoming to sub-optimal performance.Source: Lightweight SISR · p. 2

Wide-Residual Attention Weighting Learning

  • Prior SISR networks use skip connections and concatenation to reuse shallow features, while WDSR expands features before ReLU to pass more information without additional parameters.The cited discussion also notes the use of attention mechanisms for feature extraction.Source: Wide-Residual Attention Weighting… · p. 2
  • The authors introduce second-order attention into a modified wide-activation residual block to further improve feature extraction.Source: Wide-Residual Attention Weighting… · p. 3

Proposed Method

FDIWN combines feature-distillation and interaction modules to build a lightweight SISR model. Its design passes features through grouped interaction blocks, distills them across scales, and fuses them for reconstruction.

Proposed Method

  • FDIWN combines WDIB, WRDC, WIRW, WCRW, and SCF to focus on intermediate features and their interaction in a lightweight, accurate SISR model.The proposed network uses these modules to address feature use under parameter and calculation constraints.Source: Proposed Method · p. 3

Feature Distillation Interaction Weighting Network

  • Shallow features from the LR image feed a sequence of FSWGs with jump connections, and deep features are combined with LR-image features for SR reconstruction.The network has shallow feature extraction, nonlinear deep feature acquisition, and upsampling recovery parts.Source: Feature Distillation Interaction… · p. 3

Wide-Residual Distillation Connection

  • WRDC comprises WIRW, WCRW, and a distillation jump connection; wide activation in WIRW and WCRW enables distillation of richer features with fewer parameters, while Shuffle Attention further enhances feature extraction.WCRW adds a 3 × 3 convolution on its shortcut to match the original input size and enable interaction between different features. The distillation connection multiplies low-frequency attention features refined by the wide-residual unit with fine feature maps.Source: Wide-Residual Distillation Connection · pp. 3–4

Wide-Residual Distillation Interaction Block

  • WDIB uses a butterfly structure based on WRDC to interact intermediate features, with combination coefficients linking its upper and lower branches.Its SCF first fuses the branch features, then interacts and fuses features at different scales; adaptive weighting calibrates the fused features.Source: Wide-Residual Distillation Interaction… · p. 4

Feature Shuffle Weighted Group

  • Each FSWG contains three interactional WDIBs, fusing and shuffling their features before adaptively combining the interacted features with primary features.The group uses channel shuffle and group convolution, with larger groups set to reduce the parameter burden.Source: Feature Shuffle Weighted Group · p. 5

Experiments

Experiments evaluate FDIWN on standard SISR benchmarks and use ablations to assess its modules and efficiency trade-offs. The reported comparisons show competitive performance while balancing model size, computation, and inference speed.

Datasets and Evaluation Metrics

Ablation Studies

  • FDIWN improves Baseline1 by 0.12 dB; with the DC mechanism, PSNR rises from 37.53 dB to 37.59 dB for 6K additional parameters, while SCF adds 0.06 dB.Baseline1 replaces WRDC with a three-layer cascaded 3 × 3 convolution plus ReLU layer and replaces SCF with concatenation.Source: Ablation Studies · p. 5
  • Compared with basic residual blocks, WIRW and WCRW achieve better performance with fewer parameters and computational costs; increasing their channel count from 48 to 120 further improves performance.The study evaluates the wide-residual units against a basic residual-block baseline.Source: Ablation Studies · p. 5, Table 2 · p. 5
  • As WIRW and WCRW units increase, feature visualizations show more edge detail, indicating improved capture of high-frequency information.The visual comparison uses a network with one 3 × 3 convolution and varying numbers of wide-residual units.Source: Ablation Studies · pp. 5–6, Figure 5 · p. 6
  • The WDIB information-blending structure raises PSNR from 37.81 dB to 37.85 dB versus three cascaded WDIBs without interaction, and the WIRW long skip further improves performance.The comparison tests block interaction against simple cascading.Source: Ablation Studies · p. 6, Table 3 · p. 6
  • More FSWGs and WDIBs generally improve performance, but PSNR does not increase from three to four WDIBs; the final model uses six FSWGs and three WDIBs for a balance of performance, size, and computation.FDIWN is reported as not the fastest model, but its execution time is acceptable and its multi-adds comparison achieves the best balance.Source: Ablation Studies · p. 6, Figure 6 · p. 6, Figure 7 · p. 7

Comparisons with State-of-the-Art Methods

  • FDIWN-M and FDIWN perform competitively against state-of-the-art methods, with similar-sized models performing worse and similarly effective models using more parameters.The comparison covers model size, performance, and computational cost.Source: Comparisons with State-of-the-Art… · p. 6, Table 4 · p. 7

Conclusions

FDIWN reduces computational overhead while improving SR performance, and benchmark evaluations indicate a balance among model size, performance, and computational cost. Its weighting, distillation, and fusion components are presented as ways to improve feature use and interaction.

Conclusions

  • WIRW and WCRW distill useful features more strongly than ordinary residual blocks, while shuffle attention focuses feature extraction on key information.Source: Conclusions · p. 7
  • Figure 9 presents visual comparisons on the Urban100 dataset.Source: Figure 9 · p. 8
  • FDIWN reduces computational overhead while improving SR performance, with benchmark evaluations indicating a balance among model size, performance, and computational cost.Source: Conclusions · pp. 7–8, Figure 8 · p. 7
  • WRDC and SCF aggregate and distill representative features, allowing features from different scales to interact efficiently.Source: Conclusions · pp. 7–8
Loading 2112.08655v2…