Source-linked AI summary
LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyond
Wenbo Li, Kun Zhou, Lu Qi, Nianjuan Jiang, Jiangbo Lu, Jiaya Jia
TL;DR
SISR must reconstruct HR images from LR inputs, while existing approaches face a trade-off between reconstruction quality, model complexity, and optimization difficulty. LAPAR instead regresses pixel-wise coefficients over a predefined filter dictionary, producing lightweight, efficiently optimized models. It achieves state-of-the-art results among lightweight SISR methods and extends to denoising and JPEG deblocking with strong performance.
Problem
SISR is ill-posed, and deep LR-HR mapping methods can be unstable and computationally demanding when handling unconstrained image content.
Method
LAPAR regresses pixel-wise linear-combination coefficients for predefined filter bases and applies the resulting pixel-adaptive filters to a bicubic-upsampled image.
Results
LAPAR achieves state-of-the-art results among lightweight SISR methods and also obtains strong performance on image denoising and JPEG deblocking.
Takeaways & Limitations
Constraining filters to a predefined linear space makes the learning procedure easy and fast to optimize while retaining spatial variation.
Abstract
from arXiv · showhide
Single image super-resolution (SISR) deals with a fundamental problem of upsampling a low-resolution (LR) image to its high-resolution (HR) version. Last few years have witnessed impressive progress propelled by deep learning methods. However, one critical challenge faced by existing methods is to strike a sweet spot of deep model complexity and resulting SISR quality. This paper addresses this pain point by proposing a linearly-assembled pixel-adaptive regression network (LAPAR), which casts the direct LR to HR mapping learning into a linear coefficient regression task over a dictionary of multiple predefined filter bases. Such a parametric representation renders our model highly lightweight and easy to optimize while achieving state-of-the-art results on SISR benchmarks. Moreover, based on the same idea, LAPAR is extended to tackle other restoration tasks, e.g., image denoising and JPEG image deblocking, and again, yields strong performance. The code is available at https://github.com/dvlab-research/Simple-SR.
1 Introduction
Single-image super-resolution reconstructs HR images from LR inputs, but simple interpolation smooths details while deep methods can be unstable and computationally demanding. LAPAR addresses this by learning pixel-adaptive filters assembled from predefined bases, achieving strong lightweight performance and extending to other restoration tasks.
- SISR reconstructs a high-resolution image from a low-resolution input, but nearest-neighbor, bilinear, and bicubic interpolation often smooth structures and details.
- Deep LR-HR mapping methods face training instability from unconstrained image content and require substantial computational resources.
- Adaptive filters estimate neighboring-pixel importance rather than absolute values and keep outputs within the convex hull of surrounding pixels, helping avoid visual artifacts.
- LAPAR learns pixel-adaptive enhancement filters as linear combinations of predefined filter bases applied to a bicubic-upsampled image.Its dictionary uses simple anisotropic Gaussian and difference-of-Gaussians kernels.
- LAPAR achieves state-of-the-art results among lightweight networks with the least parameters and MultiAdds, while also extending to denoising and JPEG deblocking.
2 Method
LAPAR addresses SISR by replacing direct LR-to-HR estimation with pixel-adaptive filters assembled from predefined bases, preserving a lightweight and optimizable linear structure.
- Framework: The framework regresses linear combination coefficients, assembles pixel-adaptive filters, and applies them to a bicubic-upsampled image.This three-stage pipeline constructs spatially variant filters that integrate neighboring pixels to predict each HR pixel.
- Problem: SISR recovers an HR image from an LR image, but the inverse problem is ill-posed because infinitely many HR solutions may exist.The formulation uses blurring and downsampling to obtain the LR image from the HR image.
- Learning strategy: LAPAR predicts pixel-wise coefficients for predefined filter bases instead of directly estimating the LR-to-HR mapping.The filter matrix is represented using a dictionary D and coefficient matrix Φ, with only Φ optimized during learning.
- Dictionary design: The dictionary contains 72 Gaussian and difference-of-Gaussian filters, whose anisotropic settings are designed to represent structures and enhance edges and details.The filters vary through scaling, rotation, and elongation parameters; a large disparity between σ1 and σ2 is associated with edge preservation.
- Network: A lightweight residual network predicts the coefficient matrix, using local fusion blocks, PixelShuffle, convolutional tail layers, and weight normalization.The network is trained with Charbonnier loss and gradient descent.
3 Experiments
Experiments evaluate LAPAR’s dictionary design, optimization behavior, lightweight SISR performance, inference speed, and extension to denoising and JPEG deblocking. The results show that Gaussian/DoG dictionaries and pixel-adaptive filter assembly provide strong accuracy, fast optimization, and competitive restoration performance.
- Experimental Settings: The experiments train on DIV2K and Flickr2K and evaluate PSNR and SSIM on Set5, Set14, B100, Urban100, and Manga109.Only the Y channel of YCbCr images is reported.
- Filter Dictionary: A 72-filter Gaussian and DoG dictionary achieves the best result among the evaluated dictionary settings, while learned filters can produce edge artifacts or amplify noise.Random filters remain competitive, supporting the feasibility of coefficient learning with a predefined dictionary.
- Filter Dictionary: The 72-filter dictionary outperforms the 24- and 14-filter versions, indicating that a sufficiently large and diverse filter set improves results.
- Optimization: LAPAR reaches over 37.5dB on Set5 after only a small number of training iterations, with Gaussian and DoG filters providing more stable and accurate optimization than random filters.Figure 4 reports validation behavior during training.
- SISR Comparison: For ×2, LAPAR-A outperforms other lightweight methods across benchmark datasets with fewer parameters and MultiAdds; LAPAR-C remains competitive with only 80K parameters.LAPAR-A also stands out for ×3 and ×4 settings.
- Efficiency and Restoration Extensions: LAPAR-A, LAPAR-B, and LAPAR-C require 37.3ms, 29.1ms, and 22.2ms, respectively, to produce a 1280 × 720 output at ×4 on one NVIDIA 2080Ti GPU.The method also reports better visual recovery of structures and details in qualitative examples.
4 Conclusion
LAPAR achieves state-of-the-art results among lightweight methods across multiple SISR benchmarks and extends to denoising and JPEG deblocking.
- LAPAR achieves state-of-the-art results on multiple SISR benchmarks among lightweight methods.
- The framework extends easily to image denoising and JPEG deblocking while obtaining decent result quality.
- Future work will investigate compact filter-dictionary representations and joint multi-task optimization.
Broader Impact
The paper frames super-resolution, denoising, and deblocking as techniques with research and industrial applications, reporting no known negative ethical or societal impact.
- Image super-resolution, denoising, and deblocking have marked new milestones and are widely used in industry.
- The authors state that these techniques have no known negative impact on ethical and societal aspects.
A. Additional Examples and Results of Image Super-Resolution
Additional Urban100 examples show sharper and more accurate LAPAR reconstructions, while DIV2K-only training retains state-of-the-art lightweight performance and outperforms RAISR by a large margin.
- A. Additional Examples and Results of Image Super-Resolution: LAPAR recovers more accurate structures than other methods in an Urban100 example.In another example, its building transoms and mullions are sharper and straighter.
- A. Additional Examples and Results of Image Super-Resolution: Figure 11 presents Urban100 super-resolution examples at ×2 and ×4 scales.
- A. Additional Examples and Results of Image Super-Resolution: LAPAR-A retains state-of-the-art performance among lightweight SISR methods when trained only on DIV2K.The comparison is reported in Table 3, whose results are PSNR(dB).
- A. Additional Examples and Results of Image Super-Resolution: LAPAR-A outperforms RAISR by a large margin in the reported comparison.Table 4 reports PSNR(dB)/SSIM values for the two methods.
B. Additional Examples of Image Denoising
Additional Set14 denoising examples indicate that LAPAR better preserves white backgrounds and image details than the compared methods under noise standard deviation 35.
- B. Additional Examples of Image Denoising: LAPAR restores the original white background color more accurately than the compared denoising methods.
- B. Additional Examples of Image Denoising: LAPAR better preserves details across the denoised Set14 examples.
- B. Additional Examples of Image Denoising: Figure 12 visualizes Set14 denoising examples with PSNR(dB) and SSIM values beneath the images.The noise standard deviation is set to 35.
C. Additional Examples of Image Deblocking
On Set14 images compressed at JPEG quality 20, LAPAR removes compression artifacts and achieves superior results compared with DnCNN.
- LAPAR successfully removes JPEG compression artifacts in Set14 image-deblocking examples.The examples use JPEG quality 20.
- LAPAR achieves superior results compared with DnCNN on the illustrated deblocking cases.