Structure-Preserving Image Super-Resolution

1 INTRODUCTION

• Pixel-wise L1 or MSE objectives can produce blurry, high-PSNR images, whereas GAN-based SR yields sharper details but may distort geometric structure.The introduction contrasts RCAN’s straight but blurry brick edges with the sharper yet twisted edges of perceptual-driven methods; Fig. 1 describes SPSR as sharper than RCAN and more geometrically consistent than perceptual-driven methods.Source: §1 · p. 1, Figure 1 · p. 2
• SPSR-G combines an auxiliary gradient-recovery branch with gradient loss to preserve geometric structure while retaining perceptual quality.The branch integrates recovered HR gradients as structure priors, while gradient loss constrains second-order relations between neighboring pixels.Source: §1 · p. 1
• SPSR extends gradient guidance with a learnable neural structure extractor and structure loss to supervise richer, more complex local structures.The extractor is trained with contrastive prediction in SPSR-P or jigsaw puzzles in SPSR-J; these self-supervised schemes focus on local structure rather than semantic information, and the methods are model-agnostic.Source: §1 · p. 2
• Across five benchmark datasets, the proposed methods improve SR fidelity and reduce structural distortions, outperforming state-of-the-art perceptual-driven methods quantitatively and qualitatively.The paper also reports that learned neural structure extractors capture structure information and boost SR performance; structure loss can replace gradient-space losses and image-space adversarial loss.Source: §1 · p. 2

2.1 Single Image Super-Resolution

• PSNR-oriented SISR progressed from SRCNN to deeper, residual-dense, attention-based, and other architectures designed to improve reconstruction performance.Examples include DRCN, VDSR, RDN, RCAN, cross-scale attention, and holistic attention.Source: §2.1 · pp. 2–3
• Perceptual-driven methods use perceptual, adversarial, texture, and generative approaches to improve visual quality, but can still create unnatural artifacts and geometric distortions.Examples include perceptual loss, SRGAN, ESRGAN, and stochastic SR.Source: §2.1 · p. 3
• Earlier gradient-based SR estimates edge-related statistics or reconstructs high frequencies, whereas this work uses gradient maps to guide GAN-based SR and reduce geometric distortions.The authors state that, to their knowledge, no prior GAN-based SR method had used gradient-space guidance to preserve texture structures.Source: §2.1 · p. 3

2.2 Self-Supervised Learning

• Self-supervised learning uses unlabeled data and generation- or context-based pretext tasks; this work trains neural structure extractors with contrastive prediction and jigsaw puzzles to regularize SR.The cited context-based methods generally capture high-level semantics, while the proposed extractors target low-level structure features.Source: §2.2 · p. 3

3.1 Overview

• SPSR-G uses an SR branch guided by high-resolution gradient maps recovered by a second branch.The SR branch takes the low-resolution image as input, while the gradient branch supplies structural guidance for its reconstruction.Source: §3.1 · p. 4, Figure 3 · p. 4

3.2 Network Architecture

• The gradient branch uses intermediate SR features to predict gradient maps, then feeds gradient features back into fused SR reconstruction.It draws on features from the SR network’s 5th, 10th, 15th, and 20th blocks, reducing gradient-branch parameters; a fusion block combines branch features before final reconstruction.Source: §3.2 · pp. 4–5

3.3 Objective Functions

• Gradient loss adds a second-order constraint by matching SR and HR gradient maps, complementing image-space losses to discourage geometric distortions.The objective includes pixelwise gradient-map loss and adversarial discrimination of gradient patches; the gradient operation is differentiable, allowing end-to-end training, and the loss can guide other generative models.Source: §3.3 · pp. 5–6
• An adaptive fusion module lets the SR branch filter unhelpful gradient details, limiting their vulnerability to final SR performance.The authors motivate structure guidance by arguing that structural discrepancies are less acceptable than texture differences and that edges may matter more than textures in SR images.Source: §3.3 · p. 6

4.1 Neural Structure Extractor Architecture

• The NSE represents each image as a spatial feature map with 31 × 31 receptive fields and a 4× downsampled feature map to capture local geometry rather than holistic semantics.Each feature vector models neighboring image structures; adjacent feature receptive fields have a stride of 4.Source: §4.1 · p. 6

4.2 Self-Supervised Structure Learning

• Contrastive prediction trains the NSE to infer a target structure feature from neighboring features, bringing it closer to the positive target and separating it from negatives at unrelated locations.The sampled receptive fields are non-overlapping, so prediction and discrimination depend on consistency or inconsistency of local structures rather than shared overlap.Source: §4.2 · pp. 6–7
• Jigsaw training teaches the NSE local structural patterns by classifying the correct arrangement of four neighboring features among 24 permutations, rather than learning object semantics.The four features occupy a fixed 2×2 grid, have non-overlapping receptive fields, and are classified with a three-layer fully connected network.Source: §4.2 · p. 7

4.3 Structure Loss for Super-Resolution

• The structure loss combines pixelwise similarity between HR and super-resolved NSE features with an adversarial loss on those features, replacing image-space adversarial and gradient-space losses.The paper reports that this structure loss has stronger capacity to supervise SR than the gradient loss and original image-space adversarial loss.Source: §4.3 · pp. 7–8

5.1 Implementation Details

• Experiments train on DIV2K and test on Set5, Set14, BSD100, Urban100, and General100 using 4× bicubic downsampling and LPIPS, PSNR, and SSIM evaluation.SPSR-G uses an ESRGAN SR backbone and an RRDB gradient branch; structure extractors are trained using contrastive prediction or jigsaw puzzles.Source: §5.1 · p. 8

5.2 Investigation of NSEs

• Contrastive prediction’s extractor FP performs well on both structure-learning tasks, whereas jigsaw-trained FJ performs unsatisfactorily on contrastive prediction, indicating stronger structure capture by FP.The evaluation uses contrastive prediction accuracy and jigsaw classification on five benchmark datasets.Source: Figure 6 · p. 8, §5.2 · p. 9

5.3 Results and Analysis of SR

• SPSR methods achieve the best LPIPS values across all five test datasets, with higher PSNR and SSIM than other methods on most datasets; NatSR leads those metrics on most datasets.SPSR also surpasses ESRGAN, which uses the same basic reconstruction block; NatSR does not achieve second-best LPIPS on any test set.Source: Table 1 · p. 10, §5.3 · p. 10
• Visual comparisons show SPSR-G and SPSR-P recover sharper oval edges, better textures, and clearer structures than compared methods, while producing natural-looking results.Their gradient maps are described as bold and natural, whereas other methods’ maps tend to have small values or structural degradation.Source: Figure 7 · p. 9, §5.3 · pp. 10–11, Figure 8 · p. 10
• In a 1,590-vote study with 53 raters, SPSR-P receives more rank-1 votes than SPSR-G, and both methods outperform ESRGAN, NatSR, and SRGAN.Raters ranked five SR versions by perceptual quality with HR images as references.Source: §5.3 · p. 11, Figure 9 · p. 11
• SPSR-G surpasses ESRGAN on every measurement across all test sets; its gradient branch and gradient loss improve results, while combining pixelwise and adversarial losses yields the best performance among tested variants.Without the gradient branch, SPSR-G improves LPIPS over ESRGAN; removing gradient loss shows the branch can improve LPIPS or PSNR while relatively preserving the other metric.Source: Table 2 · p. 11, §5.3 · p. 12
• In SPSR-P, adversarial structure loss chiefly improves LPIPS, pixelwise structure loss mainly improves PSNR and SSIM, and their complementary combination improves all three metrics.Removing the gradient branch causes only slight degradation; removing both structure losses substantially harms LPIPS and lowers PSNR and SSIM relative to removing adversarial structure loss alone.Source: §5.3 · p. 12, Table 3 · p. 12
• In SPSR-P parameter-sensitivity experiments, changing one of the two parameters had a modest effect on SR performance, leading the authors to infer that the model is not sensitive to these parameters.Source: §5.3 · p. 12

• SPSR-G uses a gradient branch for explicit structural guidance and a gradient loss imposing second-order restrictions to alleviate geometric distortions.Source: §6 · pp. 12–13
• Experimental results on five popular benchmarks show the effectiveness of the proposed methods.Source: §6 · p. 13
• SPSR-P and SPSR-J extend SPSR-G with neural-structure-extractor losses trained through contrastive prediction and jigsaw-puzzle solving, capturing geometric relationships through gradient maps and structure features.Source: §6 · p. 13

APPENDIX A ARCHITECTURE OF NEURAL STRUCTURE EXTRAC-

• The extractor has six convolutional layers followed by ReLU layers.Source: Appendix A · p. 13
• Each extracted feature vector has a 31 × 31 receptive field.Source: Appendix A · p. 13
• Following ResNet, the extractor uses two skip connections to improve the representation ability of its features.Source: Appendix A · p. 13

B.1 Analysis of Sampling Strategies

• The three SPSR-P models have similar SR capacity on five benchmark datasets, indicating that sampling strategy does not significantly affect SR performance.The table caption describes the comparison; the passage reports the similarity and conclusion.Source: Table 5 · p. 13, §B.1 · p. 13
• The cross-position extractor FP-c performs better at structure capturing than FP-h and FP-v, while FP-v performs similarly to FP-h.FP-h and FP-v focus mainly on one direction, whereas FP-c may capture more comprehensive horizontal and vertical structural information.Source: §B.1 · p. 13
• The contrastive-prediction sampling strategies compare two horizontal positions, two vertical positions, or four end positions of a neighboring cross around an anchor.Fig. 13 illustrates the horizontal, vertical, and crossing-position strategies; the blue rectangle marks the anchor position.Source: §B.1 · p. 13, Figure 13 · p. 13

B.2 Details for Evaluation

• NSE top-1 accuracy averages repeated random-anchor scores across densely sampled test patches, counting an anchor as correct when the predictor output has the highest cosine similarity to its positive feature.For evaluation, the NSE is fixed and its autoregressive predictor is optimized on the training set; only non-overlapping feature vectors serve as negatives.Source: §B.2 · pp. 13–14

APPENDIX C EXPERIMENTS ON DIFFERENT NSES

• SPSR-VGG and SPSR-ResNet perform similarly, SPSR-R performs better than both, and all three remain inferior to SPSR-J and SPSR-P.Source: Appendix C · p. 14
• ImageNet-pretrained VGG19 and ResNet18 emphasize high-level semantics over local texture, making their features less effective for guiding fine structural recovery.Source: Appendix C · p. 14
• Self-supervised NSEs learn local structural relationships, whereas a random NSE acts as a regularizer but lacks effective structure extraction to strengthen SR capacity.Source: Appendix C · p. 14

APPENDIX D EXPERIMENTS ON OTHER NETWORK ARCHITEC-

• The experiments compare EDSR-GAN and SRResNet-GAN with gradient-loss variants and structure-loss variants, using pretrained EDSR initialization and analogous SRResNet settings.For EDSR, perceptual and adversarial losses are added during GAN training; the proposed gradient loss defines EDSR-G, and the structure loss defines EDSR-P.Source: Appendix D · pp. 14–15
• EDSR-P and SRResNet-P achieve the best SR ability among compared perceptual-driven models, while gradient-loss variants outperform their GAN counterparts on LPIPS, PSNR, and SSIM.EDSR and SRResNet outperform other models on PSNR and SSIM but perform worse on LPIPS.Source: Table 7 · p. 15, Appendix D · p. 15

E.1 SR Comparison

• The text displays more comparisons of SR performance with state-of-the-art SR methods, including SFTGAN, SRGAN, ESRGAN, and NatSR, in Figs. 16 and 17.Source: §E.1 · p. 15
• In the first two rows of Fig. 16, SPSR-G and SPSR-P produce the most accurate and pleasant-looking flower drops.Source: §E.1 · p. 15
• Across the other images, the proposed methods better preserve geometric structures while producing photo-realistic SR images than the compared methods.Source: §E.1 · p. 15
• SPSR-P recovers finer details than SPSR-G, demonstrating the effectiveness of the neural structure extractor and structure loss.Source: §E.1 · p. 15

E.2 Visualization of Gradient Maps

• The SPSR-G gradient branch converts low-resolution gradient maps into high-resolution gradient maps.Source: §E.2 · p. 15