PaperlayerView exact revision on arXiv

Source-linked AI summary

HiFiPix: Bringing High-Fidelity Real-World Super-Resolution Back to Pixel Space

Wenjie Li, Du Chen, Hui Wang, Linlong Fan, Youwei Lv, Zeyi Chen, Zehan Chen, Jinwei Chen, Heng Guo, Zhanyu Ma, Qingnan Fan

arXiv:2610.35515v1cs.CV

TL;DR

VAE-based latent compression limits reconstruction fidelity in real-world image super-resolution, while adapting pixel-space diffusion brings patch artifacts and generative degradation. HiFiPix combines boundary-focused supervision with selective LoRA pruning and reports better results than VAE-based methods. The paper also identifies residual seams in some extremely blurred inputs as a limitation.

  • Problem

    VAE-based latent representations discard subtle textures, while pixel-space Real-ISR faces patch-boundary artifacts and weakened generation of missing details.

    Source: §1 · p. 2

  • Method

    HiFiPix combines CSWD and MixCSD to suppress patch-boundary artifacts with LoRA-SP, which prunes redundant adapter layers while protecting fidelity-critical layers.

    Source: §1 · pp. 2–3

  • Results

    HiFiPix outperforms VAE-based methods, demonstrating the promise of pixel-space modeling for Real-ISR.

    Source: §5 · p. 10

  • Takeaways & Limitations

    The paper presents pixel-space modeling as a promising approach to Real-ISR, alongside a high-quality benchmark for reliable evaluation.

    Source: §5 · p. 10

    Some extremely blurred inputs retain faint seams in texture-rich areas, and prompt-free evaluation yields limited text-conditioned metric results.

    Source: §4.4 · p. 9

Abstract

from arXiv · show

Diffusion-based real-world image super-resolution suffers from limited reconstruction fidelity due to information loss in VAE-based latent representations. Recently, pixel-space diffusion models offer a promising high-faithfulness alternative by avoiding latent compression, but adapting them to Real-ISR remains challenging due to inherent patch boundary artifacts and degraded generative capability. To address the issues and maximize the potential of pretrained pixel-space diffusion models, we propose HiFiPix, a pixel-space Real-ISR framework capable of ultra high-fidelity reconstruction and fine-grained visual generation effects. To suppress patch boundary artifacts, we penalize unrealistic high-frequency periodic components with a Cross-Scale Wavelet Discriminator (CWSD) for image-level supervision, and Mixup Contrastive Self-Distillation (MixCSD) for intermediate representation regularization. Furthermore, we uncover a heterogeneous generation-fidelity trade-off across low-rank adapters (LoRA) and propose Selective LoRA Pruning (LoRA-SP) to adaptively prune LoRA layers, enhancing generative capability with minimal deterioration of reconstruction fidelity. Extensive experiments demonstrate that HiFiPix consistently outperforms existing state-of-the-art latent space methods in terms of both restoration accuracy and perceptual quality by a large margin. Code and models will be released.

1 INTRODUCTION

HiFiPix adapts pretrained pixel-space diffusion models to Real-ISR, addressing patch-boundary artifacts and loss of generative capability. It combines boundary-focused supervision, selective LoRA pruning, and a benchmark intended to support reliable evaluation.

1 INTRODUCTION

  • VAE-based latent compression discards subtle input textures, while pixel-space fine-tuning can produce patch artifacts and weaken detail generation.The paper identifies faithful texture restoration alongside perceptual quality as an ongoing Real-ISR challenge.Source: §1 · p. 2
  • HiFiPix uses a pretrained pixel-level model for high-fidelity Real-ISR without sacrificing generative capability or inference efficiency.The paper presents pixel-space modeling as an alternative to latent-space Real-ISR.Source: §1 · p. 3
  • CSWD and MixCSD provide complementary image-level and feature-level supervision to suppress patch-boundary artifacts.CSWD propagates high-frequency features across scales; MixCSD distills clean representations for patch-level discriminability.Source: §1 · pp. 2–3
  • LoRA-SP prunes functionally redundant adapter layers while protecting fidelity-critical layers, aiming to preserve the pretrained generative prior during fine-tuning.It uses representational drift to identify redundant layers and progressively prunes them.Source: §1 · pp. 2–3
  • Dual-Q-Bench is a paired real-world ISR benchmark designed to address low-quality ground truth and background de-focus for evaluating fidelity and perceptual quality.Source: §1 · p. 3

2 RELATED WORKS

Prior Real-ISR methods use varied conditioning and fidelity strategies but remain in latent spaces. Recent pixel-space diffusion advances motivate exploring pretrained pixel models for Real-ISR, despite their adaptation challenges.

2.1 REAL-WORLD IMAGE SUPER-RESOLUTION

  • Existing Real-ISR approaches use adapters, dual-stream architectures, ControlNet, restoration modules, semantic conditioning, or larger datasets, but still rely on latent spaces.Other strategies replace random noise with LQ inputs, add GAN loss, or increase VAE capacity; the passage states that latent compression limits reconstruction fidelity.Source: §2.1 · p. 3

2.2 PIXEL-SPACE DIFFUSION MODELS

  • Recent work improves pixel-space diffusion efficiency and scalability, motivating its use for Real-ISR despite patch-boundary artifacts and generative-capability trade-offs.The cited advances include efficient pixel-space generation, scalable modeling, and a large-scale pixel-space text-to-image transformer.Source: §2.2 · p. 3, Figure 3 · p. 4

3 METHOD

HiFiPix adapts pixel-space diffusion to Real-ISR with complementary controls for patch-boundary artifacts and generative capacity. It also uses a paired benchmark intended to support more reliable evaluation.

3.1 EMPIRICAL OBSERVATIONS

  • Severe degradation makes high-frequency reconstruction more challenging, and HiFiPix targets resulting boundary artifacts and weakened generative capacity with distinct strategies.The observations motivate high-frequency regularization and selective rather than uniform LoRA pruning.Source: §3.1 · p. 4

3.2 CROSS-SCALE WAVELET DISCRIMINATOR

  • CSWD uses wavelet high-frequency sub-bands and cross-scale feature injection to detect periodic boundary artifacts at multiple scales.It discards the low-frequency LL sub-band and propagates shallow-layer high-frequency details to deeper layers.Source: §3.2 · p. 4

3.3 MIXUP CONTRASTIVE SELF-DISTILLATION

  • MixCSD contrasts LQ-input patch tokens with stop-gradient tokens from a GT/LQ mixture, using InfoNCE to promote boundary-consistent representations.Other patches in the batch serve as negatives, and supervision is aggregated across layers with step-gated weighting.Source: §3.3 · pp. 5–6

3.4 SELECTIVE LORA PRUNING

  • LoRA-SP identifies redundant layers using representation similarity, protects fidelity-critical layers, and gradually freezes selected candidates within a fixed pruning budget.Selected layers move from temporary windowed freezes to permanent freezes once the redundancy signal stabilizes.Source: §3.4 · p. 6

3.5 TRAINING OBJECTIVE

  • Training replaces Gaussian noise with the LQ image, optimizes LoRA against reconstruction and adversarial losses, and warms up the wavelet loss weight.Reconstruction loss combines L1, perceptual, frequency-space, and InfoNCE terms; adversarial supervision uses main and wavelet discriminators.Source: §3.5 · p. 6

3.6 CONSTRUCTION OF DUAL-Q-BENCH

  • The user study gives Dual-Q-Bench the highest preference rate at 76%, versus 10.7% for second-best GGT.Participants assessed GT quality, LQ authenticity, and GT–LQ alignment across 50 sampled pairs per dataset.Source: §3.6 · p. 7

4 EXPERIMENTS

HiFiPix is evaluated against Real-ISR methods on fidelity, perceptual quality, and computational cost, with ablations examining its artifact-mitigation and LoRA-pruning components. The authors also identify residual seams in very blurred inputs and limits to prompt-conditioned evaluation.

4.1 EXPERIMENTAL SETUP

  • Evaluation uses DRealSR, GGT-GeneralMix, RealSR, and Dual-Q-Bench, with full-reference and no-reference metrics; training uses LSDIR and the first 10,000 FFHQ images.The authors fine-tune rank-64 LoRA layers on HiDream-O1-Image and compare against multi-step and one-step Real-ISR methods.Source: §4.1 · pp. 7–8, §4.2 · p. 8

4.2 EXPERIMENT RESULTS

  • HiFiPix consistently outperforms existing methods in restoration fidelity, notably on high-quality-GT benchmarks, and excels on most no-reference metrics.The authors report enhanced fidelity without sacrificing generation ability.Source: §4.2 · p. 8
  • Qualitative comparisons show photo-realistic results with faithfully restored fine text and textures, unlike the entangled artifacts and over-smoothed details of VAE-based methods.Figure 5 presents comparisons on real-world test sets.Source: §4.2 · p. 8, Figure 5 · p. 8
  • HiFiPix runs 40% faster than PiSA-SR, the second-fastest single-step method, and uses considerably less GPU memory than ODTSR and StrSR.The comparison is on GGT-GeneralMix at 1024² resolution using one NVIDIA H20 GPU.Source: §4.2 · pp. 8–9

4.3 ABLATION STUDY

  • Combining CSWD and MixCSD gives the best visual results and fidelity without excessive generation-quality loss; either alone degrades generation quality.The methods penalize artifacts in image and intermediate-feature domains, respectively, making them complementary.Source: Table 3 · p. 8, §4.3 · p. 9
  • Mixed LQ-HQ teacher inputs outperform an HQ-only teacher, whose domain gap from LQ student inputs degrades performance; contrastive learning also outperforms distillation.The authors attribute contrastive learning's advantage to negative samples that avoid representation collapse and maintain feature diversity.Source: §4.3 · p. 9, Table 4 · p. 9, Table 5 · p. 9
  • LoRA-SP's fidelity-aware veto improves the generation-fidelity trade-off over uniform and random pruning, while redundancy detection alone causes notable fidelity drops.The authors report only marginal gains for latent-space models, possibly because VAE-driven semantic compression leaves little layer redundancy.Source: §4.3 · p. 9, Table 7 · p. 9

4.4 LIMITATIONS

  • Extremely blurred inputs can retain faint seams in texture-rich areas; the prompt-free pipeline excludes text-conditioned metrics such as CLIPIQA.The authors say these metrics focus on text-image alignment and yield limited evaluation results in their prompt-free setting.Source: §4.4 · p. 9

5 CONCLUSION

HiFiPix removes VAE compression in a pixel-space Real-ISR framework and combines artifact mitigation, selective LoRA pruning, and a new benchmark. The authors report that it outperforms VAE-based methods, supporting pixel-space modeling for Real-ISR.

5 CONCLUSION

  • HiFiPix is a pixel-space generative framework that removes VAE compression to enable faithful restoration.The conclusion frames this as shifting Real-ISR from latent-space to pixel-space modeling.Source: §5 · p. 9
  • CSWD and MixCSD target high-frequency reconstruction and cross-patch consistency, while LoRA-SP supports fine-grained details with minimal fidelity sacrifice.The authors also construct Dual-Q-Bench for reliable evaluation.Source: §5 · p. 10
  • Experiments show HiFiPix outperforms VAE-based methods, demonstrating the promise of pixel-space modeling for Real-ISR.Source: §5 · p. 10

A APPENDIX

The appendix points readers to additional materials on Dual-Q-Bench, method comparisons, model parameters, and ablations.

A APPENDIX

  • The appendix directs readers to the Dual-Q-Bench construction details in Sec. 3.6.The appendix note refers to the main paper for those details.Source: A APPENDIX · p. 13
  • Additional quantitative and qualitative method comparisons are available in Sec. 4.2.The appendix points to the main paper for these comparisons.Source: A APPENDIX · p. 13
  • Model-parameter comparisons are listed as additional appendix material, with details referred to Sec. 4.2.Source: A APPENDIX · p. 13
  • More ablation studies of the method are listed as appendix material, with details referred to Sec. 4.3.Source: A APPENDIX · p. 13

A.1 CONSTRUCTION OF DUAL-Q-BENCH

Dual-Q-Bench is constructed as a high-quality real-world paired benchmark intended to make fidelity and generation-quality evaluation more reliable. Its GT images are selected for sharp, realistic detail, paired with screen-captured LQ images, and aligned through a multi-stage registration procedure.

A.1 CONSTRUCTION OF DUAL-Q-BENCH

  • Existing paired benchmarks can penalize realistic detail absent from imperfect GT, while de-focused backgrounds and possible GT hallucinations further complicate fidelity assessment.Dual-Q-Bench is introduced to provide realistic, sharp GT for more reliable fidelity and generation-quality evaluation.Source: §A.1 · p. 13, Figure 7 · p. 14
  • Dual-Q-Bench contains 100 manually screened GT photographs with sharp, rich details at 1024^2 resolution.Images are filtered using aggregated NR-metric scores exceeding LSDIR GT images by an average of 30%, then cropped for texture-rich regions and screened for de-focus or painterly appearance.Source: §A.1 · p. 13
  • Real-world LQ images are captured by photographing GT images displayed on an ultra-high-definition screen with a mobile phone.The capture setup follows the screen-photographing strategy in Wu et al. (2026).Source: §A.1 · p. 13
  • Each LQ is registered to GT using SIFT and RANSAC homography, with template-matching fallback, ECC refinement, and LAB-space brightness and color correction.The aligned pair is center-cropped to 1024^2; Table 8 compares Dual-Q-Bench with existing paired benchmarks.Source: §A.1 · pp. 13–14, Table 8 · p. 14

A.2 QUANTITATIVE AND QUALITATIVE COMPARISONS

HiFiPix combines competitive pixel-level restoration fidelity with strong perceptual scores and high visual quality across real-world test sets. Its comparisons also note that imperfect GT can make some full-reference perceptual metrics unreliable.

A.2 QUANTITATIVE AND QUALITATIVE COMPARISONS

  • HiFiPix achieves competitive pixel-level fidelity and the best MANIQA, Q-Align, and AFINE-ALL scores on existing test sets, improving fidelity without degrading generative performance.The authors attribute relatively lower LPIPS and DISTS scores partly to limited perceptual quality in GT images, which lack details HiFiPix reconstructs.Source: §A.2 · p. 14
  • Across varied real-world test sets, HiFiPix delivers the best visual quality while preserving high-fidelity details, including legible text and natural textures.The comparisons contrast this with twisted artifacts and over-smoothed details from VAE-based approaches.Source: §A.2 · p. 14

A.3 COMPARISON OF MODEL PARAMETERS

HiFiPix is reported to use a compact trainable component set while maintaining competitive restoration quality. The parameter comparison also distinguishes generative-model components, language-model use, and trainable parameters.

A.3 COMPARISON OF MODEL PARAMETERS

  • HiFiPix requires 207.1M trainable parameters, surpassing all multi-step methods and outperforming large-backbone single-step methods in parameter efficiency.The authors report competitive restoration quality alongside this efficiency; Table 10 compares model parameters.Source: §A.3 · pp. 14–15, Table 10 · p. 15
  • HiFiPix removes the text encoder and does not require image captions, reducing total parameters relative to several text-encoder-based approaches and the original 9B HiDream-O1-Image.Table 10 defines the generative model to include the text encoder, VAE encoder/decoder, and diffusion backbones.Source: §A.3 · p. 14, Table 10 · p. 15
  • Although some SD2.1-based single-step methods use fewer trainable parameters, the authors report poorer restoration performance.Source: §A.3 · p. 15

A.4 ABLATION STUDIES

The ablations identify settings that balance generative capability with fidelity: pruning four LoRA layers and distilling five intermediate layers are adopted in the final configuration. The authors also choose layer-selective freezing over layer-adaptive attenuation.

Number of Selected Layers in LoRA-SP

  • Pruning 4 of 35 LoRA layers achieves the optimal trade-off between fidelity and generative capability.Pruning fewer layers has limited generative benefit, while pruning more severely compromises fidelity.Source: §A.4 · p. 15

Number of Distillation Layers in MixCSD

  • Using 5 MixCSD distillation layers—roughly 1/7 of total layers—avoids the limited supervision of too few layers and redundant constraints from too many.Insufficient layers hinder high-frequency representation generation; excessive distillation reduces generative flexibility.Source: §A.4 · p. 15

Layer-Selective Freezing vs. Layer-Adaptive Attenuation

  • Layer-adaptive LoRA attenuation yielded no generative-quality gain, so the authors instead adopt layer-selective freezing.Attenuation rescales updates without removing their direction, leaving layers between the pretrained and converged-adapter states.Source: §A.4 · pp. 15–16

Related papers

Loading 2610.35515v1…