Source-linked AI summary
PixelIR: Fidelity-Perception Decoupling via Pixel-Space Image-Residual Flow Matching for Efficient One-Step Real-World Super-Resolution
Bingtian Qiao, Yue Shi, Yong Guo, Wenjun Zhang, Jiezhang Cao
TL;DR
Real-ISR must reconcile faithful structure with realistic detail, yet existing methods commonly couple these objectives and retain costly high-resolution computation. PixelIR separates them with a frozen image flow and conditional residual flow, then distills the teacher into a one-step pixel-space student. It reports leading PSNR, SSIM, and LPIPS on RealSR and DRealSR with 32.9M parameters, 89.7G MACs, and 8.5ms latency.
Problem
Real-ISR methods struggle to balance fidelity and perceptual quality because shared optimization couples the two objectives, while efficient one-step methods often retain expensive high-resolution modeling.
Method
PixelIR uses a frozen image flow for faithful reconstruction, a conditional residual flow for perceptual details, and teacher distillation into a coarse-to-fine one-step student.
Results
PixelIR achieves leading PSNR, SSIM, and LPIPS on RealSR and DRealSR, while its student uses 32.9M parameters, 89.7G MACs, and 8.5ms latency.
Takeaways & Limitations
PixelIR transfers a stronger fidelity–perception operating point to efficient one-step pixel-space deployment.
Abstract
from arXiv · showhide
Real-world image super-resolution (Real-ISR) aims to preserve structures supported by the degraded observation while reconstructing perceptually realistic details. However, existing Real-ISR methods largely optimize fidelity and perceptual quality within a shared network, causing the two objectives to interfere throughout training and making their balance difficult to control. Recent one-step methods reduce sampling steps, yet often inherit both this coupled optimization behavior and the expensive high-resolution backbone of their multi-step predecessors. We argue that efficient Real-ISR requires not only a shorter sampling trajectory, but also specialized modeling of faithful reconstruction and perceptual detail synthesis. Based on this insight, we propose PixelIR, a fidelity-perception decoupling framework built upon pixel-space image-residual flow matching. PixelIR first learns an image flow that maps the degraded observation to a faithful reconstruction. Then, a residual flow synthesizes the missing perceptual details from noise without repeatedly relearning or overwriting the complete restoration solution. We further distill the teacher into a deployment-oriented one-step student within a coarse-to-fine pyramid architecture. Extensive experiments show that PixelIR achieves leading PSNR, SSIM, and LPIPS on both RealSR and DRealSR. The final model completes pixel-space restoration in a single evaluation with only 32.9M parameters, 89.7G MACs, and 8.5ms latency, demonstrating a strong practical fidelity-perception-efficiency balance.
1 Introduction
Real-ISR must balance faithful reconstruction with perceptually realistic detail, but coupled optimization and costly high-resolution modeling make that balance difficult. PixelIR decouples these objectives in a teacher and distills the result into an efficient one-step student.
- Motivation: Pixel-wise, perceptual, and adversarial objectives trace a fidelity–perception trade-off rather than jointly improving both qualities.Pixel-wise losses favor accurate but over-smoothed outputs, while perceptual objectives can deviate from the reference.
- Motivation: Existing multi-step and one-step Real-ISR methods remain limited by coupled training and expensive high-resolution modeling.Multi-step methods repeatedly evaluate large models, while one-step methods often retain costly denoisers or auxiliary encoders.
- PixelIR: PixelIR freezes a fidelity-oriented image flow before a second flow synthesizes perceptual residual details conditioned on the faithful base.This decouple-then-distill design assigns structural reconstruction and perceptual generation to separately optimized networks.
- PixelIR: 370.1M parameters and eight evaluations make the decoupled teacher expensive, motivating distillation into a purpose-built one-step student.The student narrows channel width from 256 to 48 while increasing feature resolution from 32^2 to 512^2 and capping fine-stage attention at 1024 tokens.
- Results: 32.9M parameters, 89.7G MACs, and 8.5ms latency characterize PixelIR’s one-step pixel-space deployment.These figures are reported on an RTX PRO 6000; the model performs the complete LR-to-HR mapping in one evaluation.
- Results: PixelIR achieves the best PSNR, SSIM, and LPIPS on both real-capture benchmarks while its compact student surpasses prior methods in CLIP-IQA on DIV2K.The reported deployment efficiency is paired with leading fidelity and reference-based perceptual quality.
2 Related Work
Prior Real-ISR work spans regression, perceptual, adversarial, diffusion, and flow-based approaches, but generative methods often remain computationally demanding. PixelIR’s student instead performs complete restoration directly in pixel space without a latent autoencoder at inference.
- Perception–distortion in Real-ISR: Feed-forward regression methods favor PSNR but average plausible high-frequency completions into over-smoothed results.Perceptual and adversarial training improves realism but addresses a different point in the perception–distortion trade-off.
- Generative Real-ISR: Generative Real-ISR methods introduce diffusion, flow, and varied priors or conditions to synthesize realistic details under real-world degradation.Representative methods include StableSR, DiffBIR, SeeSR, PASD, SUPIR, and ResShift.
- PixelIR: Figure 2’s pipeline separates faithful base reconstruction, conditional residual synthesis, and one-step student distillation.The teacher uses two 4-step flows, while the student is progressively narrowed for efficient deployment.
- PixelIR: PixelIR’s compact student performs complete restoration directly in pixel space without relying on a latent autoencoder during inference.This preserves an explicit pixel-domain restoration path throughout deployment.
3 Proposed Method
PixelIR separates faithful reconstruction from perceptual refinement with two pixel-space flows, then distills their behavior into a one-step student for efficient deployment.
- 3.1 Fidelity-Oriented Image Flow: The teacher first maps the upsampled low-quality observation c to target x through a four-step fidelity image flow.The base restorer predicts the flow velocity and is frozen after training as the perceptual stage’s fidelity anchor.
- 3.2 Perception-Oriented Residual Flow: The residual stage models missing details as a flow from Gaussian noise ϵ to residual r = x − ˆxb, conditioned on the frozen base reconstruction.The detail network receives the residual-flow state and frozen base through channel-wise concatenation.
- 3.2 Perception-Oriented Residual Flow: Freezing the fidelity flow assigns faithful reconstruction and perceptual residual synthesis to separately optimized networks, while residual composition avoids relearning the complete restoration.The two teacher networks contain 370.1M parameters and require eight evaluations.
- 3.3 One-Step Flow Distillation: The student combines flow matching, teacher distillation, ground-truth supervision, inverse-flow consistency, and adversarial training to learn the one-step restoration mapping.Teacher and ground-truth paths provide complementary endpoint and velocity-field supervision.
- 3.3 One-Step Flow Distillation: 32.9M parameters enable complete LR-to-HR restoration in one pixel-space evaluation, using a five-stage pyramid that increases resolution while decreasing channel width.The stages span resolutions 32, 64, 128, 256, and 512, with the three finest stages retaining 1024 attention tokens.
4 Experiments
Experiments evaluate PixelIR across real-world benchmarks, efficiency measurements, and ablations. The compact student achieves strong real-data fidelity and perceptual alignment while substantially reducing deployment cost.
- Setup: The lightweight branch pairs a 370.1M-parameter, eight-evaluation teacher with a 32.9M-parameter one-step student.The prior-enhanced branch instead uses a 1.3B PixelDiT prior adapted with rank-32 LoRA and contains a 132.8M-parameter student.
- Main Results: PixelIR achieves the best PSNR, SSIM, and LPIPS on both RealSR and DRealSR.These results indicate strong structural fidelity and reference-based perceptual alignment under authentic degradations.
- Main Results: The compact student reaches a CLIP-IQA of 0.761 on DIV2K-Val, surpassing all prior methods.
- Inference Efficiency: 32.9M parameters, 89.7G MACs, and 8.5ms latency enable one-step 128→512 pixel-space restoration.The measurement uses an RTX PRO 6000 and requires no VAE or text encoder at inference.
- Inference Efficiency: The student uses 5.3× fewer parameters than the next-smallest deployment stack and 4.6× fewer MACs than SANA-SR.Parameter and MAC comparisons are presented as more hardware-independent than published latency values.
- Ablation Studies: Removing both direct endpoint losses increases LPIPS from 0.263 to 0.358 and DISTS from 0.216 to 0.268, while MUSIQ rises from 68.9 to 72.2.The ablation indicates that direct teacher and ground-truth losses preserve faithfulness despite weaker no-reference scores without them.
5 Conclusion
PixelIR addresses coupled fidelity and perception optimization through decoupled image and residual flows, then distills their restoration mapping into a one-step student. It achieves leading benchmark quality with a compact pixel-space deployment profile.
- The framework revisits Real-ISR through fidelity–perception decoupling and efficient one-step deployment.
- PixelIR combines a 4-step image flow for faithful reconstruction with a 4-step residual flow for complementary perceptual details.The resulting teacher is distilled into a progressively narrowed coarse-to-fine student under teacher and ground-truth anchors.
- PixelIR achieves leading PSNR, SSIM, and LPIPS on RealSR and DRealSR.
- 32.9M parameters, 89.7G MACs, and 8.5ms latency characterize the compact student's one-step deployment on an RTX PRO 6000.