Source-linked AI summary
GraftSR: Grafting Authentic Textures for Real-World Image Super-Resolution via Identical-Instance Guidance
Qifan Yu, Haoran Bai, Zongyao He, Weijie He, Sibin Deng, Honggang Qi, Ying Chen
TL;DR
Diffusion-based real-world SR can hallucinate textures, while misaligned identical-instance references complicate faithful transfer. GraftSR uses dual-mask guidance and dedicated identical-instance data to separate what textures to extract from where to apply them, achieving state-of-the-art reference-faithful restoration with a 20.2% LPIPS reduction.
Problem
Diffusion-based real-world SR inherently hallucinates textures, and spatially misaligned references create background feature leakage and ambiguous transfer targets.
Method
GraftSR combines dual-mask reference guidance with TexRefSR-141K identical-instance training data to isolate authentic textures and localize their transfer.
Results
20.2% lower LPIPS than the second-best method, ODTSR, and new state-of-the-art performance demonstrate highly faithful texture restoration.
Takeaways & Limitations
GraftSR enables robust cross-view texture transfer without brittle spatial alignment while preserving original geometric structures.
Abstract
from arXiv · showhide
Diffusion-based real-world image super-resolution (SR) achieves impressive perceptual quality but inherently suffers from severe texture hallucination. To overcome this limitation, we propose GraftSR, a texture-reference-guided generative SR framework that leverages reference images of the identical instance to anchor the restoration of authentic textures. However, severe spatial misalignment between low-quality inputs and their references poses significant challenges, often leading to ambiguous transfer targets and background feature leakage. To address these issues, GraftSR employs a novel dual-mask reference guidance mechanism that systematically decouples the cross-view texture injection process. By explicitly isolating what authentic textures to extract from the reference and precisely localizing where to apply them within the target, GraftSR achieves robust texture transfer without relying on brittle spatial alignment. Furthermore, to bridge the critical gap in appropriate training data, we construct TexRefSR-141K, the first large-scale dataset providing high-quality reference tuples equipped with complementary spatial masks. Extensive experiments on our newly established benchmark, TexRefSR-Eval, demonstrate that GraftSR sets a new state-of-the-art. Notably, it reduces LPIPS by 20.2\% over top-performing baselines, achieving superior reference-faithful restoration.
Introduction
GraftSR addresses texture hallucination and cross-view misalignment in real-world super-resolution by combining dual-mask reference guidance with identical-instance training data. Experiments show reference-faithful restoration with improved perceptual fidelity.
- Diffusion-based SR can hallucinate textures that deviate from authentic appearance, damaging fine-grained subject identity.
- Cross-view misalignment makes reference transfer difficult because irrelevant backgrounds leak into features and transfer targets remain ambiguous.
- GraftSR uses mask-modulated reference tokens to isolate textures and region-aware semantic tokens to localize their application without brittle spatial matching.
- TexRefSR-141K provides over 141K high-quality tuples across 61K identical instances with complementary masks for reference and target regions.
- 20.2% lower LPIPS than top-performing methods demonstrates improved reference-faithful restoration on TexRefSR-Eval.
- GraftSR establishes a new state-of-the-art in reference-guided SR while bridging perceptual quality and texture fidelity.
Related Work
Prior work advances diffusion-based and reference-based super-resolution, but existing approaches respectively emphasize perceptual synthesis or depend heavily on localized reference alignment.
- Diffusion-based restoration has shifted SR toward perceptual synthesis, with one-step distillation models reducing the computational overhead of multi-step approaches.
- Recent blind restoration methods improve perceptual quality and visual detail using decoupled generation branches or deep visual features.
- Reference-based SR uses auxiliary images for high-fidelity detail restoration, but established methods heavily rely on localized patch alignment.
Method
GraftSR restores degraded images by transferring authentic textures from identical-instance references while preserving the input’s spatial structure. Its dual-mask guidance separates texture extraction from target localization, supported by curated training and evaluation data.
- Reference-guided restoration: GraftSR transfers authentic textures from identical-instance references while preserving the low-quality input’s spatial structure.The framework targets precise texture fidelity without requiring aligned reference and target views.
- Dual-mask reference guidance: Dual-mask guidance uses mask-modulated reference tokens to isolate texture-bearing reference regions and region-aware semantic tokens to localize target application.The two token types jointly condition MMDiT blocks during one-step restoration.
- Dual-mask reference guidance: Region-aware semantic tokens combine visual-spatial clues with a unified text caption to guide the right textures to the right target locations.A frozen Qwen2.5-VL encoder extracts the multimodal condition tokens.
- Dual-mask reference guidance: Soft additive mask modulation isolates authentic textures while preserving the continuous latent space and the model’s general generative capability.The reference latent is encoded with a frozen VAE, spatially modulated, and projected into reference tokens.
- Training data construction: The data pipeline retrieves identical-instance triplets from multi-view galleries and equips them with captions and complementary reference-target masks.Mask dilation improves robustness near delicate boundaries, while filtering removes mismatched, degraded, or unrecognizable samples.
- Evaluation benchmark: TexRefSR-Eval contains 300 manually verified cases split into synthetic and naturally degraded subsets for paired evaluation and real-world generalization.The synthetic subset has 250 samples, and the real subset has 50 samples.
Experiments
Experiments evaluate GraftSR on reference-guided and general real-world SR benchmarks using fidelity, perceptual-quality, texture-consistency, and human-preference measures. Results show strong reference-faithful restoration, robustness without references, and advantages over competing and commercial methods.
- Experimental Setup: TexRefSR-Eval includes synthetic and real subsets, while RealSR and DRealSR assess general real-world SR under 4× upscaling.The synthetic subset uses ground-truth references; the real subset uses naturally degraded images.
- Quantitative Results: GraftSR reduces LPIPS by 20.2% versus ODTSR and is the only method with DISTS strictly below 0.10 on TexRefSR-Eval-Syn.It also achieves the highest CLIPIQA score of 0.7275.
- Quantitative Results: On TexRefSR-Eval-Real, GraftSR achieves the highest texture consistency at 2.840, perceptual quality at 3.980, and overall score at 3.410.The texture-consistency score exceeds Gemini-3-Pro at 2.700 and VOSR at 2.380.
- Quantitative Results: Compared with GPT-Image-2 and Gemini-3-Pro, GraftSR reaches 30.11 dB PSNR on the synthetic benchmark and a 3.410 VLM overall score on the real-world benchmark.The corresponding commercial-model PSNR values are 18.33 dB and 26.79 dB; overall scores are 2.560 and 3.290.
- Generalization Without References: With the reference branch empty, GraftSR achieves the highest fidelity on RealSR and DRealSR, exceeding PiSA-SR on DRealSR by 1.09 dB PSNR and 0.029 SSIM.Sequence concatenation makes the reference branch detachable when references are absent.
- Ablation and User Study: Ablations show that joint attention and both spatial masks are necessary for strong texture transfer and structural alignment.Removing the reference token or either mask disrupts alignment or causes textures to nearly vanish; the user study reports 82% and 74% wins against ODTSR and Gemini-3-Pro.
Conclusion
GraftSR addresses texture hallucination in diffusion-based real-world SR by using identical-instance references and dual-mask guidance. The paper also introduces complementary dataset and benchmark resources, with experiments reporting state-of-the-art faithful texture restoration.
- Conclusion: GraftSR restores authentic textures from identical-instance references to address texture hallucination in diffusion-based real-world SR.Its dual-mask guidance handles misalignment by separating what textures to extract from where to apply them.
- Conclusion: TexRefSR-141K and TexRefSR-Eval provide large-scale identical-instance data and benchmarking with complementary spatial masks.These resources support training and evaluation of faithful texture restoration.
Overview
The supplementary material provides additional method, implementation, user-study, and visualization details. It expands the paper’s reproducibility and result documentation.
- Supplementary Material: The supplement describes architecture details, data construction, hyperparameters, prompts, training strategies, user-study details, and additional visualizations.These materials cover both implementation and evaluation procedures.
Method Details
GraftSR’s training combines dual-noise one-step latent prediction with reconstruction, perceptual, adversarial, and regularization objectives. Its data curation further uses mask dilation and strict pair-quality filtering to support robust cross-view texture restoration.
- Dual-noise training: Dual-noise encoding balances perceptual quality and fidelity by using separate perception and control branches from the low-quality latent.The control branch samples sc between 750 and 999; larger sc values correspond to weaker noise and preserve fine details.
- One-step prediction: One-step adversarial training predicts a velocity field that updates a noised low-quality latent directly toward the high-quality target.The predicted latent is decoded for supervision, avoiding a separate consistency-distillation stage.
- Training objectives: The reconstruction objective combines RGB reconstruction with adversarial and VAE terms, while DISTS supplements MSE and LPIPS for mild cross-view misalignment.DISTS jointly measures structural and textural similarity and avoids rigid pixel-to-pixel matching.
- Adversarial training: A relativistic GAN formulation compares real and generated latents, with R1-style regularization encouraging consistent discriminator predictions under perturbations.The discriminator uses patch-wise realism scores from a Wan2.1-based architecture.
- Data curation: The curation pipeline dilates complementary masks and filters candidate tuples using semantic similarity and image-quality assessment.Mask dilation uses randomly sampled 3–6 pixel target and 5–10 pixel reference radii during training.
Implementation Details
The implementation fine-tunes a Qwen-Image-Edit generator with LoRA and fully fine-tunes a Wan2.1 discriminator, while stochastic crops and condition dropping improve robustness. VLM prompts provide captions, matched regions, and reference-based evaluation scores.
- Optimization: The generator uses LoRA fine-tuning on Qwen-Image-Edit, while the Wan2.1-1.3B discriminator is fully fine-tuned from pretrained weights.Both networks use RMSprop with constant learning rates and are trained for one epoch on 32 NVIDIA H20 GPUs.
- Robustness strategies: Training samples 512 × 512 crops, sometimes centered within the mask region, and randomly drops the reference branch and spatial mask.These stochastic strategies reduce reliance on auxiliary conditions and improve robustness and generalization.
- Annotation prompts: A VLM jointly identifies the shared product, describes its attributes, and localizes corresponding semantic regions in the source and reference images.The annotation prompt returns matched bounding boxes and structured descriptions for both views.
- Evaluation prompts: Reference-based real-world SR evaluation presents global views and key-region crops, scoring intrinsic quality and consistency with the reference on integer 1–5 scales.The global view informs content and composition, while the crop emphasizes fine detail and texture.
User Study Details
The user study compares GraftSR with competing methods through randomized two-alternative forced-choice trials. Participants judge perceptual quality against the input and texture consistency against the reference.
- Study design: 10 participants evaluated 50 diverse scenes in randomized 2AFC trials comparing anonymized GraftSR and competing outputs.Candidate positions were shuffled across trials to reduce positional bias.
- Evaluation criteria: Participants preferred candidates that recovered details and sharpness without blur, while also maintaining material and texture consistency with the reference.Highly localized regions were ignored when judging texture consistency.
More Visualization Results
Additional visualizations show GraftSR restoring sharper, more faithful textures on reference-guided synthetic and real-world benchmarks. It also generalizes to reference-free real-world SR benchmarks with competitive or superior natural-scene detail recovery.
- Overall comparison: Across the reported settings, GraftSR consistently restores sharper structures and more faithful textures while avoiding competing methods’ over-smoothing.The qualitative comparisons cover both reference-guided and general SR benchmarks.
- TexRefSR-Eval-Syn: On TexRefSR-Eval-Syn, reference guidance recovers fine patterns and structural details more faithfully than methods that over-smooth or hallucinate textures.The comparison uses synthetic degradation with available ground truth.
- TexRefSR-Eval-Real: On TexRefSR-Eval-Real, GraftSR produces sharper perceptual results whose textures remain consistent with the reference despite the absence of ground truth.Other methods tend to generate blurry or unfaithful reconstructions.
- General SR benchmarks: On RealSR and DRealSR, GraftSR generalizes without references and recovers clean, realistic details competitive with or superior to state-of-the-art general SR methods.These benchmarks evaluate natural-scene restoration in a reference-free setting.