Source-linked AI summary
RealRestorer: Towards Generalizable Real-World Image Restoration with Large-Scale Image Editing Models
Yufeng Yang, Xianfang Zeng, Zhangqi Jiang, Fukun Yin, Jianzhuang Liu, Wei Cheng, jinghong lan, Shiyu Liu, Yuqi Peng, Gang YU, Shifeng Chen
TL;DR
Real-world restoration is limited by narrow training distributions and evaluation protocols that miss diverse degradations and consistency. The paper builds realistic nine-degradation data, fine-tunes an open-source image-editing model, and introduces RealIR-Bench; RealRestorer achieves open-source state-of-the-art performance comparable to closed-source systems, while retaining limitations in compute, ambiguity, and extreme degradation.
Problem
Existing restoration data and reference-based evaluation inadequately represent diverse real-world degradations, while closed-source image-editing systems are difficult to reproduce.
Method
The paper combines realistic degradation synthesis, two-stage open-source image-editing fine-tuning, and RealIR-Bench with metrics for restoration and consistency.
Results
RealRestorer achieves open-source state-of-the-art performance across nine restoration tasks, with results highly comparable to leading closed-source systems and strong zero-shot generalization to unseen degradations.
Takeaways & Limitations
RealRestorer narrows the gap between open-source restoration and leading closed-source image-editing systems across real-world degradations.
Takeaways & Limitations
The model has substantially higher computational cost, can confuse reflections under semantic ambiguity, and struggles when severe degradation removes reliable pixel evidence.
Abstract
from arXiv · showhide
Image restoration under real-world degradations is critical for downstream tasks such as autonomous driving and object detection. However, existing restoration models are often limited by the scale and distribution of their training data, resulting in poor generalization to real-world scenarios. Recently, large-scale image editing models have shown strong generalization ability in restoration tasks, especially for closed-source models like Nano Banana Pro, which can restore images while preserving consistency. Nevertheless, achieving such performance with those large universal models requires substantial data and computational costs. To address this issue, we construct a large-scale dataset covering nine common real-world degradation types and train a state-of-the-art open-source model to narrow the gap with closed-source alternatives. Furthermore, we introduce RealIR-Bench, which contains 464 real-world degraded images and tailored evaluation metrics focusing on degradation removal and consistency preservation. Extensive experiments demonstrate our model ranks first among open-source methods, achieving state-of-the-art performance.
1. Introduction
Real-world restoration must handle diverse, co-existing degradations, but existing datasets and evaluation protocols inadequately represent these conditions. RealRestorer addresses these gaps with realistic training data, an open-source model, and RealIR-Bench for restoration and consistency evaluation.
- Real-world images contain diverse and co-existing degradations that exceed the single-degradation, single-model paradigm.
- Existing restoration methods often rely on limited synthetic distributions, while large image editing models require closed-source data and compute.
- Reference-based evaluation alone may miss perceptual quality, robustness across degradations, and detail consistency in real scenes.
- RealRestorer combines a comprehensive degradation-synthesis pipeline with open-source fine-tuning across nine restoration tasks.
- The paper presents RealRestorer, a diverse-data pipeline, and RealIR-Bench as its three main contributions.
- RealIR-Bench evaluates both degradation restoration and consistency preservation using real-world cases.
2. Related Work
Prior restoration approaches often generalize poorly because they depend on task-specific assumptions or simplified synthetic degradations. The paper instead combines realistic nine-degradation synthesis with open-source image-editing priors and evaluates restoration on real images without clean references.
- 2.1. Single-Degradation Restoration: Single-degradation models rely on predefined, relatively homogeneous corruption assumptions and may introduce artifacts on unseen or compound degradations.
- 2.1. Single-Degradation Restoration: Synthetic datasets with simplified degradation models may not faithfully represent real-world distributions, constraining robustness and practical applicability.
- 2.2. All-in-One Image Restoration: All-in-one methods unify multiple degradations but often retain narrow synthetic training distributions that weaken real-world generalization.
- 2.2. All-in-One Image Restoration: Large image-editing models provide semantic priors and often generalize better to real-world data than small specialized restoration networks.
- 2.2. All-in-One Image Restoration: The synthetic degradation pipeline includes blur, compression artifacts, moiré patterns, low-light, noise, flare, reflection, haze, and rain.
- 2.2. All-in-One Image Restoration: The proposed pipeline covers nine degradations, while RealIR-Bench contains 464 real images and metrics for degradation removal and input-content consistency.
3. RealRestorer
RealRestorer is built from realistic synthetic and web-collected degradation data, quality-controlled paired images, and a two-stage open-source image-editing fine-tuning strategy. Its construction targets diverse real-world distributions while preserving degraded-clean correspondence and content fidelity.
- 3.1. Data Construction: Existing synthetic degradation models are insufficient for the complexity and diversity of real-world distributions.
- 3.1. Data Construction: The dataset pipeline produces realistic degradation patterns while keeping paired clean images highly consistent with degraded counterparts.
- 3.1. Data Construction: Synthetic data offers scalable training and transfers general image-editing priors, but cannot fully capture real-world degradation diversity.
- 3.1. Data Construction: VLMs and quality models filter unsuitable images, and degradation alignment is checked after pairing to keep patterns learnable.
- 3.1. Data Construction: The pipeline uses task-specific synthesis, including temporal averaging for blur, web-style compression, and multi-scale moiré patterns.
- 3.1. Data Construction: Low-light, noise, flare, reflection, haze, and rain synthesis combine learned or physical transformations with collected real patterns.
- 3.1. Data Construction: Web-collected data is filtered with CLIP, watermark detection, and Qwen3-VL-8B-Instruct before clean references and pair consistency are examined.
- 3.2. Method and Training Strategy: RealRestorer fine-tunes Step1X-Edit across nine restoration tasks using transfer training followed by supervised fine-tuning.
4. Benchmark and Evaluation
RealIR-Bench evaluates restoration on diverse naturally degraded images using non-reference metrics that jointly measure degradation removal and content consistency. RealRestorer performs strongly across the benchmark, ranking first among open-source models and narrowing the gap with closed-source systems.
- Benchmark motivation: Traditional benchmarks underrepresent real-world complexity by focusing on single degradations, synthetic corruptions, or limited degradation patterns.Real-world images commonly contain diverse and mixed degradations, making broader evaluation necessary.
- Benchmark construction: RealIR-Bench contains 464 naturally degraded, non-reference images spanning nine common restoration tasks and degradation types.The benchmark uses a fixed enhancement instruction to reduce the influence of instruction variation.
- Evaluation metrics: RealIR-Bench reports LPS, Restoration Score, and Final Score to assess perceptual consistency, degradation removal, and their combined trade-off.Final Score decreases when either restoration improvement or content preservation is poor.
- Evaluation metrics: Restoration Score uses Qwen3-VL-8B-Instruct to rate degradation severity from 0 to 5, with the score defined by improvement after restoration.Higher Restoration Score indicates greater perceived restoration improvement according to the VLM evaluator.
- Experimental results: RealRestorer ranks first among open-source models and third overall, trailing Nano Banana Pro by 0.007 points while surpassing Qwen-Image-Edit-2511 by 0.019 points.It performs best on deblurring and low-light enhancement, ranks second on moiré removal, and remains competitive across the remaining tasks.
- Ablation and training analysis: The two-stage training strategy reaches a first-stage peak FS of 0.122, then improves with real-world data before overfitting beyond around 2.5K steps.Progressively-Mixed training improves the final FS by 0.004 points and supports structural consistency and content fidelity.
5. Limitations and Discussion
RealRestorer generalizes across seen and unseen restoration tasks but remains constrained by computational cost, semantic ambiguity, and extremely severe degradations.
- The 28-step denoising process makes RealRestorer substantially more computationally expensive than smaller-scale models.The authors identify this as a common limitation of large-scale image editing models.
- Strong semantic and physical ambiguity, including mirror selfies, can cause the model to confuse true scene content with unwanted reflections.
- Extremely severe degradations can leave insufficient pixel evidence, causing failures to preserve physically consistent structures such as water reflections.
6. Conclusion
The paper introduces RealRestorer, a data-generation pipeline, and RealIR-Bench to improve open-source restoration for complex real-world degradations. Experiments show open-source state-of-the-art performance across nine tasks, strong zero-shot generalization, and results highly comparable to leading closed-source systems.
- RealRestorer is an open-source image editing model for complex real-world image restoration, trained with a two-stage progressively mixed strategy.The strategy combines synthetic and real-to-clean pairs to reduce the synthetic-to-real domain gap.
- RealIR-Bench is a non-reference benchmark with authentic degraded images and a VLM-based evaluation framework for real-world restoration.
- RealRestorer achieves open-source state-of-the-art performance across nine restoration tasks, with results highly comparable to leading closed-source commercial systems.The model also exhibits strong zero-shot generalization to unseen degradations.
- The data-generation pipeline synthesizes nine major degradation patterns from clean images, including blur, rain, noise, low-light, haze, reflection, and flare.The passage also lists moiré patterns and compression artifacts among the nine degradations.
A.2. Real-World Degradation Data
The real-world degradation data uses clean images from open-source websites to cover six degradation types whose real-world patterns differ substantially from synthetic ones. Filtering combines semantic cues, watermark detection, and vision-language assessment before paired-data generation and additional failure-case filtering.
- Clean images are collected from Pexels and Pinterest for six real-world degradation types: blur, rain, low light, haze, reflection, and flare.
- Real-world degradation patterns exhibit a substantial gap from synthesized patterns, motivating dedicated real-world data collection.
- CLIP filters images using degradation-related semantic cues, while watermark detection and Qwen3-VL-8B-Instruct remove watermarked or insufficiently degraded samples.
- After editing models generate raw paired data, additional filtering removes failure cases from the training set.
B. Implementation Details
Training uses two stages: transfer learning on synthetic data covering nine degradation types, followed by progressively mixed fine-tuning with real-world and synthetic data to improve handling of complex degradations.
- Transfer Training Stage: The transfer stage trains the DiT blocks on synthetic data covering nine degradation types while freezing the VAE and text encoders.Average sampling across degradation categories balances multi-task learning; resolution is 1024 × 1024 and global batch size is 16.
- Transfer Training Stage: After approximately 500 training steps, transfer learning begins adapting high-level editing capabilities to low-level restoration but still produces artifacts on complex degradations.
- Supervised Fine-Tuning Stage: The supervised fine-tuning stage progressively mixes real-world degradation data with a small portion of synthetic data to address artifacts and complex degradations.
- Supervised Fine-Tuning Stage: The second stage freezes the first one-fourth of the DiT SingleStreamBlocks, increases global batch size to 32, and trains for 1.5K steps with cosine annealing.
C. RealIR-Bench and Metrics Details
RealIR-Bench covers diverse real-world degradation scenarios and evaluates restoration through complementary measures of degradation removal and consistency preservation.
- RealIR-Bench covers blur, rain, noise, low-light, moiré patterns, haze, compression artifacts, reflection, and flare.
- The benchmark uses Restoration Score (RS) for perceptual restoration quality and LPIPS for perceptual similarity after restoration.
- RS measures degradation removal by comparing degradation severity before and after restoration with a vision-language evaluator.
D. More Qualitative Results and Benchmark Evaluation
The paper supplements RealIR-Bench comparisons with qualitative results and evaluations on public benchmarks for deflare, moiré removal, and reflection removal.
- Qualitative Results: Additional visualizations compare RealRestorer with other image editing models on RealIR-Bench across nine degradation types.
- Public Benchmarks: Deflare evaluation uses the Flare-R subset of Flare7K++, containing 100 paired real-world flare images.
- Public Benchmarks: Moiré removal is evaluated on UHDM, while reflection removal uses SIR²+ subsets containing 50, 50, and 101 paired images.
E. Ablation Study Details
The ablation study compares synthetic-only, real-world-only, and two-stage training, showing that the two-stage strategy balances degradation removal with content consistency.
- Training Strategy: Real-world-only training tends to overfit degradation patterns, motivating comparison with the proposed two-stage strategy.
- Training Strategy: At 2.5K steps, synthetic-only training has limited real-world restoration ability, whereas real-world-only training often fails to preserve content consistency.
- Training Strategy: The two-stage strategy balances restoration capability and structural consistency, producing more stable and generalizable performance.
- Benchmark Evaluation: Table 7 reports PSNR and SSIM on Flare-R, UHDM, and SIR²+ benchmarks for deflare, moiré removal, and reflection removal.
F. User Study Details
The user study evaluates restoration outputs from five models with 32 participants, while the surrounding materials specify the benchmark examples, prompts, and comparison interface.
- User Study: Thirty-two participants rate five models’ outputs using restoration quality and consistency criteria.
- Evaluation Setup: The benchmark examples cover degradation categories evaluated with fixed bilingual prompts, including blur, compression, flare, moiré, haze, low-light, noise, and rain.
- Evaluation Setup: The degradation-evaluation instruction asks the evaluator to judge the specified degradation and its severity while ignoring unrelated quality, content, and aesthetic factors.
- Study Materials: Figure 14 compares training strategies, while Figure 15 shows the interface used for participant ratings.