PaperlayerView exact revision on arXiv

Source-linked AI summary

V-Bridge: Bridging Video Generative Priors to Versatile Few-shot Image Restoration

Shenghe Zheng, Junpeng Jiang, Wenbo Li

arXiv:2603.13089v1cs.CV

TL;DR

Image restoration typically relies on task-specific models and substantial supervision. V-Bridge instead uses pretrained video generative priors and progressive refinement for few-shot, multi-task restoration. With 1,000 samples, it achieves competitive restoration and generalizes to unseen tasks.

  • Problem

    Image restoration remains task-specific and data-intensive, leaving transferable priors from large-scale generative models underused.

  • Method

    V-Bridge reformulates restoration as progressive generative refinement, using coarse-to-fine training and lightweight drift correction with pretrained video models.

  • Results

    Using only 1,000 multi-task samples, V-Bridge achieves a 1.6dB gain over baselines trained on 15× to 1,000× more data and generalizes to unseen tasks.

  • Takeaways & Limitations

    Video generative priors can provide transferable restoration knowledge for data-efficient image restoration beyond the tasks seen during training.

  • Takeaways & Limitations

    The resolution gap between moderate-resolution video pretraining and ultra-high-resolution restoration limits faithful recovery of fine details.

Abstract

from arXiv · show

Large-scale video generative models are trained on vast and diverse visual data, enabling them to internalize rich structural, semantic, and dynamic priors of the visual world. While these models have demonstrated impressive generative capability, their potential as general-purpose visual learners remains largely untapped. In this work, we introduce V-Bridge, a framework that bridges this latent capacity to versatile few-shot image restoration tasks. We reinterpret image restoration not as a static regression problem, but as a progressive generative process, and leverage video models to simulate the gradual refinement from degraded inputs to high-fidelity outputs. Surprisingly, with only 1,000 multi-task training samples (less than 2% of existing restoration methods), pretrained video models can be induced to perform competitive image restoration, achieving multiple tasks with a single model, rivaling specialized architectures designed explicitly for this purpose. Our findings reveal that video generative models implicitly learn powerful and transferable restoration priors that can be activated with only extremely limited data, challenging the traditional boundary between generative modeling and low-level vision, and opening a new design paradigm for foundation models in visual tasks.

2 Harbin Institute of Technology, Shenzhen

V-Bridge reframes image restoration as progressive video generation and transfers pretrained video-model priors using extremely limited multi-task data. With 1,000 samples, a single model performs competitive restoration across multiple tasks and rivals specialized architectures.

  • Core approach: V-Bridge formulates image restoration as a progressive generative process that transforms degraded inputs into high-fidelity outputs.The approach uses pretrained video models to simulate gradual refinement rather than treating restoration as static regression.
  • Data efficiency: 1,000 multi-task training samples—less than 2% of existing restoration methods—induce pretrained video models to perform competitive image restoration.This demonstrates effective adaptation from extremely limited supervision.
  • Versatility: A single model achieves multiple restoration tasks while rivaling specialized architectures designed explicitly for image restoration.The result supports using one pretrained video model as a versatile restoration system.
  • Transferable priors: Video generative models implicitly learn powerful, transferable restoration priors that can be activated with extremely limited data.The finding motivates bridging generative video modeling and low-level visual restoration.

1 Introduction

V-Bridge reframes image restoration as progressive video generation, activating transferable video priors for versatile few-shot restoration. With coarse-to-fine training and lightweight drift correction, it achieves competitive or state-of-the-art performance using only extremely limited task-specific data.

  • Motivation: Video generative models encode broad visual priors from massive, diverse video data, including structure, dynamics, lighting, and long-range spatio-temporal coherence.These representations extend beyond appearance statistics and suggest general visual knowledge despite video-synthesis training objectives.
  • Motivation: Conventional image restoration remains task-specific, with denoising and deblurring methods requiring engineered architectures and substantial supervision for each degradation type.The introduction contrasts these approaches with generative modeling advances in high-level vision.
  • Method: V-Bridge treats a degraded image as the initial state and its high-fidelity reconstruction as the endpoint of a progressive quality-refinement trajectory.This formulation replaces static one-step regression with a video-generation process that integrates video generative priors into image restoration.
  • Method: <2% of typical training data is required by V-Bridge, which uses step-wise quality evolution and a coarse-to-fine curriculum to bridge moderate-resolution video pretraining with high-resolution restoration.The curriculum first establishes global structural coherence before refining high-frequency details.
  • Method: A lightweight drift correction mechanism improves fine-grained texture and color fidelity with minimal overhead and task-specific supervision.The framework combines this correction module with progressive generative refinement for data-efficient restoration.
  • Empirical validation: 1K samples suffice for V-Bridge to achieve state-of-the-art results, validating the out-of-distribution adaptability of video priors across low-level tasks.The reported empirical validation supports a unified direction for low-level visual modeling.

2 Related Work

Related work spans the evolution of scalable video generation, unified image restoration, and video models’ emerging use for complex visual inference. These directions motivate treating video generative models as broader visual learners.

  • Video Generation: Video generation progressed from UNet-based architectures with 2D VAE toward large-scale models using 3D VAE and Diffusion Transformers.Early designs struggled with scalability and temporal coherence, while Sora’s scalability helped inspire the newer direction.
  • Image Restoration: Traditional image restoration methods target individual degradations, whereas all-in-one restoration seeks one model for diverse corruption scenarios.Separate models are typically required for motion blur, rain streaks, or noise, limiting scalability and practical deployment.
  • Visual Inference: Video generation research has expanded from motion synthesis toward complex visual inference, assessed through Chain-of-Frames reasoning and benchmarks spanning spatial, logical, planning, and physical capabilities.The passage frames CoF reasoning as a way to characterize emergent intelligence in mature video generation models.

3 Methodology

V-Bridge reframes image restoration as a progressive generative trajectory by converting LQ-HQ pairs into pseudo-temporal sequences. Its multi-stage curriculum and drift correction progressively improve detail synthesis while addressing the resolution gap between video priors and high-resolution restoration.

  • Pseudo-Temporal Sequence Construction: LQ-HQ image pairs become pseudo-temporal sequences whose endpoints are the low-quality anchor and high-quality target, with linearly interpolated intermediate frames.The interpolation uses α_t = t/T and supplies temporally consistent supervision for learning a monotonic low-to-high quality trajectory.
  • Overall Training Objective: The model predicts each progressive state from the initial frame and time index using a supervised reconstruction objective.The objective is the expected reconstruction loss between f_θ(I_0, t) and the constructed target state I_t.
  • Progressive Curriculum Training: Training increases resolution across stages, guiding the model from global restoration at low resolution toward fine-grained detail generation at higher resolution.Stage-dependent downsampling uses an ordered schedule r_1 < r_2 < ⋯ < r_T, progressively increasing spatial fidelity.
  • Resolution Gap and Distribution Drift: Pretrained video models often operate at moderate resolution, creating a mismatch with restoration targets that require ultra-high-resolution detail.The method identifies this mismatch as a resolution-induced distribution drift away from the high-fidelity target manifold.
  • Drift Correction: A drift correction model learns a short trajectory from the base model’s drifted output toward the high-resolution target while preserving structural consistency.Restricting correction to a few intermediate frames keeps the process computationally efficient and reduces resolution-induced bias.

4 Experiments

Experiments show that V-Bridge achieves competitive restoration with extremely limited supervision, generalizes across datasets and unseen corruptions, and benefits from correction and progressive training design choices. Ablations indicate that moderate frame counts and progressive resolution schedules better support restoration than excessive temporal sampling or single-resolution training.

  • Main Results: Using only 1K FoundIR training samples, V-Bridge achieves competitive or superior performance against conventional restoration methods while using 0.1%–7% of their required data.The evaluation compares against Real-ESRGAN, DGUNet, TransWeather, PromptIR, DiffUIR, DA-CLIP, X-Restormer, InstructIR, AutoDIR, FoundIR, and FoundIR-Generalist.
  • Main Results: Across Dense-Haze, UHD-LL, NH-Haze, UAV-Rain1K, and HQ-NightRain, V-Bridge consistently achieves clear performance gains over competing approaches.These out-of-distribution results demonstrate strong cross-dataset generalization and transferable visual priors.
  • Correction Module: 1.4 dB PSNR and 0.024 SSIM are gained from the correction module, improving perceptual fidelity and fine-detail recovery.The correction model addresses resolution bias from video models typically trained on moderate-resolution data such as 720p.
  • Frame Count: 9 frames outperform 33 and 61 frames on the FoundIR test set, showing that performance does not monotonically improve with additional frames.The results suggest that neighboring frames become semantically redundant along smooth restoration trajectories, while excessive temporal sampling adds computational burden.
  • Progressive Curriculum Training: Increasing resolution consistently improves performance, while progressive schedules such as 512+720 and 512+720+960 outperform single-resolution training.The findings support a coarse-to-fine optimization paradigm aligned with the hierarchical nature of image restoration.
  • Few-shot Capability: 200 training samples already produce performance comparable to existing full-data baselines, while performance consistently improves as the training-data amount increases.All models in this comparison are trained for 300 epochs at a fixed 720 resolution.

5 Conclusion · Appendix for V-Bridge · A Details

V-Bridge unlocks restoration in pretrained video generative models through few-shot training by reframing restoration as progressive generative refinement. It uses chain-like frame reasoning to activate transferable visual priors with very few task-specific samples.

  • 5 Conclusion: V-Bridge unlocks restoration capability in pretrained video generative models with few-shot training.The framework is introduced as a method for activating pretrained video models for restoration.
  • 5 Conclusion: The framework reformulates image restoration as a progressive generative refinement process.This reframing underlies the approach described in the conclusion.
  • 5 Conclusion: V-Bridge exploits chain-like frame reasoning to use video foundation models as visual priors.The passage characterizes these priors as powerful and transferable.
  • 5 Conclusion: Video foundation models can serve as powerful visual priors for restoration.The conclusion presents this capability as enabled by the proposed reformulation and reasoning strategy.
  • 5 Conclusion: The activated restoration capability is transferable across visual restoration settings.Transferability is explicitly identified as a property of the visual priors.
  • 5 Conclusion: V-Bridge requires only a very small number of task-specific samples to activate strong restoration capability.The conclusion emphasizes the unusually limited amount of task-specific supervision used.

A.1 Training Details

V-Bridge trains on balanced HQ/LQ image pairs spanning diverse degradations and uses a consistent three-stage strategy that progressively increases input video resolution from 512 to 960.

  • Data: Training pairs combine high-quality and low-quality images from FoundIR and RealCE.FoundIR supplies diverse restoration degradations, while RealCE primarily provides Chinese and English text-degradation data.
  • Data: Uniform sampling across FoundIR’s degradation categories maintains a balanced training distribution.The categories include blur, lowlight, JPEG compression, haze, rain, and combinations of multiple degradations.
  • Training Strategy: Three training stages use input video resolutions of 512, 720, and 960, while other hyperparameters remain consistent.The staged strategy follows the setup described in Sec. 3.3, with resolution as the primary difference across stages.

A.2 Test Details

The evaluation uses diverse restoration benchmarks spanning blur, low-light, compression, haze, rain, snow, and combined degradations, with a unified restoration prompt and resolution-aware inference. Performance is measured using image-level PSNR and SSIM scores averaged across each dataset.

  • Benchmarks: FoundIR contains 1,500 samples covering blur, low-light, JPEG compression, haze, rain, and combinations of multiple degradations.Most samples have resolutions of 1080p or higher.
  • Benchmarks: Dense-Haze provides 33 real-world hazy and haze-free outdoor image pairs, while UHD-LL provides 115 4K image pairs across darkness and noise conditions.Dense-Haze uses densely distributed, visually uniform haze created with professional haze machines.
  • Benchmarks: WeatherBench contributes 200 snow-degraded and high-quality image pairs, making snow removal an unseen task because snow is absent from training degradations.Evaluation uses only the snow subset.
  • Inference: During inference, one unified prompt directs static-camera restoration that preserves composition while removing artifacts and improving clarity, and inputs above 2K are resized to approximately 2K before restoration.Outputs are rescaled to the original resolution afterward, while negative prompts suppress motion, scene changes, artifacts, and stylistic alterations.
  • Metrics: PSNR and SSIM are computed per image and averaged over each dataset, with higher values indicating better performance.PSNR uses reconstruction error relative to the maximum pixel value, while SSIM compares image means, variances, and covariance.

B Additional Experimental Results · B.1 Visualization Results · B.2 Failure Cases

The paper includes additional visualization results and documents failure cases, with comparisons to FoundIR-G on the same data. These materials are presented in Figures 7–10.

  • B.1 Visualization Results: Additional visualization results are presented in Figure 7.
  • B.1 Visualization Results: Additional visualization results are presented in Figure 8.
  • B.1 Visualization Results: Additional visualization results are presented in Figure 9.
  • B.2 Failure Cases: The method’s failure cases are shown in Figure 10.
  • B.2 Failure Cases: The failure cases include corresponding visualization results of FoundIR-G.
  • B.2 Failure Cases: FoundIR-G visualization results are provided on the same data as the method’s failure cases.

C Limitations and Future Work

The work demonstrates the feasibility and potential of transferring video-generation priors to low-level vision, while identifying task generalization and efficiency as key limitations. Future work will extend this transfer to broader tasks to further assess video-generation models as vision foundation models.

  • Current Scope: The study primarily investigates whether video-generation models’ prior knowledge can transfer to low-level vision tasks.It demonstrates the feasibility, effectiveness, and potential of this transfer.
  • Limitations: The current approach has two main limitations: task generalization and efficiency.
  • Future Work: Future work will transfer video-generation priors to a broader range of tasks and further validate their potential as vision foundation models.
Loading 2603.13089v1…