PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Refocusing: Flexible Defocus Control from a Single Image

Chun-Wei Tuan Mu, Cheng-De Fan, Jia-Bin Huang, Yu-Lun Liu

arXiv:2512.16923v3cs.CV

TL;DR

Single-image refocusing must recover sharp content while creating realistic bokeh, yet existing methods have limits in input flexibility, training data, and aperture control. Generative Refocusing uses two stages and a hybrid training scheme, and reports top performance across defocus deblurring, bokeh generation, and refocusing.

  • Problem

    Single-image refocusing must recover sharp detail in blurred regions and create natural-looking bokeh, while existing methods have limits in input flexibility, data realism, and aperture-shape control.

    Source: §1 · p. 2

  • Method

    GenRefocus uses DeblurNet to recover an all-in-focus image and BokehNet to synthesize controllable bokeh, with training that completes missing signals in real-world data.

    Source: §1 · p. 3

  • Results

    GenRefocus achieves top performance in defocus deblurring, bokeh generation, and refocusing, with consistent improvements across benchmarks.

    Source: §1 · p. 2, §5 · p. 14

  • Takeaways & Limitations

    The system supports aperture-shape editing and learns authentic optical behavior beyond simulators.

    Source: §5 · p. 14

    With severe blur, the diffusion method may hallucinate textures instead of recovering fine details, making it unsuitable for precision-critical or safety-sensitive applications.

    Source: Limitations and future work · p. 15

Abstract

from arXiv · show

Depth-of-field control is essential in photography, but achieving perfect focus often requires multiple attempts or specialized equipment. Single-image refocusing is still difficult. It involves recovering sharp content and creating realistic bokeh. Current methods have significant drawbacks. They require all-in-focus inputs, rely on synthetic data from simulators, and have limited control over the aperture. We introduce Generative Refocusing, a two-step process that uses DeblurNet to recover all-in-focus images from diverse inputs and BokehNet to create controllable bokeh. This method combines synthetic and real bokeh images to achieve precise control while preserving authentic optical characteristics. Our experiments show we achieve top performance in defocus deblurring, bokeh synthesis, and refocusing benchmarks. Additionally, our Generative Refocusing allows custom aperture shapes. Project page: https://generative-refocusing.github.io/

1 Introduction

Single-image refocusing aims to recover sharp details and create natural bokeh, but existing approaches face limitations in input flexibility, training data, and aperture-shape control. Generative Refocusing combines a two-stage pipeline with a training scheme for real-world data and reports top performance across three tasks.

1 Introduction

  • Existing methods often require all-in-focus inputs and accurate depth maps, while synthetic data can lack realism and real datasets may vary only focus or aperture.This leaves real datasets without the joint focus-and-aperture variation needed for flexible refocusing; many methods also lack aperture-shape control.Source: §1 · p. 2
  • GenRefocus achieves top performance in defocus deblurring, bokeh generation, and refocusing while maintaining natural scene consistency.The system is validated on DPDD, RealDOF, and the new LF-Bokeh and LF-Refocus datasets.Source: §1 · pp. 2–3
  • The two-stage pipeline accepts images in any focus state and gives users control over focus plane, bokeh intensity, and aperture shape.DeblurNet recovers an in-focus image, and BokehNet synthesizes customizable bokeh.Source: §1 · pp. 2–3
  • Its training scheme completes missing input or control signals in real-world datasets, combining synthetic data's geometric consistency with authentic lens aberrations.The approach uses auxiliary information and dedicated processing to fill missing defocus maps.Source: §1 · pp. 2–3

2 Related Work

Prior work addresses restoration, defocus deblurring, bokeh rendering, refocusing, and camera-conditioned generation, often with narrower inputs or controls. GenRefocus is positioned as single-image editing that separates deblurring from bokeh synthesis.

Diffusion Models for Image Restoration

Defocus Deblurring

  • Defocus-deblurring work progressed from sensor and disparity-aware methods to transformers, implicit representations, multi-scale attention, and diffusion-based learning.GenRefocus instead uses deblurring as the first stage of controllable refocusing.Source: Defocus Deblurring · pp. 3–4

Bokeh Rendering

  • Bokeh rendering spans physical and differentiable methods, neural rendering, variable f-stops, and diffusion, but many approaches require all-in-focus inputs and accurate depth.GenRefocus relaxes that input constraint by accepting defocused or fully in-focus images.Source: Bokeh Rendering · pp. 4–5

Single-Image Refocusing

  • Earlier refocusing methods include simulator-trained systems, real-data methods with limited controls, and 3D representations requiring multi-view capture.GenRefocus combines hybrid training, flexible single-image inputs, and control over focus, intensity, and aperture shape.Source: Single-Image Refocusing · p. 5

Camera-Conditioned Diffusion

  • Camera-conditioned diffusion work often targets generation, whereas GenRefocus edits images while maintaining scene consistency and avoiding multi-frame overhead.The paper attributes efficient single-image manipulation to decoupling the deblurring and bokeh stages.Source: Camera-Conditioned Diffusion · p. 5

3 Method

Generative Refocusing decomposes single-image refocusing into defocus deblurring followed by controllable bokeh synthesis, with separate models for recovering an all-in-focus image and producing the refocused output.

3.1 Stage 1: Defocus Deblurring

  • DeblurNet concatenates noisy latent tokens with encoded blurry-input tokens during denoising, decodes the result as an AIF image, and trains on real paired data.Source: §3.1 · pp. 5–6

3.2 Stage 2: Shallow Depth-of-Field Synthesis

  • BokehNet conditions on AIF and Ddef to synthesize refocused output, trained with simulated and real bokeh supervision whose focus planes and K labels are estimated or calibrated.Synthetic data randomly samples focus plane and K; real-data routes estimate focus plane from an in-focus mask, and LFDOF/RealBokeh use manually refined masks when initial estimates are unreliable.Source: §3.2 · p. 6, Training data and supervision · pp. 7–8

3.3 Bokeh-Shape Aware Synthesis

3.4 Pre-deblur Module

  • A variant conditions DeblurNet on both the input and a conservative pre-deblur prior from a non-generative model, whose outputs are content-faithful but overly smooth.Source: §3.4 · p. 9

3.5 Implementation Details

  • Overlapping-patch inference tiling addresses prohibitive VRAM use and quality degradation on ultra-high-resolution inputs while blending results at their original resolution.Source: §3.5 · pp. 9–10

4 Experiments

Experiments evaluate defocus deblurring, bokeh synthesis, and refocusing, alongside tests of the two-stage design, training data, and aperture-shape control. The method performs strongly across the three tasks, while ablations favor staged processing and combined training data.

4.1 Setup

  • DeblurNet uses 3.5K training pairs, while BokehNet training combines approximately 70K synthetic pairs with 26K real examples.The reported setup trains BokehNet first on synthetic data and then on real data.Source: Datasets · p. 10, Backbone and training details · p. 10
  • Evaluation covers deblurring on RealDOF and DPDD, bokeh synthesis on LF-Bokeh, and refocusing on LF-Refocus.LF-Bokeh contains 200 images, and LF-Refocus contains 400 source–target pairs; the bokeh and refocusing evaluations search for an optimal bokeh level K per image or pair.Source: Benchmarks and Metrics · p. 11

4.2 Comparison

  • DeblurNet outperforms competing methods across most referenced and non-referenced metrics.On RealDOF, it scores slightly below Bokehlicious on reference metrics but is reported to produce more visually appealing results and greater robustness to grayscale and low-quality blurry inputs.Source: §4.2 · p. 12
  • Bokeh synthesis results outperform all methods across fidelity metrics and all diffusion-based baselines in controllability.BokehMe achieves near-perfect control, attributed to its hybrid design with a classical, physically motivated renderer.Source: Bokeh synthesis · p. 12
  • GenRefocus achieves the best results on both fidelity and perceptual metrics in refocusing.The comparison uses LF-Refocus; qualitative results show fine detail at the intended focus and physically consistent bokeh elsewhere.Source: Table 4 · p. 12, Refocusing · p. 13

4.3 Ablation Study

  • The two-stage pipeline significantly outperforms direct refocusing trained on identical synthetic data.The authors attribute the gap to degraded defocus-map accuracy from blurry inputs and the direct formulation’s inability to use tailored real-data supervision for separate subtasks.Source: Direct refocusing vs. deblurring then… · p. 13, Table 5 · p. 14
  • Combining synthetic data, ITW, and LFDOF/RealBokeh yields the strongest BokehNet training results and best optical characteristics.Adding either real-data source to synthetic training also improves performance.Source: BokehNet training ablation · p. 14, Table 6 · p. 14
  • After fine-tuning on the PointLight-1K dataset, BokehNet synthesizes bokeh with user-specified aperture shapes.The demonstrated shapes include triangles, hearts, and stars; background point lights exhibit the specified geometry while maintaining scene consistency.Source: Aperture shape control · p. 14

5 Conclusion

Generative Refocusing uses a two-stage diffusion framework to turn single images into controllable virtual cameras, with focus-plane, bokeh-intensity, and aperture-shape controls. The paper reports improvements across its benchmarks and identifies failure conditions that bound its use.

5 Conclusion

  • GenRefocus handles arbitrary inputs with controls over focus plane, bokeh intensity, and aperture shape, and experiments show improvements across all benchmarks.The conclusion also reports aperture-shape editing; Fig. 7 demonstrates triangle, heart, and star shapes.Source: §5 · p. 14, Figure 7 · p. 15
  • Defocus-map errors can misassign blur; severe-blur inputs may produce hallucinated textures, making the method unsuitable for precision-critical or safety-sensitive applications.Transparent surfaces are especially challenging, and complex aperture shapes require simulator-driven training data.Source: Limitations and future work · p. 15, Figure 8 · p. 15, Figure 9 · p. 15

A Overview

  • The supplementary material adds training-dataset and PointLight-1K construction details, extended comparisons, controllability results, and in-the-wild examples.It also analyzes ITW data and the pre-deblur module.Source: A Overview · p. 21

B Additional Training Details

This section introduces its coverage of dataset processing for training DeblurNet and BokehNet.

B Additional Training Details

B.1 DeblurNet Training Data

DeblurNet training combines paired defocus-blur data with selected sharp images.

B.1 DeblurNet Training Data

  • DeblurNet uses DPDD supervised pairs and the 3,000 sharpest images selected from RealBokeh_3MP by Laplacian variance.The RealBokeh_3MP images supplement the official DPDD training split as sharp-image supervision.Source: §B.1 · p. 21

B.2 BokehNet Training Data

BokehNet training combines sharp inputs and real bokeh images, while PointLight-1K supplies night scenes with prominent point lights for controllable aperture-shape training.

B.2 BokehNet Training Data

  • BokehNet training uses about 1.7K sharp images and 26K real bokeh images, including 13K verified ITW images and 13K newly curated images.Newly collected images are focus-consistent series captured at varying apertures, with 2 to 4 images per set.Source: §B.2 · p. 21
  • New bokeh series were manually verified for in-focus masks, assigned optimized K values using a simulator, and filtered by an SSIM threshold.Mask annotation took 4 to 8 seconds per image, about 8 hours total.Source: §B.2 · p. 21

C PointLight-1K Collection Pipeline

  • PointLight-1K is a curated set of 1k night-time images with prominent point lights and minimal residual blur, used to pretrain and refine learning for controllable bokeh shape.Collected keywords are expanded into natural-language prompts with GPT-4o; fine-tuned FLUX generates images, which are then passed through DeblurNet to reduce residual blur.Source: C PointLight-1K Collection Pipeline · p. 22

D Additional Comparison with DiffCamera

The additional comparisons assess performance under controlled inputs and on diverse photographs. They report improvements over DiffCamera and qualitative comparisons with Bokehlicious and a vision-language model.

D Additional Comparison with DiffCamera

  • On DPDD, using 512 × 512 central crops to match DiffCamera’s native training resolution and fixed aspect ratio, our method achieves lower LPIPS (0.1285 vs. 0.2833) and DISTS (0.1045 vs. 0.1793).Source: D Additional Comparison with DiffCamera · p. 22

E Qualitative comparison with Bokehlicious

  • On historical and in-the-wild images, Bokehlicious fails to recover sharp content reliably, while our method restores details across these diverse conditions.Bokehlicious either has negligible deblurring effect or introduces structural artifacts; the comparison includes monochrome historical photographs.Source: E Qualitative comparison with… · p. 23

F Qualitative comparison with VLM model

  • For the prompt “focus on the man on the right,” Nano Banana Pro changes facial appearances, whereas our method preserves content while refocusing.The comparison evaluates text-driven refocusing qualitatively.Source: F Qualitative comparison with VLM model · p. 23

G Qualitative Results for Controllability

Qualitative results examine how the blur parameter K and defocus map affect refocusing. Scene blurriness increases monotonically as K grows.

G Qualitative Results for Controllability

H ITW Dataset Analysis

The ITW dataset analysis describes real optical bokeh and distributions of parameters used to estimate K. Additional figures compare blur conditions and show how pre-deblurring affects recovered detail.

H ITW Dataset Analysis

I The effect of the pre-deblur module.

  • Conditioning on a pre-deblurred estimate helps recover fine details such as “NEW YORK,” even when that estimate remains distorted.Without this condition, the example has distorted characters.Source: I The effect of the pre-deblur module · p. 24, Figure 17 · p. 29
  • Across historical and in-the-wild images, Bokehlicious either leaves blur or introduces structural artifacts, while the proposed method handles diverse blur conditions.Source: Figure 10 · p. 25
  • The controllability figure shows scene blurriness increasing monotonically across blur levels K ∈ {0, 5, 10, 15}.Source: Figure 12 · p. 26
Loading 2512.16923v3…