Source-linked AI summary
The First Controllable Bokeh Rendering Challenge at NTIRE 2026
Tim Seizinger, Florin-Alexandru Vasluianu, Jeffrey Chen, Zhuyun Zhou, Zongwei Wu, Radu Timofte, Dafeng Zhang, Yipeng Lin, Qi Yan, Junhao Chen, Yang Yang, Divyavardhan Singh, Hariom Thacker, Hammad Mohammad, Aanchal Maurya, Kishor Upla, Kiran Raja, Wei Zhou, Hongyu Huang, Yujin Cho, Grigory Malivenko, Jiachen Tu, Yaokun Shi, Guoyi Xu, Yaoxin Jiang, Jiajia Liu
TL;DR
Natural bokeh is barely noticeable on smartphones, and many prior methods lack adjustable bokeh strength. The paper presents the first NTIRE challenge assessing fidelity and perceptual quality; although many entries improved reconstruction fidelity over the baseline, none significantly outperformed it overall.
Problem
Natural bokeh is barely noticeable on smartphones, while many prior methods do not allow strength adjustment, motivating computational rendering with controllable bokeh.
Source: §1 · pp. 1–2Method
Results
Takeaways & Limitations
The challenge explored ideas advancing bokeh-rendering research, and participant feedback will inform later editions.
Source: §8 · p. 8
Abstract
from arXiv · showhide
This study presents the outcomes of the first Controllable Bokeh Rendering Challenge at NTIRE and highlights the most effective submitted methodologies. In total, 44 participants registered for the competition, of which 8 teams submitted valid solutions after the conclusion of the final test phase. All submissions were evaluated on unseen images, focusing on portraits and intricate subjects with complex and visually appealing bokeh phenomena. In addition to the first track focusing on established quantitative fidelity metrics, we conducted a qualitative user study with a panel of experts for a second track focusing on perceptual assessment. As this was the inaugural challenge on this topic, most of the participants focused on refining and extending the Bokehlicious baseline method.
1. Introduction
Smartphones need computational rendering to reproduce controllable Bokeh, whose realism and strength are difficult to model. The section motivates NTIRE’s first challenge as a step toward photorealistic, controllable rendering.
- Bokeh is a smooth, pleasing blur that emphasizes subjects by turning backgrounds into low-contrast smooth areas or intricate circular patterns in high-contrast regions.Source: §1 · p. 1
- Because smartphone optics produce barely noticeable natural Bokeh, computational systems are needed to render an equivalent effect [25] [72].Source: §1 · p. 1
- Classical image-space methods address stability, speed, and fidelity but hand-crafted kernels constrain realism [48] [52] [6] [41] [72] [83] [25] [62], while neural approaches learn paired-aperture rendering, sometimes combining both paradigms [48] [25] [75].Source: §1 · p. 1
- Many methods cannot adjust Bokeh strength, while synthetic-data approaches can face a real-image domain gap and may rely on explicit depth priors [25] [16] [34] [45].Source: §1 · p. 2
- Photorealistic rendering must preserve subject sharpness and fine structures while handling HDR light, inpainting, lens PSFs, and hierarchical effects such as eclipsing [72] [62] [83] [52].Source: §1 · p. 2
- Building on Bokehlicious and RealBokeh, the organizers launched NTIRE’s first Controllable Bokeh Rendering Challenge to engage researchers with the task.Source: §1 · p. 2
2. Challenge Data
The challenge uses RealBokeh, a real-world dataset of aligned aperture-varying image pairs, extended with new private-reference scenes for fair evaluation.
2. Challenge Data
- RealBokeh [62] provides 23K wide-/small-aperture image pairs across 4.4K scenes, captured by expert photographers with an automated protocol to maintain alignment.It was introduced to address prior datasets’ limitations in realism, scale, and alignment quality [25] [83].Source: §2 · p. 2
- Each scene has five images: an f/22.0 input, three intermediate ground truths at random f-stops from f/20.0 to f/2.2, and an f/2.0 image capturing realistic lens variability.The f-stop range is sampled in 1/3-stop increments.Source: §2 · p. 2
- For the challenge, the validation and test splits were replaced with 146 new scenes captured under the original protocol, and their reference images were kept private.The private references were intended to ensure fair competition.Source: §2 · p. 2
3. Evaluation
The challenge used two tracks with separate ranking criteria: quantitative fidelity metrics and expert assessment of perceived bokeh. The Fidelity Track weighted PSNR, SSIM, and LPIPS equally, while the Perceptual Track used expert-photographer MOS.
Fidelity Track
- LPIPS feature extraction used ImageNet-pretrained AlexNet.Source: §3 · p. 2
- The Fidelity Track weighted PSNR, SSIM, and LPIPS equally to rank rendered images.The assessment included reconstruction fidelity in PSNR, SSIM scores, and LPIPS distance from ground truth.Source: §3 · pp. 2–3
Perceptual Track
- The Perceptual Track ranked rendered bokeh using Mean Opinion Scores from expert photographers.A panel of photography experts determined the MOS.Source: §3 · pp. 2–3
4. Challenge Phases
The challenge had a development phase using RealBokeh training data and private validation ground truths, followed by a test phase with a separate private split and restricted submissions.
Development phase
- Development-phase participants received 20,500 RealBokeh training samples and 78 private-ground-truth validation samples captured with the same optical system.They submitted predictions to a remote CodaBench server to measure validation performance.Source: §4 · p. 3
Test phase
- The test phase provided 68 additional samples, evaluated against private ground truths via CodaBench, and limited submissions to restrict fine-tuning.Final submission requirements included a method description, code and checkpoints, team-member information, and predictions for all 68 test samples.Source: §4 · p. 3
- For the final submission, each team provided its method description, code with checkpoints, team-member information, and predictions for the 68-sample private testing split.These materials were submitted after the test phase, in which the number of submissions was restricted.Source: §4 · p. 3
5. User Study
The perceptual track used expert-panel Mean Opinion Scores (MOS) to assess bokeh rendering. The panel prioritized subject sharpness, attractive and accurate bokeh highlights, gradual sharpness falloff, and freedom from processing artifacts.
- Four photography experts assigned MOS on a 1–10 scale; input images were calibrated to 3, scores 1–2 marked degradation, and bokeh quality ranged from 3–10 in 0.5 increments.Source: §5 · p. 3
- The panel prioritized maintaining critical subject sharpness, especially in fine details such as hair.Source: §5 · p. 3
- The panel valued well-defined, aesthetically pleasing out-of-focus highlights (bokehballs) with accurate color and saturation.Source: §5 · p. 3
- The panel valued correctly rendered gradual sharpness falloff across continuous foreground-to-background transitions.Source: §5 · p. 3
- The panel also prioritized an overall appearance free from processing artifacts such as blockiness or ghosting.Source: §5 · p. 3
6. Challenge Results
Many challenge submissions improved reconstruction fidelity over the large Bokehlicious baseline, but only YuFans was preferred over it in the user study. Fidelity and perceptual rankings correlated weakly, and qualitative examples showed artifacts and detail loss in some submissions.
6. Challenge Results
- Eight valid submissions were received and are presented in Section 7.Source: §6 · p. 4
- Many submissions improved reconstruction fidelity over the large Bokehlicious baseline, but only YuFans was preferred over it in the user study.Table 2 reports quantitative evaluations for both fidelity and perceptual tracks; Fig. 3 shows qualitative examples.Source: §6 · p. 4
- The top fidelity method ranked fourth in the perceptual track, and fidelity and perceptual rankings correlated only weakly.The perceptual ranking showed a notable MOS difference.Source: §6 · p. 4
- NJUST-KMG and CV SVNIT showed artifacts under strong bokeh, including misaligned blended PSFs and reduced foreground detail, while YuFans slightly lost detail in PSF structure.The PSF blending may have resulted from output fusion of separate architectural branches.Source: §6 · p. 4
- The PSNR, SSIM, and LPIPS leaders—NJUST-KMG and CV SVNIT, respectively—had artifacts that reduced their user-study scores, suggesting standard metrics may miss perceptual quality.The reported observations concern this challenge iteration's commonly used qualitative metrics for Bokeh rendering.Source: §6 · p. 4
7. Challenge Methods
The teams adapt Bokehlicious- and NAFNet-based models, as well as U-Net pipelines, for controllable bokeh rendering. Their methods use aperture conditioning, depth guidance, refinement, or inference-time augmentation.
7.1. Davinci
- Davinci combines Bokehlicious coarse rendering with NAFNet refinement, using aperture-aware attention and calibrated visual prompts to refine subject boundaries, hair, and residual artifacts.The coarse stage varies its attention-mask width according to the desired f-stop; prompt-guided refinement helps preserve fine details and texture in focused regions.Source: §7.1 · pp. 4–5
7.2. NJUST-KMG
- NJUST-KMG augments BokehliciousL with a concatenated depth map, progressively masks depth during training to limit reliance on noisy predictions, and averages original and augmented model outputs.The ensemble uses a 1:1 output weighting and additional test-time augmentation.Source: §7.2 · pp. 5–6
7.3. YuFans
- YuFans enhances Bokehlicious-Large [62] with 8× geometric test-time augmentation, averaging predictions across four rotations and two flips.The team’s findings were also submitted to the workshop.Source: §7.3 · p. 6
7.4. CV SVNIT
- CV SVNIT’s HAFT encodes the target f-stop to condition a U-Net and adds depth-guided refinement using depth, foreground, focal-prior, and focus-weight maps.The aperture embedding is injected into encoder and decoder blocks; refinement targets subject boundaries and artifacts near depth discontinuities.Source: §7.4 · p. 6
7.5. BIT ssvgg
- BIT ssvgg adds continuous aperture control to a NAFNet-based restoration model by embedding the source-to-target aperture ratio and applying feature-wise modulation in its blocks.The ratio is logarithmically mapped to a feature vector, which generates channel-wise scaling and shifting parameters for intermediate features.Source: §7.5 · pp. 6–7
7.6. Centre Borelli
- Centre Borelli’s DALU-Net conditions a U-Net on monocular depth, pixel coordinates, and aperture via FiLM, and uses adversarial fine-tuning for realistic bokeh synthesis.Its depth map is estimated from the f/22 input with Marigold, and the discriminator receives a bokeh-strength map during adversarial fine-tuning.Source: §7.6 · p. 7
7.7. higasa
- higasa extends NAFNet with aperture-dependent selectable convolution kernels and decoder-skip correction blocks that sample according to aperture and focal length.The correction block uses two trainable parameters per aperture and focal-length value to control linear and quadratic grid distortions.Source: §7.7 · p. 7
7.8. NTR
- NTR fine-tunes a TimeDiffiT U-Net pretrained with Masked Diffusion Autoencoding on bokeh rendering, then uses tiled inference and an 8-way geometric self-ensemble.MDAE pretraining lasts 65 epochs; task fine-tuning uses an L1 pixel reconstruction loss.Source: §7.8 · p. 8
8. Conclusion
The first NTIRE Controllable Bokeh Rendering Challenge advanced an under-studied task through two evaluation tracks, exploring approaches to fidelity and perceptual quality. No solution significantly outperformed the baseline, but participant ideas and feedback can inform future editions.
8. Conclusion
- No solution in the challenge’s initial iteration significantly outperformed the baseline, although participants explored ideas that advanced the study of bokeh rendering.Participant feedback and recommendations will inform the design of subsequent editions.Source: §8 · p. 8
- The challenge evaluated fidelity and perceptual quality across two distinct tracks.Participants simultaneously targeted both qualities.Source: §8 · p. 8
- The challenge promoted an under-studied task, attracting many researchers, while a select group submitted solutions for final evaluation and ranking.Numerous participants also provided substantive feedback and recommendations for subsequent editions.Source: §8 · p. 8
A. Teams and Affiliations · YuFans
The YuFans team submitted Bokehlicious-Large with 8× Geometric TTA, with members Wei Zhou and Hongyu Huang affiliated with the National University of Singapore and Zhejiang University, respectively.
YuFans
- YuFans submitted Bokehlicious-Large with 8× Geometric TTA.Source: YuFans · p. 13
- The submission title specifies 8× Geometric TTA.Source: YuFans · p. 13
- Wei Zhou is listed as a member of YuFans.Source: YuFans · p. 13
- Hongyu Huang is listed as a member of YuFans.Source: YuFans · p. 13
A. Teams and Affiliations
- Wei Zhou’s listed affiliation is the National University of Singapore.Source: YuFans · p. 13
- Hongyu Huang’s listed affiliation is Zhejiang University.Source: YuFans · p. 13
Centre Borelli · higasa
The two entries describe a depth- and location-conditioned U-Net and aperture- and focal-length-dependent sampling.
Centre Borelli
- Centre Borelli presents “Bokeh simulation with depth and location conditioned U-Net,” by Yujin Cho of ENS Paris-Saclay.Source: Centre Borelli · p. 13
higasa
- higasa presents “Aperture and focal length dependent sampling,” by solo developer Grigory Malivenko.Source: higasa · p. 13
Related papers
This paper cites
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- Score-Based Generative Modeling through Stochastic Differential Equations2020
- FiLM: Visual Reasoning with a General Conditioning Layer2017
- NTIRE 2026 Challenge on Video Saliency Prediction: Methods and Results2026
- Synthetic Depth-of-Field with a Single-Camera Mobile Phone2018