Source-linked AI summary
The First Controllable Bokeh Rendering Challenge at NTIRE 2026
Tim Seizinger, Florin-Alexandru Vasluianu, Jeffrey Chen, Zhuyun Zhou, Zongwei Wu, Radu Timofte, Dafeng Zhang, Yipeng Lin, Qi Yan, Junhao Chen, Yang Yang, Divyavardhan Singh, Hariom Thacker, Hammad Mohammad, Aanchal Maurya, Kishor Upla, Kiran Raja, Wei Zhou, Hongyu Huang, Yujin Cho, Grigory Malivenko, Jiachen Tu, Yaokun Shi, Guoyi Xu, Yaoxin Jiang, Jiajia Liu
TL;DR
The challenge addresses controllable Bokeh rendering, a difficult task requiring sharp subjects, detailed separation, realistic light modeling, inpainting, and complex lens effects. It evaluates submitted methods through quantitative fidelity and expert perceptual assessment, finding that no initial solution significantly outperformed the Bokehlicious baseline. The results also indicate that fidelity metrics may not fully capture perceptual quality.
Problem
Smartphones show little natural Bokeh, while controllable rendering must handle subject sharpness, fine structures, lighting, inpainting, lens PSFs, and hierarchical effects.
Method
The study organizes the first NTIRE Controllable Bokeh Rendering Challenge, evaluating methods across quantitative fidelity and expert-based perceptual tracks.
Results
No solution in the initial challenge significantly outperformed the Bokehlicious baseline, while fidelity and perceptual rankings correlated only weakly.
Takeaways & Limitations
The challenge advanced Bokeh-rendering research and suggests that common qualitative metrics may not fully capture perceptual quality.
Abstract
from arXiv · showhide
This study presents the outcomes of the first Controllable Bokeh Rendering Challenge at NTIRE and highlights the most effective submitted methodologies. In total, 44 participants registered for the competition, of which 8 teams submitted valid solutions after the conclusion of the final test phase. All submissions were evaluated on unseen images, focusing on portraits and intricate subjects with complex and visually appealing bokeh phenomena. In addition to the first track focusing on established quantitative fidelity metrics, we conducted a qualitative user study with a panel of experts for a second track focusing on perceptual assessment. As this was the inaugural challenge on this topic, most of the participants focused on refining and extending the Bokehlicious baseline method.
1. Introduction
Bokeh creates smooth out-of-focus backgrounds that emphasize subjects, but computational rendering must address subject sharpness, fine structures, lighting, inpainting, lens blur, and complex occlusion effects. The challenge builds on Bokehlicious and RealBokeh to advance controllable, photorealistic rendering.
- Bokeh is a smooth, pleasant blur of out-of-focus backgrounds that emphasizes subjects and transforms distracting areas into visually pleasing patterns.
- Smartphones require computational systems to reproduce Bokeh because their tiny optical systems make the effect barely noticeable.
- Earlier controllable neural methods often lacked adjustable Bokeh strength or relied on synthetic data that can suffer from a domain gap on real images.
- Bokeh rendering requires subject sharpness, fine-structure separation, light and color modeling, background inpainting, space-varying PSFs, and hierarchical effects such as eclipsing.
- The challenge extends prior Bokehlicious architecture and RealBokeh work as a foundation for photorealistic and controllable Bokeh rendering.
2. Challenge Data
RealBokeh provides aligned, diverse real-world aperture data for neural Bokeh rendering, and the challenge extends it with private scenes for fair evaluation.
- RealBokeh contains 23K wide-/small-aperture image pairs across 4.4K scenes, captured by expert photographers with an automated protocol.
- Each scene includes one f/22.0 input, three randomly sampled intermediate f-stops from f/20.0 to f/2.2, and one f/2.0 reference image.
- The challenge replaced the original validation and test splits with 146 new scenes whose reference images remained private for fair competition.
3. Evaluation
The challenge used two tracks: quantitative fidelity evaluation with equally weighted PSNR, SSIM, and LPIPS, and perceptual evaluation through expert-photographer MOS scores.
- Metrics: The evaluation criteria include reconstruction fidelity, Structural Similarity Index, LPIPS distance, and expert-photographer Mean Opinion Score.
- Evaluation tracks: The Fidelity Track ranks solutions using equally weighted PSNR, SSIM, and LPIPS metrics.
- Evaluation tracks: The Perceptual Track ranks rendered Bokeh using Mean Opinion Scores from a panel of photography experts.
4. Challenge Phases
The challenge proceeded from development to restricted test evaluation, using private ground truths and remote CodaBench scoring to limit test-set fine-tuning.
- Development phase: During development, participants received 20,500 training samples and 78 validation samples with private ground truths evaluated through CodaBench.
- Test phase: During testing, participants received 68 additional samples, submitted predictions against private ground truths, and faced limits on submissions to restrict fine-tuning.
5. User Study
The user study evaluated perceptual Bokeh quality using expert ratings, emphasizing subject sharpness, attractive and accurate bokeh highlights, smooth sharpness transitions, and artifact-free appearance.
- Mean Opinion Score (MOS) was the primary perceptual-track criterion, determined by four photography experts on a 1–10 scale.The input image was calibrated to score 3, while 1–2 indicated perceptual degradation without successful bokeh rendering.
- Experts prioritized maintaining critical subject sharpness, especially for fine details such as hair.
- Experts valued well-defined, aesthetically pleasing out-of-focus highlights with accurate color and saturation.These highlights are also called bokehballs.
- Experts assessed whether rendering preserved gradual sharpness falloff across continuous foreground-to-background transitions.
- Experts also required an overall appearance free from processing artifacts such as blockiness or ghosting.
6. Challenge Results
Eight valid submissions were evaluated across fidelity and perceptual tracks. Although many methods improved reconstruction fidelity over the Bokehlicious-Large baseline, only YuFans was preferred by users, and fidelity metrics aligned weakly with perceptual quality.
- Eight valid submissions participated in the final challenge evaluation.
- Many solutions improved reconstruction fidelity over the Bokehlicious-Large baseline, but only YuFans was preferred in the user study.
- Fidelity and perceptual rankings correlated only weakly, with the fidelity leader placing fourth in the perceptual track.
- NJUST-KMG and CV SVNIT produced artifacts when rendering strong, distinct bokeh effects, while YuFans showed slight PSF-detail loss.
- PSNR, SSIM, and LPIPS leaders still received reduced user-study scores when their outputs contained perceptual artifacts.
7. Challenge Methods
Challenge submissions mainly extended Bokehlicious with aperture, depth, positional, or refinement mechanisms for controllable bokeh rendering. The methods included two-stage refinement, depth augmentation, aperture-conditioned features, lightweight conditioning, and adversarially trained U-Nets.
- Davinci: Davinci combines a Bokehlicious coarse renderer with a NAFNet refinement stage for controllable bokeh rendering.The coarse stage uses aperture-aware attention, while refinement uses visual prompts to improve boundaries and residual artifacts.
- NJUST-KMG: NJUST-KMG augments Bokehlicious with depth maps, progressive depth masking, model averaging, and test-time augmentation.The original module is frozen while the enhanced depth-integrated module is optimized during training.
- YuFans: YuFans improves Bokehlicious-Large through 8× geometric test-time augmentation using four rotations and two flips.Predictions are averaged across the augmented inputs.
- CV SVNIT: CV SVNIT’s HAFT encodes the f-stop into a 64-dimensional aperture embedding and injects it throughout U-Net feature extraction.Its inputs include RGB, positional, bokeh-strength, and Circle-of-Confusion maps, followed by depth-guided refinement.
- BIT ssvgg: BIT ssvgg introduces explicit aperture-ratio conditioning into a lightweight NAFNet-based restoration network.The source-to-target aperture ratio is logarithmically mapped and embedded into a low-dimensional feature vector for continuous blur control.
- Centre Borelli: Centre Borelli’s DALU-Net conditions a U-Net on RGB, monocular depth, coordinates, and aperture information, including aperture-conditioned adversarial fine-tuning.The generator uses FiLM conditioning, while the discriminator receives a single-channel bokeh-strength map.
8. Conclusion
The inaugural challenge advanced controllable bokeh rendering by attracting substantial participation and exploring fidelity and perceptual evaluation. No submitted method significantly outperformed the baseline, but the explored ideas and feedback can inform future editions.
- The challenge evaluated fidelity and perceptual quality across two distinct tracks, with many researchers participating and a selected group submitting final solutions.
- No solution in the initial challenge significantly outperformed the provided baseline.
- Participant ideas and recommendations are intended to inform the design of subsequent challenge editions.
NTIRE 2026 Controllable Bokeh Rendering Challenge Team
The supplied passage identifies the challenge organizers and their institutional affiliation.
- The challenge organizers are affiliated with the Computer Vision Lab, IFI & CAIDAS, University of Würzburg.
Davinci
The supplied passages list challenge teams, affiliations, and method titles associated with the challenge.
- The listed challenge methods include Dual-controllable Bokeh Rendering, HAFT, Bokehlicious-Large with 8× Geometric TTA, and lightweight conditional rendering.
- Additional listed methods include depth- and location-conditioned U-Net, aperture- and focal-length-dependent sampling, and MDAE-Pretrained TimeDiffiT.