Source-linked AI summary
LensStyle: Learning the Optical Aesthetics for Controllable Stylized Lens Effect Rendering
Yachuan Huang, Liwen Xiao, Liao Shen, Qiwen Wang, Huiqiang Sun, Zhiyu Pan, Zhiguo Cao
TL;DR
Existing lens-effect renderers largely model blur magnitude while missing diverse lens aesthetics such as aperture-shaped bokeh and diffraction starbursts. LensStyle addresses this gap with unified continuous–discrete control, a Dual Path Controller, and the physically grounded MultiLens dataset, and reports superior realism, controllability, and aesthetic quality compared with prior renderers and diffusion-based editors.
Problem
Existing lens-effect methods primarily model blur transitions and overlook stylistic effects such as diverse aperture-shaped bokeh and diffraction-induced starbursts.
Method
LensStyle uses a unified flow-based framework with a Dual Path Controller that separates continuous optical-parameter modulation from discrete lens-style conditioning.
Results
LensStyle reports superior realism, controllability, and aesthetic quality compared with existing lens-effect rendering approaches and diffusion-based image editing models.
Takeaways & Limitations
The framework supports interpretable manipulation of focus, blur, and multiple lens styles within a single model while remaining depth-free during inference.
Abstract
from arXiv · showhide
The visual aesthetics of photographs are deeply influenced by lens characteristics such as aperture shape, optical vignetting and optical diffraction, which together define a camera's unique optical style. Existing lens effect rendering methods primarily focus on accurately simulating the blur transition from small to large apertures but overlook the stylistic aspects of lens effects. As a result, they fail to produce diverse bokeh effects under large apertures or capture distinctive photographic phenomena such as starbursts that emerge under small apertures. In this work, we introduce LensStyle, a unified framework for controllable stylized lens effect rendering that explicitly models lens aesthetics through joint continuous-discrete control. Our model incorporates a Dual-Path Controller that disentangles continuous optical parameter modulation (e.g., focus distance and blur strength) from discrete lens-style conditioning (e.g., circular, polygonal, donut, cat-eye, and starburst effects), enabling fine-grained, interpretable, and physically grounded lens manipulation within a single unified framework. To support model training, we curate a comprehensive MultiLens dataset containing multi-lens image pairs synthesized under real optical constraints. Extensive experiments demonstrate that LensStyle achieves superior realism, controllability, and aesthetic quality compared with existing lens effect rendering approaches and diffusion-based image editing models, advancing computational photography toward multiple-lens-style simulation.
1 Introduction
LensStyle addresses the limited stylistic modeling of existing lens-effect renderers by jointly controlling continuous optical parameters and discrete lens styles. It introduces a Dual Path Controller and the MultiLens benchmark to support physically grounded, interpretable multi-style rendering.
- Motivation: Lens aesthetics arise from aperture geometry, optical vignetting, and diffraction, which produce distinctive bokeh and starburst patterns.Circular or polygonal apertures shape bokeh, vignetting produces cat-eye effects, and small-aperture diffraction creates radiating starbursts.
- Motivation: Existing rendering methods mainly model blur magnitude and typically produce uniform circular bokeh, while overlooking lens style and diffraction-driven starbursts.These approaches simulate blur transitions across apertures but do not address aperture shape, optical aberrations, or small-aperture diffraction spikes.
- LensStyle: LensStyle jointly controls continuous focus distance and blur strength with discrete styles including circular, polygonal, donut, cat-eye, and starburst effects.The framework represents lens behavior as a learnable style space within one unified flow-based model.
- LensStyle: Its Dual Path Controller separates continuous optical parameter modulation from discrete lens-style conditioning for fine-grained and interpretable control.Optical parameters are embedded and transformed into feature scaling and shifting factors, while lens styles are injected through a separate conditioning path.
- MultiLens Dataset: MultiLens provides physically grounded multi-lens synthetic pairs with continuous focus and blur parameters and supervision for five optical effect categories.The benchmark covers circular, polygonal, donut, and cat-eye bokeh alongside diffraction-induced starbursts.
2 Related Work
Prior work has advanced depth-aware and diffusion-based bokeh rendering, but LensStyle’s framework combines latent flow-based rendering with separate continuous and discrete lens controls. Its architecture uses a Dual Path Controller to condition the generative backbone on both optical parameters and lens style.
- Prior Lens Rendering: Traditional and neural methods primarily render shallow-depth-of-field bokeh from all-in-focus inputs using depth maps, blur kernels, or neural networks.These approaches progressively improve compositing and physics awareness, including handling of complex occlusions.
- Diffusion-Based Editing: Diffusion-based editing methods enable high-quality conditional image generation and fine-grained semantic modification through prompts or conditional denoising.Representative approaches include InstructPix2Pix, SDEdit, and SuperEdit.
- LensStyle: LensStyle encodes all-in-focus images into a latent space, transforms them with a flow-based rendering model, and decodes the resulting stylized latent.A Dual Path Controller supplies joint continuous optical and discrete lens-style conditioning to the generative backbone.
3 Method
LensStyle combines a flow-based renderer with disentangled continuous optical control and discrete lens-style conditioning. Its physically grounded synthesis models defocus and diffraction using aperture-dependent optical effects.
- LensStyle uses a latent autoencoder, Dual Path Controller, and flow-based rendering model to transform all-in-focus images into styled lens images.
- Training combines flow-matching, reconstruction, and perceptual losses to enforce transport accuracy, pixel fidelity, and high-level aesthetic alignment.
- The Dual Path Controller separates continuous parameters such as focus distance and blur strength from discrete styles such as circular, polygonal, donut, cat-eye, and starburst.
- Optical embeddings modulate intermediate features through adaptive scaling and shifting, enabling smooth control over defocus magnitude and focal-plane adjustment.
- Style embeddings enter the flow backbone through cross-attention, conditioning aperture geometry, vignetting distortion, and diffraction spike patterns.
- The unified optical forward model combines geometric defocus with wave-optics diffraction, while depth is used for synthesis but not required during inference.
- The dataset models circular, polygonal, donut, and cat-eye apertures analytically, while Fraunhofer diffraction produces starburst effects whose count and orientation depend on blade number and rotation.
- Defocus strength scales with the distance between scene depth and focal depth, and the rendering pipeline applies spatially varying blur plus diffraction responses in bright regions.
4 Experiments
Experiments compare LensStyle with bokeh renderers and diffusion-based editors across circular and diverse lens styles, user preferences, and component ablations. LensStyle shows stronger quantitative and qualitative results while its controller components support stable continuous control and structured style rendering.
- Quantitative Comparison: LensStyle outperforms diffusion-based editors across polygonal, donut, cat-eye, and starburst styles, improving PSNR by 1.1–1.8 dB and reducing LPIPS by 15–25%.The comparison uses the MultiLens dataset, where higher PSNR/SSIM and lower LPIPS are better.
- Quantitative Comparison: For circular bokeh, LensStyle achieves the highest PSNR and SSIM and lowest LPIPS across synthetic MultiLens and real-world EBB400 evaluations.Its LPIPS advantage over BokehDiff is emphasized despite a lower SSIM than that diffusion-based baseline.
- Qualitative Comparison: LensStyle produces more physically consistent and visually pleasing circle bokeh without depth input, avoiding focused-region blurring caused by inaccurate baseline depth maps.The compared baselines use Depth Anything V2 depth maps, whereas LensStyle operates depth-free during this comparison.
- Qualitative Comparison: Against diffusion editors, LensStyle produces more consistent lens styles, while fine-tuned SuperEdit* and UltraEdit* still show inconsistent hexagons or excessive starbursts.The comparison attributes LensStyle’s advantage to disentangled continuous modulation and discrete cross-attention.
- User Study: In a 58-participant pairwise study, LensStyle is consistently preferred for perceptual realism, stylistic fidelity, and structural consistency.The study compares outputs generated from matched input images, style prompts, and optical parameters.
- Ablation Study: Ablations show that affine modulation stabilizes continuous control, cross-attention preserves aperture-specific semantics, and flow transport maintains content during stylistic transformation.Removing these components weakens parameter interpolation, complex-style structure, or structural preservation, respectively.
5 Conclusion and Future Work
LensStyle unifies depth-consistent lens-effect rendering with continuous optical control and discrete lens-style conditioning in a flow-based architecture. Evaluations and user studies report improved realism, style fidelity, and controllability, while the stated future-work boundary is limited DSLR-specific bokeh simulation.
- Conclusion: LensStyle jointly models focus and blur controls with aperture-specific styles through a Dual Path Controller in a flow-based generative architecture.The framework uses a physically grounded optical forward model for dataset synthesis and remains depth-free during inference.
- Conclusion: Quantitative evaluations and user studies report improved realism, style fidelity, and controllability over existing lens rendering and image editing approaches.The conclusion summarizes the paper’s reported evaluation outcome without isolating a single metric.
- Future Work: LensStyle cannot simulate bokeh effects from different DSLR systems such as Fuji, Sony, and Nikon.The paper identifies this DSLR-specific simulation gap as a limitation and future-work direction.