Source-linked AI summary
DeepLPF: Deep Local Parametric Filters for Image Enhancement
Sean Moran, Pierre Marza, Steven McDonagh, Sarah Parisot, Gregory Slabaugh
TL;DR
Manual local photo enhancement is expressive but time-consuming and skill-dependent, while pixel-level and global automated methods have limitations in interpretability or fine-grained adjustment. DeepLPF learns and applies spatially local elliptical, graduated, and polynomial filters, achieving state-of-the-art benchmark performance with a fraction of competing models’ parameters. Its filter-based formulation supports interpretable, visually pleasing adjustments, while future work includes automatically selecting filter sequences and adding further filter types or segmentation masks.
Problem
Manual local enhancement requires time and artistic capability, while existing automated approaches may be noisy, lack interpretability, or miss fine-grained adjustments.
Method
DeepLPF regresses parameters for learned spatially local elliptical, graduated, and polynomial filters and fuses their outputs for image enhancement.
Results
DeepLPF achieves state-of-the-art image enhancement quality on two challenging benchmarks and outperforms contemporary methods with a fraction of neural-network weight capacity.
Takeaways & Limitations
The localized filter formulation provides interpretable, intuitive adjustments with visually pleasing results and reusable network blocks for improving image visual quality.
Takeaways & Limitations
Future work is needed to estimate the optimal filter-application sequence and incorporate additional filter types or segmentation masks.
Abstract
from arXiv · showhide
Digital artists often improve the aesthetic quality of digital photographs through manual retouching. Beyond global adjustments, professional image editing programs provide local adjustment tools operating on specific parts of an image. Options include parametric (graduated, radial filters) and unconstrained brush tools. These highly expressive tools enable a diverse set of local image enhancements. However, their use can be time consuming, and requires artistic capability. State-of-the-art automated image enhancement approaches typically focus on learning pixel-level or global enhancements. The former can be noisy and lack interpretability, while the latter can fail to capture fine-grained adjustments. In this paper, we introduce a novel approach to automatically enhance images using learned spatially local filters of three different types (Elliptical Filter, Graduated Filter, Polynomial Filter). We introduce a deep neural network, dubbed Deep Local Parametric Filters (DeepLPF), which regresses the parameters of these spatially localized filters that are then automatically applied to enhance the image. DeepLPF provides a natural form of model regularization and enables interpretable, intuitive adjustments that lead to visually pleasing results. We report on multiple benchmarks and show that DeepLPF produces state-of-the-art performance on two variants of the MIT-Adobe-5K dataset, often using a fraction of the parameters required for competing methods.
1. Introduction
DeepLPF addresses the difficulty of manual local photo enhancement by learning interpretable parametric filters and combining them through a neural network. The method reports state-of-the-art enhancement quality while using a fraction of competing models’ weight capacity.
- Manual local enhancement is challenging for non-experts because it requires appropriate skills, time, and aesthetic judgment.
- Automatic enhancement should reproduce spatially limited edits such as darkening skies, brightening faces, and retouching fine detail.
- DeepLPF learns graduated, elliptical, and brush-like filters from input–enhanced image pairs to reproduce local, mid-level adjustments.
- Parametric local filters provide interpretable, intrinsically regularized adjustments designed for weight efficiency and reduced overfitting.
- The multiple filter fusion block combines independent learned filter outputs and integrates with common network backbones for image quality enhancement.
- DeepLPF provides state-of-the-art image quality enhancement on two challenging benchmarks.
2. Related work
Prior enhancement methods operate globally, locally, or at both scales, while DeepLPF frames local enhancement as learning interpretable parametric filters. This parameterization constrains capacity and regularizes learning while matching familiar artistic tools.
- Earlier automated enhancement methods primarily addressed contrast, while later data-driven approaches learned adjustments for contrast, colour, brightness, and saturation.
- Related methods include local detail filtering, database matching, semantic enhancement, luminance-map prediction, and global–local architectures.
- DeepLPF differs by learning spatially local parametric filters aligned with intuitive human artistic tools.
- The filter parameterization constrains model capacity, regularizes learning, mitigates overfitting, and incurs moderate model-capacity cost.
3. Deep Local Parametric Filters (DeepLPF)
DeepLPF combines learned spatially local parametric filters through a fusion architecture designed to emulate combinations of manual editing tools. The paper instantiates elliptical, graduated, and polynomial filters.
- DeepLPF combines outputs from learned spatially local parametric filters designed to emulate combined manual-filter applications.
- The method instantiates three local filter types: Elliptical, Graduated, and Polynomial.
- The architecture learns and applies sets of filters, predicts their parameters, fuses multiple outputs, and uses a training loss for enhancement.
3.1. DeepLPF Architecture
DeepLPF learns a transformation from low-quality RGB inputs to high-quality enhanced targets using a quality-based objective. Its architecture combines fine-grained single-stream processing with higher-level local enhancement.
- Given a low-quality RGB image I and high-quality target Y, DeepLPF learns fθ so predicted output Ŷ=fθ(I) approaches Y under an image-quality objective.
- The architecture combines a single-stream network for fine-grained enhancement with a two-stream network for higher-level local enhancement.
3.2. Local Parametric Filters
DeepLPF learns parameters for three spatially local filter types—graduated, elliptical, and polynomial—from CNN features. These filters provide interpretable, regularized adjustments spanning regional enhancement and fine-grained image editing.
- Filter parameter prediction: DeepLPF estimates image-specific parameters for Graduated, Elliptical, and Polynomial filters using supervised CNN regression.A lightweight parameter prediction block regresses filter parameters from backbone features, with output size corresponding to each filter’s parameter count.
- Parameter efficiency: The filters use compact parameterizations that reduce model capacity while retaining expressive local image transformations.The cubic variants use 30 and 60 total parameters when applied independently across color channels, respectively.
- Graduated Filter: The graduated filter uses parallel lines to define localized regions whose adjustment scales decrease from a 100% area toward an unadjusted 0% area.Its parameters specify line location and orientation, offsets, and a learnable inversion that selects which side contains the fully adjusted region.
- Elliptical Filter: The elliptical filter parameterizes a center, axes, and rotation, with maximal adjustment at the center, linear decrease to the boundary, and no adjustment outside.This spatial form supports enhancement of selected regions such as faces or other objects of interest.
- Polynomial Filter: The polynomial filter emulates a brush by applying spatially smooth, fine-grained adjustments using intensity and position information.The authors empirically select cubic filters as expressive yet parameter-limited, exploring cubic-10 and cubic-20 variants with 10 and 20 parameters.
3.3. Fusing Multiple Filters of the Same Type
Multiple instances of graduated or elliptical filters are fused into a single adjustment map by multiplying their spatial effects element-wise. Cubic filters are not fused because one per-channel filter is already highly expressive.
- Same-type filter fusion: For n > 1, multiple graduated or elliptical filter instances are combined into an adjustment map through element-wise multiplication.The resulting maps aggregate the corresponding instances’ spatial adjustments.
- Same-type filter fusion: Multiple cubic filters are not fused because a single per-channel cubic filter already provides high expressive flexibility.This design choice differs from the fusion strategy used for graduated and elliptical filters.
3.4. DeepLPF Loss Function
DeepLPF trains with a loss that separates chrominance and luminance objectives in CIELab space. L1 terms address global colour differences, while MS-SSIM emphasizes local structural enhancement.
- The training loss computes L1 error on CIELab channels and MS-SSIM error on the L channel.This separates chrominance and luminance information into distinct loss terms.
- Lab(·) converts RGB inputs to CIELab channels, while L(·) extracts the CIELab L channel.
- Hyperparameters ω_lab and ω_ms-ssim weight the relative contributions of the loss terms.
4. Experiments
DeepLPF is evaluated across three benchmarks derived from MIT-Adobe-5K and SID, using standard image-quality metrics and controlled architectural settings. It outperforms competing methods across multiple datasets while using substantially fewer parameters, and qualitative results show improved colour and exposure handling.
- Datasets: Three benchmarks derive from MIT-Adobe-5K-DPE, MIT-Adobe-5K-UPE, and the Fuji portion of SID.The MIT-Adobe-5K variants use different preprocessing protocols, while SID evaluates RAW-to-RGB enhancement.
- Evaluation Metrics: Evaluation uses PSNR, SSIM, and perceptual LPIPS.
- Implementation Details: Experiments use U-Net backbones, Adam with learning rate 10^-4, and three graduated or elliptical filters per channel.
- Quantitative Results: DeepLPF surpasses 8RESBLK and supervised DPE on all three reported metrics for MIT-Adobe-5K-DPE, with nearly half their model capacity.
- Quantitative Results: On MIT-Adobe-5K-UPE, DeepLPF outperforms DeepUPE on PSNR and LPIPS, remains competitive on SSIM, and improves over other compared works on PSNR and SSIM.
- Quantitative and Qualitative Results: On Fuji SID, DeepLPF improves over the compared U-Net across all three metrics with model capacity lower by nearly a factor of four.Qualitatively, DeepLPF maintains accurate exposure and colour content and improves colour constancy in the reported comparisons.
5. Conclusion
The method automates spatially localized image enhancement by regressing parameters for graduated, elliptical, and polynomial filters from backbone features. Its filters are interpretable and reusable, while optimal filter-sequence estimation remains future work.
- DeepLPF regresses parameters for graduated, elliptical, and polynomial filters from convolutional features to automate spatially localized image enhancement.The filters can be estimated directly from a backbone network such as U-Net.
- The localized filters provide interpretable image adjustments and visually pleasing results while functioning as pluggable, reusable network blocks.
- Future work includes automatically estimating the optimal sequence of filter application from a potentially large bank of image-editing tools.The paper also proposes combining local filters with additional filter types and segmentation masks.