Source-linked AI summary
Conditional Sequential Modulation for Efficient Global Image Retouching
Jingwen He, Yihao Liu, Yu Qiao, Chao Dong
TL;DR
Photo retouching needs efficient automatic correction of global photographic defects while preserving image structure. The paper proposes CSRNet, which combines pixel-wise MLP-like processing with condition-driven global feature modulation, and reports state-of-the-art MIT-Adobe FiveK performance with fewer than 37k parameters. Its design also supports multiple styles and continuous strength control, while comparisons are scoped to global photo retouching.
Problem
Photo retouching requires automatic global color adjustment for photographic defects, but existing enhancement methods may focus on local patterns or texture changes rather than preserving high-frequency structure.
Method
CSRNet combines a 1×1-convolution base network that processes pixels independently with a condition network that generates global feature-modulation parameters.
Results
CSRNet achieves state-of-the-art performance on MIT-Adobe FiveK with fewer than 37k parameters, reported as 1/13 of HDRNet and 1/90 of DPE.
Takeaways & Limitations
The framework supports different retouching styles through separate condition networks and provides continuous strength control by interpolating input and output images.
Takeaways & Limitations
The method is designed specifically for global photo retouching, and DUPE is excluded from visual comparison because only models trained on the authors’ collected under-exposed pairs were released.
Abstract
from arXiv · showhide
Photo retouching aims at enhancing the aesthetic visual quality of images that suffer from photographic defects such as over/under exposure, poor contrast, inharmonious saturation. Practically, photo retouching can be accomplished by a series of image processing operations. In this paper, we investigate some commonly-used retouching operations and mathematically find that these pixel-independent operations can be approximated or formulated by multi-layer perceptrons (MLPs). Based on this analysis, we propose an extremely light-weight framework - Conditional Sequential Retouching Network (CSRNet) - for efficient global image retouching. CSRNet consists of a base network and a condition network. The base network acts like an MLP that processes each pixel independently and the condition network extracts the global features of the input image to generate a condition vector. To realize retouching operations, we modulate the intermediate features using Global Feature Modulation (GFM), of which the parameters are transformed by condition vector. Benefiting from the utilization of $1\times1$ convolution, CSRNet only contains less than 37k trainable parameters, which is orders of magnitude smaller than existing learning-based methods. Extensive experiments show that our method achieves state-of-the-art performance on the benchmark MIT-Adobe FiveK dataset quantitively and qualitatively. Code is available at https://github.com/hejingwenhejingwen/CSRNet.
1 Introduction
The paper frames automatic global photo retouching as pixel-independent processing and proposes CSRNet, a compact conditionally modulated network that achieves strong performance with style and strength control.
- Automatic retouching is motivated by the difficulty of manual editing for casual users and the repetitive workload of processing large photo collections.The paper positions automatic tools for smartphones and editing software.
- Unlike local enhancement tasks, photo retouching mainly adjusts global color tones while preserving high-frequency components such as edges.
- Common operations such as contrast adjustment and tone mapping are pixel-independent mappings that can be approximated with MLPs.Different operations can share network structures while using different parameters.
- CSRNet models sequential retouching with a 1×1-convolution base network and a condition network that globally modulates intermediate features.The base network processes pixels independently, while the condition vector is broadcast to different layers for feature modulation.
- CSRNet achieves state-of-the-art performance on MIT-Adobe FiveK with fewer than 37k parameters, using only six plain convolutional layers.This is reported as 1/13 the parameters of HDRNet and 1/90 those of DPE.
- CSRNet supports different retouching styles by training different condition networks without changing the base network.For a fixed style, interpolation between input and output images provides continuous retouching-strength control.
2 Related Work
The related work spans traditional image-processing filters and learning-based enhancement methods, including physical modeling, image-to-image translation, and reinforcement learning.
- Traditional retouching and enhancement methods include histogram equalization, local Laplacian operators, fast bilateral filtering, and gray-world or gray-edge color correction.
- Learning-based image enhancement methods are grouped into physical-modeling-based, image-to-image translation, and reinforcement learning approaches.
- The MIT-Adobe FiveK dataset provides input and expert-retouched image pairs that supported subsequent learning-based enhancement research.
3 Method
CSRNet models global photo retouching as sequential pixel-independent transformations, using a compact base network conditioned on global image information. Its 1×1-convolution design preserves local details while global feature modulation adapts intermediate features for retouching.
- 3.1 Analysis of Retouching Operations: Common retouching operations can be represented or approximated by MLPs because they map input pixels independently, with global statistics supplied when required.Brightness, contrast, white-balancing, saturation control, and tone mapping motivate the MLP formulation; contrast additionally uses global information such as mean luminance.
- 3.2 Network Structure: CSRNet combines a pixel-independent base network with a condition network that supplies global image priors through feature modulation.The base network generates the retouched image, while the condition network estimates global information and influences it through global feature modulation.
- 3.2.1 Network Structure: The base network uses only 1×1 filters, so it acts as an MLP sliding independently over pixels and can model pixel-independent retouching operations with few parameters.The framework uses fully convolutional layers with ReLU activations, while the condition network extracts global priors through convolution, ReLU, and downsampling blocks.
- 3.2.2 Global Feature Modulation: Global Feature Modulation scales and shifts intermediate base-network features using parameters generated from the image condition vector.GFM applies γ∗x_i+β without feature normalization and is related to AdaFM as a 1×1-filter special case.
- 3.2.4 Discussion: CSRNet preserves edges and textures through pixel-wise operations and maintains image-wide color consistency through global modulation, but is specialized for global photo retouching.The paper contrasts these design choices with methods that alter local patterns or predict spatially varying transformations, while stating that CSRNet does not generalize to tasks such as style transfer or unpaired learning.
- 3.2.5 Multiple Styles and Strength Control: Image interpolation between two stylized outputs provides continuous style and retouching-strength control through a coefficient α.Using the input and retouched image as interpolation endpoints allows users to reduce or increase overall retouching strength.
4 Experiments
CSRNet is evaluated on MIT-Adobe FiveK against established methods using quantitative, visual, user-study, multi-style, and ablation experiments. The results support compact global retouching with strong benchmark performance, style adaptation, interpolation-based control, and reliance on global conditioning.
- 4.1 Comparison with State-of-the-art Methods: CSRNet achieves state-of-the-art MIT-Adobe FiveK performance with 36,489 parameters, outperforming prior methods across PSNR, SSIM, and L2 error.Table 1 evaluates expert-C retouching; lower L2 error is better.
- 4.1 Comparison with State-of-the-art Methods: Under the same experimental setting, CSRNet outperforms DPE on all three metrics while using much fewer parameters.
- 4.1 Comparison with State-of-the-art Methods: CSRNet produces more realistic visual results than the compared methods, while baselines show over-exposure, unchanged oversaturated tones, artifacts, or unnatural regional colors.The visual comparison excludes DUPE because only a model trained on different under-exposed image pairs was released.
- 4.2 Multiple Styles and Retouching Strength: Finetuning only the condition network transfers CSRNet across expert styles with results comparable to training from scratch.The evaluated target styles are experts A, B, D, and E; the base network remains fixed.
- 4.2 Multiple Styles and Retouching Strength: Image interpolation enables continuous transitions between retouching styles and controls strength between an input image and its retouched output.Figure 6 illustrates style interpolation from expert A to B and strength control using coefficient α.
- 4.3 Ablation study: Ablations show that global conditioning and feature modulation are central: larger spatial filters add little, while GFM and global priors improve retouching.Expanding to 3 × 3 filters gives roughly no improvement with conditioning, whereas global priors improve over the base network alone.