PaperlayerView exact revision on arXiv

Source-linked AI summary

Modular Neural Image Signal Processing

Mahmoud Afifi, Zhongling Wang, Ran Zhang, Michael S. Brown

arXiv:2512.08564v3cs.CV

TL;DR

Monolithic neural ISPs can be difficult to generalize across cameras, deploy efficiently, and interpret or extend. This paper builds a functionally constrained modular ISP for raw-to-sRGB rendering, with an interactive editing tool. Its variants achieve state-of-the-art results on nearly all S24 styles and competitive results on MIT-Adobe FiveK, with lightweight-to-moderate capacity.

  • Problem

    Monolithic neural ISPs can generalize poorly to unseen cameras, require substantial compute and memory, and be difficult to interpret, debug, or extend.

    Source: §1 · p. 2

  • Method

    The framework decomposes raw-to-sRGB rendering into functionally constrained stages, using role-specific design and losses to preserve separation during end-to-end training.

    Source: §1 · p. 3

  • Results

    Across variants of ∼0.5 M to ∼3.9 M parameters, results are state-of-the-art on nearly all S24 styles and competitive on MIT-Adobe FiveK.

    Source: §4 · pp. 13–14

  • Takeaways & Limitations

    The modular framework supports easier debugging, generalization to unseen cameras, and an interactive tool for user control over styles and editing.

    Source: §4 · p. 14

    Training requires reference denoised images and camera AWB and CCM data, which can be difficult to obtain when DNG files are unavailable.

    Source: §4 · p. 14

Abstract

from arXiv · show

This paper presents a modular neural image signal processing (ISP) framework that processes raw inputs and renders high-quality display-referred images. Unlike prior neural ISP designs, our method introduces a high degree of modularity, providing full control over multiple intermediate stages of the rendering process.~This modular design not only achieves high rendering accuracy but also improves scalability, debuggability, generalization to unseen cameras, and flexibility to match different user-preference styles. To demonstrate the advantages of this design, we built a user-interactive photo-editing tool that leverages our neural ISP to support diverse editing operations and picture styles. The tool is carefully engineered to take advantage of the high-quality rendering of our neural ISP and to enable unlimited post-editable re-rendering. Our method is a fully learning-based framework with variants of different capacities, all of moderate size (ranging from ~0.5 M to ~3.9 M parameters for the entire pipeline), and consistently delivers competitive qualitative and quantitative results across multiple test sets. Watch the supplemental video at: https://youtu.be/ByhQjQSjxVM

1 Introduction and Related Work

Conventional ISPs coordinate specialized operations across a rendering pipeline, while learning-based systems have often traded modularity for camera generalization, efficiency, and interpretability. This work proposes functionally constrained stages and demonstrates benefits for debugging, camera reuse, picture styles, and interactive editing.

1 Introduction and Related Work

  • Monolithic neural ISPs can generalize poorly to unseen cameras, demand substantial resources, and be difficult to interpret, debug, or extend [45,49,50,70,71,86,90,93].Earlier modular approaches also use coarse stage definitions or post-training fine-tuning that can weaken stage functionality [6, 33, 47, 54,59,61,77,88].Source: §1 · p. 2
  • The proposed ISP aligns stages with raw-to-sRGB image formation and constrains each module with role-specific supervision or losses to preserve functional separation during end-to-end training.For photofinishing, architectural choices and role-specific losses replace per-submodule ground truth; synthetic targets and sequential stage-wise optimization support stage independence.Source: §1 · p. 3
  • Modularity supports isolating corner cases, reusing generic modules for unseen cameras, and adding picture styles without duplicating the full pipeline.The framework also integrates editing operators and supports interactive adjustment and re-rendering from embedded raw data.Source: §1 · p. 3
  • The fine-grained framework provides interpretable control across rendering and supports multiple styles with moderate capacity, alongside an interactive tool for raw and standard sRGB image editing.The tool supports style selection or interpolation, editing adjustments, and re-rendering saved images with embedded raw data.Source: §1 · p. 3

2 Method

The method decomposes raw-to-sRGB rendering into independently trainable stages, with a multi-step photofinishing module at reduced resolution and guided reconstruction. Role-specific design and losses preserve the intended separation of photofinishing functions, while the resulting pipeline supports interactive editing and post-editable re-rendering.

2 Method

  • The pipeline renders raw images through raw enhancement, color correction, downsampled photofinishing, guided upsampling, and detail enhancement, with learnable stages replaceable without retraining the whole framework.The final output is a display-referred image matching a reference rendered from the same raw input and picture style.Source: §2 · pp. 3–4, Figure 2 · p. 4

2.1 Raw Enhancement

  • Single-image raw denoising targets noise that can obscure detail, using supervised pixel-wise L1 training with pseudo-ground-truth denoised images.The paper notes these pseudo targets are imperfect but provide stable supervision and can broaden the raw data used for training.Source: §2.1 · pp. 4–5

2.2 Color Correction

  • Color correction maps denoised raw data to linear sRGB, a camera-agnostic space that makes subsequent operators less camera-dependent.The transformation uses white-balance gains and interpolated, pre-calibrated camera-specific color correction matrices.Source: §2.2 · p. 5

2.3 Photofinishing

  • Photofinishing uses five image-parameterized stages—gain, global and local tone mapping, chroma mapping, and gamma correction—predicted by lightweight networks totaling about 200K parameters.For stronger artistic color manipulation, an optional image-independent 3D LuT augments the 2D chroma LuT; it remains optional because it can interfere with other tone-mapping roles.Source: §2.3 · pp. 6–8
  • Because individual photofinishing functions lack ground-truth supervision, the networks train end-to-end with fidelity, perceptual, and regularization losses that encourage distinct roles.The tone-mapping loss encourages GTM and LTM to complement each other, while a luminance regularizer helps GTM refine contrast without changing global brightness.Source: §2.3 · pp. 6, 8

2.4 Upsampling

  • Photofinishing runs on a downscaled image; per-channel gated regularization in guided bilateral-grid upsampling avoids achromatic color crosstalk and grid blurring that trades detail for smoothness.The full-resolution linear sRGB image guides reconstruction, with empty grid cells falling back to pooled global per-channel gains.Source: §2.4 · p. 8

2.5 Detail Enhancement

  • A compact final detail-enhancement network refines the upsampled result to compensate for residual denoising and upsampling artifacts.The authors report that merging raw and detail enhancement was unstable and often failed to converge, while a separate roughly 50K-parameter stage was more robust.Source: §2.5 · p. 9

2.6 Photo-Editing Tool

  • The interactive tool exposes styles and editing adjustments, supports raw inputs from unseen cameras, and embeds compressed raw data in JPEGs for unlimited post-editable re-rendering.A lightweight linearization network also lets it process standard sRGB inputs; the tool, including photofinishing networks for multiple styles, uses about 3.9 M parameters.Source: §2.6 · p. 9

3 Experimental Results

On the S24 test set, the framework achieves state-of-the-art results across its model variants and picture styles, with qualitative comparisons, unseen-camera examples, and user preferences also reported.

3 Experimental Results

  • On the S24 dataset, 2,619 training, 205 validation, and 400 test pairs support evaluation across one default and five artistic styles.The dataset provides pseudo-ground-truth denoised raw images and six ground-truth sRGB images per raw input.Source: §3 · p. 10

3.1 Quantitative Results

  • Across Styles #1–5, the method achieves state-of-the-art results with moderate total parameters; learned LRGB consistently improves artistic styles but may slightly degrade some default-style metrics.The denoiser is shared across styles, limiting the added parameter cost per style.Source: §3.1 · p. 11, Table 2 · p. 13
  • Our lite, base, and large variants achieve state-of-the-art S24 results; even the 0.5M-parameter lite model beats 9M-parameter LiteISP by ~2 dB PSNR.Table 1 reports PSNR, SSIM, LPIPS, and ∆E 2000 for the S24 test set.Source: §3.1 · p. 11, Table 1 · p. 12

3.2 Qualitative Results

  • Qualitative comparisons report higher visual quality across styles than recent neural ISP approaches, including ISPDiffuser, LiteISP, and ParamISP.Figure 4 shows the default style and artistic styles alongside their ground truths and PSNR values.Source: Figure 4 · p. 10, §3.2 · p. 12
  • On an iPhone 13 Pro Max image, the method's generic denoiser and cross-camera AWB produce quality comparable to the native ISP and Project Indigo without iPhone training data.The paper attributes this generalization to switching between camera-specific and generic modules.Source: Figure 5 · p. 11, §3.2 · p. 12
  • In a 20-participant study of 45 scenes, the method received 51.4% of overall-preference evaluations, with significant pairwise preference over both the S24 ISP and Lightroom.The reported overall-preference tests give p = 1.9 × 10−21 versus S24 and p = 2.4 × 10−19 versus Lightroom.Source: §3.2 · p. 13

4 Conclusion and Discussion

The framework combines moderate-capacity, modular raw-to-sRGB rendering with competitive results, interactive editing, and support for unseen cameras. Its discussion also identifies edge artifacts and training-data requirements as challenges.

4 Conclusion and Discussion

  • The lite, base, and large pipelines use approximately 0.5M, 1.2M, and 3.9M parameters, respectively, and achieve state-of-the-art results on nearly all S24 styles and competitive MIT-Adobe FiveK results.A 12-megapixel image can be processed in about a second or less on a single GPU.Source: §4 · pp. 13–14
  • Interpretable stages support debugging and corner-case handling, while the user-interactive tool provides fine-grained control over picture styles and editing options.Figure 7 shows intermediate outputs and edits, including results from iPhone 15 and iPhone 13 cameras absent from training.Source: §4 · p. 14, Figure 7 · p. 15
  • In backlit scenes, halos may appear near edges; the authors identify the local tone-mapping process as their source and report mitigation through multi-scale processing and LTM-map refinement.Figure 6 also describes possible color inconsistencies in flat regions near edges.Source: §4 · p. 14, Figure 6 · p. 14
  • Training requires reference denoised images and camera AWB and CCM data; when DNG files are unavailable, obtaining the needed supervision is difficult.A Zurich Raw-to-sRGB case study reports comparable results to top-performing methods without DNG files, using roughly half or fewer parameters than alternatives.Source: §4 · p. 14

A GPU-Accelerated Iterative Bilateral Solver

The optional guided refinement addresses residual LTM artifacts with an image-space variant that preserves the FBS energy while avoiding its GPU-inefficient bilateral-grid solve. It precomputes guidance-dependent affinities and uses iterative SOR updates.

A GPU-Accelerated Iterative Bilateral Solver

  • FBS’s bilateral-grid construction and repeated pixel-grid gather-scatter operations make its PCG solve inefficient to accelerate on GPUs.The paper attributes this to memory-access costs and slow convergence under the Jacobi preconditioner.Source: A GPU-Accelerated Iterative Bilateral… · p. 16

A.1 Quadratic Objective in Image Space

  • The image-space objective balances fidelity to the initial tensor M against edge-aware smoothing of output Y using affinities from guidance image Z.The affinities are defined over a local k×k neighborhood, with spatial and range scales controlling their weights; Z is input RGB luminance.Source: §A.1 · pp. 16–17

A.2 Iterative Solver

  • The solver precomputes bilateral weights once per image, initializes Y(0)=M, then applies a fixed number of successive over-relaxation updates.The weights are normalized to sum to one and reused across channels and iterations; the relaxation parameter ω is in [1, 2).Source: §A.2 · p. 17

A.3 GPU Implementation and Efficiency

The modified solver is designed for GPU deployment and trades exact convergence guarantees for efficient local SOR updates. The section also describes artifact and training-data challenges that motivate optional remedies.

A.3 GPU Implementation and Efficiency

  • The GPU solver retains FBS energy but sacrifices exact convergence guarantees; broad evaluation across datasets and tasks is beyond this paper’s scope.The refinement is also presented as a general edge-aware post-processing method.Source: §A.3 · p. 17
  • ≈11× wall-clock speedup over original CPU-based FBS is achieved by the GPU solver at niter=80, with visually comparable refinements.On CPU, repeated dense tensor operations make the modified solver slower than FBS; the comparison uses a 750×1000 guide image.Source: §A.3 · p. 17, Figure 8 · p. 18

B Challenges and Potential Solutions

  • Rare halo artifacts and incomplete training data are challenges; optional artifact steps may reduce accuracy, and Zurich Raw-to-sRGB pairs are unaligned and may lack denoised raws or DNG metadata.The optional steps can be enabled or disabled by users.Source: B Challenges and Potential Solutions · pp. 18–19

A.3 GPU Implementation and Efficiency

  • The S24 validation comparison evaluates four multi-scale and refinement configurations by PSNR and runtime overhead.The input is a linear sRGB image generated from a pseudo-ground-truth denoised image; runtimes were measured on an Intel Core i7-14700K and NVIDIA RTX 4080 SUPER.Source: Figure 9 · p. 19
  • Fig. 10 reports that modified bilateral refinement achieves comparable or improved results to original FBS while running efficiently on GPU.Its reference image comes from the S24 test set; the high-resolution source was generated in Photoshop and downsampled to 100×75.Source: Figure 10 · p. 19

B.1 Mitigating Artifacts

Multi-scale LTM processing followed by guided bilateral refinement is used to mitigate halo artifacts in difficult scenes. Because applying these steps indiscriminately can reduce accuracy, they remain optional and user-controlled.

B.1 Mitigating Artifacts

  • The multi-scale strategy predicts LTM coefficients at progressively downsampled resolutions, smooths guidance maps, then upsamples and averages scale-specific coefficient maps.The scale factors are 1.0, 0.5, 0.25, 0.125, and 0.0625; refinement uses the digital-gain input as its guide.Source: §B.1 · pp. 19–20
  • Multi-scale LTM processing followed by guided bilateral refinement significantly mitigates halo artifacts in difficult scenes.The paper illustrates halo suppression with images captured on the S24 main camera.Source: §B.1 · p. 20, Figure 11 · p. 20
  • Because these steps can degrade accuracy when applied indiscriminately, they remain user-controlled and can be enabled or disabled as needed.The comparison evaluates the default Style #0 photofinishing module with and without multi-scale processing and refinement on S24 test data.Source: §B.1 · p. 20

B.2 Misaligned and Incomplete Datasets

This section describes how the modular ISP is trained with missing metadata and misaligned pairs, including its lightweight rendering-stage networks, differentiable histogram and guided-upsampling regularization, photofinishing losses, and separate module training. It also reports competitive Zurich-dataset performance with fewer parameters, though not state of the art.

B.2 Misaligned and Incomplete Datasets

  • For Zurich [50], the workflow substitutes an approximate denoised target, gray-world illuminant estimates, and fitted color correction matrices for unavailable DNG metadata and misaligned raw–sRGB pairs.It linearizes sRGB using inverse 2.2 gamma, fits a global color mapping after discarding saturated pixels, estimates illuminant from mean demosaiced-raw RGB, and fits a constrained CCM. Training modules separately and using 448×448 patches adapts the pipeline to the dataset.Source: §B.2 · pp. 22–23
  • On Zurich, our method is competitive despite a less than 1 dB PSNR drop versus the best-performing method, using ∼500 K–∼3.9 M rather than ∼7.6 M parameters.It does not achieve state-of-the-art results, but outperforms methods with comparable parameter counts that lack its modularity [6, 70]; the best-performing reference is [45].Source: §B.2 · p. 23, Table 4 · p. 23

C.1 Raw-Denoising Network

  • The raw-denoising network uses three NAFNet [31] variants with 4, 8, or 16 initial channels, spanning approximately 0.25 M, 0.93 M, and 3.6 M parameters.Each is a fully convolutional residual encoder–decoder with skip connections, four encoder stages, and four middle residual blocks, supporting arbitrary-resolution raw inputs.Source: §C.1 · p. 24

C.2 Attention and Multi-Branch Blocks

  • Photofinishing combines modified coordinate attention with parallel depthwise MBConv branches to model multiple receptive-field scales without increasing parameters or channels.Coordinate attention replaces Batch Normalization with Group Normalization and uses LeakyReLU; MBConv branches use undilated 3×3, dilated 3×3, and 5×5 kernels before summation and 1×1 fusion.Source: §C.2 · p. 24

C.3 Gain and Gamma-Correction Networks; C.4 Global Tone Mapping Network

  • The gain and gamma networks predict global scalar adjustments, while the GTM network predicts three positive parameters defining a global tone curve.Each gain/gamma network has 6,587 parameters and maps its scalar to gain [0.25, 4.0] or gamma [1.2, 3.0]; GTM has 28,369 learnable parameters.Source: §C.3 · p. 25, §C.4 · p. 26

C.5 Local Tone-Mapping Network

  • The 120,215-parameter LTM network combines multi-scale luminance guidance with a predicted coefficient grid to generate five spatially varying tone-mapping maps.It processes guidance at ×1, ×1/2, and ×1/4 scales, then trilinearly samples the coefficient grid using the guidance map; optional multi-scale processing and bilateral refinement can smooth maps at inference.Source: §C.5 · pp. 26–27

C.6 Chroma-Mapping Network; D Differentiable Histogram Computation

  • The chroma-mapping network predicts an image-specific 2D chroma lookup table from a differentiable CbCr histogram, with luminance-guided attention adapting the mapping.Its predicted residual lookup table is added to a learnable identity-initialized base table; the network has 45,466 learnable parameters.Source: §C.6 · pp. 28–29, D Differentiable Histogram Computation · p. 29

C.7 Detail-Enhancement Network

  • The detail-enhancement network is a compact NAFNet [31] residual encoder–decoder with approximately 50 K parameters for fine-detail correction.It uses skip connections, two encoder and decoder stages, and four middle blocks.Source: §C.7 · p. 29

D Differentiable Histogram Computation

E Guided Upsampling Regularization

  • The gated BGU regularization uses local per-channel constraints where grid cells have enough samples and global statistics in sparse cells, avoiding grid blurring, color crosstalk, and loss of sharp detail.The prior achromatic regularization could cause color crosstalk, while grid blurring could oversmooth edges and fine detail.Source: E Guided Upsampling Regularization · p. 32

F Loss Function Details

  • The chroma loss compares mapped CbCr channels with de-gammaed ground truth using the image-specific gamma predicted by the gamma-correction network.Source: F Loss Function Details · p. 34
  • Tone-mapping losses align global and local luminance with the ground truth, while a luminance-consistency term preserves average brightness and prevents exposure drift.The global tone term compares images downsampled by a factor of eight, and the local term encourages refinement of tonal and luminance variations.Source: F Loss Function Details · pp. 34–35
  • The photofinishing objective combines low-level, perceptual, and regularization losses to support stable convergence and perceptually consistent results across picture styles.When the 3D LuT is included, an auxiliary color-difference term encourages it to approximate target rendering before chroma mapping.Source: F Loss Function Details · pp. 34–35

G Training Details

G.1 Denoising Training

The denoiser is trained on 256×256 patches against pseudo-denoising targets, with three capacity variants trained for each dataset.

G.1 Denoising Training

  • Three denoiser variants—lite, base, and large—are trained for each dataset, including the generic denoisers.Their architectural configurations are provided in Sec. C.1.Source: §G.1 · p. 35
  • Training uses 256×256 patches and minimizes ℓ1 loss against pseudo-denoising ground truth.AdamW training runs for 100 epochs with batch size 32, cosine annealing from 10^-3 to 10^-5, and gradient clipping at maximum norm 0.01.Source: §G.1 · p. 35

G.2 Photofinishing Training

The photofinishing module is trained for 600 epochs on resized pseudo-denoised raw inputs and sRGB targets, using a weighted combination of loss terms.

G.2 Photofinishing Training

  • The photofinishing module is trained for 600 epochs on pseudo-ground-truth denoised raw images mapped to linear sRGB and their final sRGB targets.Both are resized to 512×512, with standard geometric augmentations.Source: §G.2 · p. 36
  • Optimization used Adam with β1 = 0.9, β2 = 0.999, and ℓ2 regularization of 10^-7; the learning rate started at 10^-4 and cosine-annealed to 10^-6.Source: §G.2 · p. 36
  • The loss combines fidelity, perceptual, color, smoothness, tone-mapping, and luminance terms with empirically set weights.The resized photofinishing output and target are 512×512 in the loss details; elsewhere, I↓GT denotes native quarter-raw-resolution images.Source: §G.2 · p. 36

G.3 Detail-Enhancement Training

Detail-enhancement training pairs guided-upsampled photofinishing outputs with corresponding ground-truth patches, with a direct-patch procedure for Zurich.

G.3 Detail-Enhancement Training

  • Training pairs are built by denoising raw images, converting them to linear sRGB, quarter-resolution photofinishing, and guided upsampling before extracting non-overlapping 512×512 patches.Lite, base, and large denoisers are used jointly to enlarge the training set as an augmentation strategy.Source: §G.3 · p. 36
  • For Zurich, downsampling and guided upsampling are omitted because the dataset contains only cropped 448×448 patches.Training is performed directly on those patches.Source: §G.3 · p. 36
  • The network is trained for 50 epochs with batch size 16 using pixel-wise ℓ1 loss between predicted and ground-truth sRGB patches.Adam uses β1 = 0.9, β2 = 0.999, ℓ2 regularization of 10^-7, and cosine-annealed learning rate from 10^-4 to 10^-6.Source: §G.3 · p. 36

H Generalization Across Cameras

The framework separates camera-specific denoising from generic stages and trains a generic denoiser on diverse noise profiles. On unseen-camera tests, the generic denoiser generalizes across devices, while the full pipeline produces competitive cross-camera renderings with fewer parameters than two compared neural ISPs.

H Generalization Across Cameras

  • The generic denoiser was trained on real S24 noise and synthetic noise modeled from 12 sensors spanning main, telephoto, ultra-wide, and front cameras.The hybrid training set comprised 20% clean pseudo-ground-truth images, 40% real noisy S24 images, and 40% synthetic noisy images.Source: H Generalization Across Cameras · pp. 37–38
  • Noise synthesis used ISO- and channel-specific shot- and read-noise parameters calibrated by linear regression, with interpolation for unmeasured ISOs.The authors fitted the parameters from patch means and variances across ISO settings.Source: H Generalization Across Cameras · p. 37
  • On SIDD images from smartphones absent from training, generic denoisers outperformed camera-specific models, demonstrating stronger cross-camera generalization.Table 6 evaluates the models on SIDD; its caption identifies the compared training sets and test set but does not provide cell values here.Source: H Generalization Across Cameras · p. 38, Table 6 · p. 38
  • On unseen Samsung Galaxy S9 and iPhone X images, the pipeline rendered competitively with commercial software; the iPhone X needed autoexposure adjustment before digital gain.The tested configurations included different denoisers and white-balance methods.Source: H Generalization Across Cameras · pp. 38–39
  • On unseen iPhone devices, the 0.5 M-parameter lite model showed stronger cross-camera generalization and greater controllability than LiteISP (about 9 M) and ISPDiffuser (about 21 M).The photofinishing and detail-enhancement modules were trained on S24; results are qualitative.Source: H Generalization Across Cameras · p. 39

I User Interface for Modular ISP Control

The interface exposes the modular ISP pipeline for user-controlled image and picture-style editing. It supports reusable edits, flexible input types, raw-backed re-rendering, and fine-grained control over processing operations and style operators.

I User Interface for Modular ISP Control

  • The GUI lets users enable or disable pipeline modules, adjust their strengths, blend styles or stages, and apply optional editing operators.Users can save manual edits as styles and batch-apply them. The tool accepts DNG raw, its own rendered sRGB with embedded raw, and third-party sRGB; it also supports frame-by-frame video processing with temporal smoothing over a 9-frame window. Embedded compressed raw enables repeated post-editable rendering without accumulating degradations.Source: I User Interface for Modular ISP Control · pp. 39, 43, Figure 20 · p. 42, Figure 21 · p. 42
  • The tool loads 3,902,604 parameters (approximately 14.9 MB / 0.015 GB) across its models and processing modules.The paper contrasts this footprint with ISPDiffuser [71] at 20,938,890 parameters for one style and LiteISP [93] at 9,094,000, with each comparison scaling 6× for six styles.Source: I User Interface for Modular ISP Control · p. 43

I.1 Denoising Adjustment

  • Denoising is adjustable in the raw domain and through optional luma and chroma operators on the color-corrected linear sRGB image.Luma denoising uses an edge-preserving guided filter; chroma denoising uses Gaussian smoothing with a blur scale adapted to image resolution and strength, plus an additional pass above strength 0.4.Source: §I.1 · pp. 44–45

I.2 Exposure Adjustment

  • Optional auto-exposure corrects brightness in linear sRGB before photofinishing, addressing cross-camera raw inputs that may otherwise be misread as near-black.It selects an exposure value by comparing a downsampled luminance histogram with a Gaussian target centered at mid-gray (0.08), searching candidates from −1.8 to +1.8 EV.Source: §I.2 · pp. 45–46

I.3 White Balance Estimation and Adjustment

  • White-balance options include camera AWB, camera-specific or cross-camera illuminant estimates, preference-aware mapping, and manual CCT and tint adjustment.For AWB settings other than the camera-provided gains, the tool recomputes the color correction matrix by interpolating calibrated matrices in the DNG metadata.Source: §I.3 · pp. 46–47

I.4 Contrast Adjustment

  • Optional contrast adjustment modifies luminance around mid-gray after local tone mapping, with positive values increasing contrast and negative values reducing it.The authors report more visually pleasing results from this luminance-channel stage than from applying the adjustment to the three processed linear sRGB channels at other stages.Source: §I.4 · p. 48

I.5 Highlights and Shadows Adjustment

  • Optional highlight and shadow controls use smooth luminance masks to boost or compress highlights and lift or deepen dark regions.The adjusted luminance is recombined with the original chroma channels, preserving color fidelity as the image continues through the pipeline.Source: §I.5 · pp. 48–49

I.6 Saturation and Vibrance Adjustment

  • Optional saturation and vibrance adjustments operate in HSV after gamma correction, and saturation is rescaled uniformly across colors.The passage describes HSV as numerically stable because its saturation channel is bounded in [0, 1], and as an intuitive space for perceptual editing.Source: §I.6 · p. 49
  • Saturation scales chroma uniformly, while vibrance boosts muted colors more strongly and preserves already saturated regions.Positive saturation increases global saturation and negative values move the image toward grayscale; positive vibrance boosts muted colors, while negative values reduce vibrance.Source: §I.6 · pp. 49–50, Figure 29 · p. 51

I.7 Detail-Enhancement Control

  • Detail-enhancement strength uses linear interpolation between the image before and after enhancement, with values from 0 (off) to 1 (full effect).Intermediate values produce proportional effects.Source: §I.7 · p. 50

I.8 Sharpening Adjustment

  • Sharpening is an optional final-stage operation that uses an edge mask to apply detail enhancement predominantly along edges and avoid uniformly sharpening noisy flat regions.The control αsharp adjusts sharpening strength; setting it to 0 disables sharpening, while higher values increase edge contrast.Source: §I.8 · pp. 50–51, Figure 30 · p. 52

I.9 Editing Picture Styles

  • Users can disable style operators, interpolate their parameters across picture styles, or replace operators; linear interpolation is applied once through the photofinishing module for consistent, efficient rendering.An example applies these operations to an iPhone 13 image, captured by a camera unseen during training, using a photofinishing module trained on S24 main-camera images.Source: §I.9 · p. 52, Figure 31 · p. 53

I.10 Re-Rendering with Embedded Raw

The paper embeds compressed raw data and editing metadata in rendered JPEGs to preserve re-rendering and editing after saving. It also describes applying the modular pipeline to ordinary sRGB inputs and swapping modules without retraining the remaining stages.

I.10 Re-Rendering with Embedded Raw

  • The tool appends JPEG-compressed raw data, operator parameters, DNG metadata, and editing configuration after the sRGB JPEG’s end-of-image marker.The visible JPEG remains standards-compliant and decodable by conventional viewers; the tool can retrieve the embedded data for re-rendering.Source: §I.10 · p. 53
  • For 12-megapixel images, the complete file averages 5–6 MB at raw-JPEG quality 95 or 3–4 MB at quality 75.At quality 95, the sRGB JPEG accounts for about 3 MB; quality 95 adds about 2–3 MB and quality 75 about 1–2 MB.Source: §I.10 · p. 54
  • Saved JPEGs can be re-rendered with the original raw-file functionality and different settings, with only slight accuracy loss from compression.The paper reports higher-quality re-rendering than prior alternatives; Fig. 32 illustrates re-rendering with the same or different settings.Source: §I.10 · p. 54, Figure 32 · p. 55

I.11 Processing Input sRGB Images

  • For standard sRGB inputs without embedded raw, the tool linearizes the image, maps it into S24 raw space, then renders it through the pipeline.This path supports sRGB images from cameras or software ISPs, as well as synthetic images, but has reduced accuracy from residual nonlinearities.Source: §I.11 · pp. 54–56

I.12 Functional Module Swapping Without Retraining

  • The tool can replace white balance, tone mapping, and chroma-LUT modules with external operators while keeping remaining modules fixed, producing visually coherent results.The tested replacements were a gray-world estimator, a parametric S-curve, and a fitted 2D LUT derived from a Fuji X-Trans III Sepia HaldCLUT.Source: §I.12 · p. 56

J User Study Details

In a 20-person study across 45 scenes, participants preferred the method across all four criteria, with significant overall preference over the S24 ISP and Lightroom. The section also demonstrates tool use on external sRGB inputs and module swapping without retraining.

J User Study Details

  • Twenty participants evaluated 45 scenes across four criteria, choosing among our method, the native ISP, and Lightroom in randomized order.Scenes covered indoor, outdoor daylight, sunset, and low-light conditions; each participant evaluated every scene under each criterion.Source: J User Study Details · p. 56
  • The tool linearizes external sRGB images into synthetic raw-like representations so its pipeline can process images without embedded raw data.Examples include images rendered by an iPhone ISP and generated by an AI image model, each processed with different tool settings.Source: Figure 33 · p. 57
  • For overall preference, the method received 456 votes versus 212 for the S24 ISP and 223 for Lightroom; pairwise tests found both differences significant.The reported p-values were 1.9 × 10−21 versus S24 and 2.4 × 10−19 versus Lightroom under the equal-preference null hypothesis.Source: J User Study Details · p. 57
  • Replacing or disabling core modules while keeping other stages fixed produces stable, visually plausible results without retraining.Demonstrations swap in gray-world AWB, parametric S-curve tone mapping, or an external HaldCLUT, and also disable tone-mapping stages.Source: Figure 34 · p. 58
  • The method was preferred across all four criteria, with margins of +13.9–27.0% over the closest competitor.Preference rates were 53.2% for color quality, 46.4% for brightness and contrast, 43.4% for sharpness and detail, and 51.4% for overall preference.Source: J User Study Details · p. 58, Figure 35 · p. 58

K.1 Ablation Studies

The ablations find that the full photofinishing design performs best overall, while individual losses, processing stages, and design choices each contribute to quality. Results also identify settings where benefits depend on style or evaluation conditions.

Denoising Ablations

Upsampling Ablations

  • Regularized BGU produces reconstructions closer to ground truth visually and quantitatively, although the guide lacks detail enhancements present in the original S24 targets.The evaluation also uses an alternative ground truth without enhancements; no guided upsampling method can fully recover details absent from its guide.Source: Upsampling Ablations · pp. 60–61, Table 9 · p. 61, Figure 37 · p. 61

Photofinishing Loss Ablations

Photofinishing Design Ablations

Local Tone-Mapping and Chroma Design

  • The full LTM and chroma configuration performs best; residual chroma-LuT learning and higher grid/LuT resolution improve reconstruction accuracy.The LTM maps model exponent modulation, compression strength, asymmetric bright/dark behavior, blending, and local exposure.Source: Photofinishing Design Ablations · pp. 64–65, Table 15 · p. 67

Photofinishing Design Ablations

  • Multi-branch convolution and coordinate attention improve photofinishing, while applying tone mapping jointly in linear RGB outperforms luma-based mapping.The luma-based alternative applies global and local tone mapping only to the Y channel of YCbCr.Source: Photofinishing Design Ablations · pp. 65–67, Table 17 · p. 68

3D Lookup Table

Comparison with HDRNet

  • Our photofinishing design achieves higher PSNR and SSIM than HDRNet replacements while requiring fewer parameters.The comparison replaces either the LTM network and operators or the entire photofinishing module with HDRNet.Source: Photofinishing Design Ablations · p. 69, Table 20 · p. 70

K.2 Additional Results

Additional evaluations report competitive results on MIT-Adobe 5K and consistent re-rendering quality, alongside cross-device examples and visualizations of intermediate stages. The visualizations show tone and chroma adjustments adapting across picture styles.

Visualization of Photofinishing Stages

Results on MIT-Adobe 5K Dataset

  • On MIT-Adobe 5K, the method ranks second in PSNR and first in SSIM; its lite version reaches 21.27 dB with ∼0.5 M parameters.The lite version exceeds ISPDiffuser at 19.44 dB with ∼21 M parameters and ParamISP at 21.17 dB with ∼1.4 M parameters.Source: Results on MIT-Adobe 5K Dataset · p. 71, Table 21 · p. 75

Re-Rendering Results with Embedded Raw

  • Embedding compressed raw data enables unlimited post-capture re-rendering without cumulative accuracy loss or reconstructing raw from rendered sRGB.The approach adds a moderate amount to the final JPEG file size compared with the stated alternatives.Source: Re-Rendering Results with Embedded Raw · p. 72
  • Re-rendering results are consistently superior and avoid other methods’ variability, where quality depends on the source picture style.The comparison uses raw-JPEG quality setting 75, introducing approximately 1–2 MB overhead for embedded raw data.Source: Re-Rendering Results with Embedded Raw · pp. 74–75

Additional Qualitative Results

  • The method produces high-quality S24 results across styles with fewer parameters overall, and iPhone examples demonstrate generalization despite no iPhone training images.Only photofinishing and detail-enhancement modules use style-specific weights.Source: Additional Qualitative Results · p. 75

L Evaluation Details

The evaluation details describe dataset-specific input preparation and baseline handling, along with adaptations needed to train or reproduce several comparison methods. Re-rendering tables cover target styles #0–#5 on S24.

L Evaluation Details

  • Except on Zurich, the authors trained and evaluated methods themselves following their original papers; Zurich results came from prior publications except CIE XYZ Net, and metrics included PSNR, SSIM, LPIPS, and ∆E2000.LPIPS was computed after resizing outputs and ground truth to 1024×1024.Source: L Evaluation Details · pp. 75, 79
  • S24 re-rendering tables report target picture styles #0–#5; for styles #0 and #1, our method uses embedded raw data directly with the base denoiser.Table 22 compares PSNR, SSIM, LPIPS, and ∆E 2000 for target style #0.Source: Table 22 · p. 76, Table 23 · p. 77, Table 24 · p. 77
  • For MIT-Adobe FiveK, comparison methods used linear sRGB inputs for stability, with the supplied split including 493 validation and 989 test images; three-channel raw inputs used black-level normalization and, when needed, Menon demosaicing.Single-channel methods used fixed-Bayer mosaics.Source: L Evaluation Details · p. 76
  • Missing proxy checkpoints and ISP-kernel libraries prevented ReconfigISP’s original architecture search, so the authors retrained a fixed architecture with 34 trainable parameters.The fixed operators were optimized for each target style.Source: L Evaluation Details · pp. 76–77
  • NPF’s missing components were reimplemented, but its rendering-engine version coupling made intermediate outputs difficult to access or interpret and module ordering non-transparent.The caveats affect information needed to construct the proxy pipeline.Source: L Evaluation Details · pp. 77–78
  • ParamISP omitted pretraining for better S24 results, ISPDiffuser patch training reduced time and improved accuracy, and FlexISP used S24 denoising followed by style-specific fine-tuning.FlexISP's stages are denoising, white balance, and Bayer-to-sRGB mapping.Source: L Evaluation Details · pp. 76, 78, 79
Loading 2512.08564v3…