Parametric Gaussian Noise Mixer (PGNM): Unified Synthesis of Camera Noise and Film Grain

Motivation

• Noise synthesis helps preserve film grain as an intended texture, including the photochemical texture identified in FotoKem’s account of Oppenheimer’s finishing process.Grain matching also helps compositing make photographed and computer-generated elements appear to share a film stock.Source: §1 · p. 1
• AV1’s film-grain synthesis encodes a denoised image with estimated grain parameters, then synthesizes grain at the decoder before display.The approach can reduce bandwidth while preserving intended texture.Source: §1 · p. 1
• Realistic noise supports denoising data augmentation: prior work shows the importance of modeling sensor noise and image processing [7], and training with samples from a learned noise model [2].These examples motivate generating noise representative of the target acquisition process, even when noise removal is the eventual task.Source: §1 · pp. 1–2

Contributions

• PGNM renders camera noise and film grain from aligned clean/noisy pairs using a convolved Gaussian field and an intensity-conditioned lookup.Its two components are fitted separately: correlations determine the convolution, and residual quantiles determine the lookup.Source: §1 · p. 2
• On SIDD, PGNM has the lowest average errors, while a denoiser trained on its noise comes closest to one trained on real pairs.For film grain, it matches a physical approximation for fine grain and leads in conditional W1 for coarser grain.Source: §1 · p. 2
• 10 ms per 1080p frame on GPU; intensity-only conditioning lets the image-independent Gaussian field be filtered once and replayed.A prototype adding grain after video decoding suggests potential bitrate savings on one film clip.Source: §1 · p. 2

Physical and statistical models

• Physical camera models combine photon-counting and readout noise, while film-grain methods include Monte Carlo rendering and analytic approximations.Other work models image processing [7], low-light electronic effects, or grain across magnification and blur; AV1 combines autoregression and intensity scaling.Source: §2 · p. 2

Neural models

• Learned camera-noise work spans normalizing flows [2] [26] [30], invertible networks [29], GANs [8] [10] [20] [37], VAEs, and diffusion models [22] [24] [25].Learned film-grain approaches include GANs, lightweight generators, style transfer, and invertible networks [4].Source: §2 · p. 2

Shared observations

• Prior models repeatedly encode intensity-dependent residual strength and spatial correlation, but processed-image residuals also vary in bias, asymmetry, and clipping [2].PGNM uses a full intensity-conditioned distribution and a separate correlation component, realized as a Gaussian copula akin to texture synthesis.Source: §2 · p. 2

3 Method

• A full RGB convolution shapes a Gaussian latent field, and an empirical lookup maps each value and clean intensity to a signed residual.The two components are fitted separately from aligned clean/noisy RGB pairs and remain fixed during synthesis.Source: §3 · p. 2, Figure 2 · p. 3

3.1 Spatial and color dependence

• The normalized convolution introduces spatial and cross-channel dependence while keeping each latent value standard Gaussian through a per-output energy constraint.Its filters are fitted to residual correlations measured on clean, flat pixels, with clipping boundaries excluded; normalized residuals and pooled products estimate the field correlations over lags up to 16 pixels.Source: §3.1 · p. 3
• The correlation fit uses spectral factorization, then converts the factors to spatial kernels cropped to 25 × 25 and normalized; projection and finite support make the fit approximate.The nonlinear lookup can also alter residual correlations.Source: §3.1 · p. 3

3.2 Intensity-conditioned residuals

• A two-dimensional lookup indexed by clean intensity and latent Gaussian value represents each intensity-conditioned residual distribution.Empirical quantiles are pooled across nearby intensity bins; since the latent is standard Gaussian, its cumulative probability selects a percentile and the lookup reproduces the empirical distribution up to interpolation and clipping.Source: §3.2 · p. 4
• During synthesis, bilinear interpolation evaluates the lookup at each clean intensity and latent value, then adds the signed residual to the clean image and clips the result to [0, 1].The Gaussian-axis values beyond the stored range use linear extrapolation.Source: Figure 2 · p. 3, §3.2 · p. 4

4.1 Noise synthesis fidelity

• Four complementary errors measure residual distributions and spatial/color dependence, with lower values better and identical clipping and 8-bit conversion for real and generated images.The measures include KL divergence, conditional W1, and spatial and color correlation errors.Source: §4.1 · p. 4

4.2 Datasets

• SIDD evaluation uses 1,248 validation blocks from 39 source images across five cameras, 16 camera–ISO classes, and eight scenes.PGNM fits one preset per camera–ISO class on SIDD-Medium sRGB pairs; the iPhone 7 ISO 1250 source is excluded because its class is absent there.Source: §4.2 · p. 4
• Film-grain evaluation uses eight Newson-rendered grain radii from 0.025 to 0.8, with conditional W1, spatial correlation error, DISTS, and LPIPS assessed on 60 natural test images.Each image is rendered with five seeds; the protocol reports all radii and separately covers radii up to 0.2, where Zhang23’s small-grain approximation applies.Source: §4.2 · p. 5

4.3 Denoising

• The denoising comparison trains the same DnCNN on real pairs and each generator’s synthetic pairs, then evaluates PSNR and SSIM on all 1,280 SIDD validation blocks.Training uses 30 epochs on 96 × 96 patches from 246 SIDD-Medium images, with shared initialization, patch order, and augmentation across noise sources.Source: §4.3 · p. 5

Camera noise

• PGNM has the lowest mean error on all four measures across 39 SIDD sources, though NM-FlowGAN has slightly lower spatial error on its 19 covered sources.The table reports camera-noise synthesis on SIDD, averaging five generator runs and weighting cameras equally.Source: Table 1 · p. 5, §5 · p. 5
• On a new image, PGNM noise changes with clean-pixel intensities but is not tied to surrounding structure, so it does not model edge-following residuals.GMDCN and NM-FlowGAN retain that edge-following component.Source: §5 · p. 5

Denoising

• A DnCNN trained on PGNM noise is 0.72 dB below one trained on real pairs, ranking PGNM first among synthetic sources.Across all 40 images, the 95% scene-bootstrap interval of its lead over each baseline excludes zero.Source: §5 · p. 5, Table 2 · p. 6

Simulated film grain

• For simulated grain, Zhang23 is on par with PGNM for small grain, while PGNM has the lowest conditional W1 from radius 0.075 upward.Zhang23 is slightly closer in spatial correlation up to radius 0.6, but its approximation degrades as grains become resolved.Source: Table 3 · p. 6, §5 · p. 6

Model size and speed

• A single PGNM model is 5.7–1,250× smaller than GMDCN, NAFlow, DANet, and NM-FlowGAN, while GPU synthesis is 5–19× faster than neural baselines.Building the reusable template bank pays back after about 30 frames on GPU and 24 on CPU; on CPU, Zhang23 remains cheaper.Source: §5 · p. 6, Table 4 · p. 6

Grain synthesis for video delivery

• On Nocturne video, coding the grain costs 9–14× the bits at equal quantizer, while matched DISTS permits at least 90% savings with PGNM synthesis.PGNM is also closer than AV1 grain synthesis at the same rate on DISTS, LPIPS, and W1, with nearly equal spatial error.Source: §5 · p. 7, Table 5 · p. 7

Fitting masks

• The flat mask alone recovers full PGNM, while the clipping guard alone does not; removing the flat mask weakens fitted cross-channel dependence and raises color-correlation error and slightly KL.Textured pixels entering the correlation fit account for the weaker neighboring-pixel dependence.Source: §6 · p. 7, Table 6 · p. 7

Kernel support

• On SIDD, kernel errors level off by 5×5; on Newson grain, 1 × 1 and 3 × 3 lose spatial structure, while 5 × 5 and 11 × 11 match 25 × 25 through the coarsest radius.The 25 × 25 support was fixed in advance; an 11 × 11 kernel has five times fewer values and is a deployment option.Source: §6 · p. 7, Table 6 · p. 7
• Across all 24 presets, two principal components explain 98% of kernel variation and 93% of lookup variation.The principal-component analysis groups camera-noise and film-grain presets along different kernel and lookup components.Source: §6 · p. 7

Conditional distribution

• More than doubled: replacing the lookup with a Gaussian of matching mean and variance at each clean intensity increases KL and conditional W1, while correlation errors stay tied.The lookup preserves the skewed, heavy-tailed SIDD residual distribution at fixed intensity.Source: §6 · p. 7

Limitations

• PGNM cannot represent image-content-dependent residual structure, capture-to-capture variation, or dependence far from Gaussian because it uses a stationary latent filter and an intensity-only lookup.Each condition needs its own fit from aligned clean/noisy pairs; one-shot fitting is not yet supported.Source: §7 · p. 7
• On SIDD, PGNM may miss real edge-following noise, although it avoids fitting that pattern if it comes from references softened by averaging captures.The text leaves unresolved whether edge-following residual structure is a dataset artifact or real noise.Source: §7 · p. 7
• A denoiser trained on PGNM noise falls short of one trained on real pairs, and the independent Newson render is closer to its reference on every measure.These comparisons mark a remaining gap to real noise and the reference grain render.Source: §7 · p. 7

Conclusion

• Each PGNM component is fitted to an observable statistic, enabling component-by-component checks and placing camera noise and film grain in one parameter space.The conclusion says two measurements account for much of the realism of larger generators; what they miss marks where learned capacity is needed.Source: §7 · p. 7

AI use disclosure

• Claude through Claude Code assisted with implementation, experiments, evaluation and figure scripts, supplementary video, and text drafting and editing.The author directed its use, reviewed its output, and takes full responsibility for the content.Source: AI use disclosure · p. 8

Explanation and evidence

• Conditional W1 compares residual distributions at similar clean intensities; spatial and color correlation errors measure dependence, tile KL compares local residual histograms, and DISTS and LPIPS compare film appearance.Lower values are better for all listed scores.Source: Explanation and evidence · p. 10

A PGNM in equations

• PGNM filters independent Gaussian inputs with full RGB convolution, then maps each latent value to a signed residual using a lookup conditioned on clean intensity.The energy constraint keeps each latent output standard Gaussian, allowing the lookup to set the residual distribution independently of the filters.Source: A PGNM in equations · p. 10
• The residual lookup uses empirical residual-code distributions from training pixels near each clean intensity, with radius at least two codes and enough samples to collect 2,000 or span all codes.It stores values at 256 clean codes and 25 Gaussian inputs, with interpolation and extrapolation specified in the supplement.Source: A PGNM in equations · p. 10, Reproducibility reference · p. 10

Reproducibility reference

• Near black and white, clipping makes observed residual spread underestimate noise strength, so the pilot fits the exact expected squared residual of a clipped Gaussian model.The fitting procedure pools flat pixels by intensity bins while evaluating predictions at each pixel’s own clean intensity; bisection solves the strength estimate.Source: Reproducibility reference · pp. 10–12
• The pilot rejects knots with too few flat pixels, excessive strength, or predicted clipping above 35%; rejected values are interpolated, then the curve is smoothed once.The pilot prepares the correlation fit; the lookup describes the observed clipped residuals directly and needs no correction.Source: Reproducibility reference · p. 12
• Spectral factorization turns measured stationary correlations into convolution filters by factoring the power spectrum frequency by frequency.The scalar relation is filter power spectrum equals the squared magnitude of its frequency response; the multichannel counterpart factors a positive-semidefinite matrix spectrum.Source: Reproducibility reference · pp. 12–13
• The covariance does not identify a unique original filter: the construction chooses zero phase, producing a real symmetric filter that need not have finite support.The fit symmetrizes correlations, tapers distant lags, projects negative spectral eigenvalues to zero, crops the inverse transform, and renormalizes filters.Source: Reproducibility reference · pp. 13–14
• The resulting nine 25 × 25 filters satisfy unit energy, but tapering, spectral projection, cropping, and renormalization make the covariance fit approximate, and the lookup can further change residual correlations.The supplement also provides algorithms, numerical conventions, data splits, seeds, and evaluation procedures for exact reimplementation.Source: Reproducibility reference · pp. 10, 14

B Ablation details

• The tested fitting variants change one step at a time, are otherwise refitted on the same pairs, and use the same random streams for rendering.The variants and which would be shown in the main paper were fixed before scoring, except the one-knot pilot variant.Source: B Ablation details · p. 14
• The moment-matched Gaussian lookup exceeds |g| ≤ 1 in dark red and blue channels of S6 ISO 800 and ISO 3200, so 29 entries are clipped to ±1.Those entries affect fewer than 10^-5 of channel values rendered on the training data.Source: B Ablation details · p. 14, Table S1
• From 5 × 5 support onward, no SIDD correlation error is worse than full PGNM’s; on film grain, changes are small and mostly improvements from 5 × 5 onward.A 3 × 3 kernel raises film-grain spatial correlation error by 76–180% at every radius; a 1 × 1 support loses grain structure.Source: B Ablation details · p. 14, Table S2 · p. 15
• The flat mask alone recovers full PGNM on every SIDD metric, while the clipping guard alone leaves color correlation error 34% higher.The text attributes weakened fitted cross-channel dependence between neighboring pixels to textured pixels in the correlation fit.Source: B Ablation details · p. 14, Table S1
• Solving the pilot at one knot per clean code raises spatial correlation error by 1.8% and tile KL by 0.15%, with other measures tied.Per-code strengths follow the pooled pilot where flat pixels are plentiful and scatter where they are few.Source: B Ablation details · p. 14, Figure S4 · p. 14

C SIDD sources, results by camera and ISO, and film grain by radius

• PGNM has the lowest value in 13 of 16 classes for tile KL, 10 for block KL, 10 for conditional W1, 10 for spatial correlation error, and 14 for color correlation error.GMDCN has the remaining lowest KL values and five of six remaining lowest conditional W1 values; NAFlow and NM-FlowGAN lead in some other classes and metrics.Source: C SIDD sources, results by camera and… · p. 15
• Zhang23 has the lowest conditional W1 at radii 0.025 and 0.05, while PGNM is lowest from radius 0.075 onward.Zhang23 has the lowest spatial correlation error through radius 0.6, GrainNet at 0.8; GrainNet is lowest in DISTS and LPIPS from radius 0.2 onward.Source: C SIDD sources, results by camera and… · p. 15
• The evaluation uses 1,248 blocks from 39 SIDD Validation images, spanning five cameras, 16 camera–ISO classes, and eight physical scenes.Each class contains one to five source images, so the camera–ISO breakdown tables are descriptive and have no intervals.Source: C SIDD sources, results by camera and… · p. 15, Table S3 · p. 16

D Further analyses

• These further analyses were computed after the main evaluation using scored outputs or unchanged fits and checkpoints, and are not additional comparisons.Source: D Further analyses · p. 16

D.1 Where the SIDD errors come from

• PGNM has the lowest SIDD bias error at 0.92 codes and shape error at 0.041σ, while GMDCN best matches per-bin strength at 1.26 versus PGNM’s 1.84 codes.GMDCN’s same-pixel channel correlation is closer to real noise, 0.361 versus 0.357 for PGNM and 0.341 for real noise; PGNM has lower neighboring-pixel color correlation error.Source: §D.1 · p. 16, Table S10 · p. 19
• With one oracle gain per channel, PGNM’s conditional W1 falls from 1.32 to 1.05 codes and is lower than GMDCN’s in 15 of 16 classes, versus 11 as scored.The oracle reads evaluation data and is diagnostic, not a method; tile KL changes little under correction.Source: §D.1 · pp. 17–18, Table S11 · p. 19

D.2 A clean image from outside SIDD

• On Kodak kodim19, PGNM’s detail slope stays within about ±0.02, consistent with using the clean image only through intensity; real SIDD slopes are +0.53 to +0.68 in some classes.GMDCN and NM-FlowGAN retain positive Kodak slopes of +0.23 to +0.30 in those classes.Source: §D.2 · p. 18, Table S12 · p. 19
• On gray Kodak, PGNM’s film-grain strength stays within 4% of the Newson reference at every radius, comparable to Zhang23 and the independent Newson render.GrainNet’s finest grain is 60% stronger than the reference.Source: §D.2 · p. 18, Table S13

D.3 Offset and strength of film-grain outputs

• PGNM’s film-grain mean residual stays within ±0.30 codes through radius 0.2 and ±0.83 codes beyond, with pixel-weighted offset within 0.06 codes of the reference at every radius.Newson runs keep every bin mean within ±0.41 codes; Zhang23 reaches 4.3 codes at radius 0.8.Source: §D.3 · p. 18, Table S14 · p. 20, Table S15 · p. 20

E Denoising details

• Training uses 246 sRGB pairs from 123 SIDD-Medium scene instances across the 15 camera–ISO classes supported by NM-FlowGAN.PGNM fits one preset per class on the same 246 captures, while each patch receives an independent latent field.Source: E Denoising details · p. 20
• All six arms use a shared DnCNN configuration, training seed, initial weights, patch order, and augmentation; only seed 1 was run.Training lasts 30 epochs, and the reported model is the epoch-30 checkpoint; no evaluation curve determined the selected checkpoint or settings.Source: E Denoising details · p. 20
• The test set contains all 1,280 blocks from 40 SIDD Validation images, including 19 images in training classes and 21 in other classes.Paired intervals resample the eight physical scenes.Source: E Denoising details · p. 20, Table S16
• PGNM’s denoiser is 0.72 dB below the real-pair reference on all 40 SIDD Validation images and above every other synthetic arm on all three evaluated sets.Its lead excludes zero at 95% confidence except against NAFlow-NAS on the 21 images outside training classes, where it is a tie.Source: E Denoising details · pp. 20–21, Table S17 · p. 21, Table S16

F Video delivery details

• The study compared direct grain coding with post-decode grain synthesis from the clean master or nlmeans-denoised video on a fixed 100-frame, 1.67-second Nocturne clip.The grain-version residual also contains ProRes and 4:2:2 coding error that no arm can remove; VVC’s grain analysis uses its own temporally filtered input rather than matched clean or nlmeans input.Source: F Video delivery details · p. 21
• With the model amortized over the 664.5-second film, PGNM on the clean master reached DISTS 0.087 at 76 kbps, below every lossy grain-coded encode’s DISTS of at least 0.089 at 73 Mbps.Charging the model to this 1.67-second clip adds about 770 kbps; amortized over the film, it adds 1.9 kbps.Source: F Video delivery details · pp. 21, 24, Figure S6 · p. 22
• At matched DISTS, PGNM saved 90.1–98.8% of bitrate on the clean master and 85.4–98.2% on nlmeans-denoised video versus the closest grain-coded baselines.The headline measure matches DISTS by interpolating the direct encode’s bitrate; the reported range covers PGNM grain-synthesis rates.Source: Table S20 · p. 24, F Video delivery details · p. 24
• At lossy rates, PGNM’s delivered grain was 10–26% stronger in RMS than the grain version, while its signed lookup represented the darker offset that zero-mean AV1 grain could not.At Q 31 on the clean master, delivered mean offsets were (0.16, −0.01, 0.17) codes for PGNM and (0.74, 1.89, 1.71) for AV1 grain.Source: F Video delivery details · p. 24

I.1 Interfaces and conventions

• A preset contains a normalized 25 × 25 × 3 × 3 convolution tensor and a 3 × 256 × 25 residual-quantile lookup; Fit creates presets and Warmup and Render synthesize images.The lookup carries residual amplitude directly, so synthesis uses no additional strength multiplier.Source: §I.1 · p. 40

I.2 Fitting

• Fitting separates dependence estimation from residual-distribution estimation: a flat-pixel filter branch estimates correlations, while a lookup branch uses all training pixels.The filter branch uses pilot normalization and a flatness mask, with an intensity-and-pilot guard unless that guard selects no pixels globally.Source: §I.2 · p. 40
• The lookup pools neighboring intensity histograms until they contain 2,000 samples or span the full intensity range, then stores residual quantiles at 25 Gaussian probabilities.It includes textured and clipped residuals and interpolates continuously between integer clean-intensity codes.Source: §I.2 · p. 40

I.3 Synthesis

• Warmup filters independent Gaussian samples into four reusable fields, and Render selects a cyclic window, applies the lookup, adds residuals, and clips intensities to [0, 1].Eval interpolates bilinearly in clean intensity and Gaussian value, extending the end segments beyond t = ±3.Source: §I.3 · p. 42
• Cyclic windows can cross nonperiodic field seams, disrupting short-range correlation; roughly three quarters of random windows do so for large images.This bank procedure is used for reported SIDD and film scores, while denoising training generates a fresh field for each patch.Source: §I.3 · p. 42

• PGNM uses one preset per SIDD camera–ISO class and one per film-grain radius, fitted with the same estimator.The fit procedure returns a convolution factor and a conditional lookup.Source: §I.4 · pp. 42–43
• For SIDD, image pairs are cropped on a 256-pixel grid, and held-out captures—at least 20% per class—are excluded from fitting.All crops from each held-out capture remain outside the fit.Source: §I.4 · p. 42
• Evaluation renders each SIDD Validation block with its class preset and a bank; denoising training instead builds a fresh field for every patch and epoch.The evaluation protocol resets the previous-field index before each output image.Source: §I.4 · pp. 42, 46
• Film-grain fitting uses GrainNet training pairs, holds out grouped transformed copies, and estimates the blurred expected image with a 7 × 7 filter.Residual fitting excludes a three-pixel border; the expected image is computed over the whole image.Source: §I.4 · p. 42
• Five generator runs use separate four-field banks, while per-image window selection uses separate seeded streams on both datasets.The SIDD and film stream seeds are derived from SHA-256 digests of protocol strings.Source: §I.4 · p. 42
• The fitting procedure estimates pilot strength, fits residual correlations from flat pixels, factors them into a convolution tensor, and fits the lookup from training pairs.The lookup uses all pixels, while the correlation step uses a flat-pixel mask.Source: §I.4 · pp. 43–44

J Metric details

• Conditional W1 compares residual bias, strength, and distribution shape at similar clean intensities, with groups weighted by pixel count and cameras weighted equally on SIDD.Adjacent intensity bins are merged until groups contain at least 512 pixels or only one group remains.Source: J Metric details · pp. 46, 48
• Spatial error compares normalized residual correlations over 288 nonzero lags, while color error compares neighboring cross-channel correlations over 24 values and excludes zero lag.Residuals are centered and scaled within clean-intensity groups before correlation comparison.Source: J Metric details · pp. 46–47
• Tile KL compares residual-value histograms pooled across channels and clean intensities, without encoding residual arrangement within each tile.Each 256 × 256 block is divided into 64 nonoverlapping 32 × 32 tiles.Source: J Metric details · pp. 47–48
• Film residuals are measured around the blur-filtered expected image, and film scoring also compares delivered-image appearance with DISTS and LPIPS.Residual scores pair five method outputs with five reference renders, averaging 25 comparisons per image and radius.Source: J Metric details · p. 48
• Denoising evaluation compares reconstructed blocks with clean images using PSNR and SSIM, averaged equally over 1,280 Validation blocks.PSNR uses peak value 255; SSIM follows specified 8-bit RGB settings and discards a three-pixel border.Source: J Metric details · p. 48

K Baselines

• SIDD baselines use released checkpoints and native settings, with five generated outputs per block; NeCA-W is excluded because its generator requires a real noisy input.The protocol does not provide a reference-free inference rule for NeCA-W.Source: K Baselines · p. 48
• Film fitting uses 159 of 199 GrainNet source images, while evaluation uses 60 separate test images; Newson references and independent renders use five seeds each.GrainNet and the independent Newson renderer receive the clean image in their native pipelines.Source: K Baselines · pp. 48–49

L Cost measurement setup

• A precomputed PGNM bank amortizes after approximately 30 GPU frames or 24 CPU frames versus new-field generation; timings use one 1080p RGB frame and include transfers.Bank playback reads four cyclic float32 fields; construction and playback are measured separately.Source: L Cost measurement setup · p. 49