PaperlayerView exact revision on arXiv

Source-linked AI summary

High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial

Mikhail Konnik, James Welsh

arXiv:1412.4031v1astro-ph.IMcs.CV

TL;DR

The paper addresses the lack of comprehensive high-level models for simulating realistic CCD and CMOS photosensor noise. It formulates a literature-based model and validates it against hardware sensor data, reporting that modeled non-linearities improve the match and that long-integration dark-signal spikes require a more complex distribution.

  • Problem

    Comprehensive high-level photosensor models are lacking, although realistic noise simulation is needed for image-processing algorithm testing and validation.

    Source: §1 · p. 1, §5 · p. 18

  • Method

    The paper formulates a literature-based high-level CCD/CMOS model and compares its simulated CMOS output with data from a custom-made hardware photosensor.

    Source: §5 · p. 18

  • Results

    The modeled V/V and ADC non-linearities better match the hardware CMOS sensor, while a Gamma distribution superimposed with uniform and Gaussian distributions reasonably approximates long-integration dark spikes.

    Source: §5 · p. 18

  • Takeaways & Limitations

    For fixed sensor parameters, superimposed distributions provide an acceptable description of dark-current FPN, whose distribution becomes more complicated as integration time increases.

    Source: §4.6.3 · p. 18

    Dark-signal modeling is constrained because distribution parameters vary non-trivially with temperature and integration time and across sensors; undisclosed CDS algorithms and additional on- and off-chip noise further complicate simulation.

    Source: §4.6.3 · pp. 17–18

Abstract

from arXiv · show

In many applications, such as development and testing of image processing algorithms, it is often necessary to simulate images containing realistic noise from solid-state photosensors. A high-level model of CCD and CMOS photosensors based on a literature review is formulated in this paper. The model includes photo-response non-uniformity, photon shot noise, dark current Fixed Pattern Noise, dark current shot noise, offset Fixed Pattern Noise, source follower noise, sense node reset noise, and quantisation noise. The model also includes voltage-to-voltage, voltage-to-electrons, and analogue-to-digital converter non-linearities. The formulated model can be used to create synthetic images for testing and validation of image processing algorithms in the presence of realistic images noise. An example of the simulated CMOS photosensor and a comparison with a custom-made CMOS hardware sensor is presented. Procedures for characterisation from both light and dark noises are described. Experimental results that confirm the validity of the numerical model are provided. The paper addresses the issue of the lack of comprehensive high-level photosensor models that enable engineers to simulate realistic effects of noise on the images obtained from solid-state photosensors.

1 Introduction

The paper addresses the need for comprehensive high-level CCD/CMOS noise simulations and presents a literature-based model intended to represent realistic sensor noise and non-linearities. Its model is demonstrated against a hardware CMOS photosensor.

1 Introduction

  • Photosensor noise arises from multiple sources, some of which cannot be modelled adequately using Gaussian noise alone.The paper frames high-level simulation as an active research area, distinct from noise estimation addressed by EMVA1288.Source: §1 · p. 1
  • The article reviews existing CCD/CMOS models and distinguishes high-level image-quality simulation from low-level solid-state-physics simulation.Existing high-level models may assume linear ADC, sense-node, and source-follower behavior, which the paper says is not always the case.Source: §1 · p. 1
  • The paper formulates a high-level CCD/CMOS model that includes multiple noise sources and conversion non-linearities, and reports validation against a hardware CMOS photosensor.The stated aim is to support realistic noisy-image simulation for image-processing algorithm testing and validation.Source: §1 · p. 2

Contents of the article

  • The article is organized around sensor noise sources, a staged simulation methodology, and experimental validation, including dark-signal performance across integration times.The listed validation analyses include radiometric response, photon transfer, signal-to-noise ratio, photo-response non-uniformity, noise spectrograms, and dark-signal behavior.Source: Contents of the article · p. 2

2 Noise sources in solid-state photosensors

The section describes noise and non-linearity across the photon-to-charge, charge-to-voltage, and voltage-to-digital stages. It distinguishes spatially fixed effects from temporal variation and outlines the mechanisms and modeling considerations at each stage.

2 Noise sources in solid-state photosensors

  • Photosensor noise includes fixed-pattern components that persist across frames and temporal components that vary from frame to frame.The section organizes noise mechanisms along the photon-to-charge, charge-to-voltage, and voltage-to-digital stages.Source: §2 · p. 2

2.1 From Photon to Charge

  • Photon shot noise follows a Poisson process, while PRNU is a signal-dependent, time-invariant spatial variation in pixel output.At high light levels, above 1000 photon-arrival events, the Poisson distribution may be approximated by a Gaussian; that approximation is not justified at low light.Source: §2.1.1 · p. 3, §2.1.2 · p. 3

2.1.4 Dark current

  • Dark current is thermally generated charge; its average varies with integration time and temperature, and its random generation produces dark-current shot noise.The section also treats pixel-to-pixel dark-current variation as fixed-pattern noise and notes that dark spikes can substantially exceed the mean dark signal.Source: §2.1.4 · p. 3, §2.1.5 · p. 3, §2.1.6 · p. 3

2.1.8 Source follower noise

  • Source-follower noise can include white, flicker, and random-telegraph components, with flicker noise reported as higher in CMOS than CCD sensors.The paper notes that flicker-noise and RTN modeling and explanation remain active research topics.Source: §2.1.8 · p. 4

2.2 From Charge to Voltage

  • Charge-to-voltage conversion adds sense-node reset noise and offset FPN, while V/V and V/e− gain non-linearities can affect the measured signal.V/e− non-linearity is mainly a CMOS issue and is usually negligible for CCD sensors; it affects both FPN and shot noise.Source: §2.2 · p. 4, §2.2.1 · p. 4, §2.2.2 · p. 4

2.2.6 Correlated double sampling

  • Correlated double sampling takes reset and signal samples and subtracts them, chiefly to reduce fixed-pattern noise from transistor threshold variations.The text states that CDS suppresses low-frequency noise components but increases thermal-noise contributions.Source: §2.2.1 · p. 4, §2.2.6 · p. 5

2.3 From Voltage to Digital Numbers

  • An ADC maps voltage to discrete codes, with its resolution determined by the available code intervals across the quantifiable voltage range.The section identifies integral and differential linearity errors, quantisation error, and ADC offset as effects to consider; it notes that integral linearity error cannot be calibrated out.Source: §2.3 · p. 5, §2.3.1 · pp. 5–6

3 Simulation Methodology

The simulator implements the sensor model as sequential functions that transform an input through photon, electron, voltage, and digital representations. It separately preserves fixed-pattern noise and recalculates temporal noise for each simulation.

3 Simulation Methodology

  • The MATLAB implementation sequentially transforms the input from photons to electrons to voltages and finally to a digital signal, with noise models selectable for CCD or CMOS simulation.The software also includes routines and scripts for estimating noise characteristics.Source: §3 · p. 6
  • Fixed-pattern noise matrices are calculated once and reused, whereas temporal-noise matrices are recalculated for each simulation.This preserves the fixed-pattern component across runs while allowing the temporal component to vary.Source: §3 · p. 6

3.1 Model: From Photon to Charge

The model simulates the path from incident light to electrons while adding temporal and fixed-pattern noise at the relevant stages. It also models source-follower noise and uses non-symmetric dark-current distributions, which experiments support for long exposures.

3.1.1 Simulating photon shot noise

  • Photon counts are perturbed by Poisson shot noise, with each pixel’s collected photon count serving as the distribution mean.The resulting photon matrix is converted to electrons using wavelength-dependent quantum efficiency.Source: §3.1.1 · p. 7, §3.1.2 · p. 7

3.1.3 Simulation of photo response non-uniformity

  • PRNU is modeled as a temporally fixed, per-pixel Gaussian variation in the electron signal, using the same saved random matrix across simulations.The PRNU matrix is calculated once because the pattern does not change from frame to frame.Source: §3.1.3 · p. 7

3.1.4 Simulation of dark signal

  • Dark signal is calculated for every pixel at the stated room temperature of +25°C, and random electron generation adds Poisson dark-current shot noise.Dark-current shot noise is recalculated for each simulation.Source: §3.1.4 · p. 7, §3.1.5 · p. 7

3.1.6 Simulation of dark current fixed pattern noise

  • For long exposures, experiments support non-symmetric dark-current FPN models, particularly Log-Normal, but a single distribution is inadequate and two or more may be needed beyond 30–60 seconds.The dark-current FPN pattern is fixed across frames; the simulations use Log-Normal for short integration times and superimpose distributions for long times.Source: §3.1.6 · p. 8

3.1.7 Simulation of Source follower noise

  • Source-follower noise is approximated from its noise spectrum and the CDS transfer function, then treated as additive, Gaussian noise in electrons.The RMS noise depends on sensor gains and sampling parameters; the paper notes flicker noise typically limits CCDs, while RTN typically limits CMOS sensors.Source: §3.1.7 · p. 8

3.2 Model: From Charge to Voltage

The charge-to-voltage stage combines light and dark electron signals before converting their total to voltage. It models reset and offset fixed-pattern noise, gain nonlinearities, and correlated double sampling (CDS).

3.2 Model: From Charge to Voltage

  • The light signal includes photon shot noise and PRNU, while the dark signal combines dark signal, dark-current shot noise, dark-current FPN, and source-follower noise.The light and dark electron matrices are summed and rounded, then truncated to the pixel full well.Source: §3.2 · pp. 8–9

3.2.1 Simulation of sense node reset noise (kTC noise)

  • Sense-node reset noise is added to the reference voltage, with inverse-Gaussian and Log-Normal distributions described for hard-reset and soft-reset techniques, respectively.The soft-reset simulation adds reset noise to the reference voltage matrix.Source: §3.2.1 · p. 9

3.2.2 Simulation of offset fixed pattern noise

  • Offset FPN is modeled as separate column and pixel components using uncorrelated first-order isotropic autoregressive processes.The column component is generated per column and repeated across rows; its parameter controls dependency on neighboring columns.Source: §3.2.2 · p. 9

3.2.3 Conversion to Voltage

  • Total collected electrons are converted at the sense node using the sensor-dependent gain, typically 1–5 µV/e−, with reset noise added to the reference when simulated.The source-follower gain is then applied, and offset FPN and optional V/V nonlinearity are added before voltage is stored.Source: §3.2.3 · p. 9

3.2.4 Simulation of the V/e−gain non-linearity

  • The model applies a V/e− gain nonlinearity parameterized by α, then models V/V nonlinearity by changing the source-follower gain before ADC conversion.The nonlinearity coefficient is typically supplied in the sensor specifications.Source: §3.2.4 · p. 9, §3.2.5 · p. 10

3.2.6 Simulation of correlated double sampling

  • CDS subtracts a zero-exposure dark image from a light image to eliminate fixed-pattern noise, but the simulated procedure may be less aggressive than hardware CDS and leave more residual noise.Hardware CDS can use different and more sophisticated techniques, contributing to mismatch with the simulator.Source: §3.2.6 · p. 10

3.3 Model: From Voltage to Digital Numbers

The final stage converts sensor voltages into digital numbers, optionally accounting for ADC nonlinearity and quantization noise. The quantization model assumes uniform error and does not prescribe an ADC architecture.

3.3 Model: From Voltage to Digital Numbers

  • After voltage conversion, the model optionally applies V/V and ADC nonlinearities, then multiplies by ADC gain and rounds to produce a digital signal.The linear case converts total voltage to digital numbers using the ADC reference voltage and gain.Source: §3.3 · p. 10

3.3.1 ADC non-linearity simulation

  • ADC integral-linearity error is modeled as signal-dependent ADC gain using a specified nonlinearity ratio.The resulting nonlinear ADC conversion gain is used in the voltage-to-digital conversion.Source: §3.3.1 · p. 10

3.3.2 Simulation of quantisation noise

  • Quantization error is assumed uniform between −0.5 and +0.5 LSB and uncorrelated with the signal; for qADC = 1, the noise is 0.29 DN.The model does not assume a particular ADC architecture.Source: §3.3.2 · p. 10

4 Experimental validation of the photosensor model

The model’s parameters were adjusted against measurements from a custom-made CMOS sensor, yielding simulated images with statistical properties close to the hardware sensor. The radiometric comparison shows similar behavior, with higher hardware noise at low integration times.

4 Experimental validation of the photosensor model

  • Parameter adjustment produced realistically noised images with statistical properties close to those of the hardware photosensor, even with incomplete manufacturer specifications.Validation used a custom-made 5T CMOS sensor, and the manufacturer’s parameters supplied the initial simulation settings.Source: §4 · pp. 10–11

4.1 Radiometric function

  • The radiometric function behaves similarly for simulated and hardware sensors, but hardware standard deviation is higher at low integration times.Figure 2 marks simulated data with circles and hardware data with bullets; the authors attribute differences partly to manufacturer-specification parameters.Source: §4.1 · p. 11, Figure 2 · p. 11
  • Undisclosed hardware CDS details may contribute to the higher hardware noise, while residual offset-FPN artifacts also contribute to the measured noise.The implemented CDS may be more aggressive than the hardware algorithm, although overall model behavior is considered consistent with the sensor.Source: §4.1 · p. 11

4.2 Photon Transfer Curve

The Photon Transfer Curve (PTC) is estimated across the sensor’s dynamic range by relating mean signal to noise. Comparing simulated and hardware curves shows that including non-linearities brings the model closer to the hardware sensor.

4.2 Photon Transfer Curve

  • A gradient scene spanning the dynamic range and 64 captured images provide mean-signal and standard-deviation pairs for the PTC.The images are averaged to estimate mean signal; each pixel’s standard deviation across the images supplies its noise value.Source: §4.2 · p. 12
  • The PTC measurement yields many data points, revealing irregularities and noise spikes at 1100 and 2000 DN.The authors note that more PTC data points may provide more information about photosensor noise.Source: §4.2 · p. 12
  • The non-linear model’s Photon Transfer Curve is closer to the hardware sensor, and the authors conclude that the model is consistent with the hardware data.The comparison includes hardware data, a fully linear model, and a model using voltage-to-voltage and ADC non-linearities.Source: §4.2 · p. 12, Figure 3 · p. 12

4.3 Signal-To-Noise Ratio

The signal-to-noise ratio (SNR) is compared between hardware and simulated sensors using data derived from the PTC. The non-linear model is closer to the hardware data, while the manufacturer’s undisclosed CDS details are cited as a source of noise-performance differences.

4.3 Signal-To-Noise Ratio

  • SNR is plotted against signal on linear and logarithmic scales using data derived from the PTC.The linear plot is more suited to analysis at large signal, while the logarithmic plot gives more information at low signal.Source: §4.3 · pp. 12–13, Figure 4 · p. 13
  • The fully linear model overestimates sensor SNR, whereas the non-linear model is closer to the hardware data.The non-linear model includes both voltage-to-voltage and ADC non-linearities, which the authors note are uncompensated in the camera’s ADC and source follower.Source: §4.3 · p. 13, Figure 4 · p. 13
  • SNR curves provide a qualitative model–hardware comparison, but image data alone make the influence of non-linearities difficult to distinguish.The authors describe PTC comparison as a simple, informative qualitative check of model consistency with hardware.Source: §4.3 · p. 13
  • The manufacturer did not disclose CDS algorithm details, which the authors cite to explain differences in simulated and hardware noise performance.The authors nevertheless report that the non-linear model is closer to the hardware sensor and produces more realistic noise in simulated images.Source: §4.3 · p. 13

4.4 Photo Response Non-Uniformity

The authors estimate PRNU from uniform-field images after subtracting averaged dark frames, then compare the hardware estimate with the manufacturer’s specification. The hardware PRNU is estimated as 0.52%, and its distribution is reported to be well approximated by a Gaussian.

4.4 Photo Response Non-Uniformity

  • The PRNU measurement uses a uniform flat-field scene, with 64 images and matching dark frames averaged before dark-frame subtraction.The exposure is set so the uniform-image mean equals half the sensor saturation level; the dark-frame subtraction reduces dark-current and offset FPN influence.Source: §4.4 · p. 13
  • Hardware PRNU is approximately 0.52%, consistent with the manufacturer’s stated 0.50% simulation value.The hardware estimate uses a mean of 1520 DN and standard deviation of 7.9, with Gaussian-parameter maximum-likelihood estimates on 95% confidence intervals.Source: §4.4 · p. 14
  • The measured hardware PRNU distribution is reported to be Gaussian with very good accuracy.The mean value of 1520 DN is subtracted in the plotted probability density.Source: §4.4 · p. 14, Figure 5 · p. 14
  • Below approximately 10000 electrons, photon shot noise dominates and may be confused with PRNU.The passage states that the percentage of PRNU in the signal should remain constant until full well is reached.Source: §4.4 · p. 14

4.5 Noise Spectrogram

Noise spectrograms are estimated from spatial-frequency content and compared for hardware and simulated sensors under dark, 50%-saturation, and 90%-saturation conditions. The read noise is reported as approximately white, while the hardware spectrogram is non-flat at the two illuminated levels and its non-whiteness coefficient is slightly higher than the simulation’s.

Method of measurements for noise spectrogram

  • The spectrogram is formed from mean FFT amplitudes across image lines, with frequency represented in cycles per pixel.For each line, the method subtracts the mean, zero-pads the pixel sequence to length 2N, and takes its FFT.Source: §4.5 · p. 14, Method of measurements for noise… · p. 14
  • The measurement compares spectrograms in darkness and at 50% and 90% saturation, using integration times of 0.05 and 0.1 seconds for the illuminated conditions.The hardware and simulated sensor spectrograms are presented in separate panels.Source: Method of measurements for noise… · p. 14, Figure 6 · p. 15

4.5 Noise Spectrogram

Results for non-whiteness coefficient of noise

  • Read noise is reported as approximately white, with F ≈ 1; the hardware non-whiteness coefficient and its measurement uncertainty are slightly greater than the simulation’s.The coefficient is calculated for each spectrogram, with uncertainty estimated from the standard deviation of 100 measurements.Source: Results for non-whiteness coefficient… · p. 15, Table 2 · p. 15

4.6 Dark signal performance for different integration times

Dark-signal statistics become more complex as integration time increases, so the tested 5T CMOS sensor requires different distribution models across exposure regimes. Superimposed distributions fit long-exposure measurements better, but their parameters are difficult to generalize across sensors and conditions.

4.6 Dark signal performance for different integration times

  • The measurements used 32 dark frames per integration time, averaged to reduce temporal noise; statistical parameters were estimated by maximum likelihood and reported within 95% confidence intervals.The study used uniform and Gaussian distributions as superimposed components.Source: §4.6 · p. 16

4.6.1 Short integration time

  • At 0.001-second integration, Gaussian fit the dark-current FPN overall but underestimated dark spikes, while Log-Normal was used to capture those outliers.At 100 seconds, the Log-Normal parameters remained similar and the distribution was still considered reasonable; Gamma underestimated dark spikes in this 5T sensor.Source: §4.6.1 · pp. 16–17

4.6.2 Long integration time

  • At 150 seconds, a Gamma distribution with a superimposed product of two uniform distributions agreed better with measured dark-signal statistics.The uniform components were chosen to fit outliers and widen the left part of the probability density.Source: §4.6.2 · p. 17, Figure 10 · p. 17
  • At very long exposures, thermal electron noise is expected to dominate over dark-current FPN, and the measured distribution may require sensor-specific modeling.A Gamma main distribution with superimposed uniform and Gaussian distributions described the example measurements; the discussion notes that mixture parameters depend non-trivially on temperature and exposure and likely vary by sensor.Source: §4.6.2 · p. 17, §4.6.3 · pp. 17–18

4.6.3 Results discussion

  • Superimposed distributions gave acceptable results for fixed sensor parameters, but additional on-chip and off-chip noise can contribute to the measured distribution.The manufacturer’s undisclosed CDS algorithm also makes hardware-to-model comparison approximate.Source: §4.6.1 · p. 16, §4.6.3 · pp. 17–18
  • For short exposures, Log-Normal models are adequate, while long exposures require superimposed distributions and exposures longer than 300 seconds may need several.The measured distribution grows more complex with exposure time, making the dark signal harder to model.Source: §4.6.3 · p. 18

5 Conclusion

The paper formulates a high-level model for CCD and CMOS photosensors from a literature review, and validates the CMOS model against a custom-made hardware sensor. The comparison highlights the role of modeled non-linearities and a more complex distribution for dark-current spikes at long integration times.

5 Conclusion

  • The paper formulates a high-level CCD and CMOS photosensor model based on an extensive review of photosensor noise models.Experimental results from a custom-made 5T CMOS photosensor are presented to validate the model.Source: §5 · p. 18
  • The hardware comparison showed that including V/V and ADC non-linearity better matched the CMOS sensor, while dark-current spikes at long integration times were reasonably approximated by Gamma, uniform, and Gaussian distributions.Photon Transfer Curve estimation also showed the importance of modeled non-linearity.Source: §5 · p. 18
  • The model can simulate either CCD or CMOS sensors using appropriate noise models and synthesize realistically noised images for image-processing algorithm development and testing.Source: §5 · p. 18
Loading 1412.4031v1…