One Photon, Many Worlds: Posteriors and Predictions with Single-Photon Cameras

• Each SPAD frame is binary, while accumulated pixel measurements have binomial statistics and increasing signal-to-noise ratio as frames accrue.At large measurement counts, the binomial distribution is approximated by a Poisson distribution, making deterministic image operators less dependent on photon noise.Source: §1 · p. 2
• At very low photon budgets, many natural images can be similarly probable given one binary measurement, making reconstruction fundamentally one-to-many.The paper studies how this ambiguity changes as measurements accumulate.Source: §1 · p. 2
• Generative models can sample images associated with SPAD measurements, enabling empirical study of reconstruction uncertainty and downstream classifier or regressor performance.The framework examines how both measurement and image-space uncertainty change with measurement count.Source: §1 · p. 2
• The paper models frame thinning with hypergeometric Markov transitions and learns an inverse Markov process that preserves Binomial marginals.The reverse process is implemented as a variational-autoencoder-style conditional sampler.Source: §1 · p. 2
• The approach is evaluated on edge regression, optical character recognition, QR decoding, and facial reconstruction as photon budgets increase.The evaluations characterize how ambiguity resolves and reconstruction and inference accuracy improve.Source: §1 · p. 3
• The work aims to quantify the minimum photon budget for reliable reconstruction or inference and provide tools to measure reconstruction uncertainty.The authors state that the minimal budget required for reliable outcomes had remained uncharacterized.Source: §1 · p. 3

• Earlier diffusion models progressively transform structured images into noise and learn a reverse denoising process for image generation.Later work connected diffusion models to stochastic differential equations.Source: §2 · p. 3
• Prior diffusion work has explored Poisson and Binomial noise, while this paper uses a Binomial process tailored to SPAD data with principled forward and reverse distributions.Related approaches include Poisson thinning and photon aggregation, and Poisson-statistics SPECT dose reduction with an intractable reverse distribution.Source: §2 · p. 3
• Diffusion models have been conditioned for inverse problems with Poisson or Binomial noise, and related SPAD work has used Gaussian diffusion or deterministic reconstruction.The cited SPAD approaches include video reconstruction, adversarially fine-tuned one-step reconstruction, and score-based denoising.Source: §2 · p. 3
• Fig. 2 presents simulated MNIST examples with a maximum expected photon level of τ=0.01 per binary frame.Source: Figure 2 · p. 3
• The paper argues that prior approaches largely treat inverse problems as one-to-one, whereas its frame-thinning forward model and learned reverse process address one-to-many ambiguity.Source: §2 · p. 3

3.1 SPAD Measurement Model

• Each SPAD exposure is binary per pixel, and independent frames sum to Binomial counts as measurement quality improves with more frames.The measurement model assumes an ideal sensor with negligible dark current.Source: §3.1 · pp. 3–4

3.2 Photon Starvation Makes Restoration One-to-Many

• At small N, many images are similarly likely given measurements, while an L2-trained deterministic denoiser returns a conditional-mean average that blurs multimodal possibilities.A deterministic output cannot represent multiple plausible reconstructions for the same input.Source: §3.2 · p. 4
• Digit identity can resolve at N=20 or 50, while high-quality pixel-level reconstruction requires much larger counts (about N=1000).The conditional distribution narrows at different rates for different image features.Source: §3.2 · p. 4

3.3 Solution Overview

• The conditional generator models discarded binary frames as a Markov chain with hypergeometric transitions, learns reverse transitions that preserve Binomial marginals, and uses sample variation to assess image- and task-level stochasticity.The paper studies this transition across image reconstruction and downstream tasks.Source: §3.3 · p. 5

4.1 Markovian Frame-Thinning Forward Chain

• Uniformly retaining frames defines hypergeometric thinning transitions that preserve the Binomial marginal and form a sensor-defined Markov chain.The transition uses count images rather than the full binary-frame stack.Source: §4.1 · pp. 5–6

4.2 Conditional Structure of the True Reverse Chain

• The true reverse transition integrates a product of Binomial count distributions over uncertainty in the hidden photon-detection map.Accordingly, the learned reverse model represents uncertainty over the map while keeping the final count-generation step Binomial.Source: §4.2 · p. 6

4.3.1 The Reverse Kernel.

• The latent Binomial reverse kernel estimates remaining uncertainty in the photon-detection map, then samples a Binomial increment to add frames.It approximates the reverse predictive distribution, not necessarily the full posterior over the photon-detection map.Source: §4.3 · p. 6, §4.3.1 · p. 6

4.3.2 Why not a direct approach?

• A direct posterior model must span tasks from near-unconditional generation to deterministic reconstruction, whereas the local reverse model targets only the missing-frame increment.This lets it learn hidden-map information relevant to the next count transition rather than the full clean probability map.Source: §4.3.2 · p. 7

4.3.3 Variational Training with a Posterior Encoder.

• During training, a posterior encoder uses target increments to infer useful latent variables, while a KL term keeps its distribution close to the prior used at sampling time.The prior encoder conditions on the source SPAD image and frame counts; the posterior encoder also receives the increment image.Source: §4.3.3 · p. 7
• Hierarchical latents combine a global variable with spatial maps at decoder resolutions, allowing uncertainty to be represented at multiple spatial scales.Spatial latent parameters are predicted within the decoder and conditioned on the global latent and coarser spatial latents.Source: §4.3.3 · p. 7

4.3.4 Practical Pairwise Loss.

• Groupwise KL weights and free bits shape latent training, while gap normalization prevents large frame-gap transitions from dominating the minibatch loss.Gap normalization reweights the reconstruction term across transitions without changing the generative model.Source: §4.3.4 · p. 8

5.1 Experimental settings

• The synthetic dataset uses 28 × 28 two-tone straight edges, and training conditions on counts from 0 to 500 frames with a 10,000-frame high-count target.The simulated light level is τ = 0.01; anchor counts are 10,000, 500, and 20.Source: §5.1 · p. 9

5.2 Results

• As conditioning frames increase, synthetic edge samples become more consistent, with remaining uncertainty concentrated near the boundary and mainly reflecting precise edge location.At zero frames, the generator produces edges with different orientations and locations; even a few frames strongly constrain geometry in the shown examples.Source: Figure 6 · p. 9, §5.2 · p. 9
• Bias estimation stabilizes around 20 conditioning frames, while accurate orientation estimation requires roughly 100 frames, relative to the high-count target performance ceiling.Figure 8 plots edge-prediction performance against conditioning frame count; the ceiling is performance on 10,000-frame target SPAD images.Source: §5.2 · p. 9, Figure 8 · p. 10
• On the real printed edge capture, generated samples have quality comparable to the simulated results after hot-pixel masking and downsizing.The capture used a Pi Imaging 512 × 512 SPAD camera and was downsampled to 28 × 28 pixels.Source: §5.2 · pp. 9–10, Figure 7 · p. 10

6.1 Experimental settings

• The study trains a conditional generator on synthetic 28 × 28 MNIST digits and tests digit classification across conditioning-frame counts.The simulated measurements use a 10,000-frame target and conditioning counts from 0 to 500.Source: §6 · p. 10, §6.1 · p. 10, §6.2 · p. 10

6.2 Results

• Most generated samples agree on the target digit class by 20 frames, while residual variation remains around strokes and boundaries.At very small frame counts, one measurement can yield samples from different digit classes.Source: §6.2 · p. 10
• Classification accuracy exceeds 99% by 250 conditioning frames.Figure 10 reports average accuracy as a function of frame count.Source: §6.2 · p. 10, Figure 10 · p. 11
• Real SPAD captures show increasing agreement between generated digits and the printed target grid as measurements accumulate.The grid was rectified and partitioned into digit patches used as conditioning inputs.Source: §6.2 · pp. 10–11

7.1 Experimental settings

• The experiment uses distorted grayscale QR codes and compares single-step with multistep reverse generation using intermediate anchor frames.The dataset contains 12,799 training images and 3,200 validation/testing images.Source: §7 · p. 11, §7.1 · p. 11

7.2 Results

• At low frame counts, multistep QR generations vary substantially; increasing conditioning counts makes samples more consistent.Variation is concentrated around QR-code structures.Source: §7.2 · p. 11
• Generated QR samples reach stable decoding at roughly 100 frames, compared with approximately 5,000 frames for raw SPAD measurements.Both inputs are evaluated with the same off-the-shelf QR-code decoder.Source: §7.2 · p. 11

8.1 Experimental settings

• The study evaluates landmark and pose estimates from generated 64 × 64 grayscale FFHQ faces against high-count targets.The dataset has 60,000 training and 10,000 validation images; MediaPipe Face Landmarker is used for evaluation.Source: §8 · p. 11, §8.1 · p. 12, §8.2 · p. 13
• The face simulations use a higher light level and a 100,000-frame target because face images contain richer multiscale structure.Conditioning counts are sampled from 0 to 10,000.Source: §8.1 · p. 12

8.2 Results

• Landmark and pose errors decrease and their distributions tighten as conditioning-frame counts increase.Figure 14 reports median errors and the 25th–75th percentile range against high-count targets.Source: §8.2 · p. 13, Figure 14 · p. 13
• Generated samples have mostly stable landmarks and pose around 20 frames, while MediaPipe fails on raw SPAD measurements until 10,000 frames.Figure 15 visualizes landmarks on generated samples and raw measurements.Source: §8.2 · p. 13, Figure 15 · p. 13
• In real captures, global face shape and pose become consistent with few frames, while finer details require more; glasses are consistent from one measurement in the shown example.The sunglasses’ appearance becomes stable by three frames in that example.Source: §8.2 · p. 13

9 Discussions

• Across tasks, classifiers and regressors are typically near saturated performance after a transition around 20–50 frames.The paper suggests 4–6-bit sensor operation and a 20× to 50× time-resolution penalty, with viable inference in the 2–5 kHz range.Source: §9 · p. 13
• The conclusions are empirical and depend on the training dataset and adopted protocols; a theoretical counterpart could deepen understanding of the conditional posterior.The paper notes that such theory has remained elusive in other instances.Source: §9 · pp. 13–14
• Training the conditional generative model is compute-intensive, prompting reduced image resolution; harder natural-image problems would require more anchor frames and increase compute concerns.Source: §9 · p. 14
• The proposed approach can in principle handle high-flux measurements, but requires diffusion models capable of generating high-dynamic-range images.In high flux, multiple photon arrivals per exposure become non-negligible and measurements do not scale with arrival rate.Source: §9 · p. 14

A Frame-Thinning Markov Chain

• The thinning transition uses the kept count, population size N_t, successes x_t,i, and draw count N_t+1.The transition is introduced as the kernel in the frame-thinning chain.Source: A Frame-Thinning Markov Chain · p. 16
• The forward path is Markov by construction, and the Hypergeometric kernel preserves the Binomial marginal independently across pixels.The proof reduces marginal preservation to the scalar case before applying it over pixels.Source: A Frame-Thinning Markov Chain · p. 16

B Reverse Frame-Addition Kernel

• For reverse frame addition, missing frames are independent of retained frames conditional on incident flux; marginalizing its posterior yields the reverse kernel.The posterior is conditioned on the retained measurements and their frame count.Source: B Reverse Frame-Addition Kernel · p. 16

C ELBO Derivation

• The latent marginal likelihood is bounded using Jensen’s inequality.The derivation identifies the inequality step as Jensen’s inequality.Source: C ELBO Derivation · p. 16

D.1 Binomial logit loss

• The first term in the Binomial logit loss is independent of model parameters and is dropped from the practical loss.Source: §D.1 · p. 17

D.2 Reparameterization

• For a diagonal Gaussian posterior, the reparameterization factor of 1/2 converts log-variance to standard deviation.Source: §D.2 · p. 17

D.3 Gaussian KL

• Substitution into the Gaussian log densities gives equation (43).Source: §D.3 · p. 17

D.4 Hierarchical latent groups

• Hierarchical latent groups use one diagonal Gaussian per group, with groupwise KL terms summed over channels and spatial positions.Source: §D.4 · p. 17

D.5 Gap-normalized reconstruction loss

• The raw reconstruction term scales linearly with frame gap M_t; dividing by M_t removes that factor as training reweighting without changing the generative kernel.Source: §D.5 · p. 17

D.6 Single-step vs. multistep generation.

• The single-step model jumps directly from x_k to x_1, while the multistep model reverses the SPAD ladder through intermediate anchor frames.Source: §D.6 · p. 17
• Multistep generation achieves higher QR decoding accuracy than single-step generation in the stochastic regime, before performance saturates.Figure 17 reports decoding accuracy as a function of conditioning frame count.Source: Figure 17 · p. 17, §D.6 · p. 17
• For the same low-count measurements, multistep outputs are sharper and more QR-like, supporting intermediate anchors for smaller reverse transitions.Figure 18 shows representative generations from the two approaches.Source: §D.6 · p. 17, Figure 18 · p. 18