Thin On-Sensor Nanophotonic Array Cameras

Motivation

• Existing metasurface optics retain chromatic and geometric aberrations, while small apertures limit angular resolution and light collection.These limitations reduce image quality and signal-to-noise ratio compared with refractive lenses.Source: §1 · p. 2

Proposed approach

• A single flat optical layer on the sensor cover glass uses an array of nanophotonic lenses to recover a single megapixel image across the broadband spectrum.The layer is positioned at 2.5 mm focal distance from the sensor, while the metasurface design is 700 nm high.Source: §1 · p. 2
• The proposed differentiable design continuously optimizes over the visible spectrum and factorizes different optical fields into individual array lenses, extending the field of view to 100°.This addresses prior computational metasurface approaches limited to narrow wavelength bands, low resolutions, and 40° fields of view.Source: §1 · p. 2
• A learned probabilistic reconstruction method combines the physical forward model with a diffusion-model prior to recover images under scene-dependent broadband aberrations.The method is trained using paired real-world captures acquired under varying illumination conditions.Source: §1 · p. 2

Results

• Simulation and prototype experiments report favorable image quality across the visible spectrum and a large field of view compared with existing metasurface optics.The authors assess both the optical design and reconstruction method against alternative thin-camera and metasurface designs.Source: §1 · pp. 2–3

Limitations

• The camera still requires GPU processing with high power consumption, has low total light efficiency from spaced sublenses, and does not explicitly model fabrication inaccuracies.The prototype also does not use the full available optical aperture.Source: §1 · p. 3

Flat Computational Cameras

• Lensless cameras enable millimeter-scale thickness but distribute each scene point across the sensor, reducing light efficiency and making reconstruction ill-posed.Their globally supported point-spread functions can limit spatial resolution and require long acquisition times.Source: §2 · p. 3
• Single refractive or diffractive elements reduce optical-stack height but commonly require back focal distances exceeding 10 mm, preventing thin cameras.Computational correction addresses introduced aberrations but does not remove the long-distance constraint.Source: §2 · p. 3

Metasurface Optics

• Existing meta-optics offer ultrathin fabrication and flexible wavefront control, yet severe chromatic and geometric aberrations make broadband imaging outside the lab difficult.Dispersion-engineered designs reduce chromatic aberrations but are limited to apertures of tens of microns.Source: §2 · p. 3
• Tseng et al. [Tseng et al. 2021a] achieve full-color imaging with a 0.5 mm aperture, but their method performs poorly beyond a 40° field and was demonstrated on tri-chromatic OLED bands.The proposed array instead optimizes image quality over the full broadband spectrum across a 100° field without increasing back focal length.Source: §2 · p. 3

Lens as a Combination of Prisms

• The array represents a large lens using re-organized wedge prisms and short-focal-length lenses, combining field-of-view expansion with reduced back focal length.The figure frames the lens as a piecewise-linear combination of local prism-like elements.Source: Figure 2 · p. 4

Differentiable Optics Design

• The proposed hybrid design first optimizes a nanophotonic lens array with an inverse-filter proxy, then uses probabilistic deconvolution conditioned on measured signals to compensate residual aberrations.This separates optical design from the final full-color reconstruction while retaining computational compensation.Source: §2 · p. 4

3 NANOPHOTONIC ARRAY CAMERA

• A nanophotonic array camera uses an array of short-back-focal-length metalenses with designed phase profiles and learned image reconstruction to enable thin on-sensor, on-chip imaging.The optic is smaller than a penny and uses 700 nm-high silicon nitride nanoposts with a 350 nm pitch.Source: §3 · p. 4, Figure 3 · p. 4, §3.2 · p. 5

3.1 Lens as a Combination of Prisms

• Co-optimized lens and prism phase elements expand the field of view while reducing back focal length, while the lens phase corrects some prism-induced dispersion aberrations.The design trades focal length against aberrations to approximate a large lens with a nanophotonic array.Source: §3.1 · pp. 4–5

3.2 Radially Symmetric Nanophotonic Array

• The metasurface represents each lens phase as a radially symmetric per-pixel function, reducing local minima and allowing full-field simulation from one axis through symmetric point-spread functions.Each nano-antenna along a radius can vary independently, while radial symmetry reduces the computational burden.Source: §3.2 · p. 5
• Wedge phases with varying angles are added to array elements to widen the field of view while retaining an optimizable radially symmetric phase.The wedge phase is fixed for selected angles, whereas the radial phase remains learnable.Source: §3.2 · p. 5
• Differentiable polynomial mappings convert target phase to nano-antenna geometry at a nominal wavelength and map geometry back to phase across other wavelengths.RCWA data fit the inverse and forward proxy operators, enabling direct optimization of broadband metasurface phase.Source: §3.2 · p. 5
• The forward model computes wavelength- and field-angle-dependent PSFs for broadband image formation using a fast Fourier transform (FFT)-based bandlimited angular spectrum method (ASM) and models per-pixel Gaussian-Poisson sensor noise.For spatially varying array formation, PSFs are computed for each metalens element at field angle θ and wavelength λ using an M×N patch grid.Source: §3.2 · pp. 5–6

4 PROBABILISTIC IMAGE RECOVERY

• Image recovery alternates physics-based inverse filtering, diffusion-prior sampling, and merging of estimates from the array’s sub-apertures in an unrolled optimization framework.The method uses calibrated PSFs and is trained end-to-end with paired measurements from a reference camera.Source: §4 · pp. 6–7, Figure 4 · p. 7

4.1 Model-based Splitting Optimization for a Single Element

• The single-element problem is posed as maximum-a-posteriori deconvolution and split into simpler data-fidelity and prior subproblems using an auxiliary variable.The penalty weight is increased iteratively while alternating updates for the latent image and auxiliary variable.Source: §4.1 · p. 7

4.2 Diffusion-based Image Prior

• The diffusion prior samples multiple plausible natural images near the current iterate, addressing ambiguity when different clear images produce the same sensor measurement.The model is conditioned on the current iterate and optimization weights during probabilistic sampling.Source: §4.1 · p. 7, §4.2 · p. 8

4.3 Learned Array Deconvolution and Blending

• Each array sub-image is deconvolved using its measured PSF, then a modified UNet corrects, stitches, and blends the recovered views into a wider-field super-resolved image.Blending is supervised with reference-camera ground truth using pixel-wise and perceptual losses to preserve color and features under misalignment.Source: §4.3 · p. 8

4.4 Implementation

• Training uses 800 × 800 ground-truth images and nine 420 × 420 metalens sub-images, with diffusion sampling performed iteratively during reconstruction.The implementation normalizes images to [0, 1] and uses ADAM optimization.Source: §4.4 · pp. 8–9, §4.2 · p. 8

5.1 Synthetic Dataset

• Synthetic measurements model metalens vignetting, simulated PSFs, sensor noise, and homography-based projection into the final 3 × 3 sensor array.The vignetting cutoff corresponds to 45° of each metalens FoV, and the sensor noise parameters are calibrated as σ_g = 1 × 10^-5 and a_p = 4 × 10^-5.Source: §5.1 · p. 9
• The synthetic dataset contains 10,000 groundtruth images, with 8,000 for training and 1,000 each for validation and testing, followed by fine-tuning on in-the-wild real data.Each metalens has a 2 mm focal length and covers a 60° FoV under broadband illumination.Source: §5.1 · p. 9

5.2 Experimental Setup and Dataset

• Paired real-world data are captured simultaneously through a 70% transmission/30% reflection plate beam splitter using the metalens camera and a synchronized conventional reference camera.Checkerboard calibration, rectification, and homography alignment map reference-camera captures to the individual metalens views.Source: §5.2 · pp. 9–10, Figure 5 · p. 10
• The experimental setup measures RGB PSFs for every metalens with fiber-coupled LEDs and preserves the point-source position when switching colors.The setup uses a point source 340 mm from the array, giving approximately 8 arc-min angular resolution, and captures all element PSFs in one frame.Source: §5.2 · p. 10

5.3 Fabrication of the Meta-Optic

• The fabricated meta-optic is a 700 nm SiN array on fused silica with approximately 7 mm^2 total area, 1 mm elements, and F#2.4.The prototype omits optical baffles, spaces lenslets with approximately 15% of the wafer area used as apertures, and uses non-continuous pixels totaling approximately 40% of the sensor area.Source: §5.3 · p. 10

6.1 Synthetic Evaluation

• The proposed probabilistic reconstruction method outperforms all compared deconvolution methods in SSIM, PSNR, and LPIPS, with more than 1 dB PSNR improvement over learned baselines.Compared methods include Wiener filtering, Richardson-Lucy iterations, FLAT-Net, and Multi-Wiener-Net; the proposed method also recovers fine details without severe reconstruction noise.Source: §6.1 · p. 11, Table 2 · p. 11, Figure 6 · p. 11
• The proposed thin imager improves on FlatCam [2017] and DiffuserCam [2017] quantitatively and qualitatively, recovering fine details across almost the entire field of view.Only peripheral corners lack some image information, which the blending network fills in.Source: §6.1 · pp. 12–13, Table 3 · p. 12
• Replacing the non-learned Tikhonov prior with the diffusion prior improves average PSNR by 7.2 dB using the same input data.The Tikhonov-prior reconstruction averaged 25.5 dB and still exceeded alternative flat-camera designs by more than 4 dB.Source: §6.1 · p. 13

6.2 MTF and FoV Analysis

• The proposed metalens has improved MTF across angles of incidence, approaches the diffraction limit at normal incidence, and retains sufficiently high values at large angles for computational reconstruction.Its array spans 70° or 80° total field of view depending on array size, compared with 40° for Tseng et al. [2021a].Source: §6.2 · p. 13, Figure 8 · p. 13

6.3 Experimental Assessment

• In broadband indoor and outdoor experiments, the proposed camera reconstructs accurate colors and spatial details, significantly outperforming Tseng et al. [2021a] and showing no apparent chromatic aberrations.Peripheral regions retain detail across the design field of view, while the center captures relatively high-quality fine detail.Source: §6.3 · p. 14, Figure 10 · p. 14, Figure 11 · p. 15
• A denser 5 × 5 prototype demonstrates increasing field of view toward peripheral lenslets, although the available sensor cropped the full array capture.A red plush toy outside the center element’s field of view enters the middle element and moves toward the center in the left element.Source: §6.3 · pp. 14–15

• A differentiable lens-design method suppresses aberrations across the visible spectrum in a metasurface array mounted on the sensor cover glass.The array collaboratively captures a larger field of view than a single element while remaining a thin, single-element optical system.Source: §7 · p. 15
• The camera’s thin form factor depends on mounting the metasurface directly on the sensor cover glass, while commodity cover glasses may require specialized tools or sensor destruction for removal.Source: §7 · p. 15
• The proposed array camera is validated experimentally and in simulation, including both its optical design and deconvolution method.The evaluation compares the optical design against existing broadband metasurface optics and the reconstruction against alternative thin-camera designs.Source: §7 · p. 15
• Probabilistic reconstruction recovers broadband images in the presence of scene-dependent aberrations, although the proposed design is not aberration-free.Source: §7 · p. 15