PaperlayerView exact revision on arXiv

Source-linked AI summary

Recovery of Continuous 3D Refractive Index Maps from Discrete Intensity-Only Measurements using Neural Fields

Renhao Liu, Yu Sun, Jiabei Zhu, Lei Tian, Ulugbek Kamilov

arXiv:2112.00002v2eess.IVcs.CV

TL;DR

IDT reconstruction is challenged by missing phase and limited-angle measurements, which make the inverse problem severely ill-posed. DeCAF fits a continuous neural field directly to each sample’s intensity measurements and reports up to 2.1× MSE reduction over existing methods.

  • Problem

    Missing phase and limited-angle measurements make IDT reconstruction a severely ill-posed inverse problem.

    Source: Main Text · pp. 2–3

  • Method

    DeCAF trains an MLP at test time using the sample’s intensity measurements and the IDT forward model, without ground-truth RI maps.

    Source: Main Text · p. 3

  • Results

    Up to 2.1× MSE reduction is reported, alongside mitigation of missing-cone artifacts while maintaining fine details.

    Source: Conclusion · p. 13

  • Takeaways & Limitations

    DeCAF can render arbitrary-density voxel grids without retraining, a capability described as useful for computational and memory bottlenecks in image reconstruction and analysis.

    Source: Conclusion · p. 13

    The current linear IDT forward model based on the first Born approximation limits DeCAF to relatively thin, weakly scattering samples.

    Source: Discussion · p. 12

Abstract

from arXiv · show

Intensity diffraction tomography (IDT) refers to a class of optical microscopy techniques for imaging the 3D refractive index (RI) distribution of a sample from a set of 2D intensity-only measurements. The reconstruction of artifact-free RI maps is a fundamental challenge in IDT due to the loss of phase information and the missing cone problem. Neural fields (NF) has recently emerged as a new deep learning (DL) approach for learning continuous representations of physical fields. NF uses a coordinate-based neural network to represent the field by mapping the spatial coordinates to the corresponding physical quantities, in our case the complex-valued refractive index values. We present DeCAF as the first NF-based IDT method that can learn a high-quality continuous representation of a RI volume from its intensity-only and limited-angle measurements. The representation in DeCAF is learned directly from the measurements of the test sample by using the IDT forward model, without any ground-truth RI maps. We qualitatively and quantitatively evaluate DeCAF on the simulated and experimental biological samples. Our results show that DeCAF can generate high-contrast and artifact-free RI maps and lead to up to 2.1 times reduction in MSE over existing methods.

Main Text

This section presents DeCAF for reconstructing continuous 3D refractive-index maps from intensity-only measurements and evaluates it on biological samples. It also describes how the continuous representation supports rendering at arbitrary voxel-grid densities.

Main Text

  • A coordinate-based MLP learns complex RI values directly from each test sample’s intensities through the IDT forward model, with radial encoding and optional anisotropic 3D regularization.The regularizer combines a pre-trained natural-image denoising CNN in x-y with one-dimensional total variation along z; DeCAF also keeps storage at 3 MB across grid densities for C. elegans, while Tikhonov storage grows with grid density.Source: Main Text · pp. 3–4, Experimental Validation · p. 11

Results

  • Across four biological samples and dense, annular, and multiplexed IDT, DeCAF resolves axial structure and reduces artifacts relative to Tikhonov and, for spirogyra, SIMBA.Reported examples include clearer separation of overlapping buccal cells and recovery of C. elegans tissues and fine features.Source: Experimental Validation · pp. 5, 7, 8, 9, 10
  • DeCAF renders continuous RI volumes on arbitrary grids without retraining, resolving small features in 26.7× x-y upsampling and producing smooth axial interpolations between training slices.The axial interpolation preserves biological-feature consistency across layers, including the appearance and disappearance of bacteria clusters.Source: Experimental Validation · p. 9
  • DeCAF improves phantom PSNR by 1.6 dB over SIMBA and 3.3 dB over Tikhonov, equivalent to 1.5× and 2.1× reductions in MSE, respectively.The comparison reports more accurate RI values, fewer missing-cone artifacts, and reduced cell shadows from axial elongation.Source: Figure 6 · p. 11, Quantitative Evaluation · p. 12

Discussion

  • Compared with SIMBA, DeCAF adapts MLP weights at test time, decouples the RI representation from a voxel grid, and combines implicit MLP and external CNN regularization.The paper reports that the best results use both regularizers, while MLP alone also provides substantial regularization.Source: Discussion · p. 12
  • DeCAF’s current implementation uses linear IDT forward models based on the first Born approximation, limiting applicability to relatively thin, weakly scattering samples.The limitation is noted in reconstructions of a relatively thick C. elegans sample.Source: Discussion · p. 12
  • DeCAF currently takes approximately 20 hours per real sample, versus several minutes for Tikhonov and hours for SIMBA; manual hyperparameter tuning may add runtime.Source: Discussion · p. 13
  • Future work will explore faster DeCAF implementations, with Instant Neural Graphics Primitives suggesting a potential order-of-magnitude acceleration.Source: Discussion · p. 13

Conclusion

  • Up to 2.1× MSE reduction was reported; DeCAF also mitigated missing-cone artifacts while maintaining fine details of small biological features.Validation used experimentally collected datasets of multiple biological samples under three different IDT setups.Source: Conclusion · p. 13
  • DeCAF’s continuous representation generates images at arbitrary voxel-grid densities without retraining the network, helping address computational and memory bottlenecks.Source: Conclusion · p. 13

Methods

DeCAF represents complex permittivity contrast with a coordinate-based neural field and fits it to intensity measurements using the IDT model. Its framework combines radial x-y encoding, regularization, and measurements collected across several IDT configurations and biological samples.

IDT Experiments

  • Experiments use dense, annular, and multiplexed IDT systems, with 89 brightfield LEDs, 24 ring LEDs, and 16 multiplexed measurements respectively.The multiplexed measurements each combine six LED lights; the setups use different microscope objectives and illumination wavelengths.Source: IDT Experiments · pp. 13–14

Sample and data preparation

  • The biological sample set includes algae, buccal epithelial cells, and C. elegans, with reconstruction volumes discretized at sample-specific spatial resolutions.The C. elegans reconstructions use two relatively steady frames selected from a 4-frames-per-second video of a moving sample.Source: Sample and data preparation · p. 14

DeCAF Framework

  • DeCAF maps 3D coordinates to real and imaginary permittivity contrast with a coordinate-based MLP, yielding a continuous representation of the refractive-index volume.The unknown permittivity contrast is related to the RI distribution; the network’s coordinate grid is normalized to [−1, 1]^3.Source: DeCAF Framework · p. 15
  • Radial encoding expands x-y coordinates to represent in-plane dependencies, while z uses positional encoding; experiments found radial encoding improved representation of small textures.The paper motivates separate encodings by the imaging system’s non-isotropic resolution across x-y and z.Source: DeCAF Framework · pp. 15–16
  • The loss combines measurement consistency with x-y noise reduction and z-dimension continuity regularizers, with α and β balancing their contributions.The x-y regularizer uses an image denoiser, and the parameters are tuned for each sample.Source: DeCAF Framework · p. 16

Data availability

Contributions

The authors divide responsibility across conception, implementation, experiments, data preparation, drafting, and review.

Contributions

  • Y.S., R.L., and U.K. conceived the project; R.L. and Y.S. implemented the model and designed the experiments.R.L. collected the numerical results, while J.Z. and L.T. handled data acquisition and preparation.Source: Contributions · p. 17
  • Y.S. primarily drafted the manuscript with assistance from R.L. and J.Z., and U.K. and L.T. revised it.All authors reviewed the manuscript.Source: Contributions · p. 17
  • R.L. collected the numerical results, while J.Z. and L.T. conducted data acquisition and preparation.Source: Contributions · p. 17

IDT forward model

The forward model expresses intensity measurements as mappings from slice-wise permittivity contrast, using transfer functions to characterize the mapping. For multiplexed illumination, the model combines measurements from individual sub-illuminations.

IDT forward model

  • The model discretizes a 3D sample into axial slices and uses an individual measurement matrix for each slice to map permittivity contrast linearly to intensity measurements.It adopts the first Born approximation to characterize scattering events throughout the sample volume.Source: IDT forward model · p. 29
  • Each measurement matrix composes 2D Fourier transforms with a transfer function, which fully characterizes the permittivity-intensity mapping.The measurement vector contains background-removed intensity, and the model includes noise.Source: IDT forward model · p. 29
  • For multipixeled illumination, the model treats the measurement as the sum of individual sub-illumination measurements and regroups their transfer functions.W denotes the total number of sub-illuminations.Source: IDT forward model · p. 30

Block-wise Training of DeCAF

DeCAF trains on coordinate blocks to limit GPU memory use while optimizing its IDT forward model. Its procedure uses block-specific predictions and measurement handling to update the neural field.

Block-wise Training of DeCAF

  • Block-wise training addresses GPU memory limits because the IDT forward-model computation scales with the total number of input coordinates.The MLP itself has a small memory footprint; the forward-model optimization is the stated bottleneck.Source: Block-wise Training of DeCAF · p. 30
  • At each step, DeCAF randomly selects a spatial block, predicts its RI volume, evaluates measurement mismatch and regularization, then updates the MLP weights.The 3D grid is divided horizontally into blocks.Source: Block-wise Training of DeCAF · p. 30
  • Padding, view enlargement, and measurement separation handle Fourier-model boundary effects, provide a sufficient measurement field of view, and reduce cross-block interference.Padding makes neighboring coordinate blocks overlap; measurement separation crops the corresponding measurement block.Source: Block-wise Training of DeCAF · p. 31
  • A dictionary tracks each block’s latest partial measurement so the procedure can reuse intermediate results; training requires only the test sample’s measurements.The block-wise procedure is implemented with customized Adam, though the optimizer can be replaced.Source: Block-wise Training of DeCAF · p. 31

Radial Encoding

Radial encoding combines features of Fourier-based positional encoding and random Gaussian encoding. Its rotation mapping gives the MLP flexibility across feature orientations while avoiding the overly smooth or noisy reconstructions associated with those alternatives.

Radial Encoding

  • Radial encoding reconciles classic Fourier-series positional encoding with Gaussian encoding's random mapping.The proposed encoding introduces rotation mappings into the positional formulation.Source: Radial Encoding · p. 32
  • The rotation mapping lets the MLP respond to features at different orientations without a completely random mapping.This design adds structural flexibility to positional encoding.Source: Radial Encoding · p. 32
  • Radial encoding improves fine-feature recovery while avoiding the oversmoothing of positional encoding and noise patterns of Gaussian encoding.The authors report visual and numerical improvement over both alternatives.Source: Radial Encoding · p. 32

Additional Technical Details

The technical details describe DnCNN's architecture and training, along with DeCAF's computational setup and measurement-error tracking during training.

Additional Technical Details

  • DnCNN uses ten 3 × 3 convolutional layers; the first nine use ReLU, and the last has no activation.Before integration into DeCAF, it is trained to predict the noise residual.Source: Additional Technical Details · p. 32
  • DnCNN training adds AWGN to BSD500 natural images and tests five noise levels, σ ∈ {1, 2, 3, 4, 5}, selecting the visually best model per sample.The noisy input and ground-truth noise are the training targets described in the loss setup.Source: Additional Technical Details · pp. 32–33
  • DeCAF training takes approximately 20 hours per sample, uses two GPUs, and applies an exponentially decreasing learning rate.Training progress is tracked by MAE between predicted and real test measurements; the predicted measurements come from the full predicted RI volume.Source: Additional Technical Details · p. 33

Ablation Experiments

The ablations examine encoding choices and implicit and explicit regularization. Radial encoding performs better than positional and Gaussian alternatives, while the regularization studies show that explicit terms are especially useful for strongly scattering samples.

Comparison of different x-y encodings on experimentally collected samples

  • On experimental C. elegans, Radial yields sharper body-wall and anterior-pharyngeal-bulb features than Positional or Gaussian, though Positional makes the grinder brighter.The comparison keeps the rest of DeCAF fixed and uses positional encoding for z.Source: Ablation Experiments · p. 33, Figure 4 · p. 39

Quantitative evaluation of different x-y encodings

  • On the simulated Yanny phantom, Radial achieves better PSNR and MSE than Positional and Gaussian; the authors attribute this to rotation mappings' added structural flexibility.The associated visual comparison shows Radial avoiding the overly smooth or noisy reconstructions of the alternatives.Source: Ablation Experiments · pp. 33–34, Figure 5 · p. 40, Table 1 · p. 43

Demonstration of implicit regularization on experimentally collected samples

  • On Spirogyra algae, DeCAF-Noreg visually outperforms SIMBA and Tikhonov and nearly matches full DeCAF, though full DeCAF better alleviates dark-shade artifacts.This comparison demonstrates implicit regularization from the MLP even without explicit regularization.Source: Ablation Experiments · p. 34, Figure 6 · p. 41

Demonstration of explicit regularization on experimentally collected samples

  • For the strongly scattering C. elegans body, noise reduction improves DeCAF-Noreg, axial continuity improves axial and lateral results but causes cross-slice interference, and combining both gives the best reconstruction.Without external regularization, DeCAF-Noreg fails to reconstruct the worm; the comparison holds the other parameters fixed.Source: Ablation Experiments · p. 34, Figure 7 · p. 42

Quantitative evaluation of different regularizers

  • 23.69 dB PSNR for DeCAF-Noreg exceeds SIMBA by 1.3 dB and Tikhonov by 2.97 dB; full explicit regularization gives the best results, and omitting either term degrades quality by at least 0.16 dB.Supplementary Table 2 reports PSNR and MSE for the baselines and DeCAF variants.Source: Ablation Experiments · p. 34, Table 2 · p. 44
Loading 2112.00002v2…