PaperlayerView exact revision on arXiv

Source-linked AI summary

Inductively Scalable, Single-Step Neural Surrogates for Wave-Scattering Inverse Problems

Charles Dove, Laura Waller

arXiv:2608.17344v1physics.opticscs.AI

TL;DR

Single-step neural surrogates for wave scattering have struggled to scale beyond a few tens of variables. This paper dynamically generates hard training examples to train a robust, inductively scalable surrogate for dense two-dimensional scattering, demonstrating a practical path to fast photonic inverse design.

  • Problem

    Existing training methods for single-step neural surrogates scale poorly, limiting their use for large wave-scattering problems.

  • Method

    The paper trains surrogates with dynamically generated hard examples that target current failure modes, alongside normalization and replay-based stabilization.

  • Results

    The trained surrogate robustly handles dense, freeform two-dimensional wave-scattering problems, while adversarial examples often exhibit structured physical properties.

  • Takeaways & Limitations

    The approach provides a practical route toward fast, robust, inductively scalable neural simulators for photonic inverse design.

  • Takeaways & Limitations

    The method does not yet address full vectorial three-dimensional scattering, broadband response, material dispersion, fabrication constraints, or arbitrary boundary conditions.

Abstract

from arXiv · show

Neural network surrogates are an emerging alternative to traditional electromagnetic wave simulators like finite-difference time-domain (FDTD); their goal is to replace rigorous physical simulations with pre-trained neural networks that solve wave-scattering forward and inverse problems orders of magnitude faster. However, nonrecurrent, single-step surrogates have scaled only to a few tens of simulation variables. Here, we show that this barrier can be overcome by dynamically generating salient training examples during training, rather than randomly sampling the large space of possible examples. We introduce an algorithm that runs in parallel with surrogate training, using gradient ascent to search refractive-index and source configurations for cases where the surrogate disagrees with a full-wave ground-truth simulator. We also use source and ground-truth normalization with an evolving replay dataset to stabilize and accelerate learning. Using this approach, we train a fast, single-step surrogate for two-dimensional wave scattering with up to 41,772 controllable variables, including dense, freely configurable grids of refractive indices and complex-valued sources. The resulting neural surrogate is robustly accurate across diverse structured and unstructured examples and generalizes inductively to larger domains, reaching over 3 million controllable variables without retraining, a $73.8\times$ increase. We demonstrate the surrogate on large-scale forward simulations and inverse design of freeform beam splitters and gradient-index (GRIN) lenses up to 98 wavelengths wide, showing comparable or better performance than FDTD-based designs, with speedups from $1.29\times$ to $26.5\times$. These results demonstrate a practical path toward fast, robustly accurate, inductively scalable neural simulators for photonic inverse design and other wave-scattering inverse problems.

Abbreviations

This section defines abbreviations for simulation methods, optical structures, neural networks, normalization, boundary conditions, error metrics, and polarization.

  • FDTD means finite-difference time-domain, and GRIN means gradient-index.
  • NN means neural network, while PML means perfectly matched layer.
  • RMS means root mean square, and TE means transverse electric.

Introduction

Electromagnetic inverse design repeatedly requires expensive, differentiable full-wave simulations, while existing single-step neural surrogates struggle to scale because they need enormous training datasets. The paper proposes dynamically targeting surrogate failure modes, enabling robust accuracy and inductive scaling to much larger domains with substantial optimization-time speedups.

  • Motivation: FDTD remains the gold standard for realistic wave-scattering simulations, but its memory and runtime costs often make iterative inverse design infeasible.Inverse problems may require forward simulations thousands to millions of times.
  • Limitations: Iterative neural surrogates require tens or hundreds of sequential inferences per simulation, with recurrence scaling with spatial size.Domain decomposition over a domain w tiles wide requires at least w sequential inference rounds, with high-index-contrast resonances potentially requiring more.
  • Limitations: Single-step surrogates avoid recurrence and can be faster and more memory-efficient, but scaling them beyond a few tens of variables is limited by training-data requirements.Prior work identifies dataset size as a central bottleneck for robust surrogate training.
  • Limitations: 109 ground-truth-labeled examples would likely be required for a 1000-variable surrogate under conventional training.At 0.1 seconds per FDTD simulation, generating this dataset would take approximately 3.17 years of serial compute time and require about 393 TB of storage.
  • Contribution: The proposed training strategy dynamically reshapes the dataset around current surrogate weaknesses, producing increasingly informative examples and improving training efficiency.This targets failure modes rather than relying on a fixed dataset or random sampling, supporting robust accuracy where random sampling would require impractically many labeled simulations.
  • Results: 64× larger trained-domain scale and 1.29× to 26.5× optimization-time speedups demonstrate inductive scaling and practical acceleration on inverse-design workloads.The method provides robust accuracy across challenging examples and comparable or better FDTD-validated performance in freeform inverse problems.

Results and Discussion · Problem scale and surrogate form

The study evaluates a single-step surrogate against challenging, structured wave-scattering cases and tests whether its fully convolutional design supports inductive scaling from training-sized domains to much larger grids. The experiments cover hard-example efficiency, robustness beyond the random distribution, scalability, and FDTD-validated inverse-design performance.

  • Results and Discussion: Pre-defined datasets can leave many wave-scattering examples on which the surrogate is severely inaccurate.The challenging cases include smooth variation, high-contrast resonance, long-range structure, and complex structured optical interactions.
  • Results and Discussion: Adversarial failures often involve structured examples such as ring resonators and ordered high-contrast refractive-index patterns.These cases expose complex, structured optical interactions that challenge a surrogate trained on randomly selected or hand-curated data.
  • Results and Discussion: The evaluation measures dynamic hard-example generation against random sampling, out-of-distribution robustness, inductive scalability, and FDTD-validated inverse-design performance.The surrogate is also intended to scale to larger simulation regions because training larger regions is limited by computation, training time, and memory.
  • Problem scale and surrogate form: Each two-dimensional TE scattering example uses dense, freely configurable complex source and refractive-index fields, with the steady-state out-of-plane electric field as output.Nonperiodic boundaries use a perfectly matched layer, while interior refractive indices are constrained to n ∈[1, 2] unless otherwise stated.
  • Problem scale and surrogate form: 3,084,588 controllable variables are evaluated on a 1024 × 1024 domain using the same trained model without retraining.The count excludes the fixed PML boundary regions.
  • Problem scale and surrogate form: The surrogate is a fully convolutional U-Net mapping real and imaginary source channels plus refractive index to real and imaginary field channels.Because no architectural component requires a fixed input size, the trained network can be evaluated on grids larger than those used during training.

Normalization for resonant examples

Resonant scattering instances can have much larger field energies than typical examples, causing raw field MSE to let a few high-energy cases dominate training and hard-example generation. The method instead normalizes each example by the RMS amplitude of its ground-truth field, yielding a relative complex-field error that makes low- and high-energy examples comparable.

  • Near-resonant configurations produce fields much larger than typical scattering examples, creating substantial variation in field energy.
  • Raw field MSE lets a small number of high-energy examples dominate both surrogate updates and hard-example search.
  • The method normalizes each example by the RMS amplitude of its ground-truth field before comparing predictions.
  • The training and generator objective is the ground-truth-RMS-normalized complex-field error.
  • This loss measures relative complex-field error and places low- and high-energy examples on a comparable scale.

Efficient training data optimization by dynamic hard-example generation

The method dynamically generates physically valid hard examples where the surrogate disagrees most with FDTD, then trains on a replay buffer of these failures. This coupled generator–trainer improves validation fidelity and continues exposing increasingly complex failure modes as training progresses.

  • Dynamic hard-example generation: The hard-example generator searches physically valid sources and refractive-index fields for configurations maximizing the surrogate–FDTD electric-field mismatch.Projected ascent respects index bounds, PML constraints, and a unit-energy source constraint.
  • Replay-buffer stabilization: A fixed-size replay buffer mixes newly generated failures with earlier examples, stabilizing training by reducing temporal correlation and distribution drift.The oldest examples are discarded when the buffer fills, while random sampling supplies training batches.
  • Replay-buffer stabilization: Without replay, immediate training on newly generated examples typically fails to converge because the adversarial generator–trainer coupling creates an unstable feedback loop.Rapidly changing, strongly correlated examples provide an insufficiently broad and stable learning signal.
  • Training configuration: The training run appended 8.28 × 10^6 teacher-labeled examples while retaining 100,000 examples in the rolling dynamic dataset.Training alternated between 64 × 64 and 128 × 128 grids, with generator and trainer batch sizes of 50.
  • Empirical results: Dynamic generation produced lower validation error than random sampling on challenging structured and unstructured examples, yielding greater physical fidelity.The comparison is reported for the validation examples in Fig. 2 and simulations in Fig. 3.
  • Empirical results: Later hard examples are more structurally complex while showing less surrogate–FDTD mismatch, indicating that the dynamic dataset continues exposing new failure modes as accuracy improves.This progression avoids reliance on a fixed hand-designed training distribution.

Robust accuracy on structured and unstructured inverse problems

The dynamically trained surrogate was evaluated on curated structured and unstructured examples designed to expose failures missed by independent random sampling. It recovered electric-field amplitude and phase across the suite, while substantially reducing structured errors relative to random sampling.

  • Robust accuracy on structured and unstructured inverse problems: The manually curated suite probes failure modes underrepresented by independent random sampling, spanning diverse structured and unstructured refractive-index and source configurations.Examples include localized and multi-source excitation, waveguide-like and ring-like structures, graded-index media, gratings, scatterers, random media, and image-derived distributions.
  • Robust accuracy on structured and unstructured inverse problems: The dynamically trained model recovers both amplitude and phase of the electric field across all tested examples.This coverage includes the suite’s structured and unstructured configurations.
  • Robust accuracy on structured and unstructured inverse problems: The largest visible errors occur near high-contrast interfaces, low-amplitude phase regions, and fine interference features.These localized errors identify the principal residual failure modes in the dynamically trained model.
  • Robust accuracy on structured and unstructured inverse problems: Compared with the typical random-sampling approach, dynamic sampling avoids large structured errors across the tested examples.The results demonstrate major improvements from dynamic sampling relative to random sampling.

Inductive scaling to larger spatial domains

A fully convolutional, multiscale surrogate exploits locality and translation-equivariant scattering relationships to inductively generalize across spatial scales. Trained on modest domains, it was evaluated without retraining on 1024×1024 grids, reaching 3,084,588 controllable variables, while accuracy changed only slightly with scale.

  • Inductive-scaling mechanism: Locality, translation equivariance, and convolutional multiscale processing provide the architectural and physical basis for reusing learned scattering relationships on larger domains.These properties support inductive scaling but do not guarantee accuracy for unrepresented propagation distances, resonances, or boundary interactions.
  • Scaling results: 3,084,588 controllable variables were evaluated without retraining after training on 64 × 64 and 128 × 128 domains.Multiple training domain sizes were used to encourage generalization across scales.
  • Scaling results: 73.8× increase in controllable variables was achieved relative to the largest training problem, with a 64× increase in total grid area.The largest evaluation used a 1024×1024 grid and corresponded to an approximately 98.1λ non-PML interior width.
  • Accuracy and limits: Very small increase with scale characterized the MSE values across evaluated domains, which remained consistent across scales.The largest tested scale was stopped by memory constraints rather than an observed accuracy failure.

Inverse design of freeform GRIN lenses

Surrogate-based optimization produced FDTD-validated GRIN lenses at both tested scales, with comparable or better performance and reduced optimization time than direct FDTD optimization.

  • Overall result: The surrogate transferred successfully to FDTD ground truth at both tested scales, producing high-performing GRIN designs while reducing wall-clock optimization time.The designs used approximately 48.5λ and 98.0λ widths, with the latter focusing to a periodic array of focal spots.
  • 48.5λ focusing lens: 1669 validated focal intensity was achieved by the NN-optimized 48.5λ focusing design, versus 1839 for FDTD optimization.The corresponding focal-plane FWHM values were 0.434λ and 0.423λ, respectively.
  • 48.5λ focusing lens: 92.2s was required for NN optimization of the 48.5λ focusing design, versus 474s for direct FDTD optimization.Both final designs were validated with the FDTD ground truth.
  • 98.0λ periodic-focus lens: 1221 mean target intensity was achieved across 13 focal points for the 98.0λ periodic-focus design, versus 1147 for FDTD optimization.The target-to-target coefficients of variation were 0.222 and 0.246, respectively, indicating slightly more uniform target intensities for the surrogate-optimized design.
  • 98.0λ periodic-focus lens: 372s was required for surrogate-based optimization of the 98.0λ periodic-focus design, versus 479s for FDTD, a 1.29× speedup.The surrogate-optimized structure achieved slightly better validated performance than the FDTD-optimized design.

Inverse design of a waveguide beam splitter

The surrogate-optimized beam splitter achieved a comparable power split with higher total guided-mode output and lower leakage than direct FDTD optimization. Its optimization loop was 26.5× faster while producing FDTD-validated designs.

  • Inverse design of a waveguide beam splitter: 51.2/48.8% split and +0.213 dB top-over-bottom balance were achieved by the surrogate design, versus 49.9/50.1% and −0.018 dB for direct FDTD optimization.The surrogate produced top and bottom guided-mode powers of 5.41 and 5.15; direct FDTD produced 5.11 and 5.13.
  • Inverse design of a waveguide beam splitter: 26.5× optimization-loop speedup was achieved, requiring 8.97s for surrogate optimization versus 238.0s for the FDTD baseline.These times exclude final FDTD validation and evaluation.
  • Inverse design of a waveguide beam splitter: The surrogate provided useful gradients for photonic inverse-design objectives, yielding FDTD-validated designs with comparable or better total guided output power.The task routes guided-mode-coupled power into top and bottom output waveguides and evaluates modal overlap with their forward-propagating fundamental modes.

Limitations and scope

The surrogate demonstrates robust accuracy and inverse-design utility within a bounded, single-wavelength, two-dimensional transverse-electric setting, but important physical and operating-regime limitations remain. Its training strategy improves difficult-case coverage and enables scalable surrogate modeling, while future work targets broader physics, larger compute, and verifiable error bounds.

  • Current scope: The demonstrated setting is limited to single-wavelength, two-dimensional transverse-electric scattering with bounded refractive-index contrast.The model does not yet address full vectorial three-dimensional scattering, broadband response, material dispersion, fabrication constraints, or arbitrary boundary conditions.
  • Generalization limits: Inductive scaling to larger domains does not ensure accuracy across substantially different refractive-index ranges, wavelength-relative grid resolutions, boundary conventions, or source parameterizations without additional training.The model generalizes spatially beyond its training domains, but not necessarily across these changed conditions.
  • Training guarantees: Dynamic hard-example generation improves empirical coverage of difficult cases but provides no formal worst-case error bound.The method shifts data collection toward the surrogate’s current weaknesses rather than exhaustive random coverage.
  • Contribution: Normalization, replay buffering, and a fully convolutional architecture produce a reusable single-step surrogate that scales inductively and supplies gradients for FDTD-validated inverse designs.The study reports these capabilities as overcoming major prior limitations in surrogate scaling, speed, and adoption.
  • Future directions: Future work could distribute training across hundreds of GPUs and extend the strategy to broadband, vectorial, three-dimensional electromagnetics with formal error bounds.The authors also anticipate much larger spatial scales and applications to other physics domains.

Methods

The methods combine online gradient-based hard-example generation with replay-buffer training and a fully convolutional surrogate architecture. The same checkpoint is evaluated on larger grids without retraining, while inverse-design paths are validated against FDTD.

  • Surrogate architecture: The surrogate is a fully convolutional U-Net-style encoder–decoder that maps Re(s), Im(s), and n to Re(Êz) and Im(Êz).The reported field-prediction network contains 253,658,118 trainable parameters and uses no dense layers tied to a particular grid size.
  • Optimization and baseline: Training minimizes teacher-RMS-normalized complex-field loss with Adam while generator and trainer run in parallel on separate GPUs.The trainer uses learning rate 10−4, batch size 50, and 200,000 optimizer steps; the random baseline retains the architecture and loss but samples the base random distribution.
  • Evaluation and inverse design: Inductive scaling evaluates one trained checkpoint across grid sizes at fixed physical discretization, applying the fully convolutional surrogate directly without retraining.Errors are reported as teacher-RMS-normalized MSE and relative L2 error, with an independently computed teacher field at each scale.

Conclusions

The study demonstrates a single-step neural surrogate for dense, freeform two-dimensional wave scattering with thousands to millions of configurable variables. Dynamic hard-example generation, normalization, and replay buffering produce robust predictions, inductive scaling, and useful inverse-design gradients with faster optimization than direct FDTD.

  • Scale and data generation: The surrogate addresses large-scale wave scattering with thousands to millions of configurable source and material variables.The conclusions identify data generation as the main obstacle because random sampling wastes teacher simulations on easy examples and leaves systematic failure modes.
  • Training method: Gradient ascent on surrogate–teacher mismatch dynamically generates hard examples, while teacher-RMS normalization, source normalization, and replay buffering stabilize training.This approach trains a surrogate robust across a diverse set of scattering configurations.
  • Model capabilities: The resulting model accurately predicts complex FDTD fields for challenging structured and unstructured examples, generalizes to larger domains, and provides useful inverse-design gradients.These capabilities support freeform GRIN-lens and waveguide beam-splitter design tasks.
  • Inverse-design results: 5.14× optimization-loop reduction was achieved for the 48.5λ single-focus GRIN lens, 1.29× for the 98.0λ periodic-focus GRIN lens, and 26.5× for the compact beam splitter.Neural-surrogate optimization produced FDTD-validated devices with performance comparable to direct FDTD optimization.
  • Future direction: The results suggest a practical route toward a wave-scattering foundation model combining faster inference, robust accuracy, and inductive scaling for large-scale photonic inverse design.The proposed direction targets scales useful for problems of current interest in photonic inverse design.
Loading 2608.17344v1…