PaperlayerView exact revision on arXiv

Source-linked AI summary

Seismic wave propagation and inversion with Neural Operators

Yan Yang, Angela F. Gao, Jorge C. Castellanos, Zachary E. Ross, Kamyar Azizzadenesheli, Robert W. Clayton

arXiv:2108.05421v2physics.geo-phcs.LG

TL;DR

Seismic wave simulations are computationally costly, and changing a source or velocity structure requires another simulation. This paper trains a Neural Operator on random 2D acoustic simulations to produce wavefields and compute tomography gradients; one 20-source iteration took ∼1 second versus ∼100 seconds with spectral-element adjoint methods.

  • Problem

    Seismic wave simulations are costly, and perturbing the source or velocity structure requires a new simulation, increasing the computational burden.

    Source: §1 · pp. 1–2

  • Method

    The paper trains a Fourier Neural Operator on random velocity models and source locations to predict 2D acoustic wavefields, with reverse-mode automatic differentiation used to compute gradients for tomography.

    Source: §3 · p. 3, Full waveform inversion with Neural… · p. 7

  • Results

    After a few thousand training simulations, new solutions required negligible time; one 20-source iteration took ∼1 second versus ∼100 seconds with spectral-element adjoint methods.

    Source: §5 · p. 9, Full waveform inversion with Neural… · p. 9

  • Takeaways & Limitations

    A single FNO model trained on random velocity models can produce solutions for arbitrary models, potentially avoiding retraining for each region of a similar size.

    Source: §5 · p. 9

    The solutions are approximate, and traditional numerical methods may be preferable when the error is unacceptable for an application.

    Source: §5 · p. 9

Abstract

from arXiv · show

Seismic wave propagation forms the basis for most aspects of seismological research, yet solving the wave equation is a major computational burden that inhibits the progress of research. This is exacerbated by the fact that new simulations must be performed when the velocity structure or source location is perturbed. Here, we explore a prototype framework for learning general solutions using a recently developed machine learning paradigm called Neural Operator. A trained Neural Operator can compute a solution in negligible time for any velocity structure or source location. We develop a scheme to train Neural Operators on an ensemble of simulations performed with random velocity models and source locations. As Neural Operators are grid-free, it is possible to evaluate solutions on higher resolution velocity models than trained on, providing additional computational efficiency. We illustrate the method with the 2D acoustic wave equation and demonstrate the method's applicability to seismic tomography, using reverse mode automatic differentiation to compute gradients of the wavefield with respect to the velocity structure. The developed procedure is nearly an order of magnitude faster than using conventional numerical methods for full waveform inversion.

1 Introduction

Seismic wave simulations are computationally costly and often limit the pace of geophysical research, especially when source or velocity conditions change. The section introduces Neural Operators as a function-space approach and explores them for seismic wave propagation and inversion.

1 Introduction

  • High simulation costs make seismic wave propagation a limiting factor in geophysical research and can lead researchers to use 1D models despite better 3D velocity models being available.Simulations support applications ranging from ground-shaking simulation to subsurface imaging and earthquake-source characterization.Source: §1 · p. 1
  • Perturbing a source or velocity structure requires a new simulation, adding costs that can make tractable problems prohibitively expensive.Prior neural approaches could solve for different source-receiver pairs but were relatively inefficient to train and learned only approximate solution operators.Source: §1 · p. 2
  • Neural networks map between finite-dimensional spaces, whereas general PDE solution operators map between infinite-dimensional function spaces.Neural Operators are presented as a paradigm for function-space maps, with shared parameters over functions and numerically discretized inputs.Source: §1 · p. 2
  • The paper develops a prototype Neural Operator framework for the 2D acoustic wave equation and explores its application to seismic propagation and inversion.The study presents the framework as a proof of concept for seismology.Source: §1 · p. 2

2 Preliminaries

The preliminaries describe a solution operator for a linear PDE and generalize the formulation to nonlinear solution operators built from sequential linear operators. Neural Operators learn components that produce solutions for prescribed input functions.

2 Preliminaries

  • For a given function A and Green’s function G, U denotes the solution to a linear partial differential equation and its solution operator.L is the corresponding linear operator.Source: §2 · p. 2
  • For the acoustic wave equation, A can describe the velocity structure and initial conditions, and Neural Operators generalize the formulation to nonlinear solution operators.An ℓ-layered Neural Operator sequentially applies linear operators to construct the nonlinear operator.Source: §2 · p. 3
  • The layered operator applies L1 through Lℓ with nonlinear activations between layers to produce U.The passage gives the layered operator construction as an equation.Source: §2 · p. 3
  • Wi(x) and Ki(x, y) are learnable components that let a Neural Operator produce a solution for any prescribed function A.In a limited sense, Neural Operators can be viewed as generalized Green’s functions.Source: §2 · p. 3

3 Methods

The methods train a Fourier Neural Operator on simulated 2D acoustic wavefields generated from random velocity models and source locations. Its frequency-domain computations produce complete wavefields that can be queried independently of the input mesh.

3 Methods

  • The Fourier Neural Operator takes a velocity model on an arbitrary, possibly irregular mesh, and point-source coordinates to produce a complete wavefield queryable anywhere in the medium.Its heavy calculations take place in the frequency domain, enabling rapid convolution computation.Source: §3 · p. 3
  • The FNO uses four sequential Fourier blocks, each transforming the input to the Fourier domain, applying a low-order truncated kernel integration, then transforming back and applying ReLU.The Fourier-mode truncation is performed after lifting function values to a higher-dimensional space.Source: §3 · p. 3
  • The training dataset contains 5,000 simulations generated from random velocity models on a 64×64 grid with 0.16 km spacing and Ricker sources at random points.The random velocity fields use a von Karman covariance function, and the acoustic equation is solved with a spectral-element method.Source: §3 · pp. 3–4
  • The FNO is trained in supervised fashion against spectral-element solutions using mean-square error and batch gradient descent.The training uses batches of 30, up to 300 epochs, with 200 simulations reserved for cross-validation; training from scratch takes approximately 18 hours on 6 NVIDIA Tesla V100 GPUs.Source: §3 · p. 4

4 Experiments

The experiments evaluate FNO wavefield predictions on validation and out-of-training-distribution velocity models, including finer grids, then test automatic-differentiation gradients for full waveform inversion. The inversion is substantially faster than SEM with adjoint methods, with a similar reported velocity-model misfit.

Initial wavefield demonstration

The number of simulations needed for training

Generalization to arbitrary velocity models

  • FNO approximated the wavefield for a constant-velocity square embedded in a homogeneous medium, despite training only on Gaussian random-field velocity models.The authors suggest better hyperparameter selection could reduce the small residual errors.Source: Generalization to arbitrary velocity… · pp. 4–5

Generalization to higher resolution grids

  • On a 128×128 velocity model with 2× finer resolution than training, FNO closely approximated the SEM solution at negligible extra compute cost.The finer-grid model had the same roughness as the training data; resolving rougher structure requires training across more correlation lengths and variances.Source: Generalization to higher resolution… · p. 7

Full waveform inversion with Neural Operators

  • 0.0319 versus 0.0289 relative ℓ2 misfit between inverted and true velocity models for FNO with automatic differentiation and SEM with the adjoint-state method, respectively.The inversion used a homogeneous initial model and 1,000 gradient-descent iterations with Adam.Source: Full waveform inversion with Neural… · pp. 7–8
  • ∼1 second versus ∼100 seconds per tomographic iteration for 20 sources: FNO with automatic differentiation was substantially more efficient than SEM with adjoint methods.The measurements used one NVIDIA Tesla V100 GPU.Source: Full waveform inversion with Neural… · p. 9

5 Discussion

The prototype applies Neural Operators to the 2D acoustic wave equation, with few thousand training simulations and negligible-time new solutions. It generalizes from lower- to higher-resolution simulations, while its approximate solutions may not suit every application; training on random velocity models may support a shared model across regions of similar size.

5 Discussion

  • A few thousand simulations train the FNO, which computes new solutions in negligible time and generalizes from lower-resolution training to higher-resolution velocity models.The paper reports substantially faster computation than traditional numerical methods at full resolution.Source: §5 · p. 9
  • The framework is a prototype for the 2D acoustic wave equation; the authors anticipate adapting it to the 3D elastic wave equation with relatively little modification.They point to successful FNO application to the Navier–Stokes equations as context for this expectation.Source: §5 · p. 9
  • FNO solutions are approximate; applications where the error is too large may be better served by traditional numerical methods.The authors characterize the reported performance metrics as a starting point and note that reasonably accurate solutions may suffice in other geophysical situations.Source: §5 · p. 9
  • Training on random velocity models lets the FNO produce solutions for arbitrary velocity models without retraining for each region.The authors propose that one model could serve the seismology community for regions of similar size, with the initial training cost incurred once for the community.Source: §5 · p. 9

Related papers

Loading 2108.05421v2…