Source-linked AI summary
EikoNet: Solving the Eikonal equation with Deep Neural Networks
Jonathan D. Smith, Kamyar Azizzadenesheli, Zachary E. Ross
TL;DR
The paper addresses how to compute first-arrival travel times efficiently in heterogeneous 3D velocity structures without relying on grids or precomputed travel-time solutions. EikoNet trains a differentiable neural network by minimizing velocity inconsistency in the factored Eikonal equation, producing continuous travel-time predictions consistent with Fast Marching solutions and scalable inference.
Problem
Computing travel-time fields for many source and receiver locations with finite-difference methods can require substantial computation, storage, and precomputed physics-based solutions.
Method
EikoNet trains a differentiable neural network on source-receiver coordinates to minimize velocity misfit from the factored Eikonal equation, without finite-difference training solutions.
Results
EikoNet produces heterogeneous-3D travel-time solutions consistent with Fast Marching results and predicts 1e6 source-point pairs in 0.424s on a single Nvidia Tesla V100.
Takeaways & Limitations
The learned model provides travel times for any two points in a continuous 3D domain, avoiding travel-time grids and enabling analytically differentiable source and receiver dependence.
Abstract
from arXiv · showhide
The recent deep learning revolution has created an enormous opportunity for accelerating compute capabilities in the context of physics-based simulations. Here, we propose EikoNet, a deep learning approach to solving the Eikonal equation, which characterizes the first-arrival-time field in heterogeneous 3D velocity structures. Our grid-free approach allows for rapid determination of the travel time between any two points within a continuous 3D domain. These travel time solutions are allowed to violate the differential equation - which casts the problem as one of optimization - with the goal of finding network parameters that minimize the degree to which the equation is violated. In doing so, the method exploits the differentiability of neural networks to calculate the spatial gradients analytically, meaning the network can be trained on its own without ever needing solutions from a finite difference algorithm. EikoNet is rigorously tested on several velocity models and sampling methods to demonstrate robustness and versatility. Training and inference are highly parallelized, making the approach well-suited for GPUs. EikoNet has low memory overhead, and further avoids the need for travel-time lookup tables. The developed approach has important applications to earthquake hypocenter inversion, ray multi-pathing, and tomographic modeling, as well as to other fields beyond seismology where ray tracing is essential.
1 Introduction
Three-dimensional ray tracing supports major seismological applications, while existing deep-learning approaches often depend on precomputed physics-based models. EikoNet instead incorporates the governing formulation into a neural network and loss function to learn travel times continuously in 3D.
- Three-dimensional ray tracing underpins earthquake hypocenter inversion, seismic tomography, and earthquake source-property studies.
- Existing deep-learning approaches for physics-based simulations can rely on precomputed models that contain modeling artifacts and input bias.
- EikoNet incorporates the governing physics into the neural-network architecture and loss function rather than relying on externally generated solutions.
- EikoNet learns travel times between any two points in a continuous 3D medium without grids, using analytically computed spatial gradients and velocity-misfit minimization.
2 Eikonal Formulation
The Eikonal equation models first-arrival travel time through heterogeneous media, with velocity determined by the travel-time gradient. EikoNet uses a factored formulation to reduce source-location singularity effects while enforcing the source-time boundary condition.
- The Eikonal equation is a nonlinear first-order PDE representing a high-frequency approximation to wave propagation in heterogeneous media.
- The equation relates the travel-time gradient to medium velocity or slowness at the receiver location.
- Travel time is obtained by minimizing misfit to the imposed velocity model while enforcing Ts→s = 0 at the source.
- The factored formulation represents travel time as a distance term multiplied by a deviation field, mitigating numerical errors caused by the source singularity.
- Finite-difference techniques are typically used to solve the partial differential terms in the factored formulation.
3 Methods
EikoNet trains a differentiable neural network on source-receiver coordinates and velocity consistency rather than known travel-time labels. Sampling, adaptive weighting, and direct network evaluation support continuous travel-time prediction after training.
- Network architecture and training: The network predicts the factored travel-time deviation from source-receiver coordinates, then analytically differentiates its output with respect to receiver position.
- Network architecture and training: EikoNet solves the Eikonal equation from scratch and must be retrained when the velocity model changes.
- Network architecture and training: The feed-forward architecture uses residual blocks with fully connected layers and nonlinear hidden units, with ELU activations selected for the model.
- Sampling: The training dataset samples source and receiver points across the continuous 3D medium and labels pairs with receiver-location velocity.
- Network architecture and training: Training minimizes the mismatch between predicted and observed velocities computed from the Eikonal formulation, not a supplied travel-time solution.
- Inference: After training, the network directly evaluates user-defined source-receiver pairs to produce travel time and predicted velocity.
- Weighted Sampling: Dynamic weighted sampling prioritizes pairs with the greatest relative velocity misfit while bounding weights between 0.1 and 0.9.
4 Baselines
Eikonal solvers include grid-based Fast Marching and iterative Fast Sweeping methods. The study uses Python Fast Marching travel times on a 0.1 km grid as a comparison baseline, not as neural-network training data.
- Fast Marching Methods: Fast Marching Methods use grid-based finite differences and an advancing interface to track minimum travel time.
- Fast Sweeping Methods: Fast Sweeping Methods solve the equation iteratively with upwind differencing and Gauss-Seidel sweeps.
- Baseline: The study uses Python Fast Marching as its baseline because it is less sensitive to sharp velocity contrasts, with only minor computational-time differences from Fast Sweeping.
- Baseline: Baseline travel times are computed on a 0.1 km grid in X, Y, and Z, and compared with EikoNet using RMS travel-time differences.
- Baseline: Fast Marching simulations are used only for comparison and are excluded from neural-network training.
- Workflow: Figure 1 organizes the workflow from network architecture and Eikonal formulation through sampling, velocity-loss training, and user-defined inference.
5 Velocity Model Experiments
EikoNet was evaluated across homogeneous, graded, block, checkerboard, and strongly heterogeneous Marmousi2 velocity models, generally recovering travel-time and velocity fields close to finite-difference or imposed solutions. The experiments also tested sharp contrasts and complex geological structure.
- Homogeneous model: 110 training epochs reduced the homogeneous-model training loss to zero, while EikoNet achieved zero RMS travel-time error against the analytical solution.The finite-matrix-method solution had an RMS travel-time error of 0.0113s.
- Graded model: 0.00209km/s mean velocity difference was obtained for the linearly depth-varying model, with an FMM travel-time RMS of 0.0189s.Velocity increased from 3km/s at the surface to 7km/s at 20km depth.
- Block model: The block model produced a 0.0311s travel-time RMS and 0.094km/s mean velocity difference despite a 20% sharp velocity contrast.Disagreement was mainly localized near sharp velocity gradients, while travel-time fronts deflected consistently with the contrast.
- Checkerboard model: The checkerboard model achieved a 0.0342s travel-time RMS and 0.19km/s mean velocity difference while reconciling alternating positive and negative anomalies.The model varied between 6km/s and 4km/s with 6km grid spacing.
- Marmousi2 model: Marmousi2 travel-time RMS values ranged between 0.34 −0.42s across source locations in a strongly heterogeneous 17km by 3.5km profile.The EikoNet and FMM fields were evaluated on a 0.001km grid.
6 Sampling Experiments
Sampling choices affect EikoNet training and recovery, with weighted random sampling giving the lowest validation loss and larger datasets substantially improving complex-structure recovery. Batch size mainly affects early training, while final best loss is comparatively insensitive to it.
- Sampling schemes: Weighted random sampling achieved the lowest validation loss, although all three sampling methods converged in a similar number of epochs.The other methods performed well because validation-loss differences were relatively small.
- Sampling schemes: Random-location sampling produced greater misfit near the source, motivating use of a random-distance metric for source-point selection.The paper attributes this bias to longer distances being overrepresented under the random-location procedure.
- Sampling schemes: The sampling figures compare sampling schematics, learned velocity models, imposed-versus-predicted misfit, and training, validation, and testing losses.These panels provide complementary views of how sampling affects both recovered fields and optimization behavior.
- Training-set size: The dataset with 10^6 samples achieved a loss about one order of magnitude lower than the dataset with 10^4 samples.The smaller dataset could not reconcile the block model's sharp velocity contrast.
- Batch size: Final best loss was insensitive to batch size, even though batch size influenced early training results.Larger batch sizes were identified as more computationally efficient.
7 Future Applications
EikoNet is applied to earthquake-location, ray-multipathing, and tomographic problems where conventional finite-difference methods face computational or storage constraints. The described procedures use continuous travel-time fields, gradients, and transfer learning to address these applications.
- Earthquake Location: Finite-difference travel-time calculations are impractical for tens of thousands of virtual receivers in distributed acoustic sensing arrays.The stated alternative scales independently of receiver count, has a compact storage footprint, and rapidly evaluates forward predictions.
- Ray Multipathing: Stationary points of a combined travel-time field identify possible secondary arrival ray paths in a low-velocity-block model.The procedure combines travel times from two source locations and evaluates the gradient at receiver locations.
- Ray Multipathing: 1 × 10^6 points require 7s on a single CPU for each possible multipathing point calculated with EikoNet.The method forms travel-time fields and their receiver-location gradients for identifying stationary points.
- Tomographic Modeling: Transfer learning is approximately 3–4× faster than training a full model from scratch for successive tomographic velocity updates.The previous iteration’s neural network is used as the starting point when velocity perturbations are relatively small.
- Tomographic Modeling: Adding an observation-based loss term could update the velocity model to fit known travel-time values during tomographic inversion.The proposed extension would incorporate known observations into the loss function rather than recomputing a finite-difference travel-time field after every update.
8 Discussion and Conclusion
EikoNet learns travel-time fields in heterogeneous 3D velocity structures and produces solutions consistent with the Fast Marching Method. Its continuous, differentiable representation supports efficient reuse across source locations, interpolation-free evaluation, and gradient-based applications.
- Core result: EikoNet solves the Eikonal equation for heterogeneous 3D velocity structures with travel-time solutions consistent with the Fast Marching Method.This is the paper’s stated demonstration and principal validation result.
- Computational advantages: EikoNet reuses learned structure across source locations instead of separately computing and storing a travel-time field for every source.Finite-difference costs and travel-time-table storage increase with source locations, receiver nodes, and grid size.
- Continuous representation: EikoNet provides travel-time solutions for any two points in a continuous 3D domain without storing or interpolating a travel-time grid.The network learns an interpolation scheme during training from context across the dataset.
- Differentiability: The learned travel-time solutions are differentiable with respect to source and receiver locations, enabling earthquake-location objectives to use analytically computed gradients.The paper specifically describes formulating the inverse problem as gradient descent over source locations.
- Broader scope: The Eikonal equation also occurs in optics, medical imaging, and video-game rendering, where the authors expect EikoNet to be suitable.These are identified as domains beyond seismology involving related wave-physics applications.
- Computational advantages: 4.047 × 10^-4s per source-point pair is the reported single-CPU forward-prediction time, while the approach is massively parallel and GPU-suited.The passage reports this timing for a 2.3GHz Intel Core i5 CPU and begins a separate million-source-point GPU timing.