Deep learning electromagnetic inversion with convolutional neural networks

Vladimir Puzyrev

Source-linked AI summary

Deep learning electromagnetic inversion with convolutional neural networks

Vladimir Puzyrev

arXiv:1812.10247v1Dec 26, 2018physics.geo-phcs.LGcs.NE

TL;DR

Geophysical EM inversion must estimate subsurface resistivity despite nonlinear, nonunique, and computationally demanding inverse problems. The paper trains fully convolutional deep networks on full-3-D synthetic simulations to map EM data directly to resistivity models. The networks accurately recover anomaly geometry and resistivity, with predictions available almost instantly and remaining feasible under noise and varied survey settings.

Problem
EM inversion is difficult because nonlinear, nonunique problems and the computational burden of conventional methods complicate estimating subsurface resistivity.
Method
Fully convolutional neural networks are trained on synthetic EM datasets generated by full 3-D simulations to learn a direct data-to-resistivity mapping.
Results
The method accurately estimates subsurface resistivity distributions and anomaly properties, with predictions made in less than one second after training.
Takeaways & Limitations
Deep learning inversion can provide accurate, nearly instantaneous resistivity estimates across CO2 plume delineation, full inversion, survey geometries, and noisy data.
Takeaways & Limitations
Training-data generation can be very time consuming because realistic resistivity structures require fine-scale full-3-D simulations, although the process is parallelizable.

Abstract

from arXiv · show

Geophysical inversion attempts to estimate the distribution of physical properties in the Earth's interior from observations collected at or above the surface. Inverse problems are commonly posed as least-squares optimization problems in high-dimensional parameter spaces. Existing approaches are largely based on deterministic gradient-based methods, which are limited by nonlinearity and nonuniqueness of the inverse problem. Probabilistic inversion methods, despite their great potential in uncertainty quantification, still remain a formidable computational task. In this paper, I explore the potential of deep learning methods for electromagnetic inversion. This approach does not require calculation of the gradient and provides results instantaneously. Deep neural networks based on fully convolutional architecture are trained on large synthetic datasets obtained by full 3-D simulations. The performance of the method is demonstrated on models of strong practical relevance representing an onshore controlled source electromagnetic CO2 monitoring scenario. The pre-trained networks can reliably estimate the position and lateral dimensions of the anomalies, as well as their resistivity properties. Several fully convolutional network architectures are compared in terms of their accuracy, generalization, and cost of training. Examples with different survey geometry and noise levels confirm the feasibility of the deep learning inversion, opening the possibility to estimate the subsurface resistivity distribution in real time.

1 Introduction

EM inversion estimates subsurface resistivity from observations, but 3-D geophysical inverse problems are nonlinear, nonunique, and computationally expensive. The paper proposes fully convolutional deep networks for rapid 2-D EM inversion using synthetic data from full 3-D simulations.

Electrical resistivity is sensitive to porosity and fluid saturation, making EM methods useful for characterizing subsurface geological formations.Applications include hydrocarbon and mineral exploration, CO2 storage monitoring, geothermal reservoirs, and hydrogeology.
Geophysical inverse problems are nonlinear and nonunique, so multiple subsurface models may fit observations and small data changes can produce large model changes.These properties complicate reliable model estimation.
3-D inversion is costly because modern surveys require many forward simulations, while accurate EM modelling can involve tens of millions of unknowns.Traditional large-scale inversion may require millions of forward problems.
Deep learning became more capable through advances in artificial intelligence, convolutional networks, and GPU computing, following earlier limited neural-network applications in geophysics.
The paper proposes fully convolutional neural networks for 2-D EM inversion trained on datasets generated by fully 3-D simulations of models with varying resistivity anomalies.

2 Problem formulation

The paper formulates EM modelling through Maxwell-based forward simulation and contrasts conventional regularized, gradient-based inversion with a data-driven neural-network mapping. The proposed approach learns resistivity models directly from synthetic examples without constructing gradients.

2.1 Forward modelling: Maxwell’s equations describe the electromagnetic fields, currents, source distribution, and 3-D computational domain used in forward modelling.The fields are represented by E and H, with J = σE and source current Js.
2.1 Forward modelling: The scattered-field formulation separates primary and secondary fields and can reduce excessive meshing near sources.Perfectly electrically conducting boundaries truncate the computational domain when sufficiently distant.
2.1 Forward modelling: Forward modelling maps a resistivity model to measured EM data and is used here to generate synthetic training data.The spatial discretization uses a 3-D finite-difference method on staggered grids.
2.2 Inversion: Traditional inversion minimizes a regularized quadratic least-squares cost involving the forward operator, observations, model regularization, and a Lagrange multiplier.The data-misfit term measures the distance between observations and simulations.
2.2 Inversion: Local optimization iteratively updates logarithmic resistivity parameters using search directions and gradients, whose computation requires forward problems for all sources and frequencies.Step length and second-order information also affect computational cost and convergence.
2.2 Inversion: Local methods depend strongly on a good starting model, whereas global methods address some issues at even greater computational expense, especially in 3-D.
2.2 Inversion: The proposed data-driven approach predicts a subsurface resistivity model directly from EM data in one step without constructing gradients.A deep neural network learns the data-to-model mapping from multiple synthetic examples and predicts previously unseen cases after training.

3 Deep Learning Inversion

Deep learning inversion uses convolutional networks to map electromagnetic measurements directly to structured resistivity models. The section covers fully convolutional architectures, training and regularization choices, and MAE, RMSE, and IoU evaluation.

Convolutional architectures: CNNs process grid-like data through learned filters, while fully convolutional networks output high-dimensional structured objects without terminal fully connected layers.This architecture supports inversion outputs whose dimensions need not match the input data dimensions.
Convolutional architectures: EM inversion CNNs can use electric or magnetic field components, amplitudes, phases, and frequencies as input channels and accommodate varying receiver configurations.The input channels represent different observed quantities at spatial locations.
Training and regularization: CNN training minimizes a loss function using gradient-based optimization, with Adam selected for computational efficiency and fast convergence; GPUs accelerate training through parallelism.The paper also compares standard optimization algorithms in an appendix.
Losses and metrics: MAE weights absolute errors equally, whereas RMSE gives relatively greater weight to large errors and is always at least as large as MAE.The difference between the metrics reflects variation in individual errors.
Losses and metrics: IoU measures overlap between sample sets, and the paper uses IoU loss only when predicting binary target/non-target masks rather than continuous resistivity values.For diffusive EM fields, IoUs of 0.4–0.5 are considered sufficiently accurate, while 0.6–0.7 or higher indicates a very good match.
Training and regularization: The networks are trained with supervised backpropagation and regularization methods such as dropout, early stopping, and data augmentation to improve generalization.Data augmentation is especially useful when training data are limited, although augmentation for modeling problems can be nontrivial.

4 Numerical examples

Numerical examples test fully convolutional CNN inversion for onshore CSEM CO2 monitoring across receiver geometries, plume models, saturation variability, and noise levels. The networks generally recover plume position, dimensions, and resistivity, while performance deteriorates for difficult structures and sufficiently high noise.

Dataset and setup: The training data combines full 3-D simulations and augmentation across 20,000 resistivity models, including primitive and realistic CO2 plume geometries.Plume dimensions range from 40 m to more than 200 m, and the simulated zone is discretized into 10,000 blocks.
Example 1: Receiver grid: The 2-D CNN achieves an average IoU of 0.67 on the test dataset, with 18% excellent and 54% very good predictions.Excellent predictions have IoU ≥0.8, while very good predictions have 0.6 ≤IoU < 0.8.
Noise robustness: 3–5% noise produces IoUs similar to the noise-free case, but reliability collapses when noise exceeds typically 23–25%.At high noise, the average IoU drops to nearly zero and badly predicted cases increase rapidly.
Example 2: Receiver lines: Using only two perpendicular receiver lines yields 64% very good or excellent predictions, despite limited data availability.Anomaly positions are more reliable than boundaries, and 39 of 800 models are badly predicted with IoU <0.2.
Example 3: Variable resistivity: With variable CO2 saturation, the CNN reliably estimates anomaly shape and position but typically underestimates resistivity.Saturation varies from 0.1 to 0.5, producing corresponding resistivity variation inside the plumes.
Architecture and uncertainty: The tested fully convolutional networks contain 0.6–5 million trainable parameters, and multiple network variants provide small output differences useful for uncertainty quantification.Using deeper networks has little effect on the relatively small dataset.

5 Discussion

The discussion emphasizes deep-learning inversion’s speed and scope, while identifying training-data representativeness and loss-function design as important boundaries for reliability.

Computational efficiency: Once trained, the networks predict resistivity models in less than one second, whereas training takes 1.5–6 hours depending on network size.The networks contain 0.6–5 million trainable parameters and were trained on NVIDIA Tesla P100 GPUs.
Data requirements: Training data generation can be time consuming because realistic resistivity structures require fine-scale full 3-D simulations, although the process is perfectly parallelizable.The paper also notes that larger public geophysical data repositories could substantially expand deep-learning applications.
Generalization: The approach depends on sufficiently representative training models so that predictions generalize to previously unobserved inputs.The network need not include every possible subsurface model, but the training set must adequately represent the target problem.
Evaluation and limitations: IoU can mis-rank EM anomaly predictions because a structurally plausible, slightly displaced result may receive a lower score than a poorer prediction.The cited examples assign IoU values of 0.12 and 0.22 to these two cases, respectively.
Evaluation and limitations: New loss functions incorporating structural similarity are needed, while model sharpness is determined by the geological features represented in the training data.This shifts a role commonly associated with regularization in traditional inversion to the training distribution.
Extensions: The convolutional output can be extended from 2-D to 2.5-D or 3-D inversion by adding anisotropy or resistivity planes in the third dimension.CNNs can also accept 1-D input data, while recurrent neural networks are identified as efficient for sequential data.

6 Conclusions

The conclusions present fully convolutional deep networks as an accurate and nearly instantaneous approach to 2-D EM inversion, supported by synthetic full 3-D simulations and examples with noise.

Conclusions: The method maps shaped electric and magnetic field arrays to resistivity models using deep fully convolutional networks trained on synthetic full 3-D simulations.The architecture accepts input dimensions and sizes without restrictions, with batch normalization and dropout used to reduce overfitting.
Conclusions: Numerical CSEM examples spanning CO2 plume delineation and full resistivity inversion demonstrate accurate, almost instantaneous 2-D model estimation.The examples use a borehole-to-surface configuration with a vertical electric dipole.
Conclusions: The quality of the deep-learning inversion does not deteriorate significantly when random noise is added to the data.The conclusion frames this result as confirmation of the method’s feasibility across the demonstrated examples.

A Data generation by fast forward modelling

The data-generation workflow uses parallel 3-D finite-difference modelling, reuses solver information for acceleration, and augments anomaly examples through symmetry.

Forward modelling: Industrial-scale CSEM data generation and deterministic inversion are computationally expensive because they require forward modelling across many source positions and frequencies.Typical problems may involve several thousand transmitter and receiver positions.
Forward modelling: The training data are generated with a hybrid MPI/OpenMP parallel 3-D finite-difference code based on the curl-curl electric-field formulation.Forward problems at different frequencies are processed independently on computational nodes.
Acceleration: Reusing ILU preconditioner factors and the previous model’s response as an initial guess produces an approximately 10x speed-up over independent system solutions.Parallelization is performed over frequencies and plume locations.
Data augmentation: Symmetry-based data augmentation creates X-, Y-, and XY-reflected versions of each anomaly, increasing the initial 5,000-example dataset fourfold.The augmentation exploits anomaly symmetry in a 1-D background.
Optimizer evaluation: Figure 14 compares IoU and RMSE training and validation errors for six adaptive learning-rate optimizers, with validation curves smoothed using a 7-point sliding window.Original validation errors are also shown as thin dotted lines.

B Comparison of training algorithms

The optimizer comparison finds Adagrad and RMSprop weakest on selected training or validation errors, while Adadelta, Adam, and its variants perform efficiently across both sets.

Optimizer comparison: Adagrad and RMSprop are the worst at minimizing training and validation error, respectively, for the compared 2-D CNN.The comparison uses a network with 1.7 million parameters and two convolutional blocks at each layer.
Optimizer comparison: Adadelta achieves the lowest training error and performs well on the validation dataset.The optimizer comparison uses default parameters from the Keras library.
Optimizer comparison: Adam and its variants show high efficiency on both training and validation sets.The paper notes that Adam is often used as a default choice, although there is no consensus on the best adaptive optimizer.