PaperlayerView exact revision on arXiv

Source-linked AI summary

Parametric generation of conditional geological realizations using generative neural networks

Shing Chan, Ahmed H. Elsheikh

arXiv:1807.05207v2stat.MLcs.LGphysics.comp-ph

TL;DR

Existing conditioning approaches may be costly or lack a parametrization of conditional sampling. This paper extends an existing generator with a posterior-sampling inference network. On binary channelized images, the method produced plausible conditioned realizations, visually comparable with snesim when reference statistics agreed with observations.

  • Problem

    Existing optimization- and MCMC-based conditioning can be costly or may not cover the full solution space, and neither parametrizes conditional sampling.

    Source: §3 · p. 7

  • Method

    A second inference network is stacked onto an existing generator and trained to sample the Bayesian posterior, yielding a conditional parametrization.

    Source: §1 · p. 2, §6 · p. 26

  • Results

    For binary channelized images, G ◦I produced plausible realizations with good conditioning; when reference statistics agreed with observations, results were visually comparable with snesim.

    Source: §4.2.2 · p. 25, §6 · p. 26

  • Takeaways & Limitations

    The extended parametrization supports reuse of an unconditional model and direct, parametric sampling of conditional realizations.

    Source: §6 · p. 26

    Alternative training methods for the inference network remain for future study.

    Source: §6 · p. 26

Abstract

from arXiv · show

Deep learning techniques are increasingly being considered for geological applications where -- much like in computer vision -- the challenges are characterized by high-dimensional spatial data dominated by multipoint statistics. In particular, a novel technique called generative adversarial networks has been recently studied for geological parametrization and synthesis, obtaining very impressive results that are at least qualitatively competitive with previous methods. The method obtains a neural network parametrization of the geology -- so-called a generator -- that is capable of reproducing very complex geological patterns with dimensionality reduction of several orders of magnitude. Subsequent works have addressed the conditioning task, i.e. using the generator to generate realizations honoring spatial observations (hard data). The current approaches, however, do not provide a parametrization of the conditional generation process. In this work, we propose a method to obtain a parametrization for direct generation of conditional realizations. The main idea is to simply extend the existing generator network by stacking a second inference network that learns to perform the conditioning. This inference network is a neural network trained to sample a posterior distribution derived using a Bayesian formulation of the conditioning task. The resulting extended neural network thus provides the conditional parametrization. Our method is assessed on a benchmark image of binary channelized subsurface, obtaining very promising results for a wide variety of conditioning configurations.

1 Introduction

Geological models have many correlated spatial variables, making reservoir simulation expensive and motivating parametrizations that preserve complex geological characteristics. This work extends a GAN generator with a Bayesian inference network to directly parameterize conditional realizations, and assesses the method on a benchmark image across varied conditioning configurations.

1 Introduction

  • Reservoir simulation is expensive for large-scale geological models, motivating parametrizations that preserve complex characteristics needed for accurate flow modeling.The introduction places GANs among existing approaches including zonation, PCA, SVD, discrete cosine transforms, level sets, and dictionary learning.Source: §1 · p. 1
  • GAN generators learn to sample geological images from a dataset, preserving high-order flow statistics, two-point statistics, and morphology while reducing dimensionality by several orders of magnitude.Prior work reports these properties across different studies.Source: §1 · p. 2
  • Earlier conditioning methods use image inpainting with per-realization optimization or Bayesian MCMC, but neither parametrizes the conditional sampling process.Repeated optimization can be expensive when realizations must be generated continuously during deployment.Source: §1 · p. 2
  • The proposed method stacks a Bayesian-posterior inference network onto an existing generator to parameterize direct conditional sampling, with positive results across varied conditioning configurations.The method is assessed on the Strebelle and Journel benchmark image.Source: §1 · p. 2

2 Background

The background frames subsurface parametrization as narrowing the search to plausible realizations, then introduces GAN-based generators and Bayesian conditioning to spatial observations.

2.1 Parametrization

  • A suitable subsurface parametrization narrows the search to plausible realizations, reducing simulations needed for uncertainty quantification and inversion.The section compares this to using a word index to rule out implausible letter arrangements.Source: §2.1 · p. 3
  • Deep neural networks provide a flexible parametrization that can learn relevant statistics from data, rather than preserving only mean and covariance as PCA does.The network is built as a composition of layers with component-wise nonlinearities.Source: §2.1 · pp. 3–4

2.2 Generative adversarial networks

  • GAN training uses a discriminator to distinguish generated from training realizations while the generator learns to approximate the target distribution; the paper uses the Wasserstein formulation.After training, the discriminator can be discarded and the generator retained as the parametrization.Source: §2.2 · pp. 4–6

2.3 Conditioning to observations

  • Conditioning seeks latent values with high posterior probability given hard observations, combining a generator-based prior with a Gaussian-noise likelihood.Earlier approaches repeatedly optimize the posterior from different initial values or sample it with Markov chain Monte Carlo.Source: §2.3 · p. 6

3 Conditional generator for geological realizations

Existing conditional samplers do not provide a functional parametrization of the sampling process, and repeated local optimization can be expensive for many realizations. The proposed method trains an inference network to map random inputs to posterior latent vectors, then composes it with the generator for direct conditional sampling.

3 Conditional generator for geological realizations

  • Repeated local optimization can be expensive for many conditional realizations, while local optimization and MCMC do not provide a parametrization of the sampling process.The paper identifies continuous deployment for uncertainty quantification and inversion as settings where repeated optimization may be costly.Source: §3 · p. 7
  • The composed generator and inference network provide a conditional parametrization for direct, efficient sampling by forward-passing random inputs.The composition is Gcond = G ◦ I, with the inference network trained to sample the Bayesian posterior.Source: §3 · pp. 7–8
  • The inference network maps random inputs to latent vectors distributed according to the conditional posterior, and its induced density is easy to sample despite being unknown and intractable.Training minimizes the KL divergence objective using automatic differentiation.Source: §3 · pp. 7–8
  • The training objective combines expected loss with an entropy term, whose sample estimate measures spread and distinguishes posterior sampling from finding only a maximum a posteriori estimate.The loss is defined as observation mismatch plus a weighted latent-vector norm; the entropy estimator uses distances to nearest neighbors.Source: §3 · pp. 7–8
  • The inference network is relatively easy to train, and its small size makes the added evaluation cost of composing it with the generator insignificant in the reported experiments.The authors report this cost pattern in their experiments.Source: §3 · p. 8

4 Numerical experiments

Experiments use the Strebelle and Journel benchmark to assess unconditional generation and nine conditional configurations. The assessments compare generated realizations with snesim and examine memorization, conditioning quality, and plausibility.

4.1.1 Architecture design

  • The generator maps a 30-dimensional standard-normal latent vector to 64 × 64 images, reducing dimensionality by two orders of magnitude.A tanh output nonlinearity represents the binary channel and background materials while keeping the generator continuous.Source: §4.1.1 · p. 9, §4.1 · p. 9

4.1.2 Quality assessment

  • G realizations show crisp, clear channels and are visually virtually indistinguishable from snesim samples.The displayed images are not thresholded.Source: §4.1.2 · p. 9, Figure 4 · p. 10
  • ANODI scores for G and fresh snesim realizations are relatively close across four resolutions, with multidimensional-scaling plots showing strong overlap.ANODI captures multipoint statistics; the analysis uses sets of 100 realizations and 4 × 4 windows.Source: Table 1 · p. 10, Figure 4 · p. 10, §4.1.2 · p. 11

4.1.3 Memorization

  • No generated realization perfectly matches the augmented training set, and latent-space interpolation yields smooth transitions between plausible outputs.The nearest-neighbor check covers 50 generated realizations and tests against flips, rotations, shearing, and small translations.Source: §4.1.3 · pp. 11, 13

4.2 Conditional parametrization

  • Nine conditioning configurations use 16 to 49 hard observations; because the Bayesian formulation treats observations as noisy, exact conditioning is not guaranteed.The tests use λ = 0.1 and a Gaussian likelihood.Source: §4.2 · p. 13

4.2.1 Architecture design – inference network

  • A fully connected inference network Iφ learns the posterior for each case, and the trained composition G◦I generates conditional realizations.The same architecture and optimization parameters are used across all nine cases, with minimal hyperparameter tuning.Source: §4.2.1 · p. 13

4.2.2 Quality assessment – conditional realizations

  • G◦I emphasizes plausibility while snesim produces more diverse realizations; G◦I is visually comparable when conditioning agrees with reference statistics, but struggles with densely enforced discordant vertical channels.The authors note that a reference image should have spatial statistics in good agreement with observations and that case-specific hyperparameter tuning could improve results.Source: §4.2.2 · pp. 13, 25, Figure 6 · p. 14, Figure 14 · p. 22
  • Pairwise plots visualize selected components of I's output distribution, while discriminator-score histograms reach the same qualitative conclusions as ANODI.Only the first and last two of the 30 latent components are plotted pairwise.Source: Figure 15 · p. 23, Figure 16 · p. 24, §4.2.2 · p. 25

5 Related work

Prior conditional GAN methods use MCMC or local optimization, while this work trains a neural sampler to parameterize posterior sampling. The authors also identify other neural-sampling approaches as future alternatives.

5 Related work

  • Earlier conditioning methods used MCMC to sample the posterior [19] or inpainting with local optimization [20] [21].The inpainting approach is described as equivalent to a Bayesian formulation using a neural-network prior.Source: §5 · p. 26
  • The proposed approach is closer to [19], but uses a neural network to sample the full posterior and parameterize that sampling process.It uses a simple prior.Source: §5 · p. 26
  • Normalizing flows, autoregressive flows, and Stein discrepancy are identified as alternative neural-sampler training methods worth exploring.Source: §5 · p. 26

6 Conclusion

The paper extends an unconditional GAN parametrization with a neural network that samples the Bayesian posterior, producing a conditional parametrization. On binary channelized benchmark images, unconditional results were positive and conditional results were plausible, with effectiveness depending on conditioning agreement.

6 Conclusion

  • The conditional parametrization composes an unconditional GAN generator with a second neural network trained to sample a Bayesian posterior.The resulting composition is denoted G ◦ I.Source: §6 · p. 26
  • Two orders of magnitude of dimensionality reduction accompanied positive unconditional results; conditional outputs were plausible, but effectiveness depended on conditioning agreement.Conditioning was difficult when observations disagreed strongly with reference spatial statistics; agreeing observations produced comparable results.Source: §6 · p. 26
  • Future work includes alternative inference-network training methods and assessment on other images and large-scale settings.Source: §6 · pp. 26–27

A Implementation details

The appendix section describes model training and hyperparameters and points to a practical neural-network training guide.

A Implementation details

A.1 Generator neural network

The generator maps a 30-dimensional latent vector to 64×64 images using a deep convolutional network. It is trained with a Wasserstein GAN formulation and generates realizations rapidly at deployment.

A.1 Generator neural network

  • The generator maps R30 to 64×64 images using stacked convolutional layers, batch normalization, intermediate ReLUs, and a final tanh output constrained to [−1, 1].Source: §A.1 · p. 32
  • Training uses the Wasserstein GAN formulation with Adam, a 10−4 learning rate, and batch size 32.The generator converges in approximately 20,000 iterations, taking around 30 minutes on a GeForce GTX Titan X GPU.Source: §A.1 · p. 32
  • At deployment, the generator produces approximately 5,500 realizations per second using a GPU.Source: §A.1 · p. 32

A.2 Inference neural network

The inference network maps a 30-dimensional input through fully connected layers to perform conditioning; the same architecture is used across conditioning experiments. The conditional generator adds little generation-time overhead.

A.2 Inference neural network

  • The inference network maps 30 inputs to 30 outputs through a 512-unit layer, five intermediate layers, and an unbounded output layer.The authors report that deeper architectures tended to produce better results in general.Source: §A.2 · p. 32
  • The inference network uses SeLU activations and is optimized with Adam at a 10^-4 learning rate and batch size 64.These settings are used for all test cases.Source: §A.2 · p. 32
  • On a Nvidia GeForce GTX Titan X GPU, conditional generator G◦I generates approximately 5,500 realizations per second, with no significant increase in generation time compared with G.Depending on conditioning, training takes between seconds and a few minutes on the reported GPU.Source: §A.2 · p. 33

B Conditioning settings

C Mixture of Gaussians

The toy tests assess whether fully connected neural networks can learn to map standard-normal samples to one- and two-dimensional Gaussian mixtures. In both cases, the network learns this transport by convergence.

C Mixture of Gaussians

  • The toy assessment trains fully connected networks to sample one- and two-dimensional Gaussian mixtures, using standard normal source distributions.The input and output dimensions are both one in the 1D case and two in the 2D case.Source: C Mixture of Gaussians · p. 33
  • The 1D test uses three Gaussian components centered at −1, 2, and 6, with standard deviations 1, 2, and 0.5, respectively.The passage describes generated samples at 100 and 1,000 training iterations.Source: C Mixture of Gaussians · p. 33
  • The 2D test uses three Gaussian components centered at (−1, −1), (1, 2), and (2, −1).Source: C Mixture of Gaussians · p. 35
  • In both the one- and two-dimensional tests, the neural network effectively learns to transport standard-normal points to the Gaussian mixture.Source: C Mixture of Gaussians · p. 35

D Comparison to related work based on inpainting

Earlier GAN-based inpainting conditions realizations through per-sample optimization, using a discriminator-based perceptual loss. The authors argue a Gaussian prior can suffice when GAN training converges, though the perceptual loss can help when generated realizations remain poor.

D Comparison to related work based on inpainting

E Convolutional neural networks

Convolutional layers impose a locality structure that reduces free weights, while deep convolutional architectures can reduce computational cost. The complexity and engineering effort of neural-network models nevertheless depend on architecture.

E Convolutional neural networks

F Computational complexity

  • Fast PCA based on singular value decomposition has complexity O(N^2d), which is favorable when N ≪ d but still grows quadratically with dataset size.Source: F Computational complexity · p. 36
  • For neural networks, complexity depends on the architecture; deep convolutional stacks can reduce the evaluation estimate from O(d^2) to O(kd), with training cost O(Tkd).The factor k is architecture-dependent and can still be sizable.Source: F Computational complexity · p. 37
  • Optimizing numerous hyperparameters, especially architecture design, can require costly engineering time because automation remains limited.The authors say whether that cost is justified depends on the model's deployment lifespan.Source: F Computational complexity · p. 37
Loading 1807.05207v2…