PaperlayerView exact revision on arXiv

Source-linked AI summary

U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow

Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, Sally M. Benson

arXiv:2109.03697v3physics.geo-phcs.LG

TL;DR

Multiphase-flow simulation is computationally intensive, motivating faster surrogates for geoscience applications. This paper introduces U-FNO, which outperforms FNO and CNN benchmarks for CO2-water predictions while using less training data and enabling substantial speedups.

  • Problem

    Numerical multiphase-flow simulation is computationally intensive because accurately capturing complex flow processes requires fine spatial and temporal discretization.

  • Method

    U-FNO enhances the Fourier neural operator with a U-Fourier layer that combines FNO-based and CNN-based paths for CO2-water multiphase-flow prediction.

  • Results

    U-FNO provides the best gas-saturation and pressure-buildup performance among the evaluated FNO and CNN models, using up to 3.4 times less training data than CNN.

  • Takeaways & Limitations

    The trained U-FNO model predicts gas saturation and pressure buildup 6 × 10^4 times faster than a traditional numerical solver, supporting repetitive forward simulations.

  • Takeaways & Limitations

    Adding the CNN-based U-Net path sacrifices the flexibility to train and test at different discretizations, so U-FNO prioritizes accuracy for this grid-sensitive problem.

Abstract

from arXiv · show

Numerical simulation of multiphase flow in porous media is essential for many geoscience applications. Machine learning models trained with numerical simulation data can provide a faster alternative to traditional simulators. Here we present U-FNO, a novel neural network architecture for solving multiphase flow problems with superior accuracy, speed, and data efficiency. U-FNO is designed based on the newly proposed Fourier neural operator (FNO), which has shown excellent performance in single-phase flows. We extend the FNO-based architecture to a highly complex CO2-water multiphase problem with wide ranges of permeability and porosity heterogeneity, anisotropy, reservoir conditions, injection configurations, flow rates, and multiphase flow properties. The U-FNO architecture is more accurate in gas saturation and pressure buildup predictions than the original FNO and a state-of-the-art convolutional neural network (CNN) benchmark. Meanwhile, it has superior data utilization efficiency, requiring only a third of the training data to achieve the equivalent accuracy as CNN. U-FNO provides superior performance in highly heterogeneous geological formations and critically important applications such as gas saturation and pressure buildup "fronts" determination. The trained model can serve as a general-purpose alternative to routine numerical simulations of 2D-radial CO2 injection problems with significant speed-ups than traditional simulators.

1. Introduction

Multiphase-flow simulations are essential across geoscience but are often time-consuming, motivating machine-learning alternatives. U-FNO extends Fourier neural operators to multiphase flow by combining FNO and CNN advantages for accurate, data-efficient predictions.

  • Motivation: Multiphase flow in porous media supports applications including contaminant transport, carbon capture and storage, hydrogen storage, oil and gas extraction, and nuclear waste storage.Its multi-physics, non-linear, and multi-scale nature makes numerical simulation the primary solution approach.
  • Motivation: Numerical simulation is essential for solving multiphase-flow conservation equations but is often time-consuming.
  • Related work: Machine-learning methods have been proposed as faster alternatives to numerical simulation, including data-driven finite-dimensional operators and physics-informed approaches.
  • Background: Neural operators learn infinite-dimensional mappings from functional parameters to solutions, require training only once, and can be evaluated on different grids.
  • Contribution: U-FNO extends FNO to multiphase flow and combines FNO-based and CNN-based models to improve accuracy and data efficiency.The design addresses cases where FNO testing accuracy is higher but training accuracy is sometimes lower because of FNO regularization.

2. Problem setting

The problem setting models immiscible, mutually soluble CO2–water multiphase flow in porous media, including phase-dependent transport, capillarity, and nonlinear fluid properties. Numerical simulations use radial super-critical CO2 injection to generate gas-saturation and pressure-buildup fields from heterogeneous geological and operating inputs.

  • Multiphase formulation: The model considers immiscible CO2 and water with mutual solubility for geological CO2 storage.Water is the wetting phase in siliciclastic rocks, while each phase contains a small amount of the other component.
  • Multiphase formulation: Phase fluxes follow multiphase Darcy flow with saturation-dependent relative permeability, pressure-dependent viscosity, gravity, and capillary pressure.Porosity, density, and CO2 solubility also depend nonlinearly on phase pressure.
  • Numerical assumptions: The simulations omit molecular diffusion and hydrodynamic dispersion, while numerical diffusion and dispersion arise from the two-point upstream algorithm.These numerical effects are intrinsic to the simulations used for neural-network training.
  • Simulation design: ECLIPSE e300 simulates constant-rate super-critical CO2 injection through a vertical well of radius 0.1 m in a radially symmetrical system.The simulator uses finite differences with upstream weighting and adaptive implicit temporal discretization.
  • Dataset construction: The dataset uses 200 radial grid cells and 24 time snapshots to represent gas saturation and pressure buildup fields.Inputs include heterogeneous horizontal and vertical permeability, porosity, perforation maps, and scalar reservoir and injection conditions.

3. Methods

The methods formulate U-FNO as a neural operator mapping discretized input fields and scalar variables to temporally varying gas-saturation and pressure-buildup fields. The architecture combines Fourier operators with U-Net paths, while grid-aware preprocessing, derivative-based relative loss, and active-cell masking address multiphase-flow resolution and heterogeneity.

  • Neural-operator formulation: The operator represents functions on n-point discretizations, mapping field and scalar inputs to temporally varying gas-saturation and pressure-buildup fields.The Fourier operator uses FFT-based Fourier coefficients with truncated modes and has nearly linear complexity.
  • U-FNO architecture: U-FNO lifts inputs with a fully connected network, applies Fourier and U-Fourier layers, then projects the representation to outputs with a fully connected network.Each U-Fourier layer combines a learnable kernel integral transformation, U-Net CNN operator, linear operator, and nonlinear activation.
  • U-FNO architecture: A U-Net path enriches local high-frequency information, and a configuration with half Fourier and half U-Fourier layers performs best for the multiphase-flow problem.The U-Net path is appended to each U-Fourier layer, unlike the original FNO Fourier layer.
  • Input and output representation: Inputs concatenate field variables, scalar variables, spatial-grid channels, and temporal-grid information after logarithmic radial projection onto a uniform (96, 200) grid.Outputs undergo the same radial conversion; pressure buildup is standardized, whereas gas saturation remains unnormalized because it ranges from 0 to 1.
  • Training objective: The relative lp-loss penalizes both outputs and their radial derivatives, while an active-cell mask restricts loss calculation to reservoir cells.Experiments report improved performance over MSE loss and over whole-field loss calculation.

4. Results

U-FNO outperformed original FNO, conv-FNO, and CNN benchmarks in gas saturation and pressure buildup prediction. Its advantages included lower errors, higher R2 scores, improved handling of heterogeneity, and strong performance across complex cases.

  • Model comparison: Four architectures were compared: original FNO, U-FNO, conv-FNO, and a state-of-the-art CNN benchmark, all predicting 3D gas saturation and pressure fields.The models were trained with the proposed loss function and directly output fields of size 96 × 200 × 24 in space and time.
  • Gas saturation: U-FNO achieved the best training and validation performance for gas saturation, while original FNO generalized better than the CNN despite higher training loss.Adding convolution enhanced expressiveness by processing higher-frequency information omitted by truncated Fourier bases.
  • Gas saturation: 46% lower average test-set MPE and R2 plume increased from 0.955 to 0.981 when comparing U-FNO with the benchmark CNN.U-FNO achieved the lowest MPE and highest R2 plume on unseen gas-saturation data.
  • Gas saturation: 70% higher average training–testing MPE difference occurred for CNN than U-FNO, with values of 1.0% and 0.3%, respectively.This comparison indicates less overfitting for U-FNO based on the reported MPE difference.
  • Pressure buildup: 24% reduced test-set average MRE was achieved by U-FNO relative to CNN for pressure buildup, alongside consistently low training and validation errors.U-FNO also produced higher R2 values and narrower 95% prediction bands across examples, despite pressure ranges of approximately 20 to 220 bar.

5. Discussion

U-FNO combines superior data efficiency and accuracy with major computational speedups for multiphase-flow prediction. It performs especially well for front prediction, heterogeneous formations, and unseen temporal resolutions, while its Fourier kernels provide full-field representations.

  • Data utilization efficiency: 3.4 times more training data may be required by CNN than U-FNO to achieve the same gas-saturation performance.U-FNO’s data-utilization efficiency reduces computational resources needed for data generation and training.
  • Front prediction: 2.7 times more accuracy is achieved by U-FNO than CNN for gas-saturation front prediction, and 1.8 times more for pressure front prediction.Gas-saturation and pressure-buildup fronts are important for CO2-storage oversight, monitoring, and history matching.
  • Heterogeneous formations: 1.7 times more accuracy is achieved by U-FNO than CNN in gas saturation for the most heterogeneous geological formations.The comparison selects formations in the testing set with the highest and lowest 10% permeability standard deviations.
  • Computational efficiency: 104 times faster than conventional numerical simulation are gas-saturation and pressure-buildup predictions from all neural-network models.FNO-based models test faster but train slower than CNN; the study prioritizes prediction accuracy and testing time, which U-FNO provides.
  • Unseen temporal resolutions: All models slightly decrease in performance at refined time steps, while FNO-based models still estimate unseen times without additional training.Conv-FNO performs best for both gas saturation and pressure buildup in the unseen-time test.
  • Fourier-kernel interpretation: Fourier kernels are full-field kernels interpretable by any grid discretization, unlike traditional CNN kernels that are generally small.The paper visualizes trained kernels for gas-saturation and pressure-buildup models.

6. Conclusion … Appendix C. Heterogeneous permeability map statistical parameters and visualizations

The paper concludes that U-FNO accurately and efficiently predicts CO2-water multiphase-flow outputs, outperforming the original FNO and CNN benchmark. The appendices document notation, simulation-grid discretization, and heterogeneous permeability-map construction and visualization.

  • 6. Conclusion: U-FNO accurately predicts complex CO2-water multiphase-flow outputs for CO2 geological storage.The conclusion presents U-FNO as an enhanced Fourier neural operator for multiphase-flow problems.
  • 6. Conclusion: U-FNO provides the best gas saturation and pressure buildup prediction performance versus the original FNO and state-of-the-art CNN benchmark.It improves original-FNO training accuracy while maintaining the original FNO’s generalizability.
  • 6. Conclusion: 6 × 104 times faster than a traditional numerical solver, the trained U-FNO predicts gas saturation and pressure buildup for repetitive simulations.The model is proposed as an alternative to full-physics simulators for probabilistic assessment, inversion, and site selection.
  • Appendix A. Table of notations: The appendix provides a table of notation used throughout the paper.The notation table is identified as Table A.6.
  • Appendix B. Grid discretization: Appendix B specifies vertical, radial, and temporal grid discretization for ECLIPSE numerical simulation runs.The radial grid width gradually coarsens according to drmin × aj−1.
  • Appendix B. Grid discretization: The temporal step size gradually coarsens according to dtmin × aj−1 for j ∈[1, ..., it].This temporal discretization complements the radial-grid specification for ECLIPSE runs.
  • Appendix C. Heterogeneous permeability map statistical parameters and visualizations: Appendix C visualizes horizontal permeability, anisotropy, and porosity maps for Gaussian, von Karman, Discontinuous, and Homogeneous media.These visualizations are presented in Figure C.9.
  • Appendix C. Heterogeneous permeability map statistical parameters and visualizations: Table C.8 defines medium appearance, spatial correlation, mean, standard deviation, and contrast ratio for diverse horizontal permeability maps.The maps are generated by Stanford Geostatistical Modeling Software (SGeMS).

Appendix D. Darcy flow comparison

Appendix D compares U-FNO with the original FNO on a steady-state 2D Darcy flow problem. U-FNO uses a 2D U-Net appended to the 2D Fourier layer and achieves lower relative loss than FNO on both training and validation sets.

  • Darcy flow comparison: U-FNO was evaluated against the original FNO on a steady-state Darcy flow problem.The comparison used the steady-state Darcy flow dataset provided in [34].
  • Model configuration: The 2D U-FNO variant appends a 2D U-Net to the 2D Fourier layer.This variant was used because the Darcy flow problem is two-dimensional.
  • Performance comparison: U-FNO achieves lower relative loss than FNO on both the training and validation sets.Figure D.10 reports the relative loss evolution versus epoch for the two architectures.
  • Benchmark comparison: Validation-set relative loss was also compared with four state-of-the-art benchmark models: FCN, PCANN, GNO, and FNO.The benchmark results are listed in Table D.9, with model descriptions and source references provided there.

Appendix E. CNN benchmark model architecture

Appendix E specifies the CNN benchmark architecture and reports its parameter counts. The model uses 3D convolutions, batch normalization, rectified linear layers, identity additions, nearest-neighbor upsampling, and reflection padding.

  • CNN benchmark model architecture: The CNN benchmark contains 33,316,481 total parameters, including 33,305,857 trainable and 10,624 non-trainable parameters.The architecture uses Conv3D, BN, ReLu, Add, UnSampling, and Padding layers.

Appendix F. FNO model architecture

The FNO architecture uses padding, linear lifting and projection, a 3D Fourier operator, bias convolution, addition, and ReLU components, totaling 31,117,541 parameters.

  • Appendix F. FNO model architecture: 31,117,541 total parameters comprise the FNO model architecture.The architecture includes padding for non-periodic boundaries, linear transformations, Fourier3d, Conv1d bias, Add, and ReLu operations.

Appendix G. Conv-FNO model architecture

Appendix G defines the Conv-FNO architecture and explains the operators used to accommodate boundaries, transform features, perform Fourier and convolutional processing, combine outputs, and apply nonlinear activation.

  • Model architecture: Conv-FNO uses Padding, Linear, Fourier3d, Conv1d, Conv3d, Add, and ReLu operations as its principal architectural components.Padding accommodates non-periodic boundaries; Linear lifts inputs to a high-dimensional space and projects outputs back, Fourier3d performs the 3D Fourier operation, Conv1d denotes the bias term, Conv3d performs 3D convolution, Add combines outputs, and ReLu provides rectified-linear activation.

Appendix H. U-FNO model architecture

The U-FNO architecture combines padding, linear transformations, Fourier and convolutional operators, a two-step 3D U-Net, addition, and ReLU layers. The model contains 33,09 total parameters.

  • Architecture components: U-FNO combines Padding, Linear, Fourier3d, Conv1d, UNet3d, Add, and ReLu operations.Padding accommodates non-periodic boundaries; Linear layers lift inputs and project outputs; Fourier3d is a 3D Fourier operator; UNet3d is a two-step 3D U-Net.
  • Model size: 33,09 total parameters are reported for the U-FNO model.

Appendix I. Training, validation, and testing set accuracy

Appendix I summarizes training, validation, and testing dataset performance using average and standard-deviation statistics, including plume and field error metrics and plume-area R2.

  • Table I.14 summarizes performance across the training, validation, and testing datasets.
  • For each metric, µ denotes the average and σ denotes the standard deviation.
  • MPE denotes plume mean absolute error, while MRE denotes field mean relative error as defined in [16].
  • R2_plume denotes the R2 score in the plume area.
Loading 2109.03697v3…