PaperlayerView exact revision on arXiv

Source-linked AI summary

PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs

Pu Ren, Chengping Rao, Yang Liu, Jianxun Wang, Hao Sun

arXiv:2106.14103v1cs.LGcs.CLmath.NA

TL;DR

Existing PINN approaches face limitations in low-dimensional spatiotemporal parameterizations and depend on penalty-based enforcement of initial and boundary conditions. The paper introduces PhyCRNet and PhyCRNet-s, which use convolutional-recurrent architectures, hard-encoded conditions, and residual-based unsupervised training. Across nonlinear PDE examples, the methods demonstrate strong solution accuracy, extrapolability, and generalizability.

  • Problem

    Existing fully connected PINN methods are limited for low-dimensional spatiotemporal parameterizations, while soft initial/boundary-condition enforcement requires hyperparameter tuning.

  • Method

    PhyCRNet and PhyCRNet-s use encoder-decoder ConvLSTM architectures, autoregressive and residual connections, discretized PDE residual losses, and hard-encoded initial/boundary conditions without labeled data.

  • Results

    The proposed methods demonstrate superiority in solution accuracy, extrapolability, and generalizability across three nonlinear PDE systems.

  • Takeaways & Limitations

    PhyCRNet and PhyCRNet-s provide physics-informed surrogate-modeling architectures for spatiotemporal PDEs and show potential for data assimilation and inverse problems.

  • Takeaways & Limitations

    The frameworks rely on prescribed spatiotemporal meshes and are limited to regular grids; PhyCRNet-s skipping can also accumulate temporal-propagation error.

Abstract

from arXiv · show

Partial differential equations (PDEs) play a fundamental role in modeling and simulating problems across a wide range of disciplines. Recent advances in deep learning have shown the great potential of physics-informed neural networks (PINNs) to solve PDEs as a basis for data-driven modeling and inverse analysis. However, the majority of existing PINN methods, based on fully-connected NNs, pose intrinsic limitations to low-dimensional spatiotemporal parameterizations. Moreover, since the initial/boundary conditions (I/BCs) are softly imposed via penalty, the solution quality heavily relies on hyperparameter tuning. To this end, we propose the novel physics-informed convolutional-recurrent learning architectures (PhyCRNet and PhyCRNet-s) for solving PDEs without any labeled data. Specifically, an encoder-decoder convolutional long short-term memory network is proposed for low-dimensional spatial feature extraction and temporal evolution learning. The loss function is defined as the aggregated discretized PDE residuals, while the I/BCs are hard-encoded in the network to ensure forcible satisfaction (e.g., periodic boundary padding). The networks are further enhanced by autoregressive and residual connections that explicitly simulate time marching. The performance of our proposed methods has been assessed by solving three nonlinear PDEs (e.g., 2D Burgers' equations, the $λ$-$ω$ and FitzHugh Nagumo reaction-diffusion equations), and compared against the start-of-the-art baseline algorithms. The numerical results demonstrate the superiority of our proposed methodology in the context of solution accuracy, extrapolability and generalizability.

1. Introduction

The paper proposes PhyCRNet and PhyCRNet-s, physics-informed convolutional-recurrent architectures for solving multidimensional spatiotemporal PDEs without labeled data. The approach targets limitations of existing methods through convolutional feature extraction, temporal learning, residual time marching, and hard enforcement of initial/boundary conditions.

  • Existing fully connected PINN approaches have intrinsic limitations for low-dimensional spatiotemporal parameterizations and require careful treatment of initial and boundary conditions.
  • Physics-informed discrete learning schemes using CNNs have shown better scalability and faster convergence in pilot studies.
  • PhyCRNet and PhyCRNet-s solve multidimensional spatiotemporal PDEs without labeled data.
  • The architectures combine encoder-decoder ConvLSTM features, global residual connections, and high-order finite-difference filtering for PDE residual construction.
  • The paper evaluates the proposed methods on nonlinear PDEs and reports superiority in solution accuracy, extrapolability, and generalizability.

2. Problem Statement

The paper formulates forward solution of nonlinear, coupled, multidimensional spatiotemporal PDE systems under specified initial and boundary conditions. Training is unsupervised: a neural network approximates the discretized solution by minimizing PDE residuals while respecting those conditions.

  • The problem concerns multidimensional nonlinear coupled PDE systems with solution variables defined over temporal and spatial domains.
  • The PDE formulation includes a first-order time derivative, spatial gradients, a nonlinear functional parameterized by λ, and initial and boundary conditions.
  • The objective is forward analysis for specified initial and boundary conditions, with potential use as a basis for inverse problems when data are available.
  • Training is unsupervised and uses physical laws, including the PDE and initial/boundary conditions, rather than labeled data.
  • The network approximates the discrete solution field and is optimized by minimizing discrete PDE residuals subject to initial and boundary conditions.

3. Methodology

PhyCRNet uses convolutional-recurrent architectures to extract low-dimensional spatial features, learn temporal evolution, and solve discretized PDEs while hard-encoding initial and boundary conditions. PhyCRNet-s reduces computation by periodically skipping the encoder, trading efficiency against possible temporal-propagation error.

  • ConvLSTM: ConvLSTM replaces fully connected gated operations with convolutions to model spatial connections while retaining LSTM cells and gates for temporal information flow.Its hidden and cell states support sequence-to-sequence temporal modeling.
  • PhyCRNet: PhyCRNet combines an encoder-decoder, ConvLSTM temporal propagator, residual connection, autoregressive process, and filtering-based differentiation.The encoder learns low-dimensional latent features, while ConvLSTM evolves them before reconstruction.
  • PhyCRNet: Residual and autoregressive connections propagate each predicted state into the next time step, explicitly extending the network’s temporal evolution.The input-output flow follows a forward-Euler-inspired update and AR(1) process.
  • PhyCRNet: Gradient-free convolutional filters approximate temporal and spatial derivatives, while boundary padding preserves differential information at domain boundaries.The method uses finite-difference-based filters and designated padding mechanisms such as periodic padding.
  • PhyCRNet-s: PhyCRNet-s periodically skips the encoder and passes low-resolution ConvLSTM features directly between steps to reduce redundant computation.The skipping-encoder cycle is controlled by T, with high-resolution inputs used only at selected steps.
  • PhyCRNet-s: Skipping the encoder makes PhyCRNet-s lighter but may introduce temporal-propagation approximation error, motivating relatively small cycle values.The tradeoff between accuracy and efficiency is discussed empirically in the experiments.
  • Hard imposition of I/BCs: Hard imposition encodes initial and boundary conditions directly into the network, while training minimizes aggregated discretized PDE residuals.Periodic padding handles the periodic Dirichlet boundary conditions considered in this work.

4. Numerical experiments

The experiments evaluate PhyCRNet and PhyCRNet-s on nonlinear PDE systems and compare them with vanilla PINN and AR-DenseED baselines. Accuracy is assessed through full-field error propagation during both training and extrapolation using the accumulative root-mean-square error.

  • Experimental design: The evaluation covers 2D Burgers’ equations and λ-ω and FitzHugh-Nagumo reaction-diffusion systems with periodic boundary conditions.The study compares PhyCRNet and PhyCRNet-s against vanilla PINN and AR-DenseED.
  • Network settings: The same PhyCRNet and PhyCRNet-s network setting is used across all PDE cases, including periodic padding in convolutional operations.The architecture includes three convolutional encoder layers and a ConvLSTM layer in latent space.
  • Baselines: Baseline training uses fully connected PINN layers with collocation-point losses and AR-DenseED settings based on its open-source implementation.The baselines receive comparable architecture and training effort for fair comparison.
  • Evaluation metric: Full-field error propagation is evaluated during both training and extrapolation using accumulative root-mean-square error.The metric aggregates root-mean-square error over time steps against a reference solution.

4.2. 2D Burgers’ equations

PhyCRNet accurately solves the 2D Burgers’ equations during training and extrapolation, whereas PINN fails to match the reference, especially near boundaries.

  • Comparison setup: Figure 5 compares ground-truth differences across representative training times t = 1.0, 2.0 and extrapolation times t = 3.0, 4.0.Err (ours) and Err (PINN) measure the difference over the entire domain.
  • Solution accuracy and extrapolation: PhyCRNet captures the reference solution in both training and extrapolation, while PINN produces substantially larger errors, particularly at the boundaries.The boundary errors are attributed to PINN’s soft imposition of initial and boundary conditions.
  • Error propagation: a-RMSE for PhyCRNet remains below 0.01 throughout the training and extrapolation phases.
  • Error propagation: PhyCRNet performs similarly to PINN during training but leads during extrapolation and outperforms AR-DenseED by two orders of magnitude in both phases.

4.3. λ-ω RD equations

For the λ-ω reaction-diffusion system, both methods fit simple periodic patterns during training, but PhyCRNet extrapolates more robustly with substantially smaller errors.

  • Comparison setup: Figure 6 evaluates solution differences at training times t = 2.5, 5.0 and extrapolation times t = 7.5, 10.0.Err(ours) and Err(PINN) compare entire-domain differences from the ground truth.
  • Training behavior: Both PhyCRNet and PINN produce good training results for the system’s simple, smooth periodic patterns.The passage associates PINN’s training performance with continuous approximation of these patterns.
  • Extrapolation behavior: PINN produces large boundary errors and does not extrapolate well, whereas PhyCRNet maintains much smaller distributed errors during extrapolation.
  • Error propagation: PhyCRNet achieves up to one order of magnitude smaller error than PINN and AR-DenseED in error-propagation comparisons.

4.4. FitzHugh-Nagumo RD equations

PhyCRNet accurately reproduces the complex FitzHugh-Nagumo dynamics in training and extrapolation, while PINN captures only parts of the patterns and accumulates much larger errors.

  • Comparison setup: Figure 7 compares entire-domain errors at training times t = 2.16, 4.32 and extrapolation times t = 6.48, 8.64.
  • Solution accuracy: PhyCRNet shows outstanding agreement with the ground truth in both training and extrapolation, whereas PINN can barely simulate parts of the dynamical patterns.
  • Error structure: PhyCRNet error maps are nearly zero, with larger errors mainly concentrated on propagating wave fronts and especially small errors for the field variable v.The passage characterizes these wave-front errors as negligible given the system’s complex patterns.
  • Error propagation: PhyCRNet surpasses PINN and AR-DenseED by up to two orders of magnitude, while its a-RMSE remains near 10^-2 and the baselines rise to magnitude 1.
  • Overall outcome: The results identify solution accuracy and extrapolation capability as salient outcomes of PhyCRNet’s evaluation.
  • Cross-system comparison: Across the three PDE systems, Figure 8 separates error propagation into training and extrapolation phases using blue and orange regions.

4.5. Generalization to different ICs

PhyCRNet generalizes across four randomly sampled initial conditions and longer temporal evolutions, maintaining accurate patterns and bounded error propagation.

  • Evaluation setup: PhyCRNet is evaluated on four initial conditions randomly sampled from a Gaussian distribution with mean 0 and standard deviation 0.1.
  • Generalization results: Across 4500 time instants spanning [0, 27], predictions capture evolutionary patterns and local details for both field variables u and v.
  • Error propagation: Error variances increase smoothly and slowly while remaining below 0.04 across the different initial-condition scenarios.
  • Overall conclusion: The experiments suggest that PhyCRNet learns the underlying physical laws and generalizes robustly to different initial conditions.

4.6. Ablation study

The ablation study evaluates how autoregressive and global residual connections contribute to PhyCRNet performance on 2D Burgers’ equations.

  • The study compares full PhyCRNet, a version without the autoregressive scheme, and a version without the global residual connection.The encoder-decoder is retained across architectures, while ConvLSTM, residual connection, input-output autoregressive scheme, and filtering-based differentiation form the remaining components.

4.7. Comparison between PhyCRNet and PhyCRNet-s

PhyCRNet and PhyCRNet-s differ through the skip-encoder parameter T, creating an accuracy–efficiency tradeoff. Smaller T improves training accuracy, while a reasonably small T supports extrapolation robustness.

  • Architecture flexibility: PhyCRNet is the T = 0 special case of PhyCRNet-s, whose skip-encoder strategy varies T to balance accuracy and efficiency.The architectures differ through the skip-encoder parameter T.
  • Architecture flexibility: Four Burgers’ equation architectures use T = 0, 10, 50, and 100 under identical training procedures.The experiments keep other neural components fixed while varying T.
  • Generalization evaluation: Four representative time instants—t = 4.32, 8.64, 12.96, and 17.28—are used to assess generalization for four testing initial conditions of FitzHugh Nagumo reaction-diffusion equations.The figure evaluates generalization across testing initial conditions and selected temporal snapshots.
  • Performance tradeoff: As T increases, computational time decreases mildly while training and extrapolation errors grow; PhyCRNet leads training accuracy, but T = 10 performs better in extrapolation.The results support a reasonably small T as a balance between accuracy, efficiency, and extrapolation robustness.
  • Evaluation overview: Figure 11 combines error propagation across initial conditions, neural-architecture ablations, and comparisons of PhyCRNet with PhyCRNet-s across T values.The plotted PhyCRNet-s settings are T = 10, 50, and 100, while PhyCRNet uses T = 0.

5. Discussion

The discussion contrasts discrete PhyCRNet methods with continuous PINNs in mesh requirements and initial/boundary-condition enforcement. Their principal limitation is reliance on prescribed regular grids.

  • Comparison framework: The comparison is organized around two aspects: mesh requirements and the encoding of initial and boundary conditions.These dimensions frame the discussion of continuous and discrete physics-informed learning.
  • Mesh requirements: PhyCRNet and PhyCRNet-s require prescribed spatiotemporal meshes and are limited to regular grids, unlike meshfree PINNs.The authors identify geometric learning methods such as graph neural networks as a possible extension to irregular meshes.
  • Learning characteristics: Convolutional kernels focus discrete methods on local physical-domain features, whereas PINNs use global basis-function approximation to capture general dynamical patterns.The discussion presents this as a distinction between local morphology and global learning.
  • I/BC enforcement: Soft I/BC penalties in PINNs require extensive hyperparameter tuning and may not guarantee boundary accuracy or generalization across initial conditions.The discrete methods instead incorporate boundary values through intrinsic padding and use the initial condition as the network’s first input state.

6. Conclusion

The conclusion presents PhyCRNet and PhyCRNet-s as physics-informed architectures for spatiotemporal PDEs, emphasizing hard I/BC enforcement and performance across accuracy, extrapolation, and generalization.

  • Hard enforcement of I/BCs through designated padding promotes a well-posed training optimization problem, solution accuracy, and convergence.The conclusion identifies hard encoding as a central design choice.
  • The proposed methods are validated for solution accuracy, extrapolability, and generalizability and are intended as universal models for spatiotemporal PDEs.The conclusion frames the networks as potential surrogate models for scientific computation.
  • The networks are positioned as potential surrogate models for data assimilation and inverse analysis when physical-system data are scarce and noisy.The authors state that this application will be demonstrated in future work.
  • Future extensions include graph neural networks for irregular spatial domains and high-order difference schemes such as Runge-Kutta for temporal evolution.These modifications are proposed to broaden spatial applicability and improve temporal modeling accuracy.
Loading 2106.14103v1…