PaperlayerView exact revision on arXiv

Source-linked AI summary

Geometry-aware Latent Autoregressive Generative Model for PDEs in Complex Domains

Zi Wang, Minghui Xu, Tapan Mukerji

arXiv:2609.00297v1cs.LGcs.AI

TL;DR

Multiphysics PDEs in highly irregular meso-scale geometries remain challenging because geometry-sensitive physical information must be modeled efficiently. GeoLAMP uses dual graph encoders, latent flow-matching autoregression, and arbitrary-point decoding; across three datasets, it reports stable long-horizon prediction with low errors and higher long-time autoregressive accuracy than regression-based models.

  • Problem

    Multiphysics PDE processes in complex meso-scale geometries have received relatively limited attention, despite their relevance to macroscopic scientific and engineering behavior.

  • Method

    GeoLAMP combines global and local graph encoders with a causal self-attention transformer trained through flow matching and an arbitrary decoder for latent autoregressive PDE prediction.

  • Results

    GeoLAMP achieves the most stable autoregression across three multiphysics datasets, with the best last-step accuracy across all three datasets and higher long-time accuracy than regression-based models.

  • Takeaways & Limitations

    The study provides a systematic geometry-aware learning framework for multiphysics PDEs in complex 2D geometries and examines block-wise temporal dependencies in latent autoregression.

  • Takeaways & Limitations

    The problem formulation is restricted to K solution variables in complex domains Ω⊆R^d over a finite interval with specified boundary and initial conditions.

Abstract

from arXiv · show

Solving multiphysics partial differential equations (PDEs) remains a major challenge in scientific computing, especially for highly complex $μ$m-scale tortuous geometries critical to energy and chemical engineering. We address this challenge by proposing a Geometry-aware Latent Autoregressive generative Model for PDEs (GeoLAMP) for solving physics within highly irregular and tortuous structures. GeoLAMP introduces a dual-encoder architecture on graph representations to jointly capture global topology and fine-scale geometric features, enabling an effective transition from real-space fields to compact latent representations. In the latent space, we propose a causal self-attention transformer with flow matching to model temporal dynamics, allowing stable and scalable block-wise autoregressive prediction. A flexible decoder reconstructs high-resolution physical fields on arbitrary points. We establish three multiphysics benchmark datasets in complex geometries, covering reactive flow, heat convection, and elasticity. GeoLAMP consistently achieves the most stable autoregression performance on these datasets, maintaining low errors throughout the entire rollout horizon. Our results provide a systematic study of geometry-aware learning for PDEs in $μ$m-scale complex geometries and offer new insights into block-wise time marching of latent autoregressive PDE modeling via a flow matching framework.

1 Introduction

Scientific machine learning has advanced PDE solution methods, but multiphysics in complex meso-scale geometries remains comparatively underexplored. GeoLAMP addresses this gap with geometry-aware latent modeling, benchmark datasets, and stable long-horizon autoregression.

  • Motivation: Multiphysics processes in complex meso-scale structures have received far less attention than widely studied physical problems.These structures can govern macroscopic behavior in applications including natural hydrogen, chip cooling, and geological engineering.
  • Related approaches: Point-cloud and graph-based methods incorporate geometric information while avoiding structured-grid requirements and modeling neighborhood connectivity.Graph representations support local physical interactions and long-range geometry-dependent transport.
  • Motivation: Latent-space methods improve efficiency and flexibility through compression, but complex domains require encoders and decoders that preserve geometry-sensitive physical information.Highly tortuous and heterogeneous meso-scale structures motivate refined geometry-aware designs.
  • GeoLAMP: GeoLAMP combines global and local graph encoders, a causal self-attention transformer, and an arbitrary decoder for latent autoregressive PDE modeling.The encoders use FPS- and CBS-sampled graph pairs, while the decoder reconstructs fields at arbitrary locations and time points.
  • Contributions: The study establishes three meso-scale physics datasets covering reactive flow, heat convection, and elasticity.The contribution is paired with stable autoregressive behavior and low errors throughout the rollout horizon.

2 Problem setup in complex geometry

The paper formulates multiphysics PDE learning for multiple solution variables in complex domains over finite time intervals. The benchmark datasets represent reactive transport, convective heat transfer, and elasticity in distinct meso-scale structures.

  • Problem formulation: The problem involves K solution variables s_K(x, t) in a complex domain Ω⊆R^d over t∈[0,T], subject to boundary and initial conditions.The governing differential operator depends on physical coefficients or source terms.
  • Datasets: The three datasets cover reactive transport in circular-packed porous structures, convective heat transfer in stochastic-field-generated structures, and elasticity in foam structures.All datasets are generated with the finite element method in COMSOL.

3 Geometry-aware latent generative model

GeoLAMP compresses irregular real-space PDE fields into geometry-aware latent representations, models their temporal evolution with flow matching, and decodes predictions at arbitrary physical-space resolutions.

  • Encoder and decoder: The physical information is organized from FEM mesh outputs as point sets, then compressed into latent representations with graph neural operators.Global and local encoders are trained together as variational auto-encoders.
  • Global encoder: Farthest-point sampling selects globally distributed points to cover the spatial domain and avoid local clustering, including inlet-to-outlet trends and boundary nodes.The greedy procedure approximately solves a max–min sampling objective.
  • Local encoder: Curvature-based sampling selects points in pores and highly curved regions, preserving fine-scale geometric details and heterogeneous multiphysics interactions.The local encoder selects points with the largest curvature scores to complement global coverage.
  • Arbitrary decoder: The arbitrary decoder reconstructs generated latent representations at arbitrary points, time steps, and resolutions while masking invalid nodes in variable-size meshes.Full fields can be reconstructed through iterative decoding.
  • Flow matching and autoregressive generation: Flow matching transports Gaussian samples to the latent data distribution, while block-wise prediction applies the objective only to the target latent block during autoregressive rollout.Inference solves a learned probability-flow ODE and repeatedly appends predicted blocks to the conditioning window before decoding.

4 Experiment

GeoLAMP is evaluated on three multiphysics datasets using autoregressive accuracy, geometry representations, super-resolution, and inference-time block flexibility. It achieves strong long-horizon stability while showing that global–local point selection and larger committed blocks improve performance or efficiency.

  • Model accuracy: GeoLAMP achieves the best last-step prediction accuracy across all three datasets, demonstrating effective long-horizon forecasting.Transolver attains lower average rollout error on elasticity, but GeoLAMP has similar average and last-step errors there.
  • Model accuracy: GeoLAMP maintains more controlled rollout error growth than representative baselines, with no significant increase over time in both prediction paradigms.Figure 3 reports mean relative L2 error over test trajectories with 10th–90th percentile bands.
  • Effects of geometry representation: The global–local dual encoder improves accuracy by more than 20% on the two flow–transport datasets versus random point sampling, while remaining comparable on elasticity.The comparison isolates the effect of FPS- and curvature-based point selection for latent encoding.
  • Effects of geometry representation: The arbitrary-point decoder enables reconstruction on meshes of arbitrary resolution, and global–local sampling improves elasticity super-resolution by about 33%.Global–local points improve both sampled-location accuracy and global super-resolution performance.
  • Rollout commit step and block length flexibility: Larger committed blocks generally reduce rollout error and time-marching cost, especially suppressing late-stage elasticity error caused by small commit strides.Increasing the commit stride reduces the number of autoregressive updates linearly.
  • Rollout commit step and block length flexibility: At C = 8 and P = 16, model evaluations and wall-clock rollout time decrease by 43–50% and 14–25%, respectively, while peak batch memory rises by 12–16%.The normalized error is 1.01 for heat convection, 0.977 for elasticity, and 1.42 for reactive flow; most useful history is captured by C = 6–7.

5 Related Work

PDE learning in complex geometries must represent irregular, unstructured domains while preserving geometry-sensitive physical information. Latent generative approaches address efficiency by modeling temporal evolution in compact latent spaces.

  • Latent generative modeling: Latent-space methods improve efficiency and flexibility by compressing real-space data before modeling temporal evolution.For complex domains, effective latent modeling requires geometry-aware backbones that preserve real-space information during compression.
  • Latent generative modeling: Diffusion-based generative models provide a framework for stable temporal prediction, while direct real-space application remains expensive for high-resolution PDEs.Latent generative modeling addresses this cost by learning dynamics in a lower-dimensional space.
  • Geometry-aware representations: Point-cloud and graph-based methods represent irregular PDE domains without relying on structured grids, with graphs adding explicit neighborhood connectivity.Graph representations support local physical interactions and geometry-dependent transport through connectivity-aware message passing or operators.
  • Geometry-aware representations: The central challenge is compressing unstructured complex-domain data into latent variables without losing geometry-sensitive physical information.Point-cloud encoders aggregate local-neighborhood features while progressively enlarging their receptive fields.

6 Conclusions

GeoLAMP combines dual graph encoders, flow-matching causal attention, and block-wise autoregression for multiphysics PDE learning in complex 2D geometries. It performs accurately over long rollouts, but reconstruction quality, training cost, and 3D scope remain open challenges.

  • Conclusions: GeoLAMP targets multiphysics PDE learning in meso-scale complex 2D geometries using a dual encoder for global physical behavior and local geometric features.The model connects real-space physics with latent representations while preserving geometric detail.
  • Conclusions: GeoLAMP uses a causal self-attention transformer with flow matching for block-wise autoregressive prediction.The reported block-wise attention analysis indicates temporal dependencies beyond simple one-step correlations.
  • Conclusions: Three complex-geometry datasets cover heat convection, reactive flow, and elasticity.These datasets support multiphysics evaluation across distinct physical processes and structures.
  • Conclusions: GeoLAMP achieves higher long-time autoregressive accuracy than regression-based models while flexibly representing geometries and reconstructing fields at different resolutions.The conclusion also reports physically meaningful temporal dependencies in block-wise attention analysis.
  • Limitations and future work: The method still struggles to reconstruct high-quality fields in highly complex geometries, incurs higher training cost with larger blocks, and is currently scoped to meso-scale multiphysics in 2D.The authors identify efficient attention mechanisms and extension to 3D as future directions.

A Dataset Generation

The study constructs three meso-scale datasets by generating distinct complex geometries and solving coupled multiphysics PDEs. The datasets cover heat convection, reactive flow, and elasticity in heterogeneous porous or foam structures.

  • Dataset generation: Each of the three datasets contains 2,000 generated structures with corresponding spatio-temporal PDE solution sequences.Datasets are split into 1,500 training, 300 validation, and 200 test sequences, with simulations implemented in COMSOL.
  • Heat convection: Heat convection uses a stochastic-field-generated heterogeneous porous structure in a 24,000 × 24,000 µm2 domain.The incompressible Navier–Stokes equations are coupled with energy transport, with water as the working fluid.
  • Reactive flow: Reactive flow uses randomly initialized and iteratively adjusted spheres of radius 14 µm in a 240 × 240 µm2 sphere-packed domain.The coupled equations describe incompressible flow and solute transport with reactants advected between inlet and outlet and consumed at the solid interface.
  • Elasticity: Elasticity uses a foam-reconstructed geometry in a 2,400 × 2,400 µm2 domain with copper as the solid material.The mechanical response follows infinitesimal linear elasticity under dynamic loading conditions.

B.1 Model hyperparameter

The hyperparameter configurations define geometry encoders, latent decoders, transformer variants, and matched baselines around a 64 × 8 × 8 latent representation. GeoLAMP-B performs non-overlapping eight-step block prediction, while comparison models use aligned evaluation protocols where specified.

  • Autoencoder: The Global & Local VAE compresses each scalar field into a 64 × 8 × 8 latent representation using two independent graph-neural-operator encoders.The encoder design supports separate global and local processing of geometry-aware inputs.
  • Decoding: The super-resolution decoder upsamples predicted latents through CNN and graph-neural-operator backends to reconstruct fields on the full mesh.The decoder produces a 32 × 32 × 128 feature grid before graph-based mapping to mesh points.
  • GeoLAMP variants: GeoLAMP-S uses a 12-block transformer with width 768 and 12 attention heads for flow-matching denoising of target latents.Its conditioning combines flow-matching time and autoregressive step embeddings through AdaLN-Zero.
  • GeoLAMP variants: GeoLAMP-B shares GeoLAMP-S settings but predicts M = 8 future latent velocity fields in one flow-matching evaluation.It uses non-overlapping block-wise prediction with causal token sequences.
  • Baselines: Geo-FNO and NUNO use geometry-aware grid-based operator strategies, while OFormer and Transolver operate directly on 4096 real-space points.Geo-FNO and NUNO follow blockwise M-to-M prediction with M = 8 under matched evaluation settings.
  • Baselines: Latent-Det. matches GeoLAMP-B’s latent architecture and protocol but replaces flow matching and iterative sampling with direct MSE regression and one forward pass per block.The control isolates the effect of the generative formulation under matched settings.

B.2 Training settings

Training settings specify dataset-dependent inputs and separate optimization of the VAE, latent dynamics model, and arbitrary decoder.

  • Heat convection and reactive flow use the velocity field as optional input context.
  • Elasticity uses only the von Mises stress field as model input.
  • The VAE and latent dynamics model are trained separately.
  • The arbitrary decoder is trained independently with partial-point latent vectors as input and full-mesh data as the target.
  • Table 4 summarizes training hyperparameters for baseline models, GeoLAMP variants, the VAE, and the arbitrary decoder.
Loading 2609.00297v1…