PaperlayerView exact revision on arXiv

Source-linked AI summary

Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domains

Shizheng Wen, Arsh Kumbhat, Levi Lingsch, Sepehr Mousavi, Yizhou Zhao, Praveen Chandrashekar, Siddhartha Mishra

arXiv:2505.18781v4cs.LG

TL;DR

Learning PDE solution operators on arbitrary domains requires balancing accuracy with computational efficiency. GAOT combines geometry-aware graph neural operator encoders and decoders with geometry embeddings and transformer processors, and is reported to perform well across diverse PDE tasks and large-scale industrial CFD benchmarks.

  • Problem

    PDE surrogates are needed for computationally expensive repeated simulations, while existing operator-learning models can trade off accuracy against computational efficiency and scalability.

    Source: §1 · pp. 1–2

  • Method

    GAOT maps arbitrary-domain inputs to PDE solutions using multiscale attentional graph neural operator encoders and decoders, geometry embeddings, and a transformer processor.

    Source: §1 · p. 2, §2 · p. 5

  • Results

    Across diverse PDE datasets, GAOT is reported more accurate, robust, and computationally efficient; it achieves state-of-the-art performance on DrivAerNet++ and superior performance to GINO on two further 3D industrial datasets.

    Source: §1 · pp. 2–3, §4 · p. 10

  • Takeaways & Limitations

    The reported efficiency and scalability make GAOT relevant to large-scale industrial CFD simulations.

    Source: §1 · p. 2

Abstract

from arXiv · show

The very challenging task of learning solution operators of PDEs on arbitrary domains accurately and efficiently is of vital importance to engineering and industrial simulations. Despite the existence of many operator learning algorithms to approximate such PDEs, we find that accurate models are not necessarily computationally efficient and vice versa. We address this issue by proposing a geometry aware operator transformer (GAOT) for learning PDEs on arbitrary domains. GAOT combines novel multiscale attentional graph neural operator encoders and decoders, together with geometry embeddings and (vision) transformer processors to accurately map information about the domain and the inputs into a robust approximation of the PDE solution. Multiple innovations in the implementation of GAOT also ensure computational efficiency and scalability. We demonstrate this significant gain in both accuracy and efficiency of GAOT over several baselines on a large number of learning tasks from a diverse set of PDEs, including achieving state of the art performance on three large scale three-dimensional industrial CFD datasets.

1 Introduction

Learning PDE solution operators on arbitrary domains is important for fast simulation, but existing approaches face an accuracy–efficiency trade-off. GAOT combines geometry-aware graph neural operators and a transformer processor to target accurate, efficient, scalable surrogates.

1 Introduction

  • Repeated numerical PDE solves make many-query problems such as uncertainty quantification, control, and inverse problems computationally expensive, motivating fast surrogates.Conventional finite element and finite difference methods are described as the preferred simulation framework.Source: §1 · p. 1
  • Many neural operators learn solution operators from data but are restricted to Cartesian grids, motivating methods for engineering and industrial PDEs on complex arbitrary domains.Existing approaches include modified spectral methods, latent-grid encoder/decoder models, graph neural networks, and transformer models.Source: §1 · pp. 1–2
  • The authors identify an accuracy–efficiency trade-off: RIGNO is accurate and robust but not necessarily efficient or scalable, while GINO [29] is more efficient but insufficiently accurate.Their goal is an accurate, computationally efficient, scalable operator for real-world industrial simulations.Source: §1 · p. 2
  • GAOT combines multiscale attentional graph neural operator encoding and decoding, geometry embeddings, and a patched transformer processor, with implementation strategies for efficiency and scalability.The decoder can generate neural fields and approximate solutions at arbitrary query points in the domain.Source: §1 · p. 2
  • GAOT is evaluated on 28 diverse PDE benchmarks and three industrial 3D datasets, with reported accuracy, efficiency, and scalability and state-of-the-art DrivAerNet++ performance.The paper also reports superior performance to GINO on the DrivaerML and NASA-CRM datasets.Source: §1 · pp. 2–3, Figure 1 · p. 2
  • The paper uses extensive ablations to assess how multiscale attentional encoders and geometry embeddings contribute to GAOT’s overall performance.Source: §1 · p. 3

2 Methods.

The method learns PDE solution operators from discretized spatial observations, with time-dependent problems also using discrete time snapshots. GAOT combines geometry-aware graph-operator encoders and decoders with transformer processing to map PDE inputs to solutions.

Problem Formulation

  • Stationary inputs combine coefficients, forcing, boundary values, and domain geometry; time-dependent inputs combine coefficients, initial state, and geometry, with data discretized over space and time.Source: §2 · p. 3

GAOT Model Architecture

  • GAOT encodes point-cloud inputs into latent tokens, processes them, then decodes solutions at query points; latent domains may be regular grids, downsampled clouds, or projected low-dimensional grids.Source: §2 · p. 3, Figure 2 · p. 4

MAGNO

  • MAGNO combines radius-specific attentional graph-operator features using token-dependent softmax weights rather than summing scale contributions uniformly.Source: §2 · pp. 4–5

Geometry Embeddings

  • Geometry embeddings add neighborhood statistics—including neighbor counts, distance moments, and PCA-based shape anisotropy—through normalized descriptors mapped by an MLP.Source: §2 · pp. 4–5

Transformer Processor

  • For grid-like latent points, GAOT uses patch-based vision transformers; randomly downsampled points use a standard transformer, with RMS normalization and skip connections.Source: §2 · p. 5

Decoder

  • The reverse MAGNO decoder combines attentional features and neighborhood geometry embeddings around each query point to predict the solution field.Source: §2 · p. 5

Extension to time-dependent problems

  • For time-dependent PDEs, GAOT adds current time and lead time as inputs and supports direct prediction, residual, or time-derivative stepping strategies.The method also uses all2all training to leverage trajectory data.Source: §2 · pp. 5–6

Efficient implementation

  • To reduce memory use on varying geometries, GAOT precomputes and stores graphs, processes encoder and decoder inputs sequentially, and can drop edges for very large datasets.Source: §2 · p. 6

3 Results.

Across PDE benchmarks on arbitrary domains and large-scale industrial CFD datasets, GAOT combines strong accuracy and robustness with high computational efficiency and scalability.

Accuracy and Robustness

  • On 14 datasets, GAOT is best on 10 and second-best on 4; it is almost 50% more accurate than RIGNO-18 on time-independent tasks and slightly more accurate on time-dependent tasks.GAOT's accuracy remains uniformly good across datasets, while baselines show significantly poorer performance on outlier datasets. GAOT is almost three times more robust than RIGNO-18 on time-independent datasets, and they are equally robust on time-dependent datasets.Source: §3 · p. 7

Computational Efficiency and Scalability

  • GAOT has the highest training throughput and fastest inference latency, nearly 50% and 15% better, respectively, than GINO, and scales more favorably with input and model size.Transolver and GNOT scale poorly enough that the authors could not train them on the large-scale time-dependent datasets with all2all training.Source: Table 2 · p. 7, §3 · pp. 7–8, Figure 3 · p. 7

Industrial scale 3D datasets

  • On DrivAerNet++, GAOT is about 30% more accurate than TripNet in wall-shear-stress MAE; it is almost twice as accurate as GINO on DrivAerML wall shear stress and outperforms GINO on NASA CRM.A neural-field GAOT trained on less than 10% of input points has accuracy comparable to some baselines trained with 10x more points. Full-resolution GAOT is almost twice as accurate as competing models trained at low resolution and inferred at high resolution.Source: Table 3 · p. 8, §3 · pp. 8–9, Table 4 · p. 9

Generality, Generalization and Scaling

  • GAOT handles regular-grid inputs and generalizes across resolutions, including to Poisson-Gauss grids with 10x more input points than training.On seven regular-grid datasets, GAOT is more accurate or comparable to RIGNO and more accurate than FNO and CNO. On unseen bluff-body shapes, fine-tuning gives almost an order-of-magnitude accuracy gain over training from scratch; GAOT also scales with model and dataset size.Source: Figure 3 · p. 7, §3 · pp. 9–10

Ablation studies

  • Ablations support the MAGNO encoder/decoder, regular equispaced latent points, time-derivative marching, statistical geometric embeddings, and multiscale features as contributors to GAOT's accuracy.MAGNO outperforms message-passing encoders/decoders; a regular equispaced latent cloud beats downsampling or a projected low-dimensional grid, and performance is robust to latent-grid size. Setting γ = 1 and δ = τ is superior to other marching choices; statistical geometric embeddings beat omitting geometry or using PointNet, while multiscale MAGNO features improve on a single-scale GNO and MAGNO works in tandem with the ViT processor.Source: §3 · p. 10

4 Discussion

GAOT combines multiscale graph neural operator encoders and decoders, geometry embeddings, and a transformer processor to learn PDE solutions on arbitrary domains. The discussion reports accuracy and efficiency gains across diverse benchmarks and inference 4 to 9 orders of magnitude faster than classical PDE solvers while retaining accuracy.

Summary

  • 4 to 9 orders of magnitude faster than classical PDE solvers, GAOT retains accuracy in the reported inference-time comparison.The speedup is quantified in Supplementary Material E.12.Source: §4 · p. 10
  • Across diverse two-dimensional PDE datasets, GAOT is reported as more accurate, robust, and computationally efficient than a large set of baselines, with SOTA performance on three industrial 3D datasets.The 3D datasets concern automobile and aerospace simulations.Source: §4 · p. 10
  • GAOT combines a multiscale GNO encoder/decoder, local-geometry statistical embeddings, and a vision-transformer processor to map point-cloud inputs to outputs at arbitrary domain query points.The implementation also uses strategies intended to improve computational efficiency and scalability.Source: §4 · p. 10

Related Work

  • Unlike RIGNO, Transolver, and GINO, GAOT avoids graph-based message passing, uses transformers only in the processor, and augments its GNO encoder/decoder with multiscale features, attention-based quadrature, and geometry embeddings.The discussion attributes GAOT’s reported accuracy and efficiency gains over these models to these design choices and its efficient implementation.Source: §4 · p. 10

Supplementary Material for: Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on · A Details of Problem Formulation. · B Details of GAOT Architecture

The supplement formalizes operator-learning tasks for time-dependent and steady-state PDEs, then outlines GAOT’s architecture components. It frames learning as approximating continuous solution operators from discretized data while supporting predictions at arbitrary query points.

Supplementary Material for: Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on

A.1 General Forms of PDEs

  • Time-dependent PDEs describe a solution u(t, x) over a spatial domain and time interval, with a differential operator, known parameter, boundary values, and initial condition.Source: §A.1 · p. 17
  • Time-independent PDEs define steady-state solutions from parameters and source terms, and can be viewed in some cases as long-time limits of time-dependent PDEs.Source: §A.1 · p. 17

A.2 Solution Operators for PDEs

  • The time-dependent solution operator maps initial data and parameters to solutions; time-shifted operators predict future solutions from current snapshots, while the steady-state operator maps parameters to boundary-value solutions.Source: §A.2 · pp. 17–18
  • The operator-learning task is to approximate the true solution operator from data distributed over initial or boundary conditions and parameters, avoiding repeated high-fidelity solver use.Source: §A.2 · p. 18

A.3 Discretizations.

  • Discretized input-output data must support learning continuous space-time solution operators, including when the domain is an input, and predictions at arbitrary spatial query points.Source: §A.3 · p. 18

B Details of GAOT Architecture

  • GAOT’s architecture comprises a latent token grid, Multiscale Attentional Graph Neural Operator encoder and decoder, geometry embeddings, and a transformer-based processor.Source: B Details of GAOT Architecture · p. 18

B.1 Choice of Latent Grid … Considered Hyperparameters

This section describes GAOT’s architecture and graph construction, including its tokenization, multiscale geometry-aware operators, and Transformer processor. It also specifies training and inference procedures, model configuration, and the baselines and experimental settings used for comparison.

B.1 Choice of Latent Grid

  • GAOT considers structured grids, downsampled unstructured points, and projected low-dimensional grids, but adopts structured stencil grids for robustness and simplicity despite empty tokens and dimensional cost.Structured grids support patching; downsampling adapts token allocation but is harder to patch and can underperform, while projection reduces token count but may lose local information and accuracy.Source: §B.1 · pp. 19–20

B.2 Multiscale Attentional Graph Neural Operator

  • MAGNO extends local GNO integration across multiple radii, gathering neighborhood information at each scale to capture multiscale interactions important to PDEs.The paper shares MLP parameters across scales.Source: §B.2 · p. 20

B.2.1 Attentional Weighting in Local Integration (AGNO)

  • AGNO uses learnable query and key matrices to assign each neighbor a contribution weight based on its relationship to the latent point.This attention-based choice of quadrature weights is applied at each scale.Source: §B.2.1 · p. 20

B.2.2 Attentional Fusion of Multiscale Features

  • MAGNO uses a coordinate-conditioned MLP to learn attention weights for fusing scale-specific features, producing an adaptively weighted multiscale representation in both encoder and decoder.This replaces simple summation or concatenation.Source: §B.2.2 · p. 21

B.3 Geometry Embeddings

  • GAOT adds local statistical or PointNet-based geometry embeddings because coordinates alone may not capture domain shape, while node-level feature merging and SDFs have practical drawbacks.Statistical descriptors include neighborhood density, distances, centroid offset, and PCA shape features; the PointNet-style alternative uses shared pointwise processing and mean pooling.Source: §B.3 · pp. 21–22

B.3.1 Integration of Geometry Embeddings in MAGNO

  • MAGNO concatenates each scale’s AGNO features with its geometry embedding before scale-attention aggregation, giving tokens scale-adapted shape cues and modular, multiscale representations.The approach is particularly useful for complex boundaries where multiple length scales and shape cues matter.Source: §B.3.1 · p. 23

B.4 Processor

  • The Transformer processor patches regular-grid tokens but feeds downsampled unstructured tokens individually, then enables global token dependencies using relative positional information.The processed tokens are passed to the MAGNO decoder.Source: §B.4 · pp. 23–24

B.5 Graph Building Tricks

  • For 3D industrial data, GAOT combines a latent-centered radius graph for locality with a physical-point-centered KNN graph that attaches every physical point to latent space.This bidirectional construction addresses the edge-count versus domain-coverage trade-off of a single global radius.Source: §B.5 · p. 24
  • GAOT uses radius graphs for 2D benchmarks and bidirectional radius-plus-KNN graphs for large 3D datasets to preserve coverage while controlling edge growth.The merged bidirectional graph uses tunable k and radius r; graphs are precomputed and cached, and edge masking is also applied during training.Source: Figure B.3 · p. 25, §B.5 · p. 25

B.6.1 Loss Function

  • Training minimizes MSE between predictions and physical targets, with time-dependent outputs formed by output, residual, or time-derivative stepping.For time-dependent PDEs, the stepping choice determines whether the network learns u(t + τ), the residual u(t + τ) − u(t), or the time derivative (u(t + τ) − u(t))/τ.Source: §B.6.1 · pp. 25–26

B.6.2 Data Normalization

  • GAOT typically uses training-set Z-score normalization, with channel-wise statistics for steady PDEs and strategy-specific target statistics for time-dependent outputs.Time-dependent inputs use global statistics across training samples and time steps; targets are normalized as solutions, residuals, or derivatives according to the selected stepping strategy.Source: §B.6.2 · p. 26

B.6.3 General Training Setup

  • Training uses one RTX 4090 GPU except for industrial 3D tasks, which use A100 GPUs, while schedules warm up, decay cosinely, and finish with a step drop.DrivAerNet++ uses four A100 GPUs for 50 epochs with batch size 1 per GPU; DrivAerML and NASA-CRM use one A100 GPU for 200 epochs.Source: §B.6.3 · pp. 26–27

B.6.4 GAOT Model Configuration

  • The default GAOT configuration combines MAGNO encoder/decoder stages with a global Transformer, using 256 hidden dimensions and radius 0.033, with larger-dataset-specific token counts and processing settings.The default also uses local Statistical Embedding, a 1024-dimensional feed-forward layer, residual connections, RMSNorm, and RoPE; industrial 3D setups use patch size 2, 10 Transformer layers, and bidirectional graphs with k = 1.Source: §B.6.4 · p. 27

B.7 Inference

  • For inference, GAOT predicts any trained-horizon future directly in one step or iterates autoregressively, trading fewer evaluations against potentially faster error accumulation.Autoregressive step sizes can be chosen up to the reliably trained maximum lead time; larger steps reduce computation, while smaller steps require more evaluations but tend to accumulate error more gradually.Source: §B.7 · pp. 28–29

C Baselines / C.1 UPT

  • The time-dependent benchmark comparisons include RIGNO-12, RIGNO-18, CNO, scOT, FNO, GeoFNO, FNO DSE, GINO, Transolver [49], GNOT [20], and UPT [1].UPT [1] encodes arbitrary-mesh or particle inputs into latent tokens, advances them with a Transformer approximator, and decodes at query positions; the reported time-independent baseline uses 64 latent tokens and 64 embedding dimensions (0.74M parameters).Source: C Baselines · p. 29, §C.1 · p. 29

C.2 Transolver / C.3 GNOT / Considered Hyperparameters

  • Transolver [49] is a transformer-based operator-learning model for unstructured PDEs using learned physics-aware slices, while GNOT [20] addresses irregular meshes, heterogeneous input functions, and multiscale problems.GNOT uses a geometric-gating mixture-of-experts FFN whose expert weights are predicted from query-point coordinates. Transolver’s repeated slicing and deslicing within each layer can cause significant computational costs and memory overhead, especially for large N.Source: §C.2 · pp. 30–31, §C.3 · pp. 31–32

D Datasets … E.1 Asymptotic Complexity

The appendices define a broad, heterogeneous PDE benchmark suite, detail newly constructed Poisson and compressible-flow datasets, and discuss model complexity alongside hardware-dependent runtime.

D Datasets

  • GAOT is tested on 28 time-independent and time-dependent PDE benchmarks spanning regular grids, random point clouds, and highly unstructured adapted grids, including five newly generated datasets.The new datasets comprise Poisson-C-Sines and four compressible-fluid-dynamics datasets; Table D.1 overviews the dataset collection.Source: D Datasets · p. 32, Table D.1 · p. 33

D.1 Poisson-C-Sines

  • Poisson-C-Sines learns the finite-element mapping from projected source-term coefficients to nodal solutions on a circular two-dimensional domain.Its Delaunay mesh has 16,431 points and 32,441 elements, with zero Dirichlet boundary conditions; the operator represents the inverse discretized stiffness operator.Source: §D.1 · p. 34

D.2 Compressible Flow Past Airfoils & Bluff Bodies

  • The compressible-flow dataset broadens airfoil benchmarks across subsonic-to-supersonic conditions, varied angles of attack, classical airfoils, and bluff-body geometries.Steady compressible Euler solutions are computed with SU2’s finite-volume solver on Delaundo unstructured grids using the JST scheme.Source: §D.2 · p. 34

D.2.1 Airfoils

  • Airfoil cases span Mach 0.5–1.4, angles of attack 0.5°–5.0°, and 500 shape perturbations per classical airfoil, with anisotropic refinement for shock resolution.The dataset samples 5,384 solutions per airfoil shape and uses an 5,000/128/256 train/validation/test split; each sample is subsampled to 8,000 points.Source: §D.2.1 · pp. 34–36

D.2.2 Bluff-Body

  • The bluff-body aerodynamics dataset covers Mach 0.3–1.3 and angles of attack 0.5°–15.0° across varied simple geometries; because steady compressible Euler solutions may not exist or are often unstable, samples are pseudo-steady in a large finite-time limit.Source: §D.2.2 · p. 36

E.1 Asymptotic Complexity

  • Graph-based message passing stores edge activations scaling with edge count and hidden channels, while latent-token models process compressed representations; theoretical costs may not predict GPU runtime.Sparse, irregular graph operations can be inefficient on GPUs, whereas dense transformer computations are throughput-efficient on current accelerators; GAOT patchifying reduces its latent-token count.Source: §E.1 · p. 36, Table E.1 · p. 36

E.2 Runtime Profiling · E.3 Accuracy, Robustness and Computational Efficiency Metrics

The sections profile GAOT’s runtime and explain how accuracy, robustness, and computational efficiency are measured. They describe GAOT’s batched execution strategy and define evaluation protocols for the reported metrics.

E.2 Runtime Profiling

  • GINO’s NACA0012 runtime is 13.79 ms per sample, with encoder and decoder graph building each taking almost 2 ms and batching prone to OOM errors.Its official implementation does not support batching when graphs change within a batch, limiting scalability with batch size.Source: §E.2 · p. 37
  • GAOT precomputes and caches graphs, processes its encoder and decoder sequentially, and batches the Transformer; edge masking controls memory on very large graphs such as DrivAerML [3].On NACA0012, the Transformer takes 10.1 ms at batch size 1 and 30.1 ms at batch size 32, with total memory usage of 7.74 GB at batch size 32.Source: §E.2 · p. 37, Table E.3 · p. 37
  • At batch size 32 on NACA0012, GAOT runs in 263.1 ms, while GINO, which lacks batching support, loops 32 times and takes 13.791 × 32 = 441.3 ms.GAOT has 5.62M parameters.Source: §E.2 · p. 37, Table E.3 · p. 37

E.3 Accuracy, Robustness and Computational Efficiency Metrics

  • For benchmarks in Tables 1 and E.5, accuracy is relative L1 error between ground-truth and predicted outputs, summarized by the test-set median and averaged across component-wise medians.For time-dependent tasks, the reported error is from the final time snapshot; the metric follows CNO [42].Source: §E.3 · pp. 37–38
  • For DrivAerNet++ pressure and wall shear stress, evaluation uses 1,154 official-leaderboard samples, normalized fields, and per-sample MSE and MAE averaged across samples following.Pressure normalization statistics come from; WSS statistics are calculated over 8,000 samples for the x, y, and z components.Source: §E.3 · p. 38
  • The Table E.4 aggregate comparison scores each model per dataset as the best model’s error divided by its error, then sums scores separately for time-dependent and time-independent categories.The comparison covers GAOT, Transolver, GINO, and RIGNO-18.Source: §E.3 · p. 38
  • Robustness Score combines a model’s mean normalized score with its coefficient of variation across datasets in each category; higher scores indicate stronger average performance and less variability.Table E.4 reports scores for GAOT, Transolver, GINO, and RIGNO-18.Source: §E.3 · p. 38
  • Training efficiency is throughput in samples per second, while inference efficiency is single-sample forward-pass latency; benchmarks use a warmed-up NVIDIA-4090 and average 100 measurements.Throughput batch size is set at approximately half the maximum that avoids OOM errors, based on the observed resource-utilization heuristic.Source: §E.3 · pp. 38–39

E.4 Results for Radar Chart in Main Text.

The radar-chart results report GAOT’s lowest peak inference memory among compared models, while scaling plots show GAOT-8 improves computational performance without substantial accuracy degradation. Scalability scores use throughput at fixed input and model sizes because some baselines could not be benchmarked larger on one NVIDIA 4090 GPU.

E.4 Results for Radar Chart in Main Text.

  • The radar chart summarizes accuracy and robustness on time-independent and time-dependent datasets, training throughput, and single-sample inference latency.Table E.4 supplies the raw data used for the chart.Source: §E.4 · p. 39, Table E.4 · p. 39
  • GAOT has the lowest peak GPU memory usage among compared models during batch-size-1 inference, and scaling plots show superior memory utilization.Figure E.1 plots peak memory against input grid size and model size.Source: §E.4 · p. 39, Figure E.1 · p. 40
  • Input and model scalability scores use training throughput at 50,000 input points and approximately 70 million parameters, respectively.These evaluation points reflect single-NVIDIA-4090-GPU benchmarking limits for models such as RIGNO-18 and Transolver; GAOT’s architecture can scale beyond the tested limits.Source: §E.4 · p. 39
  • GAOT-8, using transformer patch size 8 rather than GAOT’s default size 2, improves computational performance without substantial accuracy degradation.The efficiency improvement is reported alongside ablation studies in Section F.2.Source: §E.4 · p. 40, Figure E.1 · p. 40

E.5 Regular Grid Dataset

On time-dependent PDEs sampled on regular grids, GAOT performs strongly, placing first on five of seven benchmark datasets and in the top two on six.

E.5 Regular Grid Dataset

  • GAOT is compared on regular-grid time-dependent PDE datasets against RIGNO-18, RIGNO-12, CNO, scOT, and FNO, using baseline results from the original RIGNO paper.Source: §E.5 · p. 40
  • ViT alone matches or slightly exceeds GAOT in some cases, but GAOT is significantly superior in many more, showing contributions from both processor and encoder/decoder.The paper attributes state-of-the-art performance to their combination.Source: §E.5 · p. 41
  • GAOT ranks in the top two on six of seven regular-grid benchmarks and places first on five.The benchmark table compares the models across time-dependent datasets with regular-grid inputs.Source: §E.5 · p. 40, Table E.5 · p. 41

E.6 Model and Dataset Scaling · E.7 Resolution Invariance · E.8 Comparison of Neural Field with UPT

Across the scaling, resolution-invariance, and neural-field comparisons, GAOT benefits from increased model and dataset scale, generalizes across resolutions, and outperforms UPT under both tested NACA0012 training settings. Coarse training can nevertheless reduce accuracy relative to full-resolution training.

E.6 Model and Dataset Scaling

  • On CE-Gauss, increasing THS from 32 to 256 reduces final-time relative L1 error from 48.4% to 6.88%, while per-epoch training time rises from roughly 84 to 143 seconds.The comparison keeps TL=5 and LC=32; experiments use 1,024 trajectories and patch size 2.Source: §E.6 · p. 41, Table E.6 · p. 42
  • With TL=5 and THS=256, LC=32 consistently performs strongly; reducing LC to 16 slightly degrades results, while increasing it to 64 or 128 yields only marginal gains.The results suggest LC=32 is sufficient to capture encoder-level geometry information.Source: §E.6 · p. 41, Table E.6 · p. 42
  • On CE-Gauss, increasing TL from 1 to 10 reduces final-time error from 25.0% to 5.28%, while the largest setting uses 11.2M parameters and 260 seconds per epoch.At TL=3, the error is 9.0%; increasing TL to 10 nearly doubles training time.Source: §E.6 · pp. 41–42, Table E.6 · p. 42
  • Larger training sets consistently reduce errors for both time-dependent fluid benchmarks and time-independent PDE tasks, showing that GAOT benefits from additional training data.Fluid datasets use 128–1,024 trajectories; static tasks use up to 2,048 samples, except elasticity, which is limited to 1,024 by data availability.Source: §E.6 · pp. 42–43, Figure E.4 · p. 43

E.7 Resolution Invariance

  • Trained at 2048 points, GAOT generalizes across three lower and three higher test resolutions and achieves the best performance at 8192 points, beating baselines trained and tested there.The seven test resolutions are 256, 512, 1024, 2048, 4096, 8192, and 16384 points; baseline models are RIGNO-18, Transolver, and GINO.Source: §E.7 · p. 44, Figure E.5 · p. 44

E.8 Comparison of Neural Field with UPT

  • On Drivaernet++, GAOT trained with 10% of input points and tested on the full 500K-point resolution is less accurate than the fully trained version, but still outperforms several baselines.This illustrates that continuous neural-field evaluation across resolutions does not itself guarantee higher accuracy.Source: §E.8 · p. 44
  • GAOT consistently outperforms neural-field-based UPT on NACA0012 with both full-resolution and one-fourth-resolution training, while full-resolution training is more accurate than coarse training.UPT’s limited improvement with resolution suggests poor convergence. Table E.7 reports the comparison.Source: §E.8 · pp. 44–45, Table E.7 · p. 44

E.9 Training at full vs. low Resolution for the DrivAernet++ Dataset. · E.10 Transfer Learning

On DrivAernet++, low-resolution training followed by full-resolution interpolation sharply reduced accuracy, while full-resolution-trained GAOT was reported twice as accurate as other models. The transfer-learning experiments fine-tuned on novel bluff-body shapes and reported adaptation performance in Figure 3(c).

E.9 Training at full vs. low Resolution for the DrivAernet++ Dataset.

  • Models trained on 10K points degraded severely when interpolated from 10K-point tests to full-resolution 487K-point predictions.Even Transolver and AB-UPT, which performed well at low resolution, degraded to accuracy comparable with inaccurate RegDGCNN models.Source: §E.9 · p. 45
  • The comparison reproduces prior Carbench results for models trained on subsampled DrivAernet++ data, alongside GAOT results for one test sample and the test-set average.The reproduced study trained at 10K points from a 487K-point surface cloud and tested at both 10K and 487K points, interpolating low-resolution outputs to full resolution.Source: §E.9 · p. 45
  • GAOT trained at full resolution was reported twice as accurate as the other models and performed on par with models trained and tested at low resolution.The section presents this as evidence that training at full resolution, when possible, is the best strategy for accurate surrogates.Source: §E.9 · p. 45
  • Neural-field evaluation at full resolution does not guarantee accuracy retention after low-resolution training, and the issue needs further investigation.The passage cites AB-UPT as an example and recalls that neural-field evaluation can still involve accuracy degradation.Source: §E.9 · p. 45

E.10 Transfer Learning

  • Transfer learning fine-tuned on bluff-body shapes distinct from those used for pretraining, with adaptation performance reported in Figure 3(c).The Bluff-Body dataset varies geometry, Mach number, and angle of attack under compressible-flow simulations.Source: §E.10 · p. 46
  • The transfer-learning shape labels encode contact-surface types: C and F denote curved and flat, while L and S denote larger and smaller sides.The illustrated geometries include semicircle, cone, and rectangle shapes.Source: Figure E.6 · p. 46

E.11 Training Randomness … F.2 Tokenization Strategies

Across runtime comparisons and ablations, GAOT combines fast inference with robust performance across PDE benchmarks. The studies also identify reliable tokenization choices and tradeoffs between accuracy and computational cost.

E.11 Training Randomness

  • Across six independently seeded Bluff-Body training runs, relative L1 test error has a standard deviation of 0.12, indicating low sensitivity to training randomness.The reported statistic summarizes six repetitions; the passage does not provide the individual errors.Source: §E.11 · p. 46, Table E.9 · p. 46

E.12 Runtime Comparison between GAOT and Classical Solvers

  • 8.95–10.14 ms 2D and around 365 ms 3D DrivaerNet++ inference yields reported speedups of 1–5 orders for 2D and up to 10 orders for 3D benchmarks.DrivAerML inference takes approximately 15 seconds, versus more than 60K core hours for its LES ground truth; graph building accounts for much of its runtime.Source: Table E.10 · p. 47, §E.12 · p. 47
  • 6000× speedup on NACA0012: coarsened solver error is 4.5% at approximately 1 minute, versus GAOT error of 6.8% at approximately 10 milliseconds.The coarsened solver’s error is measured against ground truth; the comparison coarsens its mesh by two refinement levels.Source: §E.12 · p. 47

F.1 Encode-Process-Decode

  • GAOT uses MAGNO encoders and decoders with a transformer processor, while RANO, RFNO, and GINO combine different graph, transformer, or Fourier components.All four models follow an encode–process–decode pipeline; MAGNO denotes the proposed Multi-scal Attentional GNO.Source: §F.1 · p. 48, Table F.1 · p. 48
  • Across six PDE benchmarks, GAOT achieves strong, robust performance; GINO ranks second overall but struggles on NS-Gauss and Poisson-C-Sines, while RANO and RFNO are unstable on harder tasks.Models are trained for 500 epochs under the same data splits and hyperparameter conditions; GAOT is also reported to converge faster and more reliably under this protocol.Source: Figure F.1 · p. 48, §F.1 · p. 48

F.2 Tokenization Strategies

  • Strategy I performs best on Elasticity and Poisson-C-Sines; Strategies II and III can fail to converge on the harder Poisson-C-Sines benchmark.Strategies II and III perform similarly to Strategy I on Elasticity but require more careful optimization to match its reliability.Source: §F.2 · p. 48, Figure F.2 · p. 49
  • More latent tokens typically improve accuracy and convergence stability, while fewer tokens can degrade performance; larger patches reduce training time without sharply harming performance.The comparison uses a single radius per token, without multiscale radii. Attention cost grows quadratically with token count; increasing patch size can raise parameter count, and reducing token count requires a larger radius to cover the domain.Source: Table F.2 · p. 49, §F.2 · p. 49

F.3 Time-Stepping Method

Derivative time stepping usually gives the lowest final-time errors across the six fluid-dynamics benchmarks, and is used by default for time-dependent PDEs unless stated otherwise.

F.3 Time-Stepping Method

  • The study compares output, residual, and derivative stepping on six representative fluid-dynamics benchmarks with regular grids.The comparison uses median relative L1 errors at final time.Source: §F.3 · p. 49, Table F.3 · p. 50
  • Treating the operator as ∂tu may improve multi-step stability and accuracy by enforcing continuous time dependence, analogous to neural ODEs or residual networks [21] [11].The experiments use derivative time stepping by default unless stated otherwise, aligning with a differentiable, time-continuous operator design.Source: §F.3 · pp. 49–50
  • Derivative stepping yields the lowest final-time errors on all but CE-RP, where Output slightly outperforms the other strategies.Table F.3 reports median relative L1 errors (%) at final time t14 for GAOT using output, residual, and derivative stepping.Source: §F.3 · p. 49, Table F.3 · p. 50

F.4 Geometric Embedding · F.5 Multiscale Features

The ablations show that geometry embeddings and multiscale neighborhoods improve performance on unstructured domains, while encoder-only geometry embedding is preferred for 3D industrial benchmarks.

F.4 Geometric Embedding

  • On unstructured Wave-C-Sines and Poisson-C-Sines, explicit geometry encoding helps, and statistical embedding consistently beats PointNet; PointNet training is unstable on Poisson-C-Sines (23.7% error).The comparison includes an original baseline without geometry embedding.Source: §F.4 · p. 50, Table F.4 · p. 50
  • Across DrivAerNet++ and DrivAerML, geometric embeddings improve performance; encoder-only is better with less computational effort, while decoder embedding worsens DrivAerML below no embedding.The tested configurations omit geometry embedding, use it only in the encoder, or use it in both encoder and decoder.Source: §F.4 · p. 50, Table F.5 · p. 50
  • The authors attribute decoder-related overfitting to embedding geometry near final predictions, where complex graph-building techniques may amplify overfitting, especially on the 3D industrial dataset.Training loss continues to decrease while validation loss quickly saturates; the graph-building tricks lack unified principles for the model to learn.Source: §F.4 · pp. 50–51
  • For 3D industrial benchmarks, the method uses geometric embeddings only in the encoder.This choice follows the observed performance and overfitting effects of adding geometry embedding to the decoder.Source: §F.4 · p. 51

F.5 Multiscale Features

  • The multiscale comparison uses one fixed radius of 0.033 versus three neighborhood radii of 0.022, 0.033, and 0.044 per point.These are the single-scale and multiscale configurations evaluated on unstructured datasets.Source: §F.5 · p. 51
  • In Poisson-C-Sines, multiscale neighbors reduce mean relative L1 error from 4.66% to 3.04%.The comparison uses a single radius of 0.033 versus three radii [0.022, 0.033, 0.044]; multiple radii help capture fine and coarse structures where mesh density varies, whereas one fixed receptive field is often sufficient on regularly spaced grids.Source: §F.5 · p. 51, Table F.6 · p. 51

G Visualizations of Datasets

This section presents test-sample visualizations of model inputs, ground-truth solutions, and model estimates across PDE datasets. Some examples also show point distributions or compare an input at t = t0 with an estimate at t = t14.

G Visualizations of Datasets

Related papers

Loading 2505.18781v4…