Read, Write, Relax: Why Neural PDE Surrogates Need Both Global and Local Processing

Anuj Kumar; Heiko Zimmermann; Josiah Bjorgaard; Jacan Chaplais; Nikolaos Bouklas; Matteo Salvador; Alexander Lavin

Source-linked AI summary

Read, Write, Relax: Why Neural PDE Surrogates Need Both Global and Local Processing

Anuj Kumar, Heiko Zimmermann, Josiah Bjorgaard, Jacan Chaplais, Nikolaos Bouklas, Matteo Salvador, Alexander Lavin

arXiv:2608.21677v1Aug 21, 2026cs.LGcs.AIcs.CEphysics.app-phphysics.comp-ph

TL;DR

Mesh-based PDE surrogates must combine global communication with local resolution, yet latent-attention and message-passing models each fall short in one of these roles. RWR interleaves both mechanisms, lowering error across the spectrum and improving accuracy and data efficiency across the reported benchmarks.

Problem
Steady CAE surrogates need whole-domain information propagation and fine local resolution, but global and local model families each provide only one, while training data are scarce and bulk metrics poorly proxy engineering quantities.
Method
RWR unifies latent attention and message-passing relaxation in an interleaved processor whose configuration includes both operations and recovers MGN and RWP as special cases.
Results
RWR contracts error across the entire spectrum and is the most accurate model in nearly every comparison across the reported industrial and public benchmarks.
Takeaways & Limitations
The combined processor carries accuracy into gradients and wall-based engineering quantities and remains data-efficient at small training budgets.
Takeaways & Limitations
The spectral analysis uses a single steady RANS benchmark, and the datasets are steady, so transient rollout failure modes are not measured.

Abstract

from arXiv · show

Recent mesh-based simulation advances have, in no small part, relied on neural surrogates of two distinct families: global models that route information through a small set of latent tokens, and local models that perform message passing across mesh edges. Consistent with both classes is the inability to perform beyond low-dimensional problems and small-scale or oversimplified meshes, the simulation regimes where industrial problems reside. Our work shows this explicitly and presents a unified formulation. In global approaches, latent-token attention acts as a spatial low-pass filter, while local message passing lacks the global reach necessary to propagate information across large mesh spaces. Viewed through the error, the two operators are the halves of a multigrid cycle: one corrects errors at the lower end of the spectrum, the other at the higher end, and neither can do the other's job. We introduce Read-Write-Relax (RWR), which interleaves latent attention with message-passing relaxation under a unified formulation. The interleaved processor lowers error across the entire spectrum, making RWR the most accurate model in nearly every comparison across our industrial and public benchmarks. It is also markedly data-efficient in the scarce-data regimes, accurate on the engineering quantities of interest, and scales full-field predictions to challenging, large-scale problems.

1. Introduction

Mesh-based PDE surrogates require both domain-wide information exchange and fine local resolution, but global latent-attention and local message-passing models provide only one of these strengths. RWR unifies them through a multigrid-inspired processor designed for accuracy, data efficiency, and engineering quantities.

Motivation: Global models compress mesh fields into latent tokens, whereas local models update nodes from mesh neighbors with receptive fields growing one hop per layer.The two families are represented by latent-attention models and MeshGraphNets-style processors.
Motivation: Steady-state surrogates must propagate boundary and operating-condition information across the whole domain while resolving steep local gradients in one forward pass.Engineering outputs such as wall shear stress, vorticity, and integrated losses depend on derivatives.
Motivation: CAE datasets often contain hundreds rather than millions of samples because industrial RANS and hybrid RANS–large-eddy simulations cost hundreds to thousands of core-hours per geometry.This makes data efficiency a requirement for the target setting.
Contribution: The paper analyzes MGN and RWP through error spectra, interpreting their complementary failures as the relaxation and coarse-correction roles of a multigrid cycle.The read and write operate in latent space, while message-passing sweeps provide learned relaxation.
Contribution: RWR interleaves latent attention with message-passing relaxation, recovering MGN and RWP as special cases through a unified configuration.The paper reports accuracy on engineering quantities, data efficiency at small training budgets, and the highest accuracy in nearly every comparison across benchmarks.

2. Related Work

Related work frames the central tension between local graph message passing and global attention in both graph learning and mesh-based PDE surrogates. Existing latent-token models and hybrid approaches occupy different points in this design space.

Attention and message passing on graphs: Graph message passing is limited by receptive fields and over-squashing, while graph transformers exchange global information but must restore topology through structural or spectral encodings.Hybrid graph approaches combine message passing and transformer-style processing as an intermediate design.
Local and global processing in PDE surrogates: MGN-style PDE processors need depth proportional to mesh diameter, limiting long-range exchange for steady and elliptic problems where boundary information shapes the interior globally.Deep stacks can capture long-range dependencies poorly and train slowly.
Latent-token surrogates: Latent-token PDE surrogates compress fields into a small token set and process them there, including Perceiver IO, Transolver, LNO, UPT, AROMA, and FLARE.The paper places these models and RWP within a bottlenecked self-attention template for analysis.

3. The Model: Read-Write-Relax

RWR is an encode-process-decode surrogate whose processor interleaves local message-passing sweeps with persistent latent read-write iterations. Its configuration controls the local-global processing mix and includes MGN and RWP as limiting cases.

Architecture: RWR processes encoded node and edge features through n_B blocks, each configured by (m1,r,m2) for initial sweeps, read-write iterations, and final sweeps.The decoder projects the processed representation back to output fields.
Architecture: Setting r=0 recovers MGN, while setting m1=m2=0 removes relaxation and yields RWP.These special cases make the three models configurations of the same processor.
Read-write mechanism: Each read-write iteration updates persistent latents from mesh features and writes the latent state back to mesh features through cross-attention.Latents retain state across iterations, so later reads refine earlier accumulated information.
Multigrid interpretation: The multigrid interpretation treats read as restriction to an L-dimensional coarse space, write as prolongation, and message-passing sweeps as learned relaxation.An optional latent self-attention tower can be inserted between read and write.
Scalability: Geometry-anchored latent queries use physical-space coordinates and shared random-Fourier embeddings, while boundary-only encoding can restrict reads to information-dense boundary points.The design is motivated by concentrating geometry and boundary-condition information at the boundary for steady problems.

4. Datasets

The paper evaluates surrogates on two industrial steady internal-flow environments: a branched HVAC duct and a centrifugal pump. Both use finite-volume RANS simulations with sampled geometric and operating parameters.

Environments: The two environments are presented as a branched pipe on the left and a centrifugal pump on the right, each with a representative velocity field.The datasets support industrial steady-flow evaluation at different spatial scales.
Branched pipe: The branched-pipe dataset varies duct geometry, baffle angles, and inlet velocity, modeling pressure and two in-plane velocity components on roughly 60k-cell meshes.The Y-shaped HVAC junction has turbulent flow near sharp baffle turning angles.
Centrifugal pump: The centrifugal-pump dataset varies inlet and impeller radii, blade pitch, and inlet velocity, modeling three velocity components and pressure.The pump uses a moving reference frame for impeller rotation and is evaluated at multiple spatial extents.

5. Results and Discussion

The experiments show complementary spectral failures in pure latent-attention and message-passing processors, while RWR combines their strengths to reduce error across scales and improve benchmark accuracy and data efficiency.

Global models: The BSA family routes mesh updates through L≪N latent tokens, giving each layer a mixing matrix with rank at most L.This bounds each layer to at most L spatial basis fields.
Latent attention: RWP reduces low-band error by more than an order of magnitude, but its mid- and high-band errors stall, behaving as a spatial low-pass filter.Its high band separates from the low band early in training and never rejoins it.
Latent attention: Widening the latent bottleneck from 8 to 512 leaves every error band almost flat, so recovering high-frequency scales requires a second operator class.The authors attribute the limitation to compounding bottleneck, attention, loss, and optimization factors.
Message passing: MGN gains early on middle and high wavenumbers but plateaus across the spectrum because its receptive field cannot cover the domain.On the benchmark meshes, the graph diameter exceeds the depth of a typical MGN processor.
RWR: RWR interleaves latent attention with message-passing relaxation, contracting error across the entire spectrum throughout training.The two pure processors fail from opposite ends, while RWR supplies the complementary mechanism each lacks.
Benchmarks: RWR attains the lowest error at every training budget on the branched pipe and industry-scale benchmarks, with widest margins at scarce budgets.With 50 samples, it matches or improves on every latent-attention baseline trained on twice as much data for wall shear stress components.
Benchmarks: RWR is best on all Table 3 datasets and all additional benchmarks except Geo-FNO pipe, where MeshTransolver is marginally better.MGN is consistently and significantly the worst-performing model, while RWP exceeds Transolver and GeoTransolver on the three Geo-FNO problems.
Scope: The comparisons reported here use bulk metrics alone, although the paper reports larger RWR gains on gradients and wall metrics.Detailed additional engineering-quantity comparisons on industrial-scale datasets remain future work.

6. Conclusion

RWR combines complementary global and local processing to reduce error across the spectrum, with strongest margins on engineering quantities and scarce-data regimes. The analysis also identifies unresolved limits involving causality, data regime, transient problems, and hybrid-model comparisons.

RWR interleaves latent attention with message-passing relaxation, treating them as complementary multigrid-like operations and recovering MGN and RWP as special cases.Its unified configuration exposes the balance between local and global processing.
RWR contracts error across the entire spectrum, achieves the largest margins on engineering quantities and smallest training budgets, and is most accurate in nearly every public-benchmark comparison.On industry-scale meshes, it attains the lowest error at every training budget.
The analysis establishes consistency rather than causal isolation because the compounding sources of low-pass behavior are not disentangled experimentally.Controlled interventions on the loss, attention, and bottleneck would be required.
The reported spectral analysis uses a single steady RANS benchmark and steady datasets, so transient-rollout failure modes are not measured.The multigrid interpretation is an analogy of roles rather than a convergence theorem.
Comparisons with message-passing and attention hybrids are limited to the authors’ MeshTransolver-style implementation, leaving broader comparison open.

A. Datasets and Simulations

The study uses internal-flow simulations generated with finite-volume RANS setups, including branched-pipe and pump cases. Dataset-generation parameters are summarized in accompanying tables.

Both environments are internal-flow cases solved with the Finite Volume Method using a RANS turbulence model, specified inlet and outlet conditions, and no-slip walls.The branched pipe and pump use different inlet profiles, while the pump models impeller rotation with the Moving Reference Frame method.
The branched pipe uses a one-seventh power-law inlet profile, whereas the pump uses uniform inlet velocity and Moving Reference Frame impeller modeling.
Table 5 summarizes the simulation configurations.
Table 6 lists design parameters and the ranges over which each parameter is sampled to generate the simulation datasets.

B. Metric Definitions

The paper evaluates field, gradient, wall, spectral, and training-dynamics errors on held-out test cases using explicitly defined spatial, spectral, and normalization procedures.

Metric conventions: All metrics compare predicted and reference fields through physical-unit errors after inverting training normalization, with volume-weighted conversion to point data except for R2.Analysis metrics are computed per case, field, and model on held-out test cases, capped at eight cases, then averaged unweighted.
Gradient metrics: Spatial gradients use a k-nearest-neighbor weighted least-squares operator with k=20, ridge regularization, and inverse-distance weights, applied identically to truth and prediction.The operator reproduces linear fields to machine precision.
Spectral metrics: The error spectrum is computed from a masked 512-pixel raster of a mid-domain y-normal slice, using radial FFT rings and excluding the DC ring and wavenumbers above k=92.The excluded high-wavenumber range is limited by raster-interpolation noise floors.
Spectral metrics: Band-resolved fRMSE measures Fourier-space RMSE relative to reference spectral energy, with low, mid, and high bands defined as 0≤k≤4, 5≤k≤12, and 13≤k≤92.A value of 0 denotes perfect prediction, while 1 means error energy equals truth energy within the band.
Engineering quantities: Wall shear stress is evaluated through wall-normal derivative proxies τw,i=μ(∇ui·n̂) for the x and z velocity components, scored using relative L2 and R2 over wall points.Volume gradients are interpolated to wall points from their four nearest volume points.
Training and figures: Training-dynamics curves use five checkpoints on four cases and normalize each field-band value by RWP’s first-checkpoint value, so 1 means RWP-initialization error.Figure 5 varies latent count L from 8 to 512 for band-resolved test errors; Figure 6 compares MGN, RWP, and RWR dynamics.

D.2 Composition Ablation

The composition ablation holds the processing budget fixed while comparing stacked and increasingly interleaved arrangements of relaxation sweeps and latent iterations.

Composition design: Table 9 compares four arrangements using an identical budget of eight relaxation sweeps and four latent-attention iterations.The configurations range from placing all relaxation before or after the latent path to progressively finer interleavings.
Composition design: The configuration notation (m1,r,m2)×nB specifies relaxation sweeps before and after r latent iterations per block, repeated over nB blocks.The rows include (8,4,0) and (0,4,8), representing the two stacked orderings.
Evaluation: Held-out ablation results report normalized mean absolute error for pressure and both velocity components.The table expresses NMAE in units of ×10^-2.
Engineering quantities: Figure 8 reports mean absolute error for pressure and velocity components as a function of wall distance, including a near-wall zoom.The evaluation uses held-out branched-pipe cases.
Engineering quantities: Figure 7 compares parity plots of wall-normal velocity derivatives over wall cells and reports R2 for each model.The derivatives are ∂ux/∂n and ∂uz/∂n on held-out branched-pipe cases.

D.4 Analysis Training Details

The analysis trains MGN, RWP, and RWR on the branched-pipe benchmark under a shared protocol, with matched architectural accounting and geometry-anchored latent queries.

Training protocol: All analysis models use the 100-sample branched-pipe training protocol, with MGN trained for the same 550 epochs as RWP and RWR.MGN uses an initial learning rate of 3 × 10^-4.
Architecture: RWP and RWR configurations are documented in Table 10, whose hidden widths approximately match parameter counts across MGN, RWP, and RWR.The latent queries are geometry-anchored with learned anchor coordinates and use a shared random-Fourier-feature embedding.
Evaluation: Figure 8 evaluates mean absolute error against wall distance on held-out branched-pipe cases, with a dedicated near-wall zoom.The top row covers the full wall-distance range and the bottom row focuses near the wall.

E. Main Benchmark Results

The benchmark section standardizes datasets, compares model families under shared evaluation, and selects best-performing variants through capacity and locality/geometry sweeps.

Reported metrics: Table 12 reports per-field NMAE, parameter counts, peak memory, and training cost behind the benchmark summary.Table 11 provides the corresponding best-model architectural details for each mesh and data setting.
Data and protocol: Each dataset contains 300 simulations split 70/15/15 into 210 training, 45 validation, and 45 test samples.Data-limited variants use the first 50 or 100 samples with corresponding 35/7/8 and 70/15/15 splits.
Model configurations: RWR interleaves message-passing sweeps with latent read/write using steps=[2,2,2], whereas RWP performs latent read/write alone with steps=[0,2,0].Both wrap a latent-attention bottleneck within a MeshGraphNet backbone.
Model selection: RWR, RWP, Transolver, and GeoTransolver are selected after sweeping principal capacity and locality/geometry-governing hyperparameters.For RWR and RWP, sweeps include processor depth, latent-token count L, and full-volume versus boundary-only read attention.

F. Additional Benchmarks

Additional benchmarks evaluate models across AirfRANS, AhmedML, Geo-FNO airfoil and pipe, and Darcy flow using matched inputs, specified training protocols, and field-level visual and tabular metrics.

Datasets: AirfRANS contains roughly 180k-node two-dimensional RANS airfoil cases, while AhmedML contains roughly 1.1M-point three-dimensional meshes across 500 vehicle variants.AirfRANS targets velocity, pressure, and turbulent viscosity; AhmedML targets surface pressure and wall shear stress.
Input parity: Every model receives the same per-case information, with geometry and global parameters encoded through family-specific representations.MGN-family models use relative edge displacements and inject global parameters at every message-passing step, whereas global models use token-level or dedicated-token conditioning.
Model configurations: RWR uses 3 blocks on AirfRANS and AhmedML and 2 on Geo-FNO datasets, with 128 or 64 latents, hidden width 128, and 4 attention heads.The latent count is 128 for AirfRANS and AhmedML and 64 for Geo-FNO datasets.
Held-out evaluation: Table 12 reports per-field and across-field NMAE alongside parameter count, peak memory, epoch time, and epochs to mean R2 ≥ 0.95.The branched pipe is quasi-2D, so u_y is absent; n/r denotes that the target threshold was not reached.
Training protocol: Training uses AdamW with learning rate 10^-3, weight decay 0.01, linear decay to 0.1×, gradient clipping at 2.0, and dataset-dependent precision and batch sizes.The loss is MSE on standardized fields, each run reports its best validation loss, and results use a single seed.
Held-out evaluation: Figures 9–14 compare predicted pressure and velocity magnitude with signed pointwise errors for two held-out test cases per dataset.Views include the branched-pipe mid-y slice, pump-casing y = 0.04 m plane, and full-pump impeller surface; panels report R2 and NMAE.