PaperlayerView exact revision on arXiv

Source-linked AI summary

MultiScale MeshGraphNets

Meire Fortunato, Tobias Pfaff, Peter Wirnsberger, Alexander Pritzel, Peter Battaglia

arXiv:2210.00612v1cs.LGcs.CE

TL;DR

High-resolution meshes make message propagation a bottleneck for MeshGraphNets, limiting accuracy while increasing computational cost. The paper uses high-accuracy labels and a two-resolution hierarchical model; MS-MGN is more accurate and computationally cheaper than the MGN baseline at the same number of message-passing steps. Long-rollout error accumulation remains unresolved.

  • Problem

    It is unclear whether learned simulation methods, particularly message-passing GNNs such as MGN, scale effectively to high-resolution meshes where propagation requires more updates.

    Source: §1 · p. 1, §7 · p. 7

  • Method

    The paper combines high-accuracy labels interpolated from higher-resolution simulations with MS-MGN, which passes messages on fine and coarse graphs.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    MS-MGN is more accurate and computationally cheaper than the MGN baseline with the same number of message-passing steps.

    Source: §7 · p. 7

  • Takeaways & Limitations

    High-accuracy labels support accurate predictions on extremely coarse meshes, while a coarse discretization enables more efficient information exchange across the domain.

    Source: §7 · p. 7

    Multiscale models substantially ameliorate but do not resolve error accumulation in long rollouts.

    Source: §7 · p. 7

Abstract

from arXiv · show

In recent years, there has been a growing interest in using machine learning to overcome the high cost of numerical simulation, with some learned models achieving impressive speed-ups over classical solvers whilst maintaining accuracy. However, these methods are usually tested at low-resolution settings, and it remains to be seen whether they can scale to the costly high-resolution simulations that we ultimately want to tackle. In this work, we propose two complementary approaches to improve the framework from MeshGraphNets, which demonstrated accurate predictions in a broad range of physical systems. MeshGraphNets relies on a message passing graph neural network to propagate information, and this structure becomes a limiting factor for high-resolution simulations, as equally distant points in space become further apart in graph space. First, we demonstrate that it is possible to learn accurate surrogate dynamics of a high-resolution system on a much coarser mesh, both removing the message passing bottleneck and improving performance; and second, we introduce a hierarchical approach (MultiScale MeshGraphNets) which passes messages on two different resolutions (fine and coarse), significantly improving the accuracy of MeshGraphNets while requiring less computational resources.

1. Introduction

Learned simulators can accelerate costly numerical simulation, but it is unclear whether MeshGraphNets (MGN) scales to high-resolution meshes. The paper proposes multiscale message passing and high-accuracy training labels to address this bottleneck.

1. Introduction

  • Learned simulators can be faster than classical solvers and are differentiable, opening avenues for inverse design.MGN has demonstrated accurate predictions across a broad range of physical systems.Source: §1 · p. 1
  • MGN message propagation becomes a bottleneck on high-resolution meshes, increasing computational cost and potentially reducing accuracy.Finer edges require more message-passing steps to cover the same physical distance.Source: §1 · p. 1
  • MS-MGN passes messages on both fine and coarse meshes, restoring spatial convergence while improving accuracy and computational efficiency.The coarse mesh facilitates information propagation across the domain.Source: §1 · pp. 1–2
  • High-accuracy labels let models trained on coarse meshes make better predictions than the reference simulator at that resolution.The labels capture higher-resolution dynamics rather than simply reproducing the solver’s spatial convergence curve.Source: §1 · p. 2

2. MultiScale MeshGraphNets

MS-MGN extends MeshGraphNets by passing messages on fine and coarse mesh graphs, with cross-level operators to transfer information. Its processor alternates fine- and coarse-level updates so coarse updates can communicate information efficiently while outputs remain on the fine mesh.

2. MultiScale MeshGraphNets

  • MS-MGN passes messages on fine and coarse graphs, using the coarse mesh to promote efficient latent communication for fast-acting or non-local dynamics.Inputs and outputs are defined on the fine mesh.Source: §2 · p. 2
  • The fine and coarse meshes represent the same physical domain, and the coarse graph encodes geometry while field variables are encoded only on the fine graph.The fine graph follows MGN’s node and edge features; the coarse graph uses geometric edge features and node types distinguishing internal and boundary nodes.Source: §2 · p. 2
  • Downsampling connects each fine node to the corners of its containing coarse-mesh triangle, while upsampling transfers coarse information back to fine nodes.The four operators also update neighbors within each resolution; training loss is placed only on fine-graph nodes.Source: §2 · pp. 2–3, Figure 1 · p. 2
  • The processor aggregates local features on the fine graph, downsamples to update the coarse graph, then upsamples and performs fine-graph updates.Coarse updates are cheaper and propagate information farther; the sequence is similar to a multigrid V-cycle.Source: §2 · p. 3

3. High-accuracy labels

Training labels from conventional simulations reflect the resolution-dependent accuracy of those simulations. The paper instead uses high-resolution solutions interpolated onto coarse meshes to provide labels that retain smaller-scale dynamics.

3. High-accuracy labels

  • Conventional simulation labels approximate physical phenomena, and their accuracy is linked to the simulation mesh resolution.Coarse-grid solvers may use heuristic closure models for below-resolution dynamics.Source: §3 · p. 3
  • High-accuracy labels are generated by interpolating a higher-resolution reference solution onto the mesh used for prediction.The model can implicitly learn smaller-scale effects without changing its code and may outperform a classical solver on a coarse mesh.Source: §3 · p. 3
  • The Karman vortex example shows that interpolated high-resolution labels retain visible vortex shedding on the coarse mesh, unlike the coarse simulation itself.The finer reference simulation resolves the dynamics, and the interpolated labels are used to generate training data.Source: Figure 2 · p. 3

4. Experimental setup

The experiments use CylinderFlow simulations spanning a broad range of mesh resolutions. Prediction error is assessed against a highest-resolution COMSOL trajectory interpolated onto each test mesh.

4. Experimental setup

  • CylinderFlow contains 1,000 trajectories of 200 time steps, varying flow and obstacle parameters across meshes from hundreds to tens of thousands of nodes.Source: §4 · p. 3
  • The 500-trajectory test set varies mesh resolution while keeping initial conditions constant; edge_min ranges from 10^-2 to 10^-3.Mesh complexity increases strongly as edge_min decreases.Source: §4 · p. 4, Figure 3 · p. 4
  • Ground-truth error is measured against a COMSOL reference at edge_min=10^-3, interpolated onto each mesh before next-step MSE is computed.Source: §4 · p. 4

5. Results

On high-resolution meshes, MGN’s message-passing speed limits accuracy, while multiscale message passing improves convergence and efficiency. High-accuracy labels let MS-MGN outperform the reference solver across mesh resolutions, although rollout error still accumulates.

Spatial convergence of MGN

  • MGN’s error initially follows spatial convergence but eventually stagnates as finer meshes require more message-passing steps to cover the same physical distance.Increasing message-passing depth shifts the divergence point, but 25 steps—the most trainable on one GPU for this dataset—still do not restore convergence.Source: Figure 4 · p. 4, §5 · p. 4

Spatial convergence of MS-MGN

  • With the same number of message-passing steps, MS-MGN substantially reduces MSE versus MGN and at 25 mps closely tracks the reference convergence curve.The multiscale model also dampens rollout error across eigenvalues associated with slowly varying, long-range spatial correlations.Source: Figure 5 · p. 5, §5 · p. 5

Computational efficiency

  • MS-MGN offers better accuracy-versus-training-speed tradeoffs at every mesh resolution because coarse-level message-passing steps update fewer nodes and edges.The same relationship applies at inference time.Source: Figure 6 · p. 5, §5 · p. 5

High-accuracy labels

  • With high-accuracy labels on mixed-resolution meshes, MGN is more accurate than the reference simulator, with accurate predictions extending to edge lengths of 10^-2, corresponding to a very coarse mesh of around 100 nodes.Source: §5 · p. 5, Figure 7 · p. 6
  • At 25 mps, MS-MGN is more accurate than the reference solver across all mesh resolutions when trained with high-accuracy labels.The combined approach retains MS-MGN’s performance benefits while keeping its error below the reference solver curve.Source: §5 · p. 5, Figure 8 · p. 6

Error accumulation

  • For meshes with edge length below 0.0015, MS-MGN reduces rollout error versus baseline but does not stop error accumulation.Accurate one-step predictions are necessary but not sufficient for accurate simulation rollouts.Source: §5 · p. 6

Choice of coarse mesh

  • Uniform-grid coarse meshes perform worse than unstructured coarse meshes on both one-step and rollout error.The authors hypothesize that boundary-conforming meshes with node density similar to the simulation mesh make local information-transfer rules easier to learn.Source: Figure 10 · p. 6, §5 · p. 6

6. Related Work

The section situates MeshGraphNets within GNN-based simulation research and discusses related work on high-accuracy labels and hierarchical models. It distinguishes MS-MGN from prior multilevel approaches through its use of conformal meshes.

GNN-based simulation models

  • GNN models have been used to predict liquid and soft-body dynamics, aerodynamics, weather, and quantum many-body problems.The paper specifically studies and extends MeshGraphNets.Source: §6 · p. 6

High-accuracy labels

  • Mesh-based GNNs may benefit from high-accuracy labels, but whether they can learn closures and use such labels effectively was unclear.The authors report accurate predictions over a broad range of mesh resolutions using high-accuracy labels, allowing lower-cost inference.Source: §6 · p. 7

Hierarchical GNN models

  • Prior physics-simulation hierarchies include tree-based particle interactions, mesh reduction for transformers, and GNN-learned operators for classical multigrid solvers.Lino et al. (2021) is described as most closely related; unlike MS-MGN, it uses on-the-fly connectivity, proximity edges on the finest level, and regular grids on coarser levels.Source: §6 · p. 7

7. Conclusions and discussion

The discussion identifies message propagation speed as a high-resolution bottleneck for GNN simulation models and presents high-accuracy labels and MS-MGN as complementary approaches. The authors report improved accuracy and lower computational cost against MGN at equal message-passing steps, while noting that long-rollout error accumulation remains unresolved.

7. Conclusions and discussion

  • For highly resolved simulations, message propagation speed limits GNN prediction accuracy, while adding message-passing steps is costly and still may not reduce error on complex meshes.Source: §7 · p. 7
  • High-accuracy labels support accurate predictions on coarse meshes, while MS-MGN exchanges information across fine and coarse resolutions.MS-MGN is reported as more accurate and computationally cheaper than MGN with the same number of message-passing steps.Source: §7 · p. 7
  • Long-rollout error accumulation is reduced but unresolved; both models remain stable for hundreds of steps and are trained on next-step prediction with a fixed time step.The authors identify adaptive time stepping, sequence models, and training noise as future research topics.Source: §7 · p. 7

A.1. MultiScale MeshGraphNets architecture details

The architecture details describe the fine-mesh graph representation and the links used to communicate between fine and coarse meshes. Downsampling and upsampling connect nodes through the corners of containing mesh triangles.

A.1. MultiScale MeshGraphNets architecture details

  • The fine graph uses the same node and edge features and latent sizes (128) as Pfaff et al. (2021).Source: §A.1 · p. 10
  • Node and edge latents are denoted by v_i^h and e_ij^h on the fine graph, with corresponding latents on the coarse graph.The supplied passages identify these representations but do not specify their full update equations.Source: §A.1 · p. 10
  • For downsampling, each fine-mesh node connects to the three corner nodes of its containing coarse-mesh triangle.Source: §A.1 · p. 10
  • For upsampling, each coarse-mesh node connects to the three corner nodes of the fine-mesh triangle containing it.Source: §A.1 · p. 10
  • The architecture describes node and edge feature updates separately on fine and coarse graphs and on the downsampling and upsampling graphs.Source: §A.1 · p. 10

Low-resolution update

The additional results compare processor configurations and analyze rollout errors by graph frequency; the CylinderFlow figures illustrate variation in examples and mesh resolution. The update description specifies residual MLP functions and edge-wise aggregation.

Upsampling update

  • Update functions across the graphs are residual MLPs, and the right-hand summation aggregates over edges k: i → j.Source: Upsampling update · p. 10

A.2. Additional results

  • The processor sweep counts fine, coarse, downsampling, and upsampling graph updates as message-passing steps; the illustrated 15-step processor uses 1 fine update, 11 coarse updates, then 1 fine update.Source: §A.2 · p. 10
  • The analyzed 25-step processor had significantly lower accuracy than the other choices, and the comparison suggests more than two fine-level message-passing steps are beneficial.The 25-step processor performed similarly to the 15-step configuration; the authors say they did not investigate this further.Source: §A.2 · p. 10
  • After ten rollout steps, MS-MGN showed reduced error across the first 20–30 eigenvalues, corresponding to slowly varying mesh modes; no systematic difference appeared for one-step prediction.The analysis used a mesh with 10,117 nodes and edge min = 1.5 × 10^-3; the authors interpret the result as more accurate capture of slow-varying, long-range correlations for this problem.Source: §A.2 · p. 11

A.3. CylinderFlow Dataset

  • The CylinderFlow examples vary obstacle position and size, inflow initial velocity, and mesh resolution.Source: Figure A.4 · p. 13
  • The finest illustrated Fixed Obstacle test mesh has almost 30 times more nodes and edges than the coarsest mesh.Source: Figure A.5 · p. 14

Related papers

Loading 2210.00612v1…