PaperlayerView exact revision on arXiv

Source-linked AI summary

Mesh-Native Tokenisation: Retrofitting Foundation Models to Unstructured Data

Georg Maierhofer, Miles Cranmer, Payel Mukhopadhyay

arXiv:2610.12618v1cs.CEcs.AIcs.NAmath.NA

TL;DR

CNN tokenisers are efficient on regular grids, but unstructured-mesh problems require a different data regime that current mesh GNNs handle at greater training cost and with different encodings. MNC provides a CNN-like mesh tokeniser for retrofitting pretrained models, which can achieve acceptable zero-shot accuracy and outperform existing baselines after a few fine-tuning epochs.

  • Problem

    CNN tokenisers are efficient on regular grids, but many physical fields use unstructured meshes, where current GNNs are costlier to train and encode information differently, limiting emulator portability.

    Source: §1 · p. 1

  • Method

    MNC uses geometrically defined local windows and continuous convolution-like kernels to encode mesh data and transfer weights from grid-pretrained models.

    Source: §1 · p. 2

  • Results

    Retrofitted models reach acceptable zero-shot accuracy and outperform existing baselines after only a few fine-tuning epochs.

    Source: §1 · p. 2

  • Takeaways & Limitations

    MNC offers a route to transfer grid-pretrained physics emulators to unstructured-mesh problems while retaining the source processor.

    Source: §5 · p. 9

    MNC mainly improves sample efficiency and speeds fine-tuning, rather than improving overall accuracy after sufficient training.

    Source: §5 · p. 9

Abstract

from arXiv · show

Physics foundation models are an increasingly popular approach for building versatile emulators for spatiotemporal systems. They require tokenisation layers that lift physical field data into a latent representation suitable for dynamic processing. A typical choice for this layer is a convolutional encoder, which lifts field information in a locality-aware way and can be trained very efficiently. However, this choice cannot be applied to a large class of scientific computing problems where data are represented on unstructured meshes, for example simulations on domains with curved boundaries. Such data typically require graph neural network architectures, which are more expensive to train and, in the form currently used, encode information differently from their CNN counterparts, thus limiting the portability of physics emulators between these data regimes. In this work we introduce Mesh-Native Convolution (MNC), a locality-aware tokenisation layer that encodes unstructured mesh-based data in a way that is analogous to CNNs. This approach leads to models that can be trained once on mesh data and then deployed to arbitrary unstructured meshes, even if these are irregular, with minimal retraining. We show that this tokenisation approach can be used to retrofit grid-pretrained foundation models efficiently on unstructured meshes and provide theoretical results to confirm the consistency of this approach with classical CNN encoders. We demonstrate the favourable performance of this retrofitting approach, which in many cases reaches acceptable accuracy even in zero-shot deployment, and outperforms existing baselines after only a few epochs of fine-tuning, thus providing an efficient bridge between the two data regimes.

1 INTRODUCTION

Grid-based CNN tokenisers are efficient but exclude unstructured-mesh fields, while mesh GNNs are costlier and encode information differently. MNC provides a CNN-like local tokeniser for meshes, supporting pretrained-weight transfer and efficient retrofitting.

1 INTRODUCTION

  • CNN tokenisation is efficient on regular grids, but irregular boundaries and locally finer features often require unstructured meshes.Examples include engineering parts, coastal wave modelling, weather forecasting, fluid boundary layers, and turbulent structures behind jet engines.Source: §1 · p. 1
  • Graph neural networks (GNNs), though a natural choice for such data, are more expensive to train and currently encode information differently from CNNs, limiting physics-emulator portability between these data regimes (Pfaff et al., 2021).Source: §1 · p. 1
  • MNC retrofits reach acceptable zero-shot accuracy and outperform existing baselines after only a few fine-tuning epochs.The authors describe this as an efficient bridge between grid and unstructured-mesh data regimes.Source: §1 · p. 2

1.1 BACKGROUND AND RELATED WORK

  • Unlike approaches built around coordinate transforms, point-set attention, neural fields, or latent point-cloud encoders, MNC uses CNN-like physical locality for a simpler transition between data regimes.MNC combines continuous spline kernels and discretisation-invariant ideas in a mesh-native multilevel encoder-decoder designed for retrofitting grid-based surrogates.Source: §1.1 · p. 3

2 MESH-NATIVE TOKENISATION AND WEIGHT TRANSFER

MNC replaces grid windows with geometric supports on a mesh, using continuous spline kernels to gather and distribute field information. Its construction supports CNN weight transfer, exact grid reduction under stated conditions, and positional embeddings compatible with pretrained transformers.

2 MESH-NATIVE TOKENISATION AND WEIGHT TRANSFER

  • MNC extends a discrete convolution kernel continuously with splines, enabling its weights to be learned or initialized from pretrained CNN weights for mesh gather and scatter operations.Occupancy normalization preserves physical weighting during accumulation.Source: §2 · pp. 3–4
  • A mesh hierarchy of virtual anchors assigns each gathering operation a physical footprint; the encoder gathers across levels and the decoder distributes information while traversing them in reverse.The decoder reuses the hierarchy's connections in reverse, analogously to transposed convolution.Source: §2 · p. 4, Figure 3 · p. 4
  • On a source grid, MNC exactly reproduces unit-dilation Conv2d and, under the stated stride and padding conditions, ConvTranspose2d.The construction requires matching source-grid spacing, physical kernel spacing, anchors, padding, and loaded weights.Source: §2 · p. 4
  • For transformer processors using axial RoPE, MNC evaluates the same positional embedding at anchor coordinates, preserving source frequencies, channel pairing, and temporal embedding.On a regular grid, this construction reduces to standard RoPE.Source: §2 · pp. 4–5

3 A FINITE-WINDOW OPERATOR INTERPRETATION FOR MNC

MNC can be interpreted as approximating a finite-window continuous operator, with a limit that is independent of the specific mesh under equidistributed refinement. The section contrasts this behavior with graph convolution, whose outputs can depend on mesh connectivity.

3 A FINITE-WINDOW OPERATOR INTERPRETATION FOR MNC

  • Under equidistributed refinement, MNC outputs at common anchors converge to the same limit for the same field and limiting kernel.The stated result assumes a fixed-support Lipschitz kernel, zero bias, an anchor window inside the domain, and a positive limiting sampling density.Source: §3 · p. 5
  • The limit corresponds to a continuous kernel operator, so sufficiently fine equidistributed meshes yield outputs independent of the specific mesh used.The paper characterizes this as physical consistency with the underlying continuous process.Source: §3 · p. 5

4 EXPERIMENTS

The experiments test MNC retrofits on three Well datasets and the MeshGraphNets cylinder-flow benchmark, comparing zero-shot transfer and mesh fine-tuning against mesh-native baselines. Models predict the next step from six input frames and are evaluated on next-step and ten-step rollout VRMSE.

4.1 EXPERIMENT 1: ZERO-SHOT TRANSFER ONTO AN IRREGULAR MESH

  • Zero-shot Overtone+MNC retains the most pretrained skill on all three datasets under both metrics, outperforming all baselines trained from scratch on grid representations.Table 1 reports next-step and ten-step rollout VRMSE in units of 10^-2.Source: §4.1 · p. 6, Table 1 · p. 7

4.2 EXPERIMENT 2: FINE-TUNING ON IRREGULAR MESHES

  • The MNC retrofit reaches its fine-tuning plateau within 10 epochs, while the same MNC-tokenized transformer trained from scratch needs about 100 epochs to reach the same level.Grid-pretrained baselines improve little on meshes; MeshGraphNets levels off above the retrofit.Source: §4.2 · p. 7, Figure 6 · p. 8
  • After fine-tuning, MNC achieves the lowest next-step and rollout VRMSE on TRL2D, active matter, and shear flow; its errors are a factor of two to three below the best baseline on active matter and shear flow.Grid pretraining reduces MNC next-step error by about a quarter on active matter and a third on shear flow, while pretrained and scratch MNC models end at similar levels on TRL2D.Source: §4.2 · p. 7, Table 2 · p. 8

4.3 EXPERIMENT 3: ZERO-SHOT RETROFITTING OF A LARGE-SCALE FOUNDATION MODEL

  • On unseen cylinder flow, retrofitted Walrus beats every trained baseline except Transolver++ in zero-shot evaluation and achieves the lowest next-step error after fine-tuning, with a shared 7.5h compute budget.Cylinder flow was not included in Walrus’s pretraining corpus.Source: §4.3 · pp. 8–9, Table 3 · p. 9

5 DISCUSSION AND CONCLUSION

MNC retrofits grid-pretrained physics models to unstructured meshes while retaining the source processor, and the paper reports efficient fine-tuning and zero-shot transfer. Its grid consistency results rely on matched convolutional settings, while the authors identify sampling-density consistency as future work.

5 DISCUSSION AND CONCLUSION

  • MNC transfers grid-pretrained weights to mesh tokenisation while retaining the source processor, enabling zero-shot transfer and efficient fine-tuning on irregular meshes.The authors also report computational savings from repurposed weights and reduced processor training time.Source: §5 · p. 9
  • MNC mainly improves sample efficiency and shortens fine-tuning, rather than improving accuracy after sufficient training.The authors identify limited training data or computational resources as settings where its benefits are most pronounced.Source: §5 · p. 9
  • Future work includes geometric quadrature weights for varying sampling density, implicit time-stepping initial guesses, and physics symmetries in the kernel.The authors propose these as directions for further development.Source: §5 · p. 10

A.1 PROOF OF PROPOSITION 1

  • With matched grid spacing, anchors, and padding, MNC exactly reproduces unit-dilation Conv2d; transposed-convolution equivalence requires matching stride and output-padding settings.For zero padding, the window must include zero-valued nodes whose occupancy still counts.Source: §A.1 · p. 14

A.2 MULTI-LEVEL HIERARCHIES AND THE PROCESSOR INTERFACE

  • Across hierarchy levels, preserving pointwise operations, activations, normalisation, token order, and coordinate-derived rotary positions yields identical grid-model tensors and, with matching history updates, rollouts.Finite-precision scatter reductions can still cause small numerical discrepancies.Source: §A.2 · p. 14, Figure 8 · p. 15

A.3 PROOF OF PROPOSITION 2

Proposition 2 establishes a continuum limit for MNC under mesh-refinement assumptions. It implies asymptotically identical outputs at common anchors for equidistributed refinements of the same field and limiting kernel.

A.3 PROOF OF PROPOSITION 2

  • The limit assumes mesh nodes converge weakly to a positive sampling density as maximum cell spacing shrinks, with a continuous field and a Lipschitz kernel.The stated result concerns anchors whose full window lies inside the domain and uses zero bias.Source: §A.3 · p. 15
  • Two equidistributed refinements of the same field, with the same limiting kernel, have asymptotically identical outputs at common anchors.The paper describes this as independence from the specific mesh for sufficiently fine meshes when the kernel is appropriately defined.Source: §A.3 · pp. 15–16
  • For constant sampling density, density and window-area factors cancel; the proof applies the limit to two sequences to obtain asymptotic agreement.The argument uses interpolation, occupancy normalisation, and weak convergence.Source: §A.3 · pp. 15–16

A.4 A CONNECTIVITY COUNTEREXAMPLE FOR A NORMALISED GCN

On the same sampled constant field and vertex locations, a degree-normalised GCN produces different origin outputs for two triangulations, while MNC produces identical outputs.

A.4 A CONNECTIVITY COUNTEREXAMPLE FOR A NORMALISED GCN

  • The example uses two triangulations of the same vertices, with every square split along a different diagonal pattern.Both consist of right isosceles triangles with diameter √2η.Source: §A.4 · pp. 16–17, Figure 9 · p. 16
  • The degree-normalised GCN outputs 1 on mesh A but approximately 1.1518403495 on mesh B for the same constant field.Both values are positive, so ReLU leaves them unchanged; the discrepancy persists for any positive scalar weight.Source: §A.4 · pp. 16–17
  • MNC returns identical outputs on the two meshes because common anchors, physical kernel spacings, and coefficients give it the same geometric neighbours and occupancies.The example isolates connectivity: sample locations do not change, and both triangulations represent the same constant field exactly.Source: §A.4 · p. 17

B DATASETS

The experiments cover TRL2D, shear flow, active matter, and cylinder flow using grid twins or unstructured meshes. Cylinder-flow meshes are additionally refined for Experiment 3, while evaluation remains on the original nodes.

B DATASETS

  • Each sample uses six consecutive context snapshots and the following target; forecasting uses specified dataset splits, while reconstruction uses official splits throughout.Forecasting uses official TRL2D and active-matter splits and a trajectory-level resplit of shear-flow training files.Source: B DATASETS · p. 17
  • TRL2D models a radiatively cooling two-dimensional mixing layer on a 128×384 grid, with density, pressure, and two velocity channels across nine cooling times.Its mesh twin has 49,152 nodes and 146,433 edges; pressure is nearly uniform, so reconstruction reports normalized MSE rather than ill-conditioned VRMSE.Source: B DATASETS · p. 17, Figure 10 · p. 17, Figure 11 · p. 18
  • Shear flow spans 28 Reynolds- and Schmidt-number combinations, and test trajectories include both Kelvin–Helmholtz billows and near-laminar regimes.Forecasting uses an irregular mesh; reconstruction transfer also tests a uniform triangular lattice and an irregular Delaunay mesh.Source: B DATASETS · p. 18, Figure 12 · p. 18, Figure 13 · p. 19
  • Active matter uses eleven channels, including concentration, velocity, orientation-tensor, and strain-rate fields, on a 256×256 grid.Its Experiment 1 irregular Delaunay mesh has 65,632 nodes; periodicity is not built into the mesh, and no lattice twin is stored.Source: B DATASETS · p. 19, Figure 14 · p. 20
  • Cylinder flow is the only dataset without a grid representation: each trajectory has its own unstructured mesh, refined toward the cylinder and walls.The fields are pressure and velocity, with node types marking inflow, outflow, and wall boundaries.Source: B DATASETS · p. 20, Figure 15 · p. 20

B.1 MESH REFINEMENT FOR CYLINDER FLOW

  • For Experiment 3, each cylinder-flow mesh is uniformly refined three times by splitting triangles and averaging endpoint values at new midpoints.This reproduces the original piecewise-linear field without adding information; predictions are scored on the original mesh nodes.Source: §B.1 · p. 21

C MODEL DETAILS

The model and forecasting protocol pair grid-pretrained convolutional sources with mesh-native tokenisation, defined mesh sampling, and shared evaluation conventions. The protocol specifies dataset windows, normalization, mesh construction, and trajectory-level metrics.

C MODEL DETAILS

  • The baselines use their official or reference implementations where possible, and their complete-model parameter counts and shared training settings are summarized in Tables 5 and 6.The four-channel dataset counts are reported separately from the eleven-channel active-matter counts; MNC and its grid source have identical counts.Source: C MODEL DETAILS · p. 21, Table 5 · p. 21, Table 6 · p. 22
  • MNC retrofits a patch-modulated forecasting transformer by converting its convolutional encoder and decoder kernels while preserving the source token count and processor.The source’s four tokeniser kernels are converted into MNC weights, and a grid-point check verifies relative error below 10^-3 after transplantation.Source: C MODEL DETAILS · pp. 21–22
  • For cylinder flow, MNC replaces only Walrus’s outermost convolutions, reuses its kernels without adding parameters, and retains its processor and inner stages.Walrus is the released 1.3B-parameter checkpoint; its meshes are refined three times, with scores on original nodes.Source: C MODEL DETAILS · p. 23

D FORECASTING PROTOCOL

  • Forecasting experiments use one GH200 GPU per run, with official splits for TRL2D and active matter and a trajectory-stratified shear-flow resplit.Samples use six context frames and the next target; final evaluation uses 855 TRL2D windows and 1,024 windows each for active matter and shear flow.Source: D FORECASTING PROTOCOL · p. 23, §D.1 · p. 23

D.2 NORMALISATION AND TRAINING OBJECTIVE

  • Training windows are independently normalized from context frames and increments, and models predict increments that are added to the final context frame.The loss is area-weighted MSE; MNC trains at all six causal positions, while non-causal baselines predict only the final increment.Source: §D.2 · pp. 23–24

D.3 MESHES AND FIELD RESAMPLING

  • Irregular meshes are Delaunay triangulations with node counts matched to the grids, and their field values are bilinear interpolants of the same simulation arrays.MNC hierarchy spacings and window origins inherit the source grid, and source and target hierarchies retain equal token counts.Source: §D.3 · p. 24

D.4 METRICS

  • Metrics use denormalized physical fields and area-weighted statistics, with next-step and ten-step rollout VRMSE aggregated by trajectory and reported as medians.Rollouts feed predictions back into the context without further ground truth, recomputing normalization statistics at each step; rollout starts occur at five times per test trajectory.Source: §D.4 · p. 24

D.5 EVALUATION SCHEDULE AND REPORTED CHECKPOINTS

Evaluation uses fixed test windows and final checkpoints, with zero-shot scores measured before mesh updates. Cylinder-flow Experiment 3 scores predictions on the original nodes of the refined meshes.

D.5 EVALUATION SCHEDULE AND REPORTED CHECKPOINTS

  • Reported results use final checkpoints; validation is monitored but does not select models or checkpoints.Sources are evaluated every 5,000 of 50,000 updates and mesh runs every 1,000 of 20,000 updates.Source: §D.5 · pp. 24–25
  • Experiment 1 zero-shot scores evaluate the grid-pretrained source transplanted to the irregular mesh before any mesh updates.Table 1 spreads reflect three grid-source seeds; Experiment 2 mesh-training spreads reflect three mesh-training seeds starting from the seed-0 source.Source: §D.5 · p. 25

D.6 EXPERIMENT 3: CYLINDER FLOW

  • Cylinder-flow Experiment 3 scores refined-mesh predictions on original nodes, using 160 next-step windows and separate ten-step rollout starts.VRMSE weights nodes equally, averages the pressure and two velocity-component errors, and reports the median over windows.Source: §D.6 · p. 25

E ADDITIONAL NUMERICAL RESULTS

On one irregular-TRL2D test window, the retrofitted Overtone+MNC model resolves the mixing layer zero-shot and has a fainter error field than the baselines at any stage; fine-tuning reduces its error further.

E ADDITIONAL NUMERICAL RESULTS

F TOKENISER RECONSTRUCTION AND TRANSFER

In reconstruction experiments, MNC matches CNN on Well grid twins and has the lowest error on native cylinder meshes. CNN-pretrained weights transferred to MNC yield rapid mesh-training recovery, with the reported comparison limited to tokenisers and reconstruction protocols.

F TOKENISER RECONSTRUCTION AND TRANSFER

  • The appendix compares encoder-decoder tokenisers without latent processors; complete GINO and LNO forecasting models are larger and include their own processors.CNN and MNC have about 17k parameters, GINO about 22k, and LNO about 21k with 32 latent modes.Source: F TOKENISER RECONSTRUCTION AND TRANSFER · p. 25
  • MNC matches CNN on the Well grid twins and has the lowest reconstruction error on native cylinder-flow meshes.The Well models use identical samples; cylinder flow uses native meshes and requires no grid representation.Source: F TOKENISER RECONSTRUCTION AND TRANSFER · p. 25, Table 7 · p. 25
  • MNC reaches about 1.5 × 10^-3 normalised MSE after one mesh-training epoch, close to its scratch-trained error after 100 epochs.It starts near 10^-2; Figure 17 compares transferred models with scratch-trained counterparts across three seeds.Source: F TOKENISER RECONSTRUCTION AND TRANSFER · p. 25, Figure 17 · p. 27
  • CNN→MNC preserves a more accurate field representation and makes it available after far less mesh training.The comparison concerns trained-tokeniser usefulness, not remeshing penalties at matched source accuracy.Source: F TOKENISER RECONSTRUCTION AND TRANSFER · p. 26
  • Grid-to-mesh transfer pretrains CNN, GINO, and LNO on a 256 × 512 shear-flow grid, then fine-tunes on bilinearly resampled irregular-mesh data.Scratch baselines use the same target architectures, data, and mesh-training budgets.Source: F TOKENISER RECONSTRUCTION AND TRANSFER · p. 27

F.1 DATA, ARCHITECTURE AND TRAINING

Cross-domain reconstruction transfer has qualified outcomes: MNC has the lowest error after 100 target epochs, but its pretraining advantage over scratch is modest. The appendix also reports changed-discretisation controls, tokeniser configurations, execution costs, and the reconstruction protocol.

F.1 DATA, ARCHITECTURE AND TRAINING

  • The reconstruction protocol trains only the latest frame, uses six context frames for samplewise per-channel RMS normalisation, and evaluates normalised-variable MSE.Training uses Adam at 10^-3 for 100 epochs of 200 sampled windows at batch size one, without warm-up or a learning-rate schedule; reconstruction MSE is not physical-field VRMSE.Source: §F.1 · p. 27

F.2 GINO AND LNO TOKENISER CONFIGURATIONS

  • GINO kernel weights are independent of latent-grid size, while the main 32-mode LNO comparison uses a smaller bottleneck; increasing LNO modes improves shear reconstruction but does not close MNC’s gap.The larger-mode ablation is direct training, not a latent-parity transfer experiment.Source: §F.2 · pp. 27–28, Table 8 · p. 28

F.3 EXECUTION COST

  • MNC is approximately six times slower than dense CNN execution on the Well grid twins and is not uniformly faster than other mesh tokenisers.Table 9 reports per-epoch reconstruction-study timings on one NVIDIA GH200, excluding one-off geometry construction; these are not timings of the larger forecasting models.Source: §F.3 · p. 28, Table 9 · p. 28

F.4 GRID-TO-MESH TRANSFER AND COMPANION LATTICE CONTROL

  • Transferred MNC’s relative accuracy and fast initial recovery persist on a uniform triangular lattice, but changed sample positions mean neither comparison tests exactness on identical inputs.Target meshes are matched in resolution and fields are bilinearly resampled.Source: §F.4 · p. 28, Figure 19 · p. 29, Figure 18 · p. 29

F.5 CROSS-DOMAIN TRANSFER TO NATIVE CYLINDER MESHES

  • After 100 target epochs, MNC has the lowest cross-domain error, but its pretraining benefit over scratch is modest; these results do not establish transfer to forecasting or arbitrary cross-physics settings.With mixed-source pretraining, its initialisation improves substantially; after 100 epochs, the reported values are 0.023–0.025 versus 0.026 for its scratch run.Source: §F.5 · p. 29, Table 10 · p. 30

Related papers

Loading 2610.12618v1…