PaperlayerView exact revision on arXiv

Source-linked AI summary

Learnable Hodge-Laplacian Driven Mixture of Experts for Steady-PDE Surrogates on Coarse Unstructured Meshes

Stavros Nousias, Akis Nousias, Konstantinos Gkrispanis

arXiv:2610.20585v3cs.LGcs.CV

TL;DR

Steady RANS simulations make evaluating many aerodynamic geometries computationally expensive, and standard MoE routing does not prescribe expert specialization. The paper uses a learnable mesh Hodge-Laplacian controller to mix transformer experts per vertex and reports lower aggregate errors than Transolver on the AirFRANS working mesh.

  • Problem

    Steady RANS simulation makes evaluating many aerodynamic geometries computationally expensive, while standard MoE routing does not prescribe expert specialization.

    Source: §1 · p. 2

  • Method

    A learnable mesh Hodge-Laplacian controller mixes transformer experts per vertex, while a separate Laplacian-based scheme supplies auxiliary targets from reference fields.

    Source: §1 · p. 2, §3.6 · p. 6

  • Results

    31% lower mean relative L2 error (0.0436 to 0.0300) and 47% lower interior mean squared error (0.00265 to 0.00140) than Transolver on the AirFRANS working mesh.

    Source: §5 · p. 9

  • Takeaways & Limitations

    On the AirFRANS working mesh, the model has lower aggregate errors than Transolver, with its largest per-field improvement in turbulent viscosity.

    Source: §5 · p. 9

    Evaluation uses one run per configuration on a remeshed working mesh and does not establish accuracy at original CFD resolution or robustness across random seeds.

    Source: §5 · p. 10

Abstract

from arXiv · show

Current approaches for steady partial differential equations surrogate models on unstructured meshes apply a single mixing mechanism to the whole solution. This paper proposes an architecture that partitions the solution among vanilla transformer experts, each focused on a different spectral scale. The partition is built from a discrete Hodge Laplacian $L_0=\star_0^{-1} d_0^{\top}\star_1 d_0$, that serves as a fitting layer measuring high-frequency content. A gating mechanism takes the mesh as input and produces the per-vertex weights that recombine the expert outputs. AirFRANS is a steady Reynolds-averaged Navier--Stokes (RANS) airfoil benchmark. We evaluate our approach in a reduced remeshed version of the benchmark with each unstructured mesh counting 9.6K faces and 4.9K vertices. We compare our approach with the Transolver on the same dataset and dataset slit. Our approach reduces relative $L_2$ error by 31% and interior mean squared error by 47%.

1 Introduction

The section presents a transformer mixture of experts with distinct spectral-scale biases, routed by a learnable Hodge-Laplacian controller, and describes auxiliary expert targets and the losses used to train predictions and routing. It also outlines evaluation against Transolver on the official AirFRANS split under a shared data, mesh, loss, and metric protocol.

1 Introduction

  • The proposed model routes among transformer experts at each vertex using a learnable Hodge-Laplacian controller and supervises them with targets decomposed by a second Laplacian.The controller computes routing weights; the separate diagonal-star Laplacian decomposes the training target into components.Source: §1 · p. 2

2 Related Work

  • Unlike prior geometric gates for PDE experts, the model uses a mesh Hodge-Laplacian controller to mix full transformer experts and supervises auxiliary heads with mesh spectral bands.GNOT gates using query coordinates, while GAOT combines branches at different neighbourhood radii [15] [24].Source: §2 · p. 3

3.1 Overview

  • The model predicts velocity, pressure, and turbulent-viscosity fields through shared embeddings, a controller, transformer experts, and a field decoder.During training, a separate diagonal-star vertex Laplacian smooths reference fields into telescoping atoms for expert-specific supervision.Source: §3.1 · p. 3

3.2 Simplicial complex and neighborhoods

  • The controller’s vertex and edge neighborhoods are capped at 64 and 32 entries, respectively, and truncation can remove reciprocal neighbor relations.Face neighborhoods contain only the face itself; retained neighbors are ordered by element index and are not necessarily the nearest.Source: §3.2 · p. 4

3.3 Input features and input embeddings

  • Vertex inputs include mesh and flow features, and a shared two-layer perceptron embeds them into one token per vertex for all eight experts.The listed vertex features include barycentric dual area, inlet velocity, signed distance, normals, and a wall indicator; columns are standardized.Source: §3.3 · p. 4

3.4 Hodge controller

  • The star is symmetric positive-definite with eigenvalues in [1 − 2β, 1] when effective neighborhood support is reciprocal; configured caps and truncation can break reciprocity, voiding that guarantee.Under reciprocity, D−1/2ΩD−1/2 is symmetric with eigenvalues in [−1, 1].Source: §3.4 · p. 5

3.5 Transformer experts and decoder

  • Eight transformer experts differ by Fourier-feature scale, and softmax routing combines their representations without reducing computation because every expert runs at every vertex.Each expert also predicts an auxiliary output trained against its assigned target; that prediction does not form the final field prediction.Source: §3.5 · p. 6

3.6 Auxiliary targets

  • Auxiliary targets are built by repeated conductance-weighted smoothing of reference fields, with differences between smoothing levels forming components assigned to experts.The final field prediction is trained against the original reference fields, not the decomposed components.Source: §3.6 · pp. 6–7
  • The nine averaging-based atoms sum to the field, but their overlapping spectral responses mean they are not an orthogonal decomposition or disjoint frequency bands.The first atom is the field after 32 averaging steps; intermediate atoms capture changes between retained levels, and the final atom captures variation removed by the first two steps.Source: §3.6 · p. 7
  • With 9 atoms and 8 experts, learned weights form each expert’s target; the same assignment applies at every vertex and case, and the targets sum algebraically to the field.Each atom is fully distributed among expert targets, and each expert predicts the four fields in its assigned target; this assignment is separate from input-dependent routing.Source: §3.6 · pp. 7–8
  • During training, atoms are detached when targets are formed, allowing assignment scores to update without gradients flowing through atoms to analyzer conductances; expert targets are not frequency-ordered.Assignments can combine multiple atoms in one target or distribute an atom across multiple targets.Source: §3.6 · p. 8

3.7 Training Objective

  • The training objective matches final predictions to reference fields and expert outputs to assigned targets, while penalizing imbalanced average routing and experts assigned targets with very small shares of total squared norm.The loss uses λb = 0.5, λbal = 0.01, λfloor = 1.0, and ρ = 0.02; field residual MSE is computed over non-wall and wall vertices.Source: §3.7 · p. 8

4 Experiments

The experiments evaluate models on remeshed AirFRANS cases, with training meshes resampled and evaluation using a fixed mesh per case. The proposed model leads on aggregate relative L2 and interior error, while Transolver is better on the two wall quantities.

4.1 Simulation Setup

  • Each training read randomly draws one of up to eight remeshes per case, while evaluation uses one fixed mesh per case.The official split is 800/200, and the meshes have approximately 4.9k vertices with refinement near the airfoil.Source: §4.1 · p. 8
  • The reported relative L2 and R2 metrics use physical units; Volume and Surf are normalized mean squared errors on non-wall and wall vertices, respectively.All models are compared against interpolated targets on the same working meshes, so the results are not directly comparable with scores on original meshes.Source: §4.1 · p. 8, Table 1 · p. 9
  • The checkpoint was selected by test-set error, whereas baselines used validation-based selection and reserved 80 training cases for validation, limiting attribution to architecture alone.The proposed model trained on all 800 training cases.Source: §4.1 · p. 9

4.2 Results

  • 31% lower relative L2 error and 47% lower interior mean squared error than Transolver, with approximately half as many parameters.The aggregate relative L2 is the mean over four fields; Transolver has lower surface error by a factor of 1.14 and lower lift-coefficient error by a factor of 2.2.Source: §4.2 · p. 9, Table 1 · p. 9
  • The illustrated test case has per-field relative L2 errors of 0.0047, 0.0260, 0.0281, and 0.0676 for ux, uy, p, and νt, respectively.Test-set means are 0.0056, 0.0259, 0.0255, and 0.0629; visible errors occur near the airfoil and along the turbulent-viscosity wake.Source: §4.2 · p. 9
  • For νt, mean relative L2 errors are 0.063, 0.116, 0.144, and 0.155 for our model, Transolver, PointNet, and PSD-star HodgeFormer; for p, they are 0.026, 0.030, 0.066, and 0.085.The models separate more on turbulent viscosity than pressure, where the reference shows fine-scale wake structure.Source: §4.2 · p. 9

5 Discussion and Conclusion

The paper concludes that Hodge-routed transformer experts reduce aggregate errors relative to Transolver on the AirFRANS working mesh. The gains coexist with higher surface and lift errors, and the evaluation does not establish original-resolution accuracy or robustness across seeds.

5 Discussion and Conclusion

  • Mean relative L2 and interior mean squared errors decrease by 31% and 47% relative to Transolver, from 0.0436 to 0.0300 and 0.00265 to 0.00140, respectively.The largest per-field improvement is for turbulent viscosity, whose relative L2 error falls approximately 46%, from 0.1158 to 0.0629.Source: §5 · p. 9
  • Pressure relative L2 is lower than Transolver’s, but surface and relative lift-coefficient errors are higher: 0.00164 versus 0.00143 and 0.0225 versus 0.0103.The lift rank correlation remains above 0.9998 for the compared models, preserving case ordering more accurately than force magnitudes.Source: §5 · p. 9, Figure 3 · p. 10
  • Evaluation uses one remeshed working mesh of approximately 4.9K vertices and one training run per configuration; accuracy at original CFD resolution and robustness across random seeds are not established.The implementation also does not enforce the reciprocal neighborhoods required for the controller’s conditional spectral guarantee, and prediction accuracy does not validate that guarantee.Source: §5 · p. 10

Supplementary Material

The supplementary material collects notation, model background, implementation and evaluation details, configurations, and results. It separately reports controller comparisons and identifies which checkpoint underlies the analyzer and routing measurements.

Supplementary Material

  • The supplement provides notation, HodgeFormer background, implementation and training details, model configurations, and results.Controller comparisons report vertex-only and vertex-edge-face configurations separately; the analyzer and routing measurements use the vertex-only checkpoint at epoch 380.Source: Supplementary Material · p. 13

S1 Notation

The notation section groups symbols by role and flags context-specific uses. It distinguishes controller and analyzer stars, and gives array dimensions without the batch dimension.

S1 Notation

  • Controller stars and analyzer stars denote separate operators; superscripts on decoder tokens indicate block depth, and subscripts on assigned targets identify experts.In HodgeFormer background, h counts attention heads and σ denotes softmax; elsewhere these symbols have different uses.Source: S1 Notation · p. 13
  • Table S1 groups the paper’s main symbols by role, identifies HodgeFormer- and transformer-specific uses, and omits the batch dimension from array sizes.The symbols xv, xe, and xf denote input features and normalized features used to compute mesh operators.Source: Table S1 · p. 13, S1 Notation · p. 13
  • The notation distinguishes stored neighborhoods from effective neighborhoods and defines the mutual property as j in N(i) implying i in N(j).The listed configuration labels V and VEF refer to vertex-only updates and vertex, edge, and face updates.Source: S1 Notation · p. 14

S2 HodgeFormer Background

HodgeFormer extends Transformer layers to vertex, edge, and face features, using learned Hodge operators to update each feature stream. Its layer combines mesh-operator attention with residual feedforward updates.

S2.1 HodgeFormer

  • HodgeFormer [21] extends Transformer layers to vertex, edge, and face features, with separate two-layer feedforward networks and residual connections for each stream.The feature arrays share embedding dimension d; attention heads apply learned mesh operators to projected values before their outputs are joined and projected back to the embedding width.Source: S2.1 HodgeFormer · p. 16
  • Learned Hodge stars are parameterized through row-wise softmax attention, with separate linear maps for inverse stars.Products of star and signed incidence matrices define the operators used in attention updates.Source: S2.1 HodgeFormer · p. 16
  • Each attention head applies its corresponding learned operator to projected vertex, edge, or face values before the head outputs are joined.The vertex update depends on vertex and edge features, the edge update on all three streams, and the face update on edge and face features.Source: S2.1 HodgeFormer · p. 16

S3 Methodology

The methodology uses a vertex-edge-face controller to produce routing scores, while transformer experts receive positional Fourier features. A separate analyzer constructs auxiliary targets for training; the paper’s spectral guarantees are conditional on reciprocal neighbor support.

S3.1 Overall Controller Architecture

  • The VEF controller applies two PSD-Hodge layers to vertex, edge, and face features, then reads vertex features to produce routing logits converted to per-vertex softmax weights.Each stream has 64 channels; layers use four heads of width 16 and feedforward width 256, with separate normalization, projections, and feedforward networks.Source: S3.1 Overall Controller Architecture · pp. 16–17

S3.2 Conditional spectral properties

  • The vertex-operator spectral guarantee requires reciprocal retained neighbors and SPD stars; the configured neighbor procedure does not ensure reciprocal pairs, and the full controller block is not a PSD linear operator.The edge-operator proof does not establish nonnegative eigenvalues for its summed terms, and the conditional vertex bound uses ideal normalization.Source: S3.2 Conditional spectral properties · p. 17

S3.3 Expert Fourier Bandwidths

  • Experts receive eight sine-and-cosine Fourier coordinate features at fixed sampled scales, but those scales do not constrain an expert to a particular frequency band.The learned atom assignment sets auxiliary targets independently of the ordering of Fourier scales.Source: S3.3 Expert Fourier Bandwidths · pp. 17–18

S3.4 Initialization and parameter counts

  • The reported trainable-parameter totals are 3,106,490 for VEF and 3,023,962 for the vertex-only comparison, including unused parameters and analyzer parameters without training-loss gradients.The fixed Fourier matrices are stored separately from trainable parameters.Source: S3.4 Initialization and parameter counts · p. 18

S3.5 Analyzer and assignment gradients

  • A training-only analyzer builds nine target atoms from smoothed reference fields and their differences; detached atoms let auxiliary losses update assignment logits without updating conductances.The column-normalized assignment makes assigned targets sum to the reference field, without requiring orthogonal atoms or a fixed expert order.Source: S3.5 Analyzer and assignment gradients · p. 18

S4 Supplementary material on the training objective

The training objective combines field losses with auxiliary expert and routing terms. Its target-share penalty discourages nearly empty auxiliary targets but does not impose a hard minimum or an orthogonal spectral partition.

Training objective

  • Field losses compare the final prediction with the reference after channel-wise standard-deviation scaling, with separate mean squared errors for non-wall and wall vertices.The disjoint vertex sets give wall errors a contribution independent of the number of interior vertices; the auxiliary loss compares each expert output with its assigned four-channel target.Source: S4 Supplementary material on the… · p. 18
  • The routing penalty encourages each expert’s mean routing weight to approach 1/K while allowing individual vertices to use different weights.This penalty concerns how the controller combines expert features.Source: S4 Supplementary material on the… · p. 18
  • The target-share penalty discourages nearly zero auxiliary targets without requiring equal norms, but it is soft and the shares are not an orthogonal partition of the field’s squared norm.Atoms are detached before assignment, so the auxiliary term and this penalty update assignment scores without gradients passing through the atoms to analyzer conductances.Source: S4 Supplementary material on the… · pp. 18–19

S5 Experimental Protocol

The protocol evaluates Hodge-routed models on the official AirFRANS split and specifies their input normalization, training, and mesh-neighbor construction. Test-set checkpoint selection and differences in training cases limit attribution of baseline differences to architecture alone.

S5 Experimental Protocol

  • Tables S2 and S3 list training and architecture settings for the proposed VEF configuration and the vertex-only comparison.Lift uses the same contour on the working mesh for every model; the separate contour comparison uses a subset of cases and the vertex-only checkpoint.Source: S5 Experimental Protocol · p. 19, Table S2 · p. 19, Table S3 · p. 19

Input arrays and normalization

  • Vertex inputs comprise nine standardized columns; edge inputs contain endpoint and opposite-vertex coordinates, normals, and distances, while face inputs contain triangle coordinates, normals, and area.The target channels are scaled using freestream speed and fixed training statistics, then evaluation reverses those scalings for physical-unit metrics.Source: Input arrays and normalization · pp. 19–20

Training cases and checkpoint selection

  • Hodge-routed models train on all 800 official cases and select checkpoints by mean relative L2 error on the 200 test cases, while baselines train on 720 cases and select by validation error.The Hodge-routed checkpoints are selected among 30 VEF checkpoints and 20 vertex-only checkpoints; these protocol differences limit architecture-only attribution.Source: Training cases and checkpoint selection · p. 20

Neighbor lists

  • Neighbor lists are capped and truncated after reverse-pair insertion, so the final retained lists may omit one direction and fail the reciprocity condition required by the spectral bounds.The controller uses 64 vertex and 32 edge entries for VEF, versus 64 vertex and 64 edge entries for the vertex-only configuration.Source: Neighbor lists · pp. 20–21

Vertex-only training curves

  • Figure S1 reports vertex-only training curves, with the epoch-380 checkpoint selected by mean relative L2 error; the training objective includes auxiliary terms absent from test field loss.Relative L2 errors at epochs 360, 380, and 400 are 0.0307, 0.0300, and 0.0301, respectively.Source: Vertex-only training curves · p. 21, Figure S1 · p. 21

S6 The Model Family

The model family separates mesh-free predictors, frequency-mixture models, Hodge-routed controllers, Hodge predictors, and external baselines according to how mesh structure enters prediction or supervision.

Mesh-free predictors

  • Mesh-free variants range from global-attention plain transformers to geodesic-windowed or distance-biased attention, with a wider model and a lift-loss variant also included.These variants use the per-vertex channels without incidence operators or neighbor supports in prediction.Source: S6 The Model Family · p. 21

Frequency mixtures of experts with a feature-space router

  • Frequency-mixture models combine transformer experts with per-vertex weights from an MLP on shared vertex latents, while mesh structure enters auxiliary target decomposition.Fixed-band MoE uses four experts and a fixed adjacency ladder; learned-band MoE uses eight experts, a diagonal-star analyzer, learned atom assignment, and detached atoms.Source: S6 The Model Family · pp. 21–22

Hodge-routed mixtures

  • Hodge-routed VEF uses mesh operators to produce gate logits, while its transformer experts, target assignment, and decoder follow the learned-band construction.The vertex-only variant retains position-wise edge and face feedforward updates; its comparison with VEF also differs in edge support, training duration, and checkpoint count.Source: S6 The Model Family · p. 22

Hodge predictors

  • Hodge predictors keep the learned operator in the prediction path, unlike controllers that use it to generate routing weights.The cross-field decoder and disjoint-surface-loss variants modify the PSD-star HodgeFormer; the latter has the lowest mean relative L2 among the three Hodge-predictor rows in Table S6.Source: S6 The Model Family · p. 22

External baselines

  • External baselines include Transolver, PointNet, and a pointwise MLP, with input arms differing in their supplied channels.Their all-input arm has eight channels and omits the wall-indicator channel supplied to the paper's predictor.Source: S6 The Model Family · p. 22

S7 Input-Feature Ablation of the Baseline

The Transolver input-feature ablation retrains one architecture across four channel arms on the same split and metric code. The arms differ in their field-metric results and best-performing metrics.

S7 Input-Feature Ablation of the Baseline

  • All four input arms use the same Transolver architecture, split, and metric code, while differing only in per-vertex channels.The all-input arm is the one reported in the main paper.Source: S7 Input-Feature Ablation of the… · p. 23
  • 0.0436 to 0.0592 is the reported spread across Transolver arms on the field metrics.The pia arm, which adds dual area without signed distance or wall normals, has the highest relative L2; all-input has the lowest relative L2 and Volume errors.Source: S7 Input-Feature Ablation of the… · p. 23
  • The best arm varies by metric: position-plus-inlet has the highest R2 and lowest lift error, while SDF-plus-normals has the lowest Surf error.The arms contain four, five, seven, and eight channels, respectively, and none includes the predictor's wall-indicator input.Source: S7 Input-Feature Ablation of the… · p. 23

S8 Complete Results

The complete-results section reports selected-checkpoint comparisons on the official AirFRANS split and distinguishes those from separate checkpoint sweeps and a 10-case contour comparison. It also reports VEF and vertex-only lift errors over the test set.

S8 Complete Results

  • The complete comparison uses the official 800/200 split and approximately 4.9k-vertex meshes, with the checkpoint selected by mean relative L2 for each run.Table S6 reports the complete comparison; Table S7 gives selected-model per-field relative L2, and Table S8 reports per-field Volume and Surf at the selected checkpoints.Source: Table S6 · p. 24, Table S7 · p. 24, S8 Complete Results · p. 24
  • The vertex-only Volume value in the complete comparison is 0.00140 at the relative-L2-selected checkpoint, not its separate minimum of 0.00139.VEF's minimum mean relative L2 and minimum Volume occur at epoch 560; the vertex-only minima occur at epochs 380 and 400, respectively.Source: S8 Complete Results · p. 24
  • 0.0277 is the VEF lift error over 200 test cases.The vertex-only checkpoint gives CL = 0.0225 and ρL = 0.99988.Source: S8 Complete Results · p. 24
  • 0.0220 for the band and 0.0205 for the surface loop are relative lift errors from a separate 10-case contour comparison using the vertex-only checkpoint.These subset values do not establish a full-test-set comparison of contour conventions or a VEF contour comparison.Source: S8 Complete Results · p. 25

S9 Comparison of Controller Configurations

The controller comparison contrasts VEF with a vertex-only configuration, but run-setting differences limit attribution to active Hodge updates alone. Separate vertex-only diagnostics describe its analyzer atoms, assignment, and routing at epoch 380.

S9 Comparison of Controller Configurations

  • VEF applies Hodge updates to vertex, edge, and face streams, whereas vertex-only applies the vertex Hodge update and retains position-wise edge and face residual feedforward updates.Both configurations produce routing logits from the final vertex stream; VEF also uses a 32-edge support cap versus 64 for V, a 200-epoch continuation, and 30 rather than 20 checkpoints.Source: S9 Comparison of Controller… · p. 25
  • The selected VEF run has higher errors than the selected vertex-only run on the Table S10 metrics, but the comparison does not isolate active Hodge updates.Support, training duration, and checkpoint count differ, and there is one run per configuration; among Table S6 configurations, vertex-only has the lowest mean relative L2, while other models have lower surface and lift errors.Source: S9 Comparison of Controller… · p. 25

S10 Diagnostics of the Vertex-only Configuration

  • The vertex-only analyzer atoms are detached before assignment, so the conductance network receives no gradient from the assignment objective and remains at initialization.Conductance affects smoothing through weights normalized by incident degree, so larger unnormalized conductance alone does not establish faster local-structure removal.Source: S10 Diagnostics of the Vertex-only… · p. 26
  • Approximately 81% of the summed squared atom norms is in the coarsest atom and 14% in the finest residual for the illustrated pressure field at epoch 380.The atom shares overlap and are not an orthogonal partition or test-set means.Source: S10 Diagnostics of the Vertex-only… · p. 26, Figure S2 · p. 27
  • The energy-floor penalty acts on assigned targets, and its zero value at epoch 380 does not establish inactivity throughout training.Global Xavier initialization overwrites the constructor's preferred atom-to-expert grouping, so the learned assignment has no prescribed ordering.Source: S10 Diagnostics of the Vertex-only… · p. 26

Related papers

Loading 2610.20585v3…