PaperlayerView exact revision on arXiv

Source-linked AI summary

Scale-Split Neural Operator for Memory- and Data-Efficient 3D Turbulence Prediction

Shaoxiang Qin, Yucheng Zhao, Zongyi Li, Liangzhu Leon Wang, Xiongye Xiao

arXiv:2609.38977v1cs.LGcs.AI

TL;DR

High-resolution turbulence surrogates are difficult to train because full-field memory grows with resolution and full-resolution data are costly and scarce. ScaleSplit-NO pairs a coarse-grid Parent with a conditioned patch-based Child; on two high-resolution benchmarks it outperforms competing baselines in accuracy, with lower error and training memory on JHTDB256.

  • Problem

    Full-field turbulence surrogates face resolution-dependent training memory, while high-resolution simulation data are costly to generate and store and remain scarce.

    Source: §1 · p. 1

  • Method

    ScaleSplit-NO separately trains a coarse-grid Parent to forecast the next coarse field and a Child to predict local full-resolution patches conditioned on that forecast.

    Source: §1 · p. 2

  • Results

    ScaleSplit-NO surpasses competing baselines in prediction accuracy on two high-resolution turbulence benchmarks; at 256^3 its error is 53% lower and training memory 79% lower than the respective strongest and most memory-efficient baselines.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Additional coarse fields can improve prediction by upgrading the Parent without retraining the Child, at a small storage cost.

    Source: §1 · p. 3

    The method is validated only on structured grids, and overlapping patches trade inference speed against accuracy.

    Source: §5 · p. 10

Abstract

from arXiv · show

Neural surrogates have emerged as fast alternatives to the numerical simulation of three-dimensional turbulence. However, training them at high resolution remains challenging, since the memory of full-field models grows with the resolution. In addition, full-resolution training data are expensive to simulate and store, and therefore scarce. We introduce ScaleSplit-NO (Scale-Split Neural Operator), which exploits the scale structure of turbulence with two neural operators: a Parent predicts the global coarse field at the next time step, and a Child predicts full-resolution local patches conditioned on this prediction. Neither model operates on the full-resolution field. The Child is pretrained alone and then attached to the Parent's coarse prediction through zero-initialized connections. On two complex high-resolution turbulence benchmarks, ScaleSplit-NO surpasses all competing baselines in both prediction accuracy and data efficiency. On the higher-resolution dataset JHTDB256 ($256^3$), its normalized mean squared error (NMSE) is 53% lower than that of the strongest baseline, and its training memory is 79% lower than that of the most memory-efficient baseline. We further demonstrate its effectiveness for urban wind prediction in a real district of Montreal on a $500\times150\times500$ grid, reducing one-step NMSE by 65.8% relative to the baseline. Moreover, swapping in a Parent trained on additional coarse fields improves prediction without retraining the Child, providing further accuracy gains at a small storage cost.

1 INTRODUCTION

High-resolution turbulence surrogates face costly memory and scarce training data because full-field models process every grid point. ScaleSplit-NO separates coarse global forecasts from local patch predictions, improving benchmark accuracy while using less training memory and making efficient use of additional coarse data.

1 INTRODUCTION

  • Full-field models must store activations across the grid, while high-resolution trajectories are costly to simulate, store, and transfer, leaving only hundreds to thousands of fields in common datasets.A single high-resolution field contains tens of millions of values, and one trajectory can take hours to days to simulate.Source: §1 · p. 1
  • ScaleSplit-NO assigns global coarse forecasts to a Parent and conditioned full-resolution patches to a Child, so neither operator processes the whole fine-resolution field.The Parent's coarse prediction supplies large-scale context; overlapping Child predictions are assembled into the full field.Source: §1 · p. 2
  • Spatially homogeneous patch positions provide many Child training examples per frame, while each frame supplies only one large-scale example for the Parent.Separate training lets each model's capacity and schedule match the data available to it.Source: §1 · p. 2
  • At 256^3, ScaleSplit-NO lowers error by 53% versus the strongest baseline and uses 79% less training memory than the most memory-efficient baseline.The introduction presents these as the headline high-resolution results.Source: §1 · pp. 2–3
  • The Parent can be trained on additional coarse fields and swapped in without retraining the Child, improving accuracy at a small storage cost.The introduction states that coarse fields require only a small fraction of full-resolution storage.Source: §1 · pp. 2–3

2 RELATED WORK

Prior turbulence surrogates span convolutional and recurrent networks as well as neural operators, with recent work adding multiscale and local structure. ScaleSplit-NO differs by supervising its scales separately rather than training end to end on full-resolution fields.

2 RELATED WORK

  • Turbulence surrogates have used convolutional networks, recurrent networks on compressed latent states, and neural operators such as FNO; one convolutional simulator was reported more accurate than a same-resolution numerical solver.The passage also describes learned corrections to solvers and improvements to spectral solvers.Source: §2 · p. 3
  • Neural operators add multiscale or local structure through multiwavelets, U-Net pathways, local kernels, high-frequency modules, or separate global and local streams.Examples include LOGLO-FNO, EddyFormer, patch-based models, and P3D.Source: §2 · p. 3
  • Unlike these end-to-end multiscale models, which train on full-resolution fields, ScaleSplit-NO supervises each scale separately so no model processes the full domain at full resolution during training.Source: §2 · p. 3

3 METHOD

ScaleSplit-NO splits turbulence prediction across a coarse-grid Parent and a patch-based Child, then assembles the Child's local predictions into a full field. Separate supervision, imperfect but informative coarse conditioning, and fixed-size model inputs shape its training procedure and memory use.

3 METHOD

  • A coarse-grid Parent forecasts global evolution while a shared Child predicts full-resolution patches conditioned on the Parent's coarse forecast; overlapping patches are assembled into the field.The Parent operates on the whole coarse domain, while the Child operates on local fine-grid patches.Source: §3 · p. 3, Figure 2 · p. 4

3.1 PROBLEM FORMULATION

  • The formulation represents a field on a fine grid, uses D to truncate and sample low frequencies onto a coarse grid, I to interpolate back, and Tj to extract patches.The Parent, Child, and assembly map compose into a predictor from the input field to the next-step field.Source: §3.1 · pp. 3–4

3.2 COARSE PREDICTION AS THE CONDITION

  • The Child receives the Parent's predicted coarse field cropped to its location as context, because patch evolution depends on large-scale motion absent from the patch alone.The Parent is directly supervised and can be trained and evaluated independently of the Child; with sufficient data, conditioning on the coarse input instead of the prediction yields higher error.Source: §3.2 · pp. 4–5

3.3 COMPLETE-PATCH PREDICTION FROM AN IMPERFECT CONDITION

  • The Child predicts a complete target patch rather than adding a residual to the imperfect coarse condition, and Hann-weighted overlap averaging assembles patch predictions smoothly.It is pretrained on patches alone, then fine-tuned with the condition attached through zero-initialized weights; fine-tuning uses predictions from the frozen Parent, matching inference conditions.Source: Figure 2 · p. 4, §3.3 · p. 5

3.4 SEPARATE SUPERVISION

  • Parent training, Child pretraining, and Child fine-tuning are separate stages, with model capacity and schedules matched to their different amounts of training data.Each frame yields many patches for the Child but only one coarse field for the Parent; extra coarse fields can be stored cheaply and used to retrain the Parent without retraining the Child.Source: §3.4 · pp. 5–6

3.5 TRAINING MEMORY

  • For fixed patch size, coarse-grid size, and batch size, ScaleSplit-NO's training memory does not grow with field resolution.The Child processes p^3 voxels per patch and the Parent at most R^3 voxels per sample, with each stage trained separately.Source: §3.5 · p. 6

4 EXPERIMENTS

ScaleSplit-NO is evaluated on two high-resolution turbulence datasets, with low-data and full-data comparisons, memory measurements, and ablations. It achieves lower errors than the baselines, uses less training memory at high resolution, and supports accuracy gains from additional coarse fields without retraining the Child.

4.1 MAIN RESULTS

  • On both datasets and data settings, ScaleSplit-NO has the lowest one-step and rollout error; on JHTDB256 its one-step error is 53% lower than P3D’s.On MHD64, one-step error is 23% lower than EddyFormer’s with all eight trajectories and 30% lower with one; JHTDB256 rollout error is 10% and 35% lower.Source: §4.1 · p. 6, Table 1 · p. 7
  • At 256^3, training memory is 3.0 GiB, 79% lower than the lightest baseline.ScaleSplit-NO’s training memory depends on patch size and coarse-grid size, not field resolution.Source: §4.1 · pp. 6–7
  • With 1/8 of the data, JHTDB256 error is 43% lower than the best baseline trained on all data.On MHD64, one trajectory gives error within 6% of the best baseline trained on all eight trajectories.Source: §4.1 · p. 7
  • On MHD64, adding coarse fields from all eight trajectories to a one-trajectory Parent lowers error by 19.5% using 4.6% more storage, without retraining the Child.One additional full trajectory adds 12.5% storage and lowers error by 10.7%; the coarse-field gain is small on JHTDB256, though more efficient per unit storage.Source: §4.1 · pp. 7–8
  • On Montreal urban wind, one-step NMSE is 65.8% lower than U-Net’s, with training memory of 2.8 rather than 27.1 GiB.The simulated district uses a 500 × 150 × 500 grid and is nonperiodic, with buildings deflecting and channeling wind.Source: §4.1 · p. 8

4.2 ABLATION STUDIES

  • With four or eight MHD64 trajectories, replacing the Parent’s coarse prediction with downsampled input raises error by 14% or 15%; with one trajectory, errors are nearly equal.The authors attribute the one-trajectory result as likely reflecting too few distinct coarse-field changes for the Parent to learn.Source: §4.2 · p. 8, Figure 5 · p. 9
  • On MHD64, joint training through the Child loss raises error by 23.2%, while fine-tuning the Child on the coarse target raises it by 75.2%.Removing Child fine-tuning, using residual correction, or omitting pretraining and zero initialization raises error by 5–9%.Source: §4.2 · p. 8, Table 2 · p. 9
  • On MHD64, 20% patch overlap lowers NMSE by about a quarter, while 50% overlap makes inference nearly four times slower than 20%.The default overlap is 33%; the final configuration uses a 24^3 coarse grid and 16^3 Child patches, and a 16^3 Parent grid matches its error while 32^3 is slightly worse.Source: §4.2 · p. 9, Table 3 · p. 9

5 CONCLUSION

ScaleSplit-NO couples global coarse forecasts with local full-resolution predictions, and additional coarse fields can improve accuracy by upgrading the Parent without retraining the Child. The conclusion also identifies inference-speed and grid-structure limitations.

5 CONCLUSION

  • Additional coarse fields can improve prediction accuracy by upgrading the Parent without retraining the Child.This is the conclusion’s stated route to further accuracy gains.Source: §5 · p. 10
  • Overlapping patches make inference slower as overlap grows, and the method has been validated only on structured grids.The overlap must balance accuracy against inference speed; the FNO cannot serve directly as the backbone on unstructured meshes.Source: §5 · p. 10

A RELATED WORK ON NEURAL OPERATORS

  • Neural-operator research spans spectral, local, multiscale, attention-based, equivariant, and pretrained designs, as well as generative approaches for time-dependent problems.The cited review also describes models that sample turbulent states from geometry and boundary conditions without an initial flow field.Source: A RELATED WORK ON NEURAL OPERATORS · p. 17

B.1 MAIN COMPARISON IN NRMSE

  • The one-step NRMSE ranking matches Table 1 in all four settings.Table 4 repeats the one-step comparison in NRMSE.Source: §B.1 · p. 17, Table 4 · p. 17

B.2 VISUALIZATIONS

  • The visualizations cover first test pairs for models trained with 1/8 of the data, showing predictions, normalized errors, and complete-field views.Panels include one-step and rollout comparisons, multichannel predictions, and isosurfaces.Source: §B.2 · p. 17, Figure 10 · p. 21, Figure 11 · p. 22

B.3 ENERGY SPECTRA

  • ScaleSplit-NO’s error spectrum is at or near the bottom across wavenumbers on both datasets, with its JHTDB256 advantage over P3D concentrated at high wavenumbers.On MHD64 its error is below every baseline at all wavenumbers; on JHTDB256 it is below every baseline except P3D and comparable to P3D at low to intermediate wavenumbers.Source: §B.3 · p. 26, Figure 15 · p. 26

B.4 SENSITIVITY TO REPEATED TRAINING

Repeated low-budget training runs preserve ScaleSplit-NO’s lowest mean error, while swapping in a Parent trained on more coarse fields yields a small JHTDB256 gain.

B.4 SENSITIVITY TO REPEATED TRAINING

  • ScaleSplit-NO retains the lowest mean error on both datasets across the repeated low-data runs, despite short training and seed variation.For 20 of 23 configurations, standard deviation is below 5% of the mean; the maximum is 13%.Source: §B.4 · p. 26, Table 5 · p. 27

B.5 SWAPPING IN MORE ACCURATE PARENTS

  • On JHTDB256, a Parent trained on additional coarse fields improves the result by 1.9% for 1.4% of full-data storage with the Child of 50 frames.The added coarse fields are downsampled from the high-resolution simulation, so the experiment measures storage efficiency rather than data-generation efficiency.Source: §B.5 · pp. 26–27

B.6 SIZES OF THE PARENT AND THE CHILD

The reported configurations show that finer Parent grids do not guarantee better results, while the urban-wind test favors ScaleSplit-NO over U-Net in both NMSE and training memory.

B.6 SIZES OF THE PARENT AND THE CHILD

  • On JHTDB256, a 323 global coarse grid is 10.5% worse, while a 1283 grid requiring eight times more coarse data brings no improvement.Increasing coarse-grid resolution adds detail but also increases the values predicted from few coarse fields, making overfitting more likely.Source: §B.6 · p. 27

B.7 OVERLAP OF THE PARENT PATCHES

  • Changing Parent-patch overlap on JHTDB256 changes error by less than 3% and inference time by less than 2%, with Child overlap fixed at 27%.Source: §B.7 · p. 28, Table 9 · p. 29

B.8 URBAN WIND FIELDS

  • Urban-wind test NMSE (×10−3) is 1.87 for ScaleSplit-NO and 5.46 for U-Net; peak GPU memory per training update is 2.8 GiB and 27.1 GiB, respectively.Both methods are trained on the same data within 40 GB of GPU memory.Source: §B.8 · pp. 28–29

C.1 DATASETS AND SPLITS

The evaluation uses MHD64 trajectories for unseen-trajectory generalization and JHTDB256 frames for later-time prediction on one flow. The low-data MHD64 setting removes most initial conditions.

C.1 DATASETS AND SPLITS

  • MHD64 contains ten 100-frame trajectories of 643 compressible magnetized turbulence, with density, velocity, and magnetic-field channels.The released split uses eight training trajectories, one validation trajectory, and one test trajectory; training budgets use one, two, four, or all eight trajectories.Source: §C.1 · p. 29
  • JHTDB256 uses every fourth point of a 10243 simulation to form 2563 fields, and every tenth archived snapshot gives a 0.02 time interval.Training budgets contain 50, 100, 200, or 400 frames; validation and test use later time windows.Source: §C.1 · p. 29
  • MHD64 tests generalization to an unseen trajectory, whereas JHTDB256 tests prediction at later times of one flow; the 1/8 MHD64 setting retains only one initial condition.Source: §C.1 · p. 30

C.2 EVALUATION PROTOCOL

The evaluation uses held-out frame pairs for single-step tests and fixed-length autoregressive rollouts, with horizons chosen to represent similar decorrelation. NMSE is the main metric; NRMSE is also reported.

C.2 EVALUATION PROTOCOL

  • JHTDB256 rollouts span 15 steps, while MHD64 rollouts span 9; the reported horizons correspond to a similar loss of correlation.Single-step evaluation uses all 49 JHTDB256 and 99 MHD64 test pairs; rollouts are segmented and each method feeds back its own predictions.Source: §C.2 · p. 30
  • MHD64’s 9-step and JHTDB256’s 15-step rollout horizons reach the same field-to-two-steps-earlier cosine similarity of 0.81.The paper notes that longer MHD64 rollouts have limited use because benchmark errors exceed mean-field prediction within a dozen steps on the complete The Well MHD dataset.Source: §C.2 · p. 30

C.3 METRICS

  • NMSE is the main metric, and NRMSE is also reported in Appendix B.Both metrics are computed on the complete field in physical units and averaged over test pairs, or over rollout segments at each step.Source: §C.3 · p. 30

D.1 ARCHITECTURE

The architecture separates coarse-grid global prediction from fine-grid patch prediction, then assembles overlapping Child patches into the full field. Zero-initialized condition connections let the Child begin fine-tuning with its pretrained output unchanged.

D.1 ARCHITECTURE

  • JHTDB256 uses a 256^3 fine grid, a 64^3 coarse grid, 32^3 Parent patches, and 16^3 Child patches; MHD64 uses a 64^3 fine grid and 24^3 coarse grid.MHD64 uses 16^3 Child patches, with the Parent processing the whole coarse field.Source: §D.1 · p. 30
  • The Parent is half the Child’s depth and width, while on JHTDB256 their parameter counts remain similar because the Parent uses more Fourier modes per axis.Both models use FNO backbones with residual Fourier blocks; the Child architecture is otherwise shared across datasets up to physical-channel count.Source: §D.1 · p. 30
  • The Parent predicts a coarse-grid increment, which is added to the coarse input; on JHTDB256, overlapping Parent patch increments are assembled first.The Child’s condition pathway receives the normalized coarse prediction cropped to the patch location, through zero-initialized condition weights and maps.Source: §D.1 · p. 31
  • At fine-tuning’s start, zero condition-pathway weights preserve the pretrained Child’s output for every input patch and condition.The proof states that the lift, hidden states, blocks, projection, and output therefore match those of the pretrained Child.Source: §D.1 · p. 31
  • Inference places patches at nearly uniform periodic intervals, averages overlaps with Hann weights, and uses 22 Child patches per axis on JHTDB256.The average Child-patch overlap is 27% on JHTDB256 and 33% on MHD64.Source: §D.1 · p. 31

D.2 TRAINING PROCEDURE

The Parent and Child are trained in three successive stages, with normalized-root-mean-squared-error objectives. The section also specifies ablation variants and baseline configurations used for comparison.

D.2 TRAINING PROCEDURE

  • The three stages run sequentially on one GPU, and checkpoints are selected by physical-unit validation MSE before one test evaluation.Parent training uses cube rotations and reflections for augmentation; Child pretraining and fine-tuning do not, and each Child is paired with a Parent trained on the same data.Source: §D.2 · p. 31
  • All three stages minimize NRMSE on normalized network outputs, with the Child’s patch objective averaged over training frame pairs and patch positions.On JHTDB256, the Parent objective uses patches of the coarse grid.Source: §D.2 · pp. 31–32
  • Ablations vary Child fine-tuning and pretraining, condition choice, residual prediction, joint Parent–Child updates, and model size.The listed variants are trained on one-eighth of MHD64 data; the no-pretraining and residual variants train for 350 epochs, while joint training updates both models for 300 epochs.Source: §D.2 · p. 32

D.3 BASELINE CONFIGURATIONS

  • Baselines use the same data splits, but JHTDB256 configurations are constrained to 40 GB GPU memory and several are scaled down or checkpointed.P3D uses 128^3 crops without a context network on JHTDB256; MG-TFNO is extended from two to three dimensions and still processes a whole field per update.Source: §D.3 · p. 32, Table 12 · p. 33

D.4 TRAINING TIME, INFERENCE TIME, AND MEMORY

Appendix D.4 compares training memory, training time, and inference time; the supplementary theory analyzes forecast conditions, weighted patch assembly, and coarse-prediction error. These theoretical results specify information and error boundaries for the composed predictor.

D.4 TRAINING TIME, INFERENCE TIME, AND MEMORY

  • Child fine-tuning takes about 80% of ScaleSplit-NO’s training time; with overlap, inference takes longer roughly in proportion to predicted voxels.Without overlap, ScaleSplit-NO is about as fast as baselines on MHD64 and slower on JHTDB256, where baselines use smaller configurations.Source: §D.4 · p. 33
  • 4.6 GiB is the maximum training memory for any ScaleSplit-NO stage on JHTDB256 through batch size 16, while a baseline sample needs at least 7.0 GiB.Figure 17 plots peak GPU memory per update against batch size, with the input size of one sample in its legend.Source: §D.4 · p. 33, Figure 17 · p. 35

E.2 CONDITIONAL INFORMATION AND COARSE PREDICTION

  • A forecast condition can expose relevant nonlocal information to a patch predictor, but cannot add information beyond the patch and the complete current coarse field.Strict improvement over using the patch alone occurs exactly when the corresponding conditional means differ with positive probability; replacing the complete coarse field with a restricted patch changes the comparison.Source: §E.2 · p. 35
  • If the future is not determined by the current field, coarse-prediction error can include uncertainty that remains even without conditional-mean information loss from coarsening.For deterministic evolution the uncertainty term is zero; otherwise it may be positive even when coarse graining loses no information relevant to the conditional mean.Source: §E.2 · p. 36

E.3 WEIGHTED ASSEMBLY

  • Under the stated weighted-assembly conditions, assembled prediction error is bounded by the coverage-weighted patch error.The result concerns fixed nonnegative partition-of-unity weights.Source: §E.3 · pp. 36–37

E.4 PARENT ERROR AND THE COARSE CLOSURE RESIDUAL

  • The Child receives the coarse forecast as an input rather than adding it as a residual, allowing its output to correct forecast errors.The passage links this design to the inexactness of the coarse prediction and fine-scale information in each patch.Source: §E.4 · p. 37

E.5 ONE-STEP ERROR DECOMPOSITION AND PARENT REPLACEMENT

The section decomposes deployed one-step error into an oracle-conditioned residual and a forecast-induced change, then bounds assembled error and fixed-Child Parent replacement under stated sensitivity conditions.

E.5 ONE-STEP ERROR DECOMPOSITION AND PARENT REPLACEMENT

  • The deployed error is exactly the oracle-conditioned residual plus forecast-induced change; their interaction can have either sign, so the terms are not additive causal shares.The oracle-conditioned diagnostic depends on the target and is not a deployable predictor.Source: §E.5 · p. 37

E.6 COMPOSITION BOUND

  • A small oracle residual, accurate Parent, and controlled Child sensitivity suffice for a small error upper bound, but are not necessary because residuals can cancel.The composition bound uses the exact decomposition and stability bounds under the theorem's assumptions.Source: §E.5 · p. 37, §E.6 · p. 38

E.7 PARENT REPLACEMENT

  • A sufficiently close compatible Parent replacement changes fixed-Child error only slightly, and forecast disagreement can be evaluated without future labels.Source: §E.5 · p. 38, §E.7 · p. 40

E.6 COMPOSITION BOUND

  • The bound assumes finite Parent error and oracle-conditioned residual; finite oracle-conditioned error is not required, though it provides an additional upper bound when available.Finite input moments alone do not justify the L2 argument for arbitrary nonlinear predictors.Source: §E.6 · pp. 38–39
  • When patch sensitivities are at most ℓ and no voxel belongs to more than m_max patches, L_cov ≤ ℓ√m_max; with constant multiplicity and uniform weights, L_cov ≤ ℓ.Source: §E.6 · p. 39

Related papers

Loading 2609.38977v1…