PaperlayerView exact revision on arXiv

Source-linked AI summary

RheOPT: Few-Shot Non-Newtonian Flow Prediction via Newtonian Pretraining of RheOFormer

Maedeh Saberi, Amir Barati Farimani, Safa Jamali

arXiv:2610.31465v1cs.LG

TL;DR

Non-Newtonian flow prediction is costly because stress and velocity evolve together, and neural-operator surrogates still need expensive target simulations for training. RheOPT pretrains a transformer-based neural operator on Newtonian simulations, then fine-tunes it on a few geometry-matched non-Newtonian samples. It improves few-shot contraction prediction and recovers wake-shedding dynamics with phase-insensitive diagnostics, while transfer reliability depends on normalization.

  • Problem

    Neural-operator surrogates for non-Newtonian flow still require many expensive target simulations, limiting learning when only a few can be afforded.

    Source: §I · p. 1

  • Method

    RheOPT pretrains a transformer-based neural operator on Newtonian simulations and fine-tunes it on a small number of geometry-matched non-Newtonian samples.

    Source: B. RheOPT architecture, inputs, and… · pp. 4–5, §V · p. 13

  • Results

    Newtonian pretraining improves few-shot contraction prediction under RevIN, while phase-insensitive diagnostics show better recovery of shedding-frequency trends and wake fluctuation amplitudes in the obstacle wake.

    Source: §IV · p. 7, §V · p. 13

  • Takeaways & Limitations

    Newtonian simulations provide a physically grounded pretraining source for data-scarce non-Newtonian prediction by supplying geometry-conditioned transport structure useful after the stress model changes.

    Source: §V · pp. 13–14

    The geometry-matched experiments isolate constitutive transfer but do not establish whether the representation transfers when geometry also changes.

    Source: §V · p. 13

Abstract

from arXiv · show

Despite advances in numerical methods, non-Newtonian flow simulations remain computationally expensive because velocity and stress must be advanced together through rate- and history-dependent constitutive dynamics. Neural-operator surrogates reduce inference cost after training, but their training data require the same expensive simulations they aim to replace, making data-efficient learning essential when only a few non-Newtonian simulations can be afforded. Here, we introduce the Rheological Operator Pretrained Transformer (RheOPT), which pretrains a transformer-based neural operator on inexpensive Newtonian simulations and fine-tunes it on a small number of non-Newtonian samples. Rather than transferring the Newtonian constitutive response itself, RheOPT reuses geometry-conditioned transport structure that remains useful after the stress model changes. We evaluate RheOPT in two geometry-matched settings: a $4{:}1$ contraction channel for Giesekus, Oldroyd-B, and Bingham fluids, and the oscillatory wake behind a triangular obstacle for Oldroyd-B flow. In the contraction, Newtonian pretraining substantially improves few-shot prediction, and a model fine-tuned on four target samples can match or exceed a from-scratch model trained on four times as many. This benefit is reliable under reversible instance normalization (RevIN) but dataset dependent under Global normalization, a difference we trace to the dynamic range of each fluid's velocity and stress fields. In the wake, where relative $L_2$ error alone cannot verify that vortex shedding is recovered, phase-insensitive diagnostics show that pretraining recovers the shedding frequency and wake fluctuation amplitude that are suppressed by the from-scratch model in the few-shot regime. Overall, RheOPT establishes Newtonian simulation as a physically grounded pretraining source for data-efficient non-Newtonian operator learning.

I. INTRODUCTION

Non-Newtonian flow prediction is costly because velocity and stress evolve together, while neural-operator surrogates still require many expensive target simulations. RheOPT tests whether geometry-matched Newtonian pretraining can support few-shot non-Newtonian prediction, with results depending on normalization and flow dynamics.

I. INTRODUCTION

  • Stress must be evolved alongside velocity in non-Newtonian simulations, making solver-based prediction computationally demanding.The cost recurs when geometry, boundary conditions, or constitutive parameters change.Source: §I · p. 1
  • Neural operators reduce inference cost, but their data-hungry training can shift the computational burden to generating many expensive target simulations.This makes learning with only a handful of non-Newtonian samples a practical bottleneck.Source: §I · p. 1
  • Newtonian simulations are easier to generate and share geometry- and boundary-driven transport structure with non-Newtonian flows, motivating their use as pretraining data.The proposed transfer targets the kinematic scaffold, not the Newtonian constitutive response itself.Source: §I · pp. 1–2
  • Whether Newtonian-learned representations remain useful across a change in constitutive law is an open question for non-Newtonian prediction.The work frames this as a transfer across a constitutive-law boundary.Source: §I · p. 2
  • Field normalization and phase-insensitive diagnostics matter because field magnitudes vary and pointwise error can miss recovered vortex-shedding dynamics.A phase-shifted shedding prediction may be penalized, while a nearly steady prediction may remain close to the time-averaged field.Source: §I · p. 2
  • RheOPT tests Newtonian pretraining in a 4:1 contraction for Giesekus, Oldroyd-B, and Bingham fluids and in an Oldroyd-B triangular-obstacle wake.In the contraction, pretraining improves few-shot prediction; in the wake, phase-insensitive diagnostics show improved recovery of shedding dynamics.Source: §I · p. 2

II. RELATED WORK

Prior complex-fluid learning generally uses target-fluid or target-configuration data, while neural operators and other transfer approaches address related but different forms of generalization. RheOPT addresses the untested question of transferring neural-operator representations from Newtonian flow across a constitutive-law change.

II. RELATED WORK

  • Complex-fluid learning methods often target a specific fluid, constitutive family, or configuration, so changing these typically requires new target data and retraining.Existing approaches include physics-informed learning, constitutive response modeling, and stress prediction.Source: §II · p. 2
  • Neural operators learn mappings between functions and can generalize to new conditions within their training distribution, but their training data still come from target physics.RheOFormer extends operator learning to spatiotemporal stress and velocity prediction in irregular complex-fluid geometries.Source: §II · p. 2
  • Prior neural-operator pretraining transfers across coefficients, boundary conditions, equation families, and data fidelities, but not systematically across Newtonian and non-Newtonian constitutive laws.Other related strategies include heterogeneous-PDE pretraining and transfer from cheaper lower-dimensional data.Source: §II · pp. 2–3
  • RheOPT addresses the gap in testing whether Newtonian-flow representations remain useful for viscoelastic and viscoplastic prediction.Related physics-informed transfer methods generally address instances of a single physical problem and still require target-specific re-optimization.Source: §II · p. 3

A. Problem setup and transfer design

RheOPT is evaluated as geometry-matched transfer from Newtonian to non-Newtonian flow, isolating a change in constitutive closure rather than geometry. The experiments span three target fluids in a contraction and Oldroyd-B flow in an oscillatory obstacle wake.

A. Problem setup and transfer design

  • The model takes velocity and symmetric-stress fields with node coordinates as input and predicts the remaining spatiotemporal trajectory on an irregular mesh.The fields are vx, vy, σxx, σxy, and σyy.Source: A. Problem setup and transfer design · p. 3
  • The contraction uses two snapshots over 0–0.2 s to predict through 5 s, while the obstacle wake uses ten snapshots over 0–1.8 s to predict through 10 s.These settings compare short-observation constitutive transfer with transfer in an unsteady vortex-shedding flow.Source: A. Problem setup and transfer design · p. 3
  • The transfer design keeps each geometry fixed while changing from Newtonian source flow to Giesekus, Oldroyd-B, or Bingham target flow.The source and target share boundary-driven transport structure and momentum balance, but differ in constitutive relation.Source: A. Problem setup and transfer design · p. 3
  • The same few target samples fine-tune RheOPT from Newtonian-pretrained weights or train an otherwise identical model from random initialization.The comparison tests whether geometry- and transport-constrained structures can be reused after the constitutive closure changes.Source: A. Problem setup and transfer design · p. 3, Figure 1 · p. 4
  • The study uses 200 Newtonian source simulations per geometry, and downstream models are compared on matched target samples.The contraction covers Giesekus, Oldroyd-B, and Bingham; the obstacle case covers Oldroyd-B with Re from 10 to 100 and Wi = 0.01 Re.Source: A. Problem setup and transfer design · pp. 3–4

B. RheOPT architecture, inputs, and training

RheOPT uses RheOFormer to encode irregular-mesh flow states, propagate latent dynamics, and decode predictions at query coordinates. Pretraining and fine-tuning share the field-prediction objective, while matched training conditions make initialization the intended comparison.

B. RheOPT architecture, inputs, and training

  • RheOFormer encodes spatial dependencies on irregular meshes, propagates the latent state recurrently, and decodes fields at arbitrary query coordinates.This supports long-horizon prediction without remeshing, interpolation, or geometry-specific architectural changes.Source: B. RheOPT architecture, inputs, and… · p. 4
  • Each mesh-node input combines the observed physical fields with node coordinates, and dimensionless numbers are not added as node features.The physical channels provide the observed trajectory, while coordinates identify mesh locations.Source: B. RheOPT architecture, inputs, and… · p. 5
  • Global normalization uses fixed training-set channel statistics, whereas RevIN computes statistics separately for each sample and physical channel.RevIN normalizes and then restores sample-specific offsets and scales using the observed input sequence.Source: B. RheOPT architecture, inputs, and… · p. 5
  • Newtonian pretraining and non-Newtonian fine-tuning use the same field-prediction objective rather than a separate self-supervised task.The loss is relative error averaged over samples and output channels.Source: B. RheOPT architecture, inputs, and… · p. 5
  • Pretrained and from-scratch models share architecture, target samples, normalization, optimizer, schedule, batch size, training steps, and validation set.All model parameters are updated during fine-tuning; initialization is the only intended difference.Source: B. RheOPT architecture, inputs, and… · p. 5
  • Each condition uses four training seeds, with 500 epochs for contraction experiments and 2000 for obstacle experiments.At a given target-data budget, pretrained and from-scratch models use identical optimization-step counts.Source: B. RheOPT architecture, inputs, and… · pp. 5–6

C. Evaluation metrics and dynamical diagnostics

The section evaluates prediction errors in physical space and adds phase-insensitive diagnostics for the triangular-obstacle wake. These diagnostics test whether predicted shedding recovers both its characteristic time scale and fluctuation strength.

Evaluation metrics

  • Errors are evaluated in physical space after reversing normalization, over the full predicted horizon, and summarized across the fixed validation set and three training seeds.Velocity and stress errors are kept separate to avoid obscuring field-specific effects of pretraining.Source: C. Evaluation metrics and dynamical… · p. 6
  • The percentage improvement measures the reduction in real-space relative L2 error from Newtonian pretraining relative to training from random initialization.Positive values indicate lower error for the pretrained model.Source: C. Evaluation metrics and dynamical… · p. 6

Dynamical diagnostics

  • For the wake, dominant shedding frequency and RMS transverse-velocity fluctuations complement relative L2 error by testing temporal scale and fluctuation strength.The frequency resolution is about 0.12 Hz, so a peak at the lowest non-zero frequency indicates no resolved oscillation.Source: C. Evaluation metrics and dynamical… · p. 6

IV. RESULTS AND DISCUSSION

The results examine whether Newtonian pretraining transfers to few-shot non-Newtonian prediction, first in a geometry-matched contraction and then in an oscillatory wake. The section frames normalization and phase-insensitive diagnostics as conditions relevant to interpreting that transfer.

Results and discussion

  • The results are organized around transfer to few-shot non-Newtonian prediction and the conditions under which it is visible.The study uses a geometry-matched 4:1 contraction and an oscillatory triangular-obstacle wake.Source: §IV · pp. 6–7

Contraction flow

  • In the contraction, the Newtonian pretraining simulation, ground truth, pretrained prediction, and from-scratch prediction are compared for three fluids at N = 4 under RevIN.The figure shows velocity magnitude at t = 5 s and downstream profiles at x = 1.Source: Figure 2 · p. 7

Triangular-obstacle wake

  • The wake analysis supplements relative L2 error with phase-insensitive diagnostics to assess whether vortex-shedding dynamics are recovered.The section also contrasts Global normalization and RevIN because target field-magnitude ranges differ.Source: §IV · p. 7

A. Newtonian pretraining improves few-shot contraction-flow prediction

In the short-observation, geometry-matched contraction test, Newtonian pretraining improves few-shot prediction, with transfer most consistent under RevIN. Four-sample pretrained models can match or outperform from-scratch models trained with 16 samples.

Representative flow fields

  • With only two input snapshots, the pretrained model follows representative ground-truth profiles more closely than the from-scratch model trained on the same four target samples.At x = 1, from-scratch Oldroyd-B and Bingham centerline velocities are 2.10 and 2.14 cm/s versus 2.32 cm/s in both ground truths; pretrained predictions reach 2.26 and 2.30 cm/s.Source: A. Newtonian pretraining improves… · pp. 7–8
  • For the plotted velocity fields, pretraining reduces relative L2 error from 13% to 6% for Giesekus, 7% to 3% for Oldroyd-B, and 8% to 4% for Bingham.The corresponding shear-stress errors are 18% versus 48% for Giesekus, 8% versus 20% for Oldroyd-B, and 6% versus 19% for Bingham, for pretrained and from-scratch models, respectively.Source: A. Newtonian pretraining improves… · p. 8

Few-shot results

  • With RevIN, pretrained models show positive transfer for every tested fluid, field grouping, and fine-tuning budget, with stress gains of 44–76% for Giesekus, 75–81% for Oldroyd-B, and 5–49% for Bingham.Figure 3 plots real-space relative L2 velocity and stress errors against the number of fine-tuning samples; its solid and dashed curves distinguish pretrained and from-scratch models.Source: A. Newtonian pretraining improves… · p. 8, Figure 3 · p. 9

Transfer interpretation

  • The transfer is attributed to geometry and momentum-transport structure that remains relevant after changing the stress response, not to transfer of the Newtonian constitutive law.The benefit is consistent across fluids under RevIN, while Global-normalization results are more variable.Source: A. Newtonian pretraining improves… · pp. 8–9

Few-shot results

  • At N = 4, pretrained RevIN models reach stress errors of 0.059 for Giesekus and 0.092 for Bingham, below from-scratch errors of 0.083 and 0.149, respectively, at N = 16.The results show that in several cases four target samples suffice to match or exceed the from-scratch model trained on four times as many.Source: A. Newtonian pretraining improves… · p. 8

B. Normalization controls transfer reliability

Newtonian pretraining improves few-shot prediction reliably under RevIN, while its effect under Global Normalization varies by fluid. The variation is consistent with differences in field-magnitude range and how well shared dataset statistics represent each sample.

B. Normalization controls transfer reliability

  • Global Normalization reduces Giesekus stress error by 8–36%, but gives no Oldroyd-B stress benefit and nonmonotonic Bingham results.The direction of the Global Normalization effect therefore depends on the fluid dataset.Source: B. Normalization controls transfer… · p. 9
  • At N = 16, Oldroyd-B stress error is 0.017 with RevIN versus 0.050 with Global normalization; Bingham stress errors are 0.109 and 0.175, respectively.For Bingham velocity at N = 4, pretrained RevIN error is 0.040 versus 0.369 under Global normalization; Giesekus instead favors Global normalization for N ≥8.Source: B. Normalization controls transfer… · pp. 9–10
  • Global statistics can compress low-magnitude samples in wide-range datasets, while RevIN rescales each sample; Oldroyd-B normal-stress standard deviation is about 12 times Newtonian's.Oldroyd-B and Bingham span wider field-magnitude ranges; Giesekus is comparatively narrower, where global statistics remain representative and competitive.Source: B. Normalization controls transfer… · p. 10
  • RevIN yields reliable pretraining improvements across all three fluids, whereas Global Normalization gives weaker, variable, and sometimes negative effects.Global Normalization remains competitive when field magnitudes span a comparatively narrow range.Source: B. Normalization controls transfer… · pp. 9–10

C. Wake dynamics transfer in an oscillatory flow

In the triangular-obstacle Oldroyd-B wake, Newtonian pretraining improves few-shot predictions and recovers unsteady dynamics that few-shot from-scratch models often suppress. Frequency and fluctuation diagnostics show this recovery beyond what pointwise error or a snapshot can establish.

C. Wake dynamics transfer in an oscillatory flow

  • The triangular-obstacle test uses a 10-snapshot input history from t ∈ [0, 1.8] s to predict the remaining trajectory to t = 10 s.The test targets sustained unsteady dynamics, not just pointwise field accuracy.Source: C. Wake dynamics transfer in an… · p. 11
  • Pretraining reduces stress error by 18–41% across six budgets and velocity error by 43–50% for N ≤16, but scratch velocity error is lower at N = 24 and 32.Under Global normalization, the pretrained stress curve remains below the from-scratch curve at every budget.Source: C. Wake dynamics transfer in an… · p. 11
  • At N = 12, the pretrained prediction reproduces the ground-truth wavy downstream wake, whereas the from-scratch prediction is smooth and nearly steady.The fine-tuned prediction follows the Oldroyd-B wake rather than the Newtonian reference flow.Source: C. Wake dynamics transfer in an… · p. 11, Figure 5 · p. 11
  • Snapshot-wise relative L2 error cannot verify temporal content because a smooth prediction may suppress shedding, while phase-shifted vortices can still be penalized.Frequency and RMS-fluctuation diagnostics test temporal scale and fluctuation strength.Source: C. Wake dynamics transfer in an… · pp. 11–12
  • Pretrained models recover the ground-truth peak-frequency bin in 87–95% of 127 resolved-shedding cases at every budget; from-scratch models recover none for N ≤12.Figure 6 shows pretrained models tracking the frequency–Re trend and shedding onset, while few-shot from-scratch predictions remain at the lowest resolved frequency.Source: Figure 6 · p. 12, C. Wake dynamics transfer in an… · p. 12
  • Pretrained models reproduce the wake-fluctuation envelope across budgets, while from-scratch models at N = 4, 8, and 12 remain near zero over most of the wake.At N = 16, from-scratch amplitude reaches only about 0.05–0.08 cm/s where the reference is roughly 0.25–0.30 cm/s; fine-tuned predictions follow the weaker Oldroyd-B rather than Newtonian envelope.Source: Figure 7 · p. 12, C. Wake dynamics transfer in an… · pp. 12–13

V. CONCLUSION

RheOPT shows that Newtonian pretraining can improve few-shot non-Newtonian prediction by transferring geometry-conditioned transport structure rather than the Newtonian constitutive response. The benefit depends on normalization, and the geometry-matched study leaves cross-geometry transfer open.

V. CONCLUSION

  • Because the experiments match pretraining and fine-tuning domains, they isolate constitutive transfer; the gain reflects geometry-conditioned transport information, not a transferred Newtonian stress response.The pretrained and from-scratch models differ only in initialization, and the fine-tuned model reproduces the target rather than Newtonian flow.Source: §V · p. 13
  • The study tests geometry-matched transfer in a 4:1 contraction for Giesekus, Oldroyd-B, and Bingham fluids and in an Oldroyd-B triangular-obstacle wake.The wake extends the evaluation to sustained unsteady dynamics.Source: §V · p. 13
  • With RevIN, Newtonian pretraining improves few-shot contraction predictions across tested fluids; in several cases, 4 target simulations match or beat a from-scratch model trained on 16.Global normalization is dataset dependent and can weaken or reverse the gain.Source: §V · p. 13
  • Normalization governs how reliably transfer is expressed: RevIN works across the three fluids, whereas Global normalization can weaken or reverse gains when source and target field scales differ.Wide variation in velocity and stress magnitudes can cause fixed dataset-level statistics to compress low-magnitude but important structures.Source: §V · p. 13
  • In the triangular-obstacle wake, pretraining better preserves shedding-frequency trends and wake fluctuation amplitudes than from-scratch training, while relative L2 error alone cannot verify recovered dynamics.The from-scratch model largely misses the unsteady response in representative fields.Source: §V · p. 13
  • Because the experiments are geometry matched, whether the representation transfers across geometries remains open; a systematic theory of field-scaling interactions with pretraining is also lacking.The authors identify broader geometries, constitutive models, parameter ranges, and three-dimensional flows as future extensions.Source: §V · pp. 13–14

Related papers

Loading 2610.31465v1…