PaperlayerView exact revision on arXiv

Source-linked AI summary

Learnable composition for neural operators

Zituo Chen, Baiming Zhang, Sili Deng

arXiv:2609.03069v1cs.LG

TL;DR

Neural operators can lose accuracy when geometry, size, or operating conditions change, and supervised adaptation can require costly high-fidelity simulations. LATENTDDM pretrains a local operator and adapts a learned composition module while freezing that operator. It lowers field errors on larger Darcy domains and improves fast-pitching airfoil rollouts in the reported settings.

  • Problem

    Neural operators often lose accuracy when domain geometry, size, or operating conditions differ from training, while adapting to a new setting can require costly high-fidelity simulations.

    Source: §1 · p. 1

  • Method

    LATENTDDM pretrains a local operator on subdomains, then freezes it and trains a separate composition module to coordinate local predictions for new settings.

    Source: §1 · p. 2

  • Results

    Across larger Darcy domains and faster-pitching airfoil rollouts, LATENTDDM improves reported field-error outcomes; with 16 target simulations, its errors are 36–56% below the best capacity-matched LoRA result on Darcy tests.

    Source: §4.2 · p. 7, §4.3 · p. 8

  • Takeaways & Limitations

    The results support co-designing local pretraining and composition-level transfer as a promising design principle for physical foundation models.

    Source: §5 · p. 8

Abstract

from arXiv · show

Neural operators are fast, differentiable surrogates for physical simulation, but their accuracy often degrades when domain geometry, size, or operating conditions differ from training. Supervised adaptation can recover accuracy, but even a small target set requires costly high-fidelity simulations. We therefore ask how pretraining and transfer can be designed together to reduce this deployment cost. LatentDDM first pretrains a neural operator to predict fields on small subdomains. For a new setting, it freezes this operator and trains only a lightweight module that composes the local predictions. We evaluate our method on two complementary problems: steady Darcy flow, where long-range pressure coupling must extend across increasingly large porous domains, and unsteady incompressible flow around a pitching airfoil, where rollout errors compound as target pitching frequencies exceed the training range. Compared with the capacity-matched models that process the full domain at once, LatentDDM's error is 36-56% lower on larger Darcy domains after adaptation with 16 target simulations. It also improves 20-step field rollouts in fast-pitching airfoil flow, both zero-shot and after few-shot calibration. These results identify the co-designed local pretraining and composition-level transfer as a promising design principle for physical foundation models.

1 Introduction

Neural operators can lose accuracy when geometry, domain size, or operating conditions shift, while target adaptation can require costly high-fidelity simulations. LATENTDDM co-designs local-operator pretraining and a separate learned composition module, and tests transfer across domain size and operating conditions.

1 Introduction

  • Neural-operator errors often grow when domain geometry, size, or operating conditions differ from training, and adapting with labeled target examples can require costly high-fidelity simulations.The passages cite losses in accuracy on larger domains and sensitivity to unseen initial conditions, PDE coefficients, or physical processes.Source: §1 · p. 1
  • Pretraining across multiple physical systems or geometries broadens prior exposure, but joint design of pretraining and transfer for new settings has received less attention.The introduction frames this as a gap in physics foundation-model design.Source: §1 · p. 1
  • Neural DDM work has learned interface conditions or local solvers, while a separate data-learned composition rule coordinating local predictions remains unexplored.Existing approaches may retain repeated meshing and modeling, fixed boundary updates, or reliance on governing equations.Source: §1 · pp. 1–2
  • LATENTDDM pretrains a local operator, learns a separate composition module from data, and adapts to new settings by freezing the operator and supervising only composition on few samples.The method updates local inputs from preliminary predictions, evaluates the local operator again, and assembles the full field.Source: §1 · p. 2
  • LATENTDDM is tested on steady Darcy flow and pitching-airfoil flow, targeting long-range pressure coupling across larger porous domains and rollout errors under faster pitching.The experiments compare against full-domain Transformers and a neural DDM with a fixed composition rule.Source: §1 · p. 2

2 Related work

Related work spans field representation and pretraining, adaptation to new settings, and domain decomposition. LATENTDDM combines local-operator pretraining with transfer confined to a learned composition rule that coordinates subdomains.

Physical pretraining

  • Physics models represent fields with latent spaces, point, slice, or patch tokens, and use multi-system objectives; LATENTDDM instead pretrains local operators on labeled solutions from fixed-size subdomains.Other geometry-focused approaches learn from occupancy reconstruction or synthetic dynamics before downstream physics tasks.Source: §2 · p. 2

Model adaptation and transfer learning

  • Transfer methods include fine-tuning pretrained networks, conditioning on equations or physical parameters, inference-time in-context examples, and composing operators for different dynamics.For changing geometries and domains, Geo-FNO learns a deformation to a regular computational grid.Source: §2 · pp. 2–3

Neural domain decomposition methods

  • Neural DDMs use learned interface conditions or local and coarse computations, while SNI and NEST reuse local solvers and repeatedly update artificial boundary data.LATENTDDM instead coordinates local subdomains through learnable composition without iterations.Source: §2 · p. 3

3 Learnable composition for neural operators

LATENTDDM applies a frozen local operator to overlapping subdomains, then uses a learned composition module to correct local inputs before a second evaluation and assembly. Its error analysis separates local-response coverage from composition's ability to select a useful response, with finite target-sample estimation also affecting adaptation.

3.1 Formulation of LATENTDDM and comparison methods

  • A lightweight Transformer uses preliminary subdomain predictions to jointly produce local input corrections, after which the same frozen local operator is evaluated again and outputs are assembled.The composition module exchanges information among one token per subdomain; adapting to new settings tunes this module while retaining the local predictor.Source: §3.1 · pp. 3–4
  • The local operator is pretrained on pairs of local computational setups and their target fields, with each setup drawn from a subdomain.The full-domain input includes geometry, coefficients, sources, boundary or control data, and any required state history.Source: §3.1 · p. 3
  • LATENTDDM's corrected local inputs represent missing influence from the rest of the domain, such as pressure values on interior Darcy crop edges or coarse airfoil velocity-pressure inputs.The frozen local family maps collections of local conditions to a full-domain field.Source: §3.1 · p. 3
  • Full-domain Point, Slice, and Patch models tokenize the entire domain and decode a field, with their weights jointly learning prediction and cross-domain information exchange.These baselines are trained end to end on full-domain examples.Source: §3.1 · p. 4
  • SNI coordinates subdomains by repeatedly correcting inputs from assembled-field discrepancies along shared boundaries, making sequential calls to the same frozen local operator.Its update blends the current input with a correction, controlled by τ, and SNI–K uses K calls.Source: §3.1 · p. 4

3.2 Error analysis for learnable composition

  • Full-field error depends on both local-response coverage from pretraining and the composition module's selection accuracy; finite-sample estimation adds a third term for few-sample adaptation.In the Darcy diagnostic, selecting local inputs is the dominant error while the frozen operator retains accurate target responses; for SNI, extra sweeps can reduce finite-iteration error but not fixed-point mismatch.Source: §3.2 · p. 5

4 Experiments: domain and operating-condition transfer

The experiments evaluate LATENTDDM under larger Darcy domains and faster-pitching airfoil conditions. Results compare local composition with iterative and full-domain baselines, including zero-shot and target-data adaptation.

4 Experiments: domain and operating-condition transfer

  • The evaluation enlarges Darcy covers from 3×3 to 4×4, 8×8, and 16×16 at fixed spacing and tests airfoil rollouts under faster pitching.For each evaluated OOD airfoil trajectory, adaptation uses 16 sampled transitions that do not overlap scored rollout windows.Source: §4 · p. 5, §4.1 · p. 6, §4.2 · p. 7

4.1 Darcy: structural domain adaptation

  • 2.34×–4.91× faster, LATENTDDM approaches SNI’s selected accuracy range with a fixed two-call path; SNI requires 9–10 sequential sweeps as Darcy covers grow.On 8×8 and 16×16 pressure fields, LATENTDDM retains spatial variation around perforations while Point and Slice suppress it and Patch adds grid-aligned artifacts.Source: Figure 2 · p. 6, §4.1 · p. 6

4.2 Pitching airfoil: regime transfer and rollout

  • At H20, LATENTDDM lowers field error 13% versus SNI and 28% versus the best full-domain model; vorticity follows the same ordering, with separation increasing over rollout.LATENTDDM and SNI retain coherent vortices, whereas full-domain baselines develop unphysical patterns.Source: Figure 3 · p. 7, §4.2 · p. 7

4.3 Cross-case transfer summary

  • At 16×16, Darcy error drops from 0.127 zero-shot to 0.074 after 16-simulation adaptation; airfoil H20 error reaches 0.25–0.27 after calibration, 19–25% below label-free SNI–10.Darcy errors are also 36–56% below the best capacity-matched LoRA result.Source: Figure 4 · p. 8, §4.3 · p. 8

5 Conclusion

The conclusion presents LATENTDDM as a co-designed approach to pretraining and transfer: a local operator is pretrained, then only its composition module is adapted. Experiments report improved transfer on larger Darcy domains and faster-pitching airfoil rollouts, while future work will study module design, broader pretraining, and control applications.

5 Conclusion

  • LATENTDDM pretrains a local neural operator and adapts only a learned composition module when domain size or operating conditions change.The conclusion frames reusable local field responses and setting-specific composition adaptation as a design principle for physical foundation models.Source: §5 · p. 8
  • On larger Darcy domains, LATENTDDM is 36–56% more accurate than capacity-matched full-domain methods; on faster-pitching airfoil trajectories, it improves H20 rollouts zero-shot and after calibration.Source: §5 · p. 8
  • Future work will study composition-module design, scale local pretraining across larger datasets and physical systems, and use LATENTDDM as a differentiable simulator for real-world control.Source: §5 · p. 8

A Extended results and evaluation ledger

The evaluation ledger records the scope and protocols for Darcy domain extension and airfoil zero-shot rollout. Darcy results expose an accuracy–cost trade-off, while airfoil rankings vary by predicted quantity.

A Extended results and evaluation ledger

  • The appendix collects accuracy, timing, and protocol records, and distinguishes the diagnostic Local/Core field from evaluated models.LATENTDDM and SNI share a frozen local operator; Point, Slice, and Patch are trained independently on the whole domain.Source: A Extended results and evaluation ledger · p. 12

A.1 Darcy domain extension

  • Table 1 reports protected-test relative L2 after 16 target simulations per scale, while Table 2 compares critical-path timing at matched validation-selected error.Table 1 averages adapted entries over three seeds and the same 64 cases; Table 2 includes local calls, communication, and assembly.Source: Table 1 · p. 12, Table 2 · p. 12
  • Deep SNI has the lowest Darcy field error, while LATENTDDM reaches comparable validation-selected error in two local rounds and is 2.34×–4.91× faster.Target adaptation changes 0.796M composition parameters for LATENTDDM, versus 6.612M–9.729M parameters in full-domain baselines.Source: §A.1 · p. 12, Table 2 · p. 12, Table 1 · p. 12

A.2 Airfoil zero-shot rollout

  • LATENTDDM has the lowest accumulated H20 field and vorticity errors, but full-domain models are faster on one GPU and more accurate in force prediction.Table 3 covers frozen source-trained checkpoints on protected ID and faster-pitching OOD tests; Table 4 reports force and execution time, whose ranking differs from field and vorticity.Source: §A.2 · p. 12, Table 3 · p. 12, Table 4 · p. 13

A Extended results and evaluation ledger

  • The representative covers use a 32-cell owned core and four-cell input collar, with normalized overlap for assembling fields.Darcy’s 32-cell footprint is retiled across 3×3, 4×4, 8×8, and 16×16 covers; the airfoil uses a shifted 13×7 cover.Source: Figure 5 · p. 13

B Implementation details

The implementation separates local field prediction from learned communication: a frozen local operator is evaluated before and after one communication step. The same schedule applies across Darcy and airfoil, with task-specific local-condition corrections and pretraining.

B.1 Forward algorithm and invariants

  • Each prediction uses two frozen local-operator calls separated by one learned communication step, followed by assembly of overlapping responses.The first call produces preliminary local fields; communication updates each local input before the second call.Source: §B.1 · p. 13

B.2 Attention over local summaries

  • The composition module exchanges information among one summary token per subdomain and returns local-condition updates, not physical fields.Relative-position bias yields translation-invariant communication and equivariance to joint token-position permutations; full attention costs O(N^2), or O(q^4) for a q×q cover.Source: §B.2 · pp. 13–14
  • Communication can recover only information represented by local inputs, first-pass fields, and token positions, limiting attainable accuracy jointly with the frozen response family.The training order first learns local responses, freezes them, and then trains composition; joint fine-tuning is outside the evaluated protocol.Source: §B.2 · p. 14, §B.3 · p. 14

B.4 Local neural-operator pretraining

  • Pretraining learns local field responses on cropped subdomains, after which communication training loads the local checkpoint frozen.Darcy uses supervised subdomain fields and a four-cell collar; the airfoil uses source trajectories, with a 12.804M-parameter local model and 0.822M-parameter composition module.Source: §B.4 · p. 15, Table 5 · p. 15

B.5 Local-condition charts

  • Darcy corrects artificial face values, whereas airfoil corrects a low-order local state before reevaluating the frozen operator.For airfoil, validation favors a volume chart over a face-only chart, and correction distillation avoids directly adding a field residual to the final prediction.Source: §B.5 · pp. 15–16

C Error bounds for composition and target adaptation

The bounds separate error from the frozen local response family and error from selecting its condition. They also characterize when composition-level adaptation can be favored over adapting a full-domain model.

C Error bounds for composition and target adaptation

  • With few target examples, adapting composition is favored when its smaller estimation error outweighs its approximation penalty.The bounds connect pretraining, which controls available target responses, with transfer, which controls response selection and the adapted function class size.Source: C Error bounds for composition and… · p. 16

C.1 Composition error

  • The composition error bound separates the best response available from the frozen local family from the cost of selecting an imperfect condition.For a fixed input, the derivation applies the triangle inequality around the best available condition and then uses the frozen field map’s Lipschitz property.Source: §C.1 · p. 16

C.2 Darcy evaluation of the composition bound

The appendix separates composition coverage from target-time selection and estimation, and distinguishes iterative convergence from field accuracy. Its comparisons test whether composition-limited adaptation can compete with full-domain models under matched target data and adaptation budgets.

C.2 Darcy evaluation of the composition bound

  • At 4×4, 8×8, and 16×16, oracle error is 4.5–5.2% of final error, while the learned–oracle difference is 99.7–99.9%.The frozen local family contains low-error fields on these targets, but the nonlinear oracle is diagnostic, not a certificate of the global minimum.Source: §C.2 · p. 16

C.3 Target adaptation with few examples

  • Composition adaptation has a smaller target-risk bound when its selection cost relative to the best full-domain predictor is outweighed by the smaller finite-sample generalization gap.The theorem’s field-loss framing applies to normalized, nonzero evaluation fields and does not compare the two complexity terms.Source: §C.3 · p. 17

C.4 Finite-iteration and fixed-point diagnostics for SNI

  • SNI contraction and field accuracy are distinct: the bound separates finite-sweep error from the fixed point’s distance to the best condition available to the frozen local family.The appendix recommends checking contraction, physical consistency, and assembled-field accuracy separately.Source: §C.4 · pp. 17–18

C.5 Model-owned rollout states

  • The airfoil comparison advances each method’s own (u, v, p) state through twenty fixed-cadence steps, measuring accumulated model-owned rollout error.A separate native-step audit propagates (u, v, p, ∇p).Source: §C.5 · p. 18

D.1 Independent full domain baselines

  • Point, Slice, and Patch globally process tokens and decode full-domain fields, without local halos or overlap assembly.These controls omit positional encoding to isolate local pretraining and learned composition; the specialized domain-extension Transformer is excluded because it combines locality bias with rotary positional embeddings.Source: §D.1 · p. 19
  • The capacity-matched controls use centered low-rank updates near LATENTDDM’s 796,400-parameter composition budget, with the same target data and training schedule.The validation-best checkpoint is frozen before evaluation on the same 64 protected domains per scale.Source: §D.1 · p. 19
  • At 4×4, 8×8, and 16×16, LATENTDDM errors are .0621, .0709, and .0744, below every listed PEFT baseline.Every paired PEFT–LATENTDDM interval remains positive; the intervals quantify evaluation-unit variation, not retraining variation.Source: §D.1 · pp. 19–20, Table 8 · p. 20

D.2 Airfoil source-to-OOD protocols

The airfoil protocol tests transfer to faster pitching within the represented airfoil family, using protected trajectory splits and fixed rollout rules. Its SNI and timing procedures distinguish sequential communication stages from batched subdomain work.

D.2 Airfoil source-to-OOD protocols

  • The 256 airfoil designs are split by pitch-rate severity: the lower 70% form the source pool and the upper 30% the faster-pitching OOD pool.The design tests operating-regime transfer within represented NACA sections and pitching motions, not transfer to a new airfoil topology.Source: §D.2 · p. 20
  • OOD few-sample adaptation is trajectory-specific: each method uses 16 transitions per trajectory, excluded from its scored 20-step model-owned rollout.LATENTDDM updates only composition, full-domain models update all weights, and SNI uses no labeled transitions.Source: §D.2 · p. 20
  • Figure 6 shows ground-truth vorticity across eight fixed phases of the second pitching cycle, with rows spanning evenly ranked airfoil thicknesses.Its eight source-training and eight OOD adaptation-pool trajectories share a near-body crop and color scale.Source: Figure 6 · p. 21

D.3 Critical-path accounting

  • Inference latency counts dependent local-response rounds, while all subdomains within a round are evaluated in one batched call.Timing includes local calls, communication, and assembly; stages remain sequential.Source: §D.3 · p. 22

D.4 SNI operating points

  • One airfoil SNI stage at τ = .025 reduces port defect from .113689 to .110851 but changes field error only from .0417766 to .0417788.SNI reads directional face data from the current assembled field, updates each subdomain with a shared local BVP, and reassembles before another sweep.Source: §D.4 · p. 22

D.5 Darcy data and extended validation

The Darcy data preserve local grid spacing and geometry as domain covers grow, increasing the number and separation of interacting features. Local pretraining and target adaptation use matched scalar-pressure label volumes, while composition learns boundary-trace corrections without explicit interface constraints.

D.5 Darcy data and extended validation

  • Darcy target covers grow from 3×3 to 16×16 while grid spacing and local geometry stay fixed, increasing the number and separation of interacting features.Each scale has 64 geometry-disjoint training, validation, and protected-test domains; adaptation uses 16 labeled training domains.Source: §D.5 · p. 22
  • Figure 7 presents eight validation geometries per resolution across 3×3, 4×4, 8×8, and 16×16 covers, with white regions marking outside-domain cells or prescribed-pressure inclusions.All panels share a color scale.Source: Figure 7 · p. 23
  • Local and full-domain source supervision each use 36.864M scalar pressure labels, while local training converts them into nine times as many local field examples.The local set contains 36,000 problems extracted from 4,000 source domains; full-domain selection uses 250 additional domains.Source: §D.5 · pp. 22, 24
  • The composition module writes low-dimensional corrections to local boundary traces, but learned traces are not explicitly constrained to satisfy interface continuity or flux balance.The reported evaluation measures assembled-field error; direct interface and flux diagnostics remain future work.Source: §D.5 · p. 24

D.6 Auxiliary native-step SNI mechanism audit

The native-step audit separates SNI interface contraction from assembled-field accuracy and compares model-owned rollouts under controlled validation conditions. The appendix also records release plans and broader scope boundaries for interpreting the surrogate’s use.

D.6 Auxiliary native-step SNI mechanism audit

  • The auxiliary audit uses six training and three validation trajectories, with three case-3 test trajectories held unopened and 20 model-owned steps.Its scope is whether a learned transport iteration can contract interface defect while missing the best response in the frozen local family.Source: §D.6 · p. 24
  • Paired Point-minus-LATENTDDM field differences are .009724 over H1–H20 and .022270 at H20.H20 vorticity/divergence also favor LATENTDDM (.323/.510) over Point (.378/.804).Source: §D.6 · p. 24
  • Across one to 16 SNI stages, field error changes from .027492 to .026661 as port defect contracts from .006198 to .000364.At H100, the SNI–LATENTDDM field gap remains .001535 over all steps and .003237 terminally; LATENTDDM is 3.3× faster than SNI–16, while global Point remains faster on one GPU.Source: §D.6 · p. 24

D.7 Reproducibility, release, and asset licenses

  • The planned acceptance release includes implementation, evaluation code, datasets, generators, schemas, split manifests, and normalization metadata.Documentation will map scripts and data products to experiments, retaining the specified Darcy and airfoil partitions.Source: §D.7 · pp. 24–25

E Broader impacts

  • The potential benefit is reduced marginal simulation cost and improved access, but the experiments do not measure total energy use or demonstrate safety-critical deployment.The authors identify misplaced trust outside validated regimes as the principal risk.Source: E Broader impacts · p. 26
  • The study covers two two-dimensional systems, keeps airfoil transfer within the represented NACA family, and does not explicitly enforce interface continuity or flux conservation.The authors advise treating LATENTDDM as a surrogate within a validation pipeline, not as a replacement for a trusted solver in high-consequence decisions.Source: E Broader impacts · p. 26
Loading 2609.03069v1…