Source-linked AI summary

Latent Twin Operator

Deepanshu Verma, Riley Chen, Matthias Chung

arXiv:2609.35531v1cs.CEcs.NAmath.NA

TL;DR

PDE surrogates often face changing sensor configurations and resolutions, motivating models that are not tied to a fixed discretization. The LTO encodes observations into a latent state, evolves it, and decodes at arbitrary query points, with an explicit context-discretization bound. Experiments verify predicted heat-equation decay and report strong matched comparisons across time-dependent benchmarks, including lower long-horizon error with direct evolution on Navier–Stokes.

  • Problem

    PDE workflows use sensor locations and densities that vary across settings and may change over time, while learned surrogates need to handle differing input and output discretizations.

    Source: §1 · p. 1
  • Method

    The LTO encodes context observations into a latent state, advances it with a learned evolution map, and decodes predictions at arbitrary query points.

    Source: §1 · p. 1
  • Results

    LTO verifies predicted discretization decay on 2D heat, beats matched FNO one-step error on Burgers and FNO/LNO on Navier–Stokes, and direct evolution lowers long-horizon error.

    Source: §1 · p. 2
  • Takeaways & Limitations

    The explicit context-discretization bound provides a theoretical guarantee of discretization invariance under quasi-uniform refinement.

    Source: §1 · p. 2

    The bound covers only SetConv reconstruction error under C2 states, a periodic domain, and well-distributed contexts; flexible evaluation alone does not ensure accurate above-native transfer.

    Source: §5 · p. 9

Abstract

from arXiv · show

Surrogate models deployed on real physical systems rarely see data at fixed resolutions: sensor configurations vary across deployments and may evolve over time as sensing infrastructure changes. We introduce the Latent Twin Operator (LTO), a latent-space surrogate for time-evolving PDEs with a Convolutional Conditional Neural Process (ConvCNP)-style encoder and decoder that accepts a context set of $N$ sensor observations with arbitrary placement and can be queried at any resolution. A learned latent evolution map advances the encoded state directly between arbitrary time points, decoupling temporal evolution from the observation and query discretizations. We derive an explicit $\mathcal{O}(N^{-2/(3D)})$ rate for the context discretization error in spatial dimension $D$ under quasi-uniform refinement. We verify the predicted decay empirically on a 2D heat-equation benchmark. Across time-dependent PDE benchmarks, LTO achieves strong accuracy under one-step comparisons and transfers from native to coarser spatial resolutions with fixed parameters. On Navier--Stokes, its direct latent evolution further reduces error over longer prediction horizons relative to recursive evaluation.

1 INTRODUCTION

The LTO targets PDE workflows where sensor locations and resolutions vary, using a latent encode–evolve–decode surrogate and an explicit context-discretization error bound. Experiments verify the predicted heat-equation decay and report matched benchmark comparisons across time-dependent PDEs.

  • The LTO maps observations to a latent state, advances it with a learned evolution map, and decodes predictions at arbitrary query points.Its resolution-flexible encoding and decoding accept context sets of varying size and placement and produce predictions at arbitrary query resolutions.Source: §1 · pp. 1–2
  • Unlike standard FNO and DeepONet, LTO directly accepts variable-size, irregularly placed contexts without resampling or modification; LNO instead uses learned cross-attention.LTO uses spatially structured kernel smoothing and convolutional latent dynamics, making its discretization error directly amenable to analysis.Source: §1 · pp. 1–2
  • The SetConv-based encoder has a Lipschitz stability constant independent of context-set size and placement.Source: §1 · p. 2
  • The SetConv reconstruction bound is O(N^-2/(3D)) for N context points in dimension D, yielding a discretization-invariance guarantee under quasi-uniform refinement.The bound makes explicit how approximation error depends on context-point number and placement.Source: §1 · p. 2
  • LTO verifies predicted discretization decay on 2D heat, beats matched FNO one-step error on Burgers and FNO/LNO on Navier–Stokes, and direct evolution lowers long-horizon error.It also transfers from native to coarser spatial resolutions with fixed parameters.Source: §1 · p. 2

2 THE LATENT TWIN OPERATOR

The Latent Twin Operator maps variable context observations into a latent state, evolves it independently of context and query discretizations, and decodes predictions at arbitrary query locations. Its SetConv reconstruction supports an explicit discretization-error analysis, with invariance established under quasi-uniform refinement.

2 THE LATENT TWIN OPERATOR

  • LTO encodes context observations, evolves the latent state independently of observation and query discretizations, and decodes at arbitrary query locations.The encoder and decoder accommodate variable context and query discretizations.Source: §2 · p. 3
  • SetConv maps an unordered, variable-size context set to a fixed-grid field using distance-based kernels, then the encoder maps that representation into latent space.The representation stacks local context concentration and kernel-weighted state values.Source: §2 · p. 3
  • The normalized SetConv reconstruction is Lipschitz-stable to observed-state perturbations, with a constant independent of context-point number and placement.This gives uniform preprocessing stability across admissible context discretizations.Source: §2 · p. 4
  • Under the stated regularity assumptions, the reconstruction error combines O(ell^2) kernel-smoothing bias with finite-context quadrature error of order delta(c_ctx)/ell.The assumptions require bounded state curvature, a periodic domain, and context points that do not become arbitrarily clustered; the equal-volume partition is stronger than standard quasi-uniformity.Source: §2 · p. 5
  • With context fill distance δ(c_ctx)=Θ(N^-1/D) and the bandwidth choice from Corollary 2.4, Corollary 2.5 establishes discretization invariance.The supplied passage does not include the resulting bound's formula.Source: §2 · p. 6

3 RELATED WORK

LTO combines flexible input and output discretizations with spatially structured kernel smoothing and convolutional latent dynamics. Related work differs in its discretization requirements or representation, while LTO's construction supports explicit discretization-error analysis.

3 RELATED WORK

  • Standard FNO uses regular grids through the FFT, while DeepONet's branch network is tied to fixed input sensor locations, despite arbitrary output queries [Li et al., 2020; Lu et al., 2019].The broader discussion notes that relaxing FNO's grid requirement needs additional machinery.Source: §3 · p. 6
  • Like LTO, LNO maps arbitrary input sets to fixed-size latent states and decodes at arbitrary locations, but uses learned cross-attention rather than spatially structured kernel smoothing [Wang & Wang, 2024].LTO's construction admits classical error analysis; the cited discussion says no analogous discretization-error rate has been established for irregular-input encoders generally.Source: §3 · p. 6
  • Prior Latent Twin theory leaves the discretization-error term abstract, whereas LTO derives an explicit rate using its kernel-smoothing construction [Chung et al., 2026; 2027].LTO's latent evolution can therefore be a plain CNN, and the construction supports classical error analysis.Source: §3 · p. 6

4 EXPERIMENTS

The experiments test predicted discretization behavior, one-step accuracy, long-horizon evolution, and fixed-parameter resolution transfer across four PDE benchmarks. LTO shows predicted decay on the heat equation, strong results on time-dependent benchmarks, and a remaining accuracy gap on Darcy flow.

4 EXPERIMENTS

  • The time-dependent PDEBench models are trained from one source state to its corresponding target state, with context-size augmentation resampling only the encoder’s context set.Evaluation errors are relative L2; query and target remain at full native resolution during augmentation.Source: §4 · p. 7

4.1 2D HEAT EQUATION: RESOLUTION SWEEP

  • Both closed-form discretization error and a trained LTO’s forecasting error decrease with context count at a rate consistent with the predicted slope.The trained model is evaluated across context counts without retraining, with query count fixed; the test is averaged over five runs.Source: §4.1 · p. 7, Figure 2 · p. 7

4.2 BURGERS’ EQUATION

  • LTO has lower one-step error than FNO at every tested Burgers resolution from 64 to 2048, while training in about one quarter of FNO’s time.The models use comparable parameter counts: 29,395 for LTO versus 23,937 for FNO.Source: §4.2 · pp. 7–8, Table 1 · p. 8

4.3 NAVIER–STOKES (2D)

  • At 16 × 16, 32 × 32, and 64 × 64, LTO has the lowest one-step error and is nearly insensitive to coarsening; at 128 × 128, all models deteriorate and LTO increases most.The 128 × 128 result does not support reliable extrapolation to finer-than-training resolution.Source: §4.3 · p. 8, Table 2 · p. 8
  • By five and ten recursive steps, both LTO evaluations have lower error than FNO and LNO, and direct evaluation gives the lowest ten-step average.The comparison fixes the context at one time point, unlike the one-step comparison, which averages over six context start points.Source: §4.3 · p. 8, Table 3 · p. 9

4.4 DARCY FLOW

  • On Darcy flow, Gaussian LTO trails LNO by 11.80× on average, while the attention variant narrows the ratio to 4.86× but remains less accurate at every resolution.Table 4 matches LNO’s parameter count to each LTO variant.Source: §4.4 · p. 9, Table 4 · p. 9

5 DISCUSSION AND LIMITATIONS

LTO is less accurate than FNO and LNO at the first Navier–Stokes step, but has lower long-horizon errors, with direct evaluation improving over recursive evolution. The discretization guarantee is limited to SetConv reconstruction under stated regularity and sampling assumptions; above-native transfer and fixed-kernel weighting remain limitations.

5 DISCUSSION AND LIMITATIONS

  • On Navier–Stokes, FNO and LNO are more accurate at the first step, while LTO gives lower long-horizon errors, with direct evaluation further improving over recursive evolution.The discretization result covers only SetConv reconstruction; it assumes C2 states, a periodic domain, and sufficiently well-distributed context points.Source: §5 · p. 9
  • At 128×128 on Navier–Stokes, a sharp error increase shows that resolution-flexible evaluation does not ensure accurate above-native transfer.Autoencoder and latent-evolution errors remain separate from the SetConv reconstruction bound.Source: §5 · p. 9
  • The fixed Gaussian kernel weights context by physical distance; learned attention reduces Darcy approximation error but is not covered by the SetConv analysis.This motivates adaptive reconstruction operators that retain quantitative error control.Source: §5 · p. 9

6 CONCLUSION AND FUTURE WORK

The conclusion presents LTO as a resolution-flexible surrogate with an explicit SetConv discretization rate and benchmark results that include lower one-step errors and improved long-horizon prediction. It also identifies transfer and reconstruction limits, outlines future directions, and records reproducibility, AI-use, and ethics statements.

6 CONCLUSION AND FUTURE WORK

  • LTO combines resolution-flexible encoding and decoding with an explicit O(N^-2/(3D)) SetConv discretization rate under quasi-uniform refinement.The heat-equation experiment supports the predicted decay; LTO has lower one-step error than matched FNO on Burgers and the lowest one-step error at native and sub-native resolutions on Navier–Stokes.Source: §6 · p. 9
  • On Navier–Stokes, LTO has lower error at longer horizons, and direct temporal evaluation further reduces error accumulation.The conclusion also identifies above-native Navier–Stokes deterioration and Darcy results as limits of the current fixed-resolution latent representation and local SetConv reconstruction.Source: §6 · p. 9
  • Future work includes broader domain and sampling analyses, improved above-native transfer, adaptive reconstruction with error control, adaptive sensing, temporal-consistency constraints, and physical or stochastic latent structure.Source: §6 · p. 10

REPRODUCIBILITY STATEMENT

  • The authors state that architecture code, training and evaluation scripts, and exact Burgers and Navier–Stokes PDEBench splits will be released after acceptance.The FNO Burgers baseline was trained from scratch on the same split because the PDEBench published number could not be independently reproduced.Source: REPRODUCIBILITY STATEMENT · p. 10

AI USE STATEMENT

  • LLM assistance revised prose and notation; the authors state that theorem statements, proofs, experimental design, and numerical results are their own.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The work uses synthetic and publicly available PDEBench data, with no human subjects, personal data, or original data collection.The authors anticipate no work-specific ethical concerns beyond those generally associated with scientific machine learning research.Source: ETHICS STATEMENT · p. 10

A.1 DEFERRED PROOFS

The deferred proofs establish stability for normalized SetConv reconstruction and derive a discretization bound by separating smoothing bias from finite-context quadrature error. The resulting rate depends on quasi-uniform refinement and the stated regularity conditions.

Deferred lemmas

  • For periodic domains, translation in the integral shows that the kernel normalization κ̄ is constant across spatial locations.Source: §A.1 · p. 12

Encoder stability

  • The normalized SetConv reconstruction is 1-Lipschitz uniformly over context count, bandwidth, and placement, whereas the raw-channel bound can grow without limit under clustered contexts.The normalized channel divides by local density; this keeps the output change bounded by the largest input perturbation.Source: §A.1 · pp. 12–13

Discretization/quadrature error

  • The quadrature argument uses cellwise estimates for smooth, decaying kernel terms, avoiding a dimension-dependent worst-case bound.The proof notes that the curvature contribution is smaller by Θ(δ/ℓ) than the linear-term bound.Source: §A.1 · p. 14
  • Under the proof’s assumptions, finite-context quadrature error is of relative order δ/ℓ, while the state-dependent term adds no change to that rate.The additional gradient contribution is smaller than the kernel term as ℓ tends to zero.Source: §A.1 · p. 14

Explicit discretization error

  • Balancing O(ℓ²) smoothing bias against the finite-context error yields an optimal bandwidth ℓ*=Θ(δ^(1/3)) and an error of order δ^(2/3).The smoothing-bias bound follows from symmetry of the second-order kernel and the Hessian bound.Source: §A.1 · p. 15

A.2 EXTENDED RELATED WORK

The related-work discussion positions LTO among operator-learning, implicit-representation, conditional-process, and latent-space approaches. It emphasizes that the paper’s contribution is an explicit discretization-error rate for a SetConv-style construction, alongside differences in architecture and temporal evolution.

Neural operators

  • Unlike standard FNO, which uses FFT-based global convolutions on regular grids, LTO targets discretization-flexible input and output.The standard FNO formulation requires regular-grid inputs or resampling.Source: §A.2 · p. 15

Implicit neural representations

  • Implicit neural representations encode individual fields as coordinate networks, but do not themselves provide an encode→evolve→decode surrogate for time-evolving states.Source: §A.2 · p. 15

Conditional neural processes and SetConv

  • CNP and ConvCNP use kernel-weighted embeddings for variable-size observations; this paper contributes an explicit discretization-error rate for that construction, not the construction itself.Source: §A.2 · p. 15

Latent-space operator learning

  • Prior latent-space methods show empirical multi-resolution behavior, while earlier Latent Twins provide the encode→evolve→decode framework but leave its architecture-specific discretization term abstract.LTO instantiates the framework with resolution-flexible encoding and decoding and a concrete bound.Source: §A.2 · p. 15

Broader attention-based operator architectures

  • Among attention-based models, some compress inputs into latent tokens while others retain a full token set; the cited architectures do not report explicit input-point discretization rates.The discussion notes ONO, like LNO, reports zero-shot Darcy resolution transfer.Source: §A.2 · pp. 15–16

Discretization-invariance critiques

  • Theorem 2.3 supplies an explicit rate for SetConv-style encoders on arbitrary, not necessarily gridded, context sets, analogous to a regular-grid FNO rate.The comparison distinguishes the SetConv result from an FNO bound expressed in terms of input Sobolev regularity.Source: §A.2 · p. 16

How LNO differs from our construction

  • Compared with PhCA, SetConv uses parameter-free kernel embedding and a spatial latent grid, enabling CNN latent dynamics; LTO also evolves without intermediate decoding.The discussion says existing attention-based encoder results establish universal approximation rather than an explicit input-point rate.Source: §A.2 · p. 16

A.3 DARCY FLOW: ATTENTION MECAHSNISM AND NATIVE LNO PARAMETER BUDGET

The section contrasts fixed distance-based Gaussian weighting with learned attention, which can weight distant context points. A native LNO parameter-budget comparison indicates the reported gap is not primarily explained by LNO’s larger budget.

A.3 DARCY FLOW: ATTENTION MECAHSNISM AND NATIVE LNO PARAMETER BUDGET

  • The fixed Gaussian kernel weights context points by distance, whereas learned attention uses position embeddings and can weight distant points when that reduces training loss.The attention score uses learned query and context embeddings with scaled dot-product attention.Source: §A.3 · p. 16
  • Table 5 reports Darcy-flow relative L2 error across evaluation resolutions for LTO variants using Gaussian-kernel and learned-attention encoders.Source: Table 5 · p. 17
  • 12.99× for the plain kernel and 3.69× for the attention encoder are close to parameter-matched ratios, so the gap is not primarily an artifact of LNO’s larger parameter budget.The ratios use LNO’s originally reported 762,113-parameter configuration.Source: §A.3 · p. 17
Loading 2609.35531v1…