PaperlayerView exact revision on arXiv

Source-linked AI summary

MENO: Memory-Efficient Neural Operator

Shengyang Xu, Weijun Zhang, Jun Hu, Pengzhan Jin

arXiv:2609.27739v1cs.LG

TL;DR

Varying-domain PDE operator learning faces efficiency bottlenecks in integration-based approaches and generalization limits in an integration-free alternative. MENO uses the Manifold Function Encoder to address these challenges, and reports strong benchmark accuracy with substantially lower peak GPU memory.

  • Problem

    Varying-domain neural operators face resolution-related memory and computation bottlenecks, while integration-free MIONet with MFE has weaker generalization than integration-based architectures.

    Source: §1 · p. 2

  • Method

    MENO uses the Manifold Function Encoder to process manifold functions without integration operators during training.

    Source: §3 · p. 8, §5 · p. 12

  • Results

    MENO achieves the best accuracy on Poisson and NASA-CRM, second-best on AhmedML, and significantly lower peak GPU memory than other architectures in all tested cases.

    Source: §5 · p. 13

  • Takeaways & Limitations

    MENO handles arbitrary geometric domains and discretizations, including cross-geometry inputs where functions and solutions lie on different manifolds.

    Source: §5 · pp. 12–13

    The stated MFE setting is a finite collection of k-dimensional Lipschitz manifolds in a d-dimensional box, with pairwise intersections of dimension below k or empty.

    Source: §2 · p. 3

Abstract

from arXiv · show

We propose the Memory-Efficient Neural Operator (MENO) as a high-performance PDE neural solver based on the Manifold Function Encoder (MFE). MENO features three primary advantages: (1) MENO has a significantly smaller memory footprint and much faster training speed than other popular architectures, with the memory footprint being independent of the data resolution, and therefore holds the potential for scaling up to large-scale models. (2) MENO can accept PDE inputs of arbitrary form, including arbitrary geometric domains and arbitrary discretizations. In particular, it is capable of handling cross-geometry scenarios, i.e., where the input functions and the output solutions are defined on different manifolds. (3) MENO exhibits strong generalization capability, and achieves the best accuracy on most of the benchmarks we tested, compared with the results reported in the literature. The code is available on GitHub at https://github.com/jpzxshi/MENO, and all numerical examples in this paper can be run with a single command to reproduce the reported results.

1 Introduction

Neural operators learn solution operators for PDE families, but varying domains add a challenge beyond varying coefficients, forcing terms, and boundary conditions. MENO targets the efficiency and generalization limits of existing approaches while handling arbitrary geometries, including cross-geometry inputs.

1 Introduction

  • Neural operators learn mappings between function spaces and have developed through alternative architectures, physics-informed training, and integration into classical numerical solvers.Examples include DeepONet and FNO, physics-informed DeepONets, and hybrid iterative methods.Source: §1 · p. 1
  • Varying-domain methods use deformation to map equivalent domains to a canonical domain or extension to embed domains and functions in a larger canonical domain.Deformation cannot handle domains that are not topologically homeomorphic; extension is not constrained by domain topology.Source: §1 · pp. 1–2
  • Integration-based encoders can incur severe GPU-memory and computational costs as resolution grows, while integration-free MIONet with MFE still lags in generalization.Direct characteristic-function and signed-distance-function extensions also face efficiency bottlenecks in three-dimensional problems because they compute over physical points throughout the space.Source: §1 · p. 2
  • MENO combines resolution-independent memory use and fast training with strong generalization, achieving the best accuracy on most tested benchmarks and handling arbitrary geometries, including cross-geometry cases.The architecture is motivated by the need for an integration-free operator with generalization performance no worse than integration-based neural operators.Source: §1 · p. 2

2 Manifold Function Encoder

The Manifold Function Encoder (MFE) encodes functions defined on manifolds into latent codes and decodes them into functions. Its stated theoretical guarantee is super-algebraic convergence for common spectral bases.

2 Manifold Function Encoder

  • MFE's target space comprises manifold-function pairs; for zero-dimensional manifolds, the manifold is treated as a finite set of points with real-valued functions on those points.The setting describes finite collections of k-dimensional Lipschitz manifolds in a d-dimensional box, with pairwise intersections of dimension less than k or empty.Source: §2 · p. 3
  • MFE encodes manifold functions using fixed continuous basis functions, with the encoding dimension increasing as the number of basis modes increases.Examples of basis functions include orthogonal polynomials, Fourier bases, and finite elements.Source: §2 · p. 3
  • For PDE solutions on a known manifold, the decoded solution can be obtained on that manifold using a restriction operator.The paper notes that the solution manifold may coincide with one of the input manifolds.Source: §2 · p. 4
  • For Legendre- or Fourier-based MFE, the encoder-decoder composite converges to the identity mapping super-algebraically.The paper connects MFE's ability to compress manifold-function information into a low-dimensional vector to MENO's memory efficiency.Source: §2 · p. 4
  • MFE tokens encode multiple manifold functions, including functions whose domains have different dimensions, as a sequence of tokens.The text describes each token as having dimension equal to the sum of the functions' latent dimensions; the Poisson example uses n^2 tokens of dimension 5.Source: §2 · p. 4

3 Architecture of MENO

MENO combines global and local branches, transferring global information into the local branch before summing their outputs. Its MFE-based architecture supports cross-geometry inputs and avoids integration operators during training, making GPU memory usage independent of data resolution.

Architecture overview

  • MENO’s global and local branches exchange information through global-to-local blocks, then sum their outputs for the final solution.The global branch supplies information to the local branch; the transfer is unidirectional.Source: §3 · p. 5, Figure 1 · p. 7

Global branch

  • The global branch encodes MFE tokens with spectral position information, processes them through Transformer blocks, then projects and decodes preliminary solutions.The spectral position information uses t frequency modes.Source: §3 · p. 5

Global-to-local transfer

  • The global branch injects processed information into the local branch, where decoded values and derivatives can carry information of arbitrary derivative order.The described first-order derivative information is an example; higher-order derivatives can also be incorporated.Source: §3 · pp. 5–6

Local branch

  • At each output query point, the local branch evaluates input information even when the output manifold differs from the input manifolds.When the output manifold matches an input manifold, the corresponding cross-evaluation can be replaced by direct evaluation.Source: §3 · p. 6

Memory efficiency

  • MENO’s training architecture has no integration operator, so GPU memory usage is independent of data resolution.The passage contrasts this with integration-based architectures that involve physical points throughout the mesh or point cloud.Source: §3 · p. 8

Cross-geometry inputs

  • MFE unifies PDE inputs defined on different manifolds, including geometry, material properties, boundary conditions, and contact or friction conditions.The stated examples span 0-, 1-, 2-, and 3-dimensional manifolds.Source: §3 · p. 8

4 Numerical Experiments

The experiments compare MENO with recent varying-geometry operators and report a cross-geometry Poisson result alongside the training setup.

4 Numerical Experiments

  • Experiments use Legendre-based MFE, AdamW, GeLU, and 500 training epochs, comparing MENO with recent neural operators using original-paper results.All experiments run on an NVIDIA A100 PCIe GPU; memory comparisons use the same batch size and number of query points.Source: §4 · pp. 8–9, Table 1 · p. 9

4.1 Poisson equation.

  • The cross-geometry Poisson dataset has 9,000 training and 1,000 test examples in [0, 1]2, split evenly between disk-like and annulus-like domains.Inputs encode the domain and functions on the domain and its boundary; redundant boundary-domain information is omitted.Source: §4.1 · p. 9
  • 2.07% relative error for MENO, one-third of MIONet’s 6.49%, on cross-geometry Poisson with disk-like and annulus-like domains.The dataset contains 9,000 training examples and 1,000 test examples; Transolver-3 and PCNO are not applicable to this case.Source: Table 3 · p. 9, §4.1 · p. 9

Gsingle

MENO is evaluated on single-geometry Poisson, NASA–CRM, and AhmedML benchmarks. It combines competitive accuracy with low resource use, though its accuracy rank varies across tasks.

Gsingle

  • MENO slightly exceeds PCNO in single-geometry Poisson accuracy while using 1.7% of its parameters, 5.1% of its peak GPU memory, and 8.3% of its training time.Source: Gsingle · p. 10, Table 4 · p. 10

4.2 NASA–CRM.

4.3 AhmedML.

  • On AhmedML, MENO achieves the second-best accuracy; PCNO exceeds the 80 GB single-GPU memory limit under the reported hyperparameters.A small MENO version also achieves second-best accuracy with very few parameters and short training time.Source: §4.3 · p. 12, Table 6 · p. 12

5 Conclusion

MENO uses the Manifold Function Encoder to handle varied PDE inputs while reducing memory use and training time. It performs strongly across the reported benchmarks, and the authors identify general PDE solvers as future work.

  • MENO's GPU memory usage is independent of data resolution because its training architecture has no integration operator, reducing memory cost especially for large-scale problems.Source: §5 · p. 12
  • MFE encodes functions on different manifolds, allowing MENO to accept arbitrary PDE inputs, geometric domains, discretizations, and cross-geometry scenarios.Source: §5 · pp. 12–13
  • MENO achieves the best accuracy on Poisson and NASA-CRM and the second-best accuracy on AhmedML, with peak GPU memory significantly below other architectures in all cases.Source: §5 · p. 13
  • The authors plan to develop general PDE neural solvers based on MENO's memory efficiency, cross-geometry handling, and generalization capability.Source: §5 · p. 13
Loading 2609.27739v1…