PaperlayerView exact revision on arXiv

Source-linked AI summary

Physics-Informed Neural Plasticity: PDE Solvers That Reshape Themselves

Chun-Wun Cheng, Bingcheng Hu, Angelica I. Aviles-Rivero

arXiv:2610.09510v1cs.LGcs.NAmath.NA

TL;DR

Physics-informed PDE solvers often keep their representation fixed even when solution complexity varies across space and space–time. ReCAP adapts localized capacity during training, and achieves the lowest relative L2 error on every benchmark across five 3D/4D PDE benchmarks and 11 competing physics-informed solvers.

  • Problem

    Physics-informed solvers often keep their representation fixed, although heterogeneous PDE solutions can require different approximation resources across space or space–time.

    Source: §1 · p. 1

  • Method

    ReCAP is a Gaussian-localized solver that adapts capacity through component enrichment, residual-directed splitting, function-aware merging and pruning, and coordinate-aware quiet-child refinement.

    Source: §1 · p. 2

  • Results

    ReCAP achieves the lowest relative L2 error on every benchmark across five 3D/4D PDE benchmarks and 11 competing physics-informed solvers.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The results support treating representational capacity as something to reorganize during physics-informed learning, alongside learning the solution.

    Source: §1 · p. 2, §5 · p. 10

    Reliability of the localized residual indicators requires a PDE stability estimate and appropriate mass weighting; reusing training points for independent certification requires uniform generalization control or continuous-domain verification.

    Source: §3.4 · p. 7

Abstract

from arXiv · show

Physics-informed neural PDE solvers adapt their parameters to satisfy governing equations, yet their representational structure typically remains fixed throughout training. This rigidity is poorly matched to PDE solutions with strongly heterogeneous complexity across space and space--time, leaving capacity insufficient where the physics is difficult and redundant where it is simple. We introduce physics-informed neural plasticity, a paradigm in which the representation itself reshapes during optimization in response to unresolved physics. We instantiate this principle with Representation Capacity Adaptation for PDEs (ReCAP), a Gaussian-localized solver that dynamically redistributes capacity through local enrichment, residual-directed splitting, gate-based pruning, and function-aware merging. ReCAP uses responsibility-weighted error indicators and the geometry of residual energy to determine where and how to refine. To limit the disturbance introduced by splitting, we introduce quiet-child refinement, which initializes new components by transporting the parent representation while controlling instantaneous functional perturbation. We further establish conditional a posteriori reliability and structural-stability guarantees linking localized physics residuals to solution error and stable refinement. Across five challenging 3D and 4D PDE benchmarks against 11 physics-informed solvers, ReCAP achieves the lowest relative $L^2$ error on every problem, reducing error by $10.7\%$--$27.5\%$ relative to the strongest competing result. These results suggest that physics-informed solvers need not merely learn their parameters---they can learn how their representational capacity should be organized.

1 INTRODUCTION

ReCAP addresses fixed representational structure in physics-informed solvers by adapting capacity to unresolved physics. Across five 3D/4D PDE benchmarks, it achieves the lowest relative L2 error among 11 competing physics-informed solvers.

1 INTRODUCTION

  • Fixed representations can leave capacity redundant in resolved regions and insufficient where heterogeneous, difficult physics needs to be represented.PDE solutions may mix smooth regions with oscillatory waves, localized peaks, or rapidly evolving fronts.Source: §1 · pp. 1–2
  • Physics-informed neural plasticity reshapes representational capacity during training in response to unresolved physics.The solver jointly learns the solution and reorganizes the capacity used to express it.Source: §1 · p. 2
  • ReCAP is a Gaussian-localized solver that enriches, splits, merges, and prunes components using local responsibilities to redistribute capacity.These operations implement p-like enrichment, h-like refinement, and coarsening.Source: §1 · p. 2
  • Responsibility-weighted errors identify unresolved components, while residual-weighted covariance orients splits along the dominant spread of unresolved residual energy.Merging requires both effective overlap and agreement of local predictions.Source: §1 · p. 2
  • Quiet-child refinement transports the parent polynomial into child coordinates and uses a small gate to control immediate disturbance to the learned function.The paper also establishes conditional reliability and a gate-dependent bound on instantaneous loss increase for normalized mixtures with a linear transport operator.Source: §1 · pp. 2–3
  • ReCAP achieves the lowest relative L2 error on every benchmark across five 3D/4D PDE benchmarks and 11 competing physics-informed solvers.The paper also reports theoretical analysis connecting local adaptation signals to global solution accuracy and characterizing refinement stability.Source: §1 · pp. 2–3

2 RELATED WORK

Related approaches improve physics-informed optimization, sampling, and local representations, while ReCAP coordinates local basis changes with component insertion and removal. Its analogy to hp refinement is motivational and does not transfer finite-element convergence guarantees.

2 RELATED WORK

  • Optimization and residual-adaptive sampling methods alter training signals or equation-evaluation locations, whereas ReCAP also changes the available local basis.The paper describes sampling and representation adaptation as complementary and examines their cumulative effect.Source: §2 · p. 3
  • Prior local-representation approaches include domain decomposition, trainable Gaussian features, and residual-driven addition of local subnetworks; ReCAP coordinates insertion and removal with local polynomial-order changes.Its splitting uses residual geometry, merging checks local functional agreement, and child initialization accounts for polynomial-coordinate dependence.Source: §2 · p. 3
  • ReCAP's learned Gaussian components use hp-like refinement operations, but the analogy does not transfer finite-element convergence guarantees.Classical hp methods change element size and approximation order; hp-VPINNs bring related refinement into variational physics-informed learning.Source: §2 · p. 3
  • Residual adaptation scores are not local solution-error bounds; their reliability requires PDE stability and appropriate mass weighting.The analysis makes this condition explicit rather than asserting that the stability estimate holds for every benchmark.Source: §2 · p. 3

3 METHODOLOGY

ReCAP alternates between fitting continuous parameters and changing the representation according to localized physics errors. Its adaptation combines local enrichment, component splitting, pruning, and function-aware merging, with conditional analyses of localized residuals and refinement stability.

3 METHODOLOGY

  • ReCAP alternates parameter optimization at fixed structural state with representation updates, separating fitting the current approximation space from reallocating capacity.Source: §3 · p. 3

3.1 REPRESENTATION AND LOCAL ERROR INDICATORS

  • Responsibility-weighted indicators localize PDE errors; their normalization separates violation intensity from assigned mass, while the overlap disagreement score compares local models, not a global predictor's interface jump.Source: §3.1 · pp. 4–5

3.2 POLYNOMIAL ENRICHMENT, PRUNING, AND FUNCTIONAL MERGING

  • High-scoring components can gain polynomial order; pruning uses gate thresholds, while merging requires effective overlap and prediction agreement and remains a heuristic that may change the predictor.Source: §3.2 · p. 5

3.3 RESIDUAL-DIRECTED SPLITTING AND COORDINATE-AWARE INITIALIZATION

  • Residual-directed splitting orients refinement along the leading eigenvector of a responsibility- and residual-weighted covariance, with geometric fallback when residual mass or covariance is inadequate.Source: §3.3 · pp. 5–6
  • Quiet-child refinement translates polynomial coefficients to preserve the parent's local function, then limits the child's gate to control its initial contribution to the aggregate prediction.Source: §3.3 · p. 6

3.4 CONDITIONAL RELIABILITY AND CONTROLLED ADAPTATION

  • The conditional reliability result transfers an assumed PDE stability estimate to mass-weighted localized residual and condition indicators; it does not establish stability for arbitrary PDEs or losses.Source: §3.4 · pp. 6–7
  • The child-perturbation analysis concerns additive insertion into a normalized gated mixture and depends on mixture coverage, controlled child derivatives, and parent–mixture agreement.Source: §3.4 · p. 7, The post-split predictor satisfies · p. 7

The post-split predictor satisfies

  • When child gates can approach zero, the refined class contains the pre-split class in its closure, so optimal achievable loss cannot increase; this is not a monotonic training-loss guarantee.Source: The post-split predictor satisfies · p. 7

4 EXPERIMENTAL RESULTS

Across five three- and four-dimensional PDE benchmarks, ReCAP reports the lowest relative L2 error against the strongest baseline on each task. KG3D ablations and comparisons further examine the benefits and sensitivity of its adaptive allocation.

Main Results

  • 27.5% (FM3D vs. PirateNets), 10.7% (H3D vs. SPINN), 12.1% (ND3D vs. PIG), 15.7% (KG3D vs. SPINN), and 12.2% (KG4D vs. JAXPI) lower relative L2 errors; ReCAP led all five.The benchmarks include five PDE problems spanning three and four input dimensions.Source: Table 1 · p. 8, §4 · p. 8

Qualitative solution and error comparisons

  • Figure 2 shows ReCAP resolving localized solution structures, with small errors concentrated near rapidly varying regions in the FM3D comparison.The visualizations include ND3D at t = 0.4, KG3D, and FM3D at t = 1.5.Source: Figure 2 · p. 8, §4 · p. 9

Splitting-threshold sensitivity

  • 3.772 × 10−3: on the KG3D threshold sweep, τsplit = 1.2 had the lowest tested relative L2 error, a 20.9% reduction relative to τsplit = 0.8.The response was non-monotonic; the other tested thresholds were within 4.5% of the minimum.Source: Figure 3 · p. 9, §4 · p. 10

Cumulative component ablation

  • 45.0%: KG3D relative L2 error fell from 6.75×10−3 (M1) to 3.71 × 10−3 (M4) as splitting, pruning and merging, and adaptive sampling were added cumulatively.The reductions after splitting reflect incremental gains conditional on preceding components, not independent effects.Source: §4 · p. 9

Adaptive allocation versus particle-count scaling

  • ReCAP achieved a relative L2 error of 3.181 × 10−3, improving on PIG-large by 24.4%, Fixed-pmax by 43.1%, and Split-only by 17.8%.Matching ReCAP’s final particle count or using uniform maximum polynomial order or splitting alone did not close the accuracy gap.Source: §4 · p. 9

5 CONCLUSION

The conclusion presents ReCAP as a way to treat representational capacity as an adaptive resource for physics-informed PDE solving. It summarizes benchmark performance and conditional theoretical guarantees, and discloses the use of generative AI tools.

Conclusion

  • ReCAP jointly adapts local approximation order, component geometry, and component count to redistribute capacity during physics-informed PDE solving.The conclusion frames representational capacity as an adaptive resource.Source: §5 · p. 10
  • Across five three- and four-dimensional PDE benchmarks, ReCAP achieved the lowest reported relative L2 error; under stated assumptions, analysis connects mass-weighted physics indicators to solution error and bounds additive-refinement perturbation.The paper reports that capacity-controlled comparisons support targeted allocation beyond increasing component count alone.Source: §5 · p. 10
  • The authors report using generative AI tools to assist with proofs, methodological exposition, interpretation and presentation of results, and writing, and state that they reviewed the AI-assisted work.They state that they take responsibility for the final content and artifacts.Source: AI USE STATEMENT · p. 10

Supplementary Material

The supplementary material expands the representation and adaptation rules, beginning with the PDE problem and training objective and then describing the adaptive Gaussian representation.

A DETAILED REPRESENTATION AND ADAPTATION RULES

  • The supplement expands the representation and component-update rules summarized in Section 3.It introduces the detailed representation and adaptation rules.Source: A DETAILED REPRESENTATION AND… · p. 13

A.1 PROBLEM FORMULATION

  • The problem formulation defines the PDE residual from the learned solution and states that continuous parameters are optimized against a base objective between structural adaptation events.The adaptation mechanism is independent of the precise composition of that objective.Source: §A.1 · p. 13

A.2 ADAPTIVE GAUSSIAN LOCAL REPRESENTATION

  • The adaptive representation uses Gaussian-localized components with centers, scales, gates, local polynomial coefficients, and active polynomial orders.Responsibilities and, when needed, component-wise local values are inputs required by the adaptation procedure.Source: §A.2 · p. 14

A.3 PHYSICS-GUIDED LOCAL ERROR ESTIMATION

The section localizes physics errors to individual components using their responsibilities, with optional condition-error and local-disagreement information contributing to adaptation.

A.3 PHYSICS-GUIDED LOCAL ERROR ESTIMATION

  • When value-based initial or boundary observations are available, the section adds a condition-error indicator; softly imposed initial conditions use initial and boundary points together.If initial conditions are hard-constrained, only remaining boundary points are used.Source: §A.3 · p. 14
  • A nearest-neighbor graph between Gaussian centers supports a local disagreement score that compares component gradients where both components are relevant.The score is not a literal interface jump of the global predictor.Source: §A.3 · pp. 14–15
  • Responsibility-weighted physics error assigns current equation and condition mismatches to individual components, supporting targeted changes in representation complexity.The solution-error interpretation additionally requires the stated stability assumptions.Source: §A.3 · pp. 14–15

A.4 ADAPTIVE REPRESENTATION REFINEMENT AND COARSENING

ReCAP redistributes representation capacity through local polynomial enrichment, component pruning and merging, and residual-directed Gaussian splitting. Its refinement rules use localized physics errors and aim to limit the functional disturbance caused by adding a component.

Polynomial-order refinement

  • High-error components can first receive local p-enrichment, activating additional polynomial terms without changing Gaussian geometry or component count.Setting newly activated coefficients to zero preserves the current predictor; otherwise, the passage describes the enrichment operation but does not guarantee that property.Source: §A.4 · p. 15

Gate-based pruning

  • Pruning removes components with small effective gate probabilities, with an optional low-localized-error criterion and a minimum-component constraint.When candidates exceed the removal budget, the smallest gates are prioritized, with localized error as an additional ordering criterion.Source: §A.4 · p. 15

Functional merging

  • Merging coarsens low-error components only when they have both effective overlap and similar local predictions; geometric proximity alone is insufficient.Candidate pairs are ranked by large overlap, small functional discrepancy, and small combined error, then selected as disjoint pairs. The merged parameters use regularized, moment-based initialization, which does not guarantee function preservation.Source: §A.4 · pp. 15–16

Residual-guided Gaussian splitting

  • Residual-directed splitting adds localized capacity where unresolved physics is concentrated, orienting each split along the principal direction of responsibility-weighted residual spread.If residual mass is negligible or the covariance is invalid, the method falls back to a geometric direction, typically the largest-scale axis.Source: §A.4 · pp. 16–17

Near-function-preserving split initialization

  • Quiet-child refinement translates the parent polynomial into child coordinates, keeps the trained parent unchanged, and gives the child a small gate to limit insertion perturbation.With a complete monomial basis through the active degree, translation preserves the polynomial as a function of global coordinates; the full model prediction need not remain exactly unchanged because responsibilities and gates also affect aggregation.Source: §A.4 · pp. 17–18

B FURTHER THEORETICAL RESULTS

The theoretical results give conditional reliability for mass-weighted local indicators and analyze residual-covariance splitting and quiet-child refinement. Their guarantees depend on stated stability, sampling, and model-form assumptions.

Reliability certificate

  • Mass-weighted local indicators reconstruct the global physics loss, while converting residual accuracy into solution accuracy requires stability in the chosen residual norm.The stability premise is not automatic for arbitrary PDEs or losses.Source: B FURTHER THEORETICAL RESULTS · pp. 18–19
  • Independent certification samples support a probabilistic bound for the local indicators; reusing training points requires a uniform generalization argument or continuous-domain verification.The predictor is frozen before certification samples are drawn; if κΓ = 0, condition terms and their sample-count requirements may be omitted.Source: B FURTHER THEORETICAL RESULTS · p. 19

Optimal residual-covariance direction

  • The residual-covariance split direction maximizes responsibility- and residual-weighted projected coordinate variance for the selected component.The split direction is the principal eigenvector of the weighted covariance.Source: B FURTHER THEORETICAL RESULTS · p. 20

Quiet-child refinement

  • Exact polynomial translation preserves the parent polynomial in child coordinates when the complete monomial basis through its degree is used.With component-wise local orders, translation must use only active parent coefficients or have inactive stored coefficients set to zero; otherwise exact preservation need not hold.Source: B FURTHER THEORETICAL RESULTS · p. 20
  • The quiet-child perturbation bound applies to additive, parent-preserving splits under its normalized-mixture and transport assumptions, not to symmetric replacement, pruning, or merging.If admissible child gates can approach zero, the refined class contains the old class in its closure, so its optimal achievable loss cannot increase.Source: B FURTHER THEORETICAL RESULTS · pp. 20–21

C PDE BENCHMARKS

The benchmark set comprises five PDE problems spanning three- and four-dimensional inputs, with the suffix counting all input coordinates, including time. It covers flow mixing, Helmholtz, nonlinear diffusion, and Klein–Gordon equations.

C PDE BENCHMARKS

  • Five benchmarks cover flow mixing, Helmholtz, nonlinear diffusion, and Klein–Gordon equations, with input dimensions determined by spatial coordinates and time.H3D has three spatial coordinates; FM3D, ND3D, and KG3D have two spatial coordinates and time; KG4D has three spatial coordinates and time.Source: C PDE BENCHMARKS · p. 21, §C.4 · p. 22

C.1 (2+1)D FLOW MIXING

  • FM3D models linear transport and rotational mixing of a passive scalar in a two-dimensional velocity field.Its domain is [-4, 4]^2 × [0, 4], with an initial condition specified by the cited passage.Source: §C.1 · p. 21

C.2 3D HELMHOLTZ EQUATION

  • The 3D Helmholtz benchmark uses a manufactured solution satisfying a homogeneous boundary condition exactly.The passage specifies k = 1 and coefficients (a1, a2, a3) = (4, 4, 3).Source: §C.2 · p. 22

C.3 (2+1)D NONLINEAR DIFFUSION

  • ND3D is a solution-dependent diffusion problem in two spatial dimensions and time, initialized with three localized peaks of differing amplitudes, widths, and locations.Its gradient and Laplacian act only on spatial coordinates.Source: §C.3 · p. 22

C.4 KLEIN–GORDON EQUATIONS

  • KG3D and KG4D are manufactured Klein–Gordon problems with two or three spatial coordinates, respectively, plus time.The full specification includes second-order time dynamics, initial displacement and velocity, and boundary values from the manufactured solution.Source: §C.4 · p. 22

D EXPERIMENTAL SETTINGS

The experimental-settings section points readers to Table 2 for benchmark hyperparameters, including training epochs and other model-specific settings.

D EXPERIMENTAL SETTINGS

E VISUALIZATION

Visualizations compare ReCAP predictions with reference fields and show concentrated or reduced errors across the illustrated benchmarks. On KG3D, its reported operation count matches SINN and is below several displayed methods, while Scale-PINN has the lowest count.

E VISUALIZATION

  • ReCAP captures FM3D’s evolving rotational mixing structures, with remaining errors concentrated near complex spiral interfaces and lower, more concentrated errors than representative baselines.Figure 4 shows solution and absolute-error snapshots over evaluated time points.Source: E VISUALIZATION · p. 23, Figure 4 · p. 24
  • On H3D and KG3D, ReCAP reproduces dominant spatial patterns across slices and time instances, with reduced deviations from reference fields compared with SPINN and PirateNets.Figures 5 and 6 show reference solutions, predictions, and absolute-error maps on representative two-dimensional slices.Source: E VISUALIZATION · p. 23, Figure 5 · p. 25, Figure 6 · p. 25

F COMPUTATIONAL COST

  • 0.53 GFLOPs: ReCAP matches SINN and is below ActNet (1.64), PirateNets (2.59), and CoupledNet (4.38) on KG3D; Scale-PINN has the lowest displayed count at 0.27 GFLOPs.These operation counts do not establish wall-clock training-time savings.Source: F COMPUTATIONAL COST · p. 26, Figure 7 · p. 26
Loading 2610.09510v1…