Source-linked AI summary
PINNMorph: Evolving Online Adaptation Policies for Physics-Informed Neural Networks
Xu Yang, Mingyang Yu, Jun Zhang, Keqian Li, Jing Xu
TL;DR
PINNs can face changing training bottlenecks, while architectures and major training mechanisms are often fixed before training. PINNMorph adapts the current PINN and its intervention policies online using LLM-guided, state-conditioned actions and execution feedback. Across 13 PDE benchmarks, it achieves the lowest mean MSE among the evaluated methods.
Problem
PINN training behavior and bottlenecks can change during optimization, while architecture and major training mechanisms are typically determined beforehand.
Source: §1 · pp. 1–2Method
PINNMorph uses an LLM-guided population of state-conditioned policies to select interventions for the PINN under training and evolves those policies using execution feedback.
Source: §1 · p. 2Results
PINNMorph achieves the lowest mean MSE on all 13 PDE benchmarks against SA-PINN, ConFIG, RoPINN, HARMONIC, and PINNsAgent.
Source: §1 · p. 2, §4.2 · p. 6Takeaways & Limitations
Abstract
from arXiv · showhide
Physics-informed neural networks (PINNs) provide a learning-based framework for solving partial differential equations (PDEs), yet their training behavior can change substantially throughout optimization. Residual distributions, gradient interactions, regional learning difficulty, and model-capacity requirements may evolve over time, while the network architecture and major training mechanisms are typically determined before training. We propose PINNMorph, an online PINN adaptation framework based on large language model (LLM)-guided policy evolution. PINNMorph maintains a population of state-conditioned adaptation policies that map execution diagnostics to controlled interventions over topology modification, additive representation augmentation, objective balancing, gradient handling, adaptive sampling, and optimizer-phase control. At each intervention opportunity, candidate programs are instantiated from the current policy population, selected according to the observed training state, and applied directly to the PINN under training. The resulting model inherits its existing parameters and training state and continues optimization along the same trajectory. Execution outcomes are subsequently used to evaluate interventions and evolve the policy population. Unlike pre-training architecture search or fixed adaptation rules, PINNMorph jointly adapts the current PINN and the policies governing its interventions using feedback from actual training. Experiments on 13 PDE benchmarks show that PINNMorph achieves lower solution errors than SA-PINN, ConFIG, RoPINN, HARMONIC, and PINNsAgent across all evaluated problems. Ablation studies further examine the effects of online adaptation, state-conditioned intervention selection, and execution-feedback-driven policy evolution.
1 Introduction
PINNMorph addresses the need to adapt PINNs as training bottlenecks change, rather than fixing adaptation rules or configurations in advance. It applies state-conditioned interventions to the current model and uses execution feedback to evolve the policies that generate them.
1 Introduction
- PINN training bottlenecks can shift from physical-objective imbalance to gradient conflicts, localized residual concentration, sampling needs, or limited representational capacity.Their effects also vary across PDEs and training regimes.Source: §1 · p. 1
- Existing adaptive methods typically fix the intervention family, while automated PINN design searches configurations before training or across independent runs.Neither directly selects among intervention families within the training trajectory of the same PINN.Source: §1 · p. 1
- PINNMorph uses an LLM-guided population of state-conditioned policies to turn training diagnostics into interventions across six model and training dimensions.The intervention families cover topology, representation, objectives, gradients, sampling, and optimizer control; selected actions apply directly to the PINN under optimization while preserving learned parameters and transferable training states whenever possible.Source: §1 · p. 2
- PINNMorph compares each intervention with an estimated no-intervention continuation and feeds execution outcomes back into policy evolution for later decisions.This addresses the fact that continued training may improve the PINN even without an intervention.Source: §1 · p. 2
- PINNMorph achieves the lowest mean MSE on all 13 PDE benchmarks against SA-PINN, ConFIG, RoPINN, HARMONIC, and PINNsAgent, with a Friedman average rank of 1.00.Ablations examine online adaptation, state-conditioned action selection, and execution-feedback-driven policy evolution.Source: §1 · p. 2
2 Related Work
Prior PINN work adapts training mechanisms or searches for configurations, while PINNMorph selects intervention families within the evolving training trajectory of one model. It also applies execution-feedback-driven evolution to state-conditioned adaptation policies.
2.1 Physics-Informed Neural Networks and Adaptive Training
- Training-time PINN methods adapt losses, sampling, regions, or gradient directions, but primarily operate within a predefined intervention family.PINNMorph instead selects among multiple families according to the current training state.Source: §2.1 · pp. 2–3
2.2 Automated PINN Design
- Automated PINN design methods generate and evaluate candidate configurations as separate models or runs, whereas PINNMorph modifies the current model without restarting optimization.Its policies observe the evolving state of that PINN within one ongoing training process.Source: §2.2 · p. 3
2.3 LLM-Driven Algorithm Evolution
- PINNMorph evolves state-conditioned policies mapping the current PINN training state to online adaptation actions, rather than evolving standalone algorithms or independently evaluated models.Policy evolution and PINN optimization interact within the same continuous training trajectory.Source: §2.3 · p. 3
3 Method
PINNMorph formulates PDE solving with PINNs and adapts a model during its ongoing optimization. It diagnoses training state, selects among policy-generated interventions, and uses audited execution feedback to evolve the policy population.
3.1 Problem Formulation
- The problem formulation represents a PDE through governing-equation, boundary-condition, and, when required, initial-condition constraints on the unknown field.The PINN approximates the solution with a neural network and minimizes corresponding loss terms.Source: §3.1 · p. 3
3.2 Method Overview
- PINNMorph maintains the PINN under optimization and forms a training state from the current model and its architecture, diagnostics, history, and remaining budget.A population of five state-conditioned policies generates interventions, which are executed on the current PINN and inform subsequent policy updates.Source: §3.2 · p. 4
3.3 Training-State Diagnosis
- A fixed decision collocation set supplies physics residual, residual-concentration, constraint, gradient, and stagnation diagnostics, while a separate audit set evaluates intervention effects.Neither held-out set participates in gradient updates; separating them reduces direct coupling between action selection and feedback estimation.Source: §3.3 · p. 4
3.4 Adaptation Policies
- The policy population maps the current training state to adaptation actions and is expanded through refinement, physics-guided, architecture-guided, synthesis, and novelty modes.These modes improve effective policies, respond to physical errors, adjust representational capacity, combine mechanisms, and explore new directions.Source: §3.4 · p. 5
3.5 Online Adaptation
- At each diagnosis point, PINNMorph scores candidates from six intervention families against the current bottleneck and prior outcomes, and may continue without intervention.At most one selected action is applied directly to the current PINN; structural changes preserve learned parameters and transferable optimizer states whenever possible.Source: §3.5 · p. 5
3.6 Policy Evolution
- PINNMorph estimates an intervention’s incremental effect by comparing the post-intervention trajectory with a no-intervention continuation and uses the resulting feedback to evolve policies.Beneficial interventions provide positive experience for later generations, while ineffective or narrowly applicable actions constrain future updates.Source: §3.6 · p. 5
4 Experiments
Experiments compare PINNMorph with five baselines on 13 PDE benchmarks, then examine component ablations, online adaptations, and computational cost. PINNMorph has the lowest mean MSE on every benchmark, while ablations and process analyses examine the contributions and behavior of its adaptation mechanisms.
4.1 Experimental Setup
- On 13 diverse PDEs, the experiments compare PINNMorph with five baselines using mean squared error over five random seeds and a fixed 20,000-step training budget.The baselines are SA-PINN, ConFIG, RoPINN, HARMONIC, and PINNsAgent.Source: §4 · p. 5, §4.1 · p. 6
4.2 Main Results
- PINNMorph achieves the lowest mean MSE on all 13 PDE benchmarks, with a Friedman average rank of 1.00; representative MSE reductions versus each PDE’s best baseline range from 61.0% to 67.6%.The advantage also persists on Burgers 1D, Heat 2D MS, and Poisson 5D, where margins are smaller.Source: Table 1 · p. 6, §4.2 · p. 6
4.3 Component Ablation
- Removing online adaptation raises the median error ratio to 12.65×; Random Selection reaches 10.77× and loses to No Morph on 8 of 13 tasks, while Random Evolution reaches 10.30×.The full method improves performance on all 13 PDEs and reduces median error by approximately one order of magnitude relative to each ablation.Source: §4.3 · pp. 7–8
4.4 Analysis of the Online Adaptation Process
- Training loss and Decision and Audit physics scores evolve non-uniformly, while different diagnosed bottlenecks favor different responses, including representation changes, adaptive sampling, or structural specialization.The Decision score reflects the state used for online decisions; the Audit score is computed on a fixed audit set, and lower residual-based scores indicate better equation satisfaction.Source: §4.4 · p. 8, Figure 2 · p. 9
4.5 Computational Cost
- PINNMorph incurs approximately 4.6% additional wall-clock time versus No Morph while achieving substantially lower solution error, under the same optimization budget.The additional cost mainly comes from periodic diagnosis, LLM inference, intervention execution, and policy updates.Source: §4.5 · p. 8, Table 5 · p. 8
5 Conclusion
PINNMorph integrates training-state diagnosis, state-conditioned interventions, and execution-feedback-driven policy evolution into a single PINN training trajectory. Across 13 PDE benchmarks, it achieves the lowest mean MSE, with ablations supporting the value of its adaptation mechanisms.
5 Conclusion
- Across 13 PDE benchmarks, PINNMorph achieves the lowest mean MSE, while ablations confirm the value of online adaptation, state-conditioned selection, and feedback-driven policy evolution.With the same 20,000-step budget, it incurs about 4.6% wall-clock overhead over No Morph.Source: §5 · p. 9
- PINNMorph selects among heterogeneous intervention families according to the evolving training state and applies each selected action directly to the current model.It preserves learned parameters and transferable training state whenever possible, then uses execution feedback to improve subsequent adaptation policies.Source: §5 · p. 9
- The framework combines training-state diagnosis, state-conditioned intervention, and execution-feedback-driven policy evolution within one training trajectory.Source: §5 · p. 9
A PDE Benchmark Specifications
The benchmark specifications cover Burgers, Gray–Scott, heat, and Kuramoto–Sivashinsky problems, with the supplied passages identifying selected initial conditions, domains, parameters, and boundary conditions.
A.1 Burgers 1D
- The Burgers 1D specification gives u(x, 0) = −sin(πx) and zero boundary values at x = −1 and x = 1.Source: §A.1 · p. 9
A.2 Burgers 2D
- The Burgers 2D specification prescribes an initial field and periodic value conditions on opposite spatial sides.Source: §A.2 · p. 10
A.3 Gray–Scott 2D
- The Gray–Scott 2D specification sets (εu, εv) = (10^-5, 5 × 10^-6), b = 0.04, and d = 0.1, and introduces initial conditions.Source: §A.3 · p. 10
A.4 Heat 2D CG
- The Heat 2D CG domain is [−8, 8] × [−12, 12] with eleven radius-1 disks and six radius-0.4 disks removed; the boundary coefficient c varies by boundary.The passage specifies c = 5 on radius-1 holes, c = 1 on radius-0.4 holes, and c = 0.1 on the outer boundary.Source: §A.4 · p. 10
A.5 Heat 2D MS
- The Heat 2D MS specification gives an anisotropic diffusion equation with κx = 1/(500π)^2 and κy = 1, plus a sinusoidal initial field and zero boundary values.Source: §A.5 · p. 10
A.6 Heat 2D VC
- The Heat 2D VC specification states that a(x, y) is prescribed and sets the initial field and boundary values to zero.Source: §A.6 · p. 10
A.7 KS 1D
- The KS 1D specification gives the equation ut + αuux + βuxx + γuxxxx = 0 and initial field u(x, 0) = cos(x)[1 + sin(x)].Source: §A.7 · p. 11
A.8 NS 2D CG
These benchmark specifications describe a two-dimensional Navier–Stokes problem and three Poisson problems, including their domains, equations, and selected boundary or coefficient conditions.
A.8 NS 2D CG
- The NS 2D CG domain is an L-shaped region, with viscosity ν = 0.01 and specified inlet, wall, and outlet conditions.The inlet velocity is (4y(1 − y), 0), solid-wall velocity is zero, and outlet pressure is zero.Source: §A.8 · p. 11
A.9 Poisson 2D C
- The Poisson 2D C specification sets Δu = 0, with u = 1 on the outer boundary and u = 0 on hole boundaries.Source: §A.9 · p. 11
A.10 Poisson 2D CG
- The Poisson 2D CG domain is [−1, 1]^2 with four circular holes, and the specified boundary values are 0.2 outside and 1 on the holes.The passage lists the hole centers and radii.Source: §A.10 · p. 11
A.11 Poisson 2D MA
- The Poisson 2D MA specification uses a(x, y)Δu + f(x, y) = 0 with a Robin boundary condition and prescribed coefficients on a 5 × 5 cell grid.The passage identifies the cell-local coordinates and coefficients aij and Fij,mn as prescribed.Source: §A.11 · p. 11
A.12 Poisson 3D CG
The Poisson 3D CG benchmark specifies a three-dimensional domain with spherical holes, a variable-coefficient equation, and Neumann boundary conditions.
A.12 Poisson 3D CG
- The Poisson 3D CG domain is [0, 1]^3 with four spherical holes and equation −µ(z)Δu + k(z)^2u = f(x, y, z), with ∂nu = 0 on all boundaries.The forcing term and coefficient functions are specified in the passage.Source: §A.12 · pp. 11–12
B Implementation and Training Details
The implementation uses a fixed base PINN and optimizer setup, with scheduled online interventions and separate held-out sets for diagnosis and auditing. Model changes preserve compatible training state, and language-model configuration is held consistent across PDE problems.
B.1 Base PINN and Optimization
- The base PINN is a four-hidden-layer, width-128 tanh MLP; training uses Adam at 10^-3 followed by L-BFGS at 1.0 without a scheduler.Source: §B.1 · p. 12
B.2 Online Adaptation Schedule
- Interventions are diagnosed every 200 steps, enabled from step 400 through step 14,000, and followed by 10 activation-only and 50 new-parameter optimization steps after structural changes.Feedback is evaluated at +100, +500, and +1000 steps; another intervention waits until persistent feedback is available.Source: §B.2 · p. 12
B.3 Decision and Audit Sets
- Independent, fixed decision and audit collocation sets support diagnosis/action selection and outcome/checkpoint evaluation, respectively, without contributing gradients or using reference solutions.The decision set has up to 128 points per component and 4,096 total; the audit set has up to 2,048 per component and 32,768 total.Source: §B.3 · p. 12
B.4 State Transfer
- Structural changes operate on the existing PINN, retaining unchanged parameters and compatible optimizer states while adding initialized parameters to the active optimizer.Source: §B.4 · p. 12
B.5 LLM Configuration
- PINNMorph uses DeepSeek-V4-Flash for policy generation/evolution and Qwen3.8-27B for candidate scoring, with the same LLM configuration across PDE problems.Deterministic code validates and executes selected interventions.Source: §B.5 · p. 13
C.1 Action space
The action space provides registered, compatible interventions and a declarative policy format, then applies validated changes to the live model while retaining compatible state. Validation and rollback rules constrain execution and allow a no-op when intervention is unavailable.
C.1 Action space
- The action space combines 16 high-level actions, including KEEP, with 23 primitive mechanisms organized into six descriptive families, while compatibility checks determine valid action–mechanism–target combinations.Source: §C.1 · p. 13
- Macro-actions span structural changes, objective and gradient interventions, sampling adaptation, and optimizer-phase switching; operations in a multi-operation program must be compatible and execute as one ordered transaction.Examples include local experts for LOCAL SPECIALIZATION and component-gradient balancing or residual-region reweighting for REGION AWARE REBALANCING.Source: §C.1 · p. 14
C.2 Structured policy output and execution semantics
- Policies are declarative requests rather than executable code, specifying an action, mechanism, target, strength, and optional ordered operations for executor validation and translation.Historical action-only policies can use a fixed state-dependent primitive choice when their operation list is empty.Source: §C.2 · pp. 14–15
C.3 Parameter and optimizer-state transfer
- Interventions modify the incumbent model: unchanged parameters and compatible optimizer moments are retained, new parameters are initialized, and supported function-preserving changes undergo output, derivative, and residual checks.Failed execution or required checks restore the pre-action model and runtime snapshot; reduction or reinitialization operations are not asserted to preserve the function exactly.Source: §C.3 · p. 15
C.4 Validation, rejection, and no-op decisions
- The parser and executor enforce registered fields, compatibility, budgets, and enabled capabilities; incompatible operations are rejected or rolled back, while KEEP leaves the training trajectory unchanged.Source: §C.4 · p. 16
D Counterfactual Continuation and Execution Feedback
The feedback procedure estimates a no-intervention continuation from recent held-out observations, then evaluates intervention outcomes against that forecast at separate horizons. Numerical policy credit is gated by forecast eligibility and cross-set agreement, and the forecasts remain estimated counterfactuals rather than observed controls.
D Counterfactual Continuation and Execution Feedback
- At intervention time, a frozen log-linear forecast is fitted to each held-out physical loss component using the last seven pre-intervention observations, with analogous fits for total score and stable spatial regions.The forecast is not refitted using post-intervention measurements.Source: D Counterfactual Continuation and… · p. 16
- Forecast credit requires eligible fits with sufficient confidence, no clipped predictions, and matching region sets; fewer than seven observations or other specified data inconsistencies invalidate the forecast.Forecast ineligibility withholds numerical policy credit but does not by itself prevent a safe intervention.Source: D Counterfactual Continuation and… · p. 16
- A set accepts an outcome when aggregate utility is at least 0.01 and no component or region degrades by more than 0.10 relative to forecast; both sets must accept for positive credit.When available, the least-improved region enters the aggregate as an additional group.Source: D Counterfactual Continuation and… · p. 16
- The +100, +500, and +1000 observations are separate feedback events, not combined across horizons; eligible outcomes update policy credit using the smaller set utility, while conflicting set directions withhold numerical credit.The forecasts reduce contamination from recent trajectory trends but remain estimated counterfactuals, not observed causal controls.Source: D Counterfactual Continuation and… · p. 17