Physics-Informed Stochastic Configuration Machine: A Backpropagation-Free Neural Network with Fast Training for Nonlinear Differential Equations

Yuehao Song; Zhong Chen; Lihui Cen; Liang Wu; Kai Zhang

Source-linked AI summary

Physics-Informed Stochastic Configuration Machine: A Backpropagation-Free Neural Network with Fast Training for Nonlinear Differential Equations

Yuehao Song, Zhong Chen, Lihui Cen, Liang Wu, Kai Zhang

arXiv:2608.26549v1Aug 27, 2026math.NAcs.AIcs.LGeess.SY

TL;DR

PINNs are limited by nonconvex gradient optimization and the computational overhead of backpropagation and automatic differentiation. PI-SCM replaces this process with analytically linearized physics-informed updates and progressive least-squares solvers for forward and inverse problems. Across reported benchmarks, the framework maintains competitive accuracy while substantially reducing training time.

Problem
PINNs face nonconvex optimization challenges and substantial computational overhead from backpropagation and automatic differentiation when solving nonlinear differential equations.
Method
PI-SCM analytically evaluates local operator Jacobians, linearizes the physical residual, and solves successive updates with generalized least-squares methods across localized, windowed, and global algorithms.
Results
PI-SC-III consistently achieved competitive or lower errors than deep gradient-based PINNs while reducing training time by one to two orders of magnitude across the reported tasks.
Takeaways & Limitations
PI-SCM supports forward approximation, sparse-observation assimilation, state reconstruction, and physical-parameter identification within a backpropagation-free framework.
Takeaways & Limitations
Future work includes noisy parameter-identification settings, adaptive candidate-range and update-scope selection, and extending the shallow constructive architecture while retaining analytical training and numerical stability.

Abstract

from arXiv · show

While Physics-Informed Neural Networks (PINNs) have emerged as a transformative paradigm for solving complex differential equations, their reliance on backpropagation-based gradient descent and automatic differentiation (AD) imposes significant computational bottlenecks and severe non-convex optimization challenges. To overcome these fundamental limitations, we propose the Physics-Informed Stochastic Configuration Machine (PI-SCM), a novel backpropagation-free framework for both forward and inverse problems in differential equations. The core mathematical contribution lies in the analytical evaluation of local Jacobians for nonlinear differential operators, which facilitates a linearized representation of the physical loss and projects it into a unified, linearized algebraic subspace. This reformulation allows for the explicit determination of optimal network weights via a sequence of generalized linear least squares solvers, effectively bypassing the iterative traps of traditional nonlinear optimizers. We develop a progressive algorithmic suite comprising localized construction (PI-SC-I), sliding-window updating (PI-SC-II), and global updating (PI-SC-III), and rigorously establish their universal approximation properties. Extensive experiments demonstrate that PI-SCM achieves high-fidelity predictive accuracy and robust parameter identification while accelerating the training process by orders of magnitude compared to standard PINNs. Our work provides a highly efficient and scalable foundation for next-generation, real-time Scientific Machine Learning applications.

I. INTRODUCTION

The paper identifies nonconvex optimization and automatic-differentiation costs as major PINN bottlenecks, then proposes PI-SCM as a backpropagation-free alternative using analytically solved linear updates for physics-informed learning.

Backpropagation and automatic differentiation impose substantial memory and computational costs, slowing convergence and training.
Standard PINNs face nonconvex optimization landscapes that can trap gradient-based optimizers in sub-optimal local minima.
PI-SCM addresses these bottlenecks with a backpropagation-free framework for unsupervised and semi-supervised physics-informed learning.
Analytical local Jacobians linearize nonlinear physical operators and combine physical residuals with observational constraints into a unified algebraic system.
Pseudoinverse-based linear least-squares solvers determine output coefficients without automatic differentiation while retaining governing equations as explicit constraints.
PI-SC-I, PI-SC-II, and PI-SC-III provide localized, sliding-window, and global updates with different computational-cost and approximation-accuracy trade-offs.
The framework extends to inverse problems by jointly updating physical parameters and network approximations for state reconstruction and parameter identification.

C. Inverse Problem

The inverse problem jointly identifies network parameters and unknown physical parameters while fitting empirical measurements and satisfying parameterized physical laws. PI-SCM linearizes nonlinear residuals using local Jacobians, then solves progressively updated least-squares systems with damping, stability controls, and supervisory selection.

The inverse task jointly determines network parameters {w_k, b_k, β_k} and the unknown physical parameter λ while fitting measurements and enforcing parameterized physical laws.
Nonlinear dependence on the generalized differential state makes direct linear least-squares determination of newly added output weights impossible.
Analytical first-order Taylor expansion with local Jacobians converts the nonlinear physical residual into a linear mapping with respect to the new output weights.The Jacobians are evaluated with respect to derivative components up to order q.
PI-SC-I: PI-SC-I minimizes physical, boundary, and data residuals in a unified weighted linear least-squares system, with pseudo-inverse solutions for output-weight updates.The objective balances boundary conditions and observational data against the physical domain using penalty parameters.
Damping and TSVD stabilize updates, while supervisory node selection is used to enforce monotonic residual descent.The damping factor α ∈ (0, 1] controls the Taylor-based step, and TSVD filters unstable singular components.
PI-SC-II and PI-SC-III: Global updating improves accuracy but grows rapidly in computational complexity, whereas sliding-window updating bounds the design-matrix width independently of total network size.The sliding-window strategy reduces computational burden while preserving accelerated convergence benefits.

B. Inverse Problem

For inverse problems, PI-SCM jointly linearizes state and parameter changes, solves augmented least-squares systems, and applies line search to select damped updates. Local-only parameter updates can restrict identification accuracy, motivating global joint updating.

The inverse formulation uses F(x, D^(q)u; λ)=0 and jointly estimates the local output-weight increment β_L and physical parameter λ.
PI-SC-I: A simultaneous first-order Taylor expansion with respect to D^(q)u and λ produces a linearized physical equation for the current update.The expansion is taken around the previous state and parameter configuration.
The augmented design matrix concatenates the state Jacobian and parameter Jacobian, while boundary and data matrices are zero-padded for joint identification.
TSVD solves the augmented linear least-squares problem to obtain a candidate joint solution, followed by damping and supervisory filtering.The adopted configuration is selected after candidate configurations are extracted and filtered for monotonic residual descent.
Updating λ only through localized node addition restricts identification accuracy, motivating global joint identification of output weights and physical parameters.
PI-SC-II and PI-SC-III: Global and windowed inverse updates augment their respective state matrices with the local parameter Jacobian and solve corresponding joint least-squares systems.The windowed formulation uses a predefined window size W to reduce full-batch inversion cost.
A discrete line search evaluates true residual norms across candidate scaling factors and adopts the configuration with the minimizing factor.

C. Universal Approximation Property

PI-SCM’s universal approximation analysis establishes residual convergence for its localized, sliding-window, and global update schemes under regularity, boundedness, and residual-completeness assumptions.

Universal approximation results: PI-SC-I, PI-SC-II, and PI-SC-III are shown to guarantee asymptotic convergence of nonlinear residuals to zero.The analysis builds from PI-SC-I as a convergent baseline and extends the guarantees to the global and sliding-window schemes.
Assumptions: The proofs assume a Cq activation, a twice continuously Fréchet-differentiable differential operator, bounded generalized residuals, and a residual-complete hidden-node family.Residual completeness requires a nonorthogonal linearized direction for every nonzero discrete residual.
PI-SC-I construction: Each accepted hidden node uses an analytically computed output-weight update based on the weighted design matrix ML and target vector vL.The pseudoinverse-based construction is combined with a discrete search over admissible nodes and damping factors.
Residual descent: For sufficiently small positive damping factors, the bounded second-order Taylor remainder is dominated by first-order descent, producing strict residual reduction.The analysis establishes a positive upper bound on the damping factor for which the true residual decreases.
Algorithmic extensions: The sliding-window algorithm inherits the same convergence conclusion through an identical discrete line-search mechanism, with its proof stated directly after PI-SC-III.PI-SC-III’s proof is established first, followed by the corresponding PI-SC-II theorem.

D. Hyperparameter Configuration Strategy

PI-SCM configures randomized hidden parameters through a progressively scaled candidate range, balancing matrix conditioning against nonlinear expressivity and convergence.

Conditioning–expressivity trade-off: Large τ expands the basis-function space but causes derivative blocks of order |γ| to scale as O(τ^|γ|), worsening matrix conditioning as q increases.Higher-order derivative terms can dominate lower-order state-dependent terms, creating column-wise magnitude disparity.
Numerical stability: Highly ill-conditioned matrices can force TSVD to discard excessive singular components, destroying physical information embedded in the Jacobian operators.TSVD mitigates ill-conditioning but cannot fully preserve information when the input matrix is severely ill-conditioned.
Conditioning–expressivity trade-off: Restricting τ to small values improves matrix conditioning but reduces expressive capacity, potentially requiring many hidden nodes and slowing convergence on complex problems.The resulting architectures can exhibit structural redundancy.
Adaptive candidate pool: The candidate pool uses the scaled sequence Υ = a × {1, 2, 4, 8, 16, 32, 64}, where a > 0 is tailored to the governing differential equations.The scaling factor equilibrates numerical magnitudes among state-dependent and derivative Jacobian terms.
Adaptive candidate pool: PI-SCM tests candidate ranges in ascending order and escalates only when the current range fails to produce a valid residual descent.This ordering prioritizes narrower, more stable configuration spaces before broader exploratory ranges.

IV. EXPERIMENTAL RESULTS

The experiments evaluate PI-SCM across forward state approximation, inverse state-and-parameter reconstruction, and hidden-node activation studies.

Forward problems: The evaluation covers forward problems for the Van der Pol oscillator, two-dimensional Helmholtz equation, and Allen–Cahn equation.These tasks assess state approximation across nonlinear dynamical and differential-equation settings.
Inverse problems: Inverse problems assess joint state reconstruction and parameter identification on the Van der Pol and Helmholtz systems.The inverse evaluation tests both recovered states and physical parameters.
Activation study: An activation-function study isolates the effect of the hidden-node basis.This separates activation choice from the broader forward and inverse problem evaluations.

1) Van der Pol oscillator:

The Van der Pol oscillator is used as a nonlinear, non-conservative test system, with μ prescribed for the forward task and identified in the inverse task.

Problem definition: The Van der Pol oscillator is a nonlinear, non-conservative dynamical system with applications in circuit modeling, seismology, biological neuron modeling, and optimal control.The paper uses it as one of the forward and inverse benchmark systems.
Problem definition: The controlled second-order oscillator uses μ = 1 in the forward task, whereas μ is identified in the inverse task.The initial conditions are u(0, U) = −0.25 and ut(0, U) = −2.0.

2) Two-dimensional Helmholtz equation:

The two-dimensional Helmholtz benchmark models time-harmonic wave propagation with homogeneous Dirichlet boundaries and a manufactured source. The forward task prescribes k2 = 1, whereas the inverse task identifies k2.

The Helmholtz equation is used as a canonical elliptic model of time-harmonic wave propagation.
Homogeneous Dirichlet conditions impose zero values on all four domain boundaries.The conditions are u(−1, y) = u(1, y) = u(x, −1) = u(x, 1) = 0.
The benchmark uses a manufactured source and an analytical solution for evaluation.
k2 = 1 is prescribed in the forward task, while k2 is identified in the inverse task.

3) Allen–Cahn equation:

The Allen–Cahn benchmark represents phase separation with a stiff reaction term and sharp moving interfaces. A small diffusion coefficient creates scale disparity, addressed by coordinate stretching with ωn = 100.

The Allen–Cahn equation models phase separation and combines a stiff reaction term with sharp moving interfaces.
ϵ = 10^-4, with initial condition u(x, 0) = x^2 cos(πx) and periodic state and derivative boundary constraints.
Coordinate stretching defines X = ωn x with ωn = 1/√ϵ = 100 to address the scale disparity caused by small diffusion.
For n = 1, the stretching balances state- and derivative-dependent Jacobian contributions and improves ML conditioning without changing the physical solution.
State accuracy is measured by RMSE over Ntest test inputs, using predicted and reference states.The implementation uses a workstation with an NVIDIA RTX 4060 Ti GPU, Intel Core i5-14600KF CPU, and 16 GB of RAM.

B. Forward Problems: Experimental Setup

The forward study evaluates PI-SCM on Van der Pol, Helmholtz, and Allen–Cahn problems using unsupervised and semi-supervised settings with matched PINN baselines. Across the benchmarks, broader correction scopes improve accuracy, while PI-SC-III retains a substantial speed advantage.

Van der Pol: The Van der Pol experiment uses 1000 interior points and 100 initial points, plus 100 Runge–Kutta observations in the semi-supervised setting.Penalty weights are ωbc = 1 and ωdata = 1.
Helmholtz: The Helmholtz setup uses 2000 interior collocation points and 1000 boundary points, with shallow and deep PINNs plus PIELM as baselines.PIELM uses the same 1 × 400 architecture as PI-SCM.
Allen–Cahn: The Allen–Cahn experiment sets Lmax = 500 and compares PI-SCM with shallow and deep VS-PINN baselines.
Baseline protocol: PINN baselines use the same collocation, initial or boundary, and observational points as PI-SCM and train with Adam for 5,000 epochs.The initial learning rate is 10^-2 for Van der Pol and Helmholtz and 10^-3 for Allen–Cahn.
Evaluation: Predictive accuracy is evaluated on 10,000 random test coordinates, with tables reporting training time and RMSE under both training paradigms.Values are reported as mean ± standard deviation.
Forward results: PI-SC-I has the lowest construction cost but may underfit difficult structures, while PI-SC-II improves accuracy and PI-SC-III consistently achieves the lowest errors.The hierarchy is reported across the three forward benchmarks.
Forward results: PI-SC-III reduces Van der Pol RMSE from 5.49 × 10^-3 for deep PINN to 1.93 × 10^-3 and training time from 40.0 s to 1.44 s.
Forward results: On semi-supervised Allen–Cahn, PI-SC-III reaches RMSE 1.57 × 10^-3, approximately 49× lower error and 19× faster than deep VS-PINN.

F. Inverse Problems: Results

Across inverse benchmarks, PI-SC-III delivers the strongest overall recovery, combining accurate parameter identification and state reconstruction with substantially lower training time. The activation study further identifies sine as the most effective basis for the oscillatory benchmark, while the conclusion notes remaining scalability and robustness questions.

Inverse results: PI-SC-III provides the most accurate overall recovery in the inverse problems.PI-SC-I is less effective, while PI-SC-II improves reconstruction through windowed correction.
Inverse results: PI-SC-III matches or exceeds the deep PINN in parameter-identification accuracy across both inverse benchmarks.It also improves state reconstruction and requires substantially less training time.
Activation-function study: Sine activation achieves the highest predictive accuracy and fastest residual convergence among the tested activation functions.It reduces the nonlinear residual more rapidly and reaches the lowest residual level.
Evaluation scope: The empirical study covers inverse identification on the Van der Pol and Helmholtz systems alongside forward benchmarks and activation ablation.The tables report parameter-identification error, state RMSE, and training time.
Conclusion: The reported conclusion identifies PI-SC-III as the strongest accuracy–efficiency trade-off among the proposed variants.Across tasks, it achieves competitive or lower errors than deep gradient-based PINNs while reducing training time by one to two orders of magnitude.
Limitations and future work: Future work must examine higher-dimensional geometries, more severe multiscale stiffness, noisy parameter identification, and adaptive configuration choices.The paper also calls for extending the shallow constructive architecture while retaining analytical training and numerical stability.