Densely Multiplied Physics Informed Neural Networks

• PINNs can suffer from low precision or fail to obtain the correct PDE solution.The introduction identifies imbalanced residual distributions and spatial-temporal causality as possible failure sources.Source: §1 · pp. 1–2
• Prior approaches address residual imbalance through resampling or by assigning higher weights to high-residual points.The cited approaches differ in whether training points are resampled or kept fixed.Source: §1 · p. 2
• Other prior approaches address spatial-temporal causality by reformulating the loss or training on sequences of time points.The sequential approach uses each sequence's endpoint as the next sequence's initial state.Source: §1 · p. 2
• Other structural changes include exact boundary-condition imposition and trainable activation parameters.The introduction also describes ResNet-based PINNs and multiplicative interactions, including a design requiring two additional single-layer networks.Source: §1 · pp. 2–3
• Only a few researchers have explored improving PINNs by changing their neural-network structure.The original PINN simply used a fully connected neural network, although more complex structures may increase expressive power.Source: §1 · p. 2
• The proposed densely multiplied network structure improves PINN accuracy without introducing extra trainable parameters.Its effectiveness is demonstrated on benchmark PDE problems.Source: §1 · p. 3

• The PDE is stated in a general form, with initial data specified over the spatial domain and boundary data on its boundary.The initial and boundary conditions are given for times from 0 to T.Source: §2 · p. 3
• The spatial domain, its boundary, space and time coordinates, and the initial and boundary data are identified in the formulation.The formulation also defines the nonlinear operator and its parameter.Source: §2 · p. 3
• A PINN represents the latent PDE solution with a neural-network output and incorporates the governing equation, initial condition, and boundary conditions into its loss.The loss combines separate terms for the initial condition, boundary conditions, and PDE residual.Source: §2 · pp. 3–4
• The loss terms have separate weighting coefficients, which are used to address imbalance between the terms.Collocation points are distributed across the space-time domain.Source: §2 · p. 4
• Training adjusts the network parameters to minimize the loss, typically using a gradient-descent algorithm such as Adam or SGD.The optimization is expressed as minimizing the loss over the trainable parameters.Source: §2 · p. 4

• DM-PINN reuses hidden-layer outputs by multiplying each element-wise with the outputs of all subsequent hidden layers.This DenseNet-inspired structure is intended to enhance expressive ability without increasing trainable parameters.Source: §3 · pp. 4–5, Figure 1 · p. 5
• The dense multiplication may cause gradient vanishing as hidden-layer depth increases, so the method adds batch normalization after the input and adopts a ResNet structure.Each layer performs as a residual block in the adopted structure.Source: §3 · p. 5, Figure 2 · p. 5
• The method modifies dense multiplication into skip multiplication and names the resulting structure SDM-PINN.Figure 3 identifies the skip densely multiplied structure.Source: §3 · p. 5, Figure 3 · p. 5
• The reported training setup samples data points with Latin Hypercube, initializes weights with Xavier normal distribution, and uses tanh activation.Training uses one NVIDIA GeForce RTX 4090 GPU, and accuracies are averaged over five independent trials per model.Source: Figure 3 · p. 5

4. Numerical examples

• DM-PINN achieves the highest accuracy on Allan-Cahn and uses only 2/3 of the time required by the second-highest-accuracy modified MLP.The reported comparison says this efficiency is achieved without extra trainable parameters.Source: Table 1 · p. 6, Figure 4 · p. 6, 4.1.Allan-Cahn equation · p. 7
• DM-PINN obtains a minimum average L2 error of 5.29e-3 on the Helmholtz equation and has smaller absolute error at the boundaries than vanilla PINN.The comparison also reports smaller absolute error distributions in the interior domain.Source: Figure 5 · p. 7, 4.2.Helmholtz equation · p. 8, Table 2 · p. 8
• On Burgers’ equation, the proposed method and modified MLP achieve close accuracy, while the proposed method takes 15 ms less per iteration.The example adopts the skip densely multiplied structure.Source: 4.2.Helmholtz equation · pp. 8–9, Table 3 · p. 9, Figure 6 · p. 9
• Only the proposed method solves the 1D convection equation, with error over one order of magnitude smaller than all other structures.The comparison uses skip densely multiplied PINN.Source: 4.2.Helmholtz equation · pp. 9–10, Table 4 · p. 10, Figure 7 · p. 10

5. Discussion

• DM-PINN has the smallest λmax, indicating less-stiff gradient flow and fewer learning-rate restrictions, though computing Hessian eigenvalues can be costly.The authors note that an optimum learning rate should lie between 0 and 2/λmax, but they do not use that prescription in practice.Source: §5 · p. 11
• Near the loss minima, DM-PINN shows the lightest oscillations among four models, a stability pattern associated with easier training.The discussion says the L2-error curves verify this training-stability observation.Source: Figure 8 · p. 11, Figure 9 · p. 12
• Across tested learning rates, DM-PINN achieves the best outcomes among four models, with L2 error decreasing faster than modified MLP's.Table 5 covers L2 errors across models and learning rates; Figure 9 shows loss and L2-error curves.Source: §5 · p. 12, Table 5 · p. 12, Figure 9 · p. 12
• Element-wise multiplication reuses hidden-layer outputs to enhance expressive power without extra parameters; the architecture is compatible with advanced training processes.The authors report more accurate predictions with the same training resources.Source: §5 · pp. 12–13