A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks

1. Introduction

• Existing alternatives include finite-difference, coupled automatic–numerical, smoothing-kernel, first-order reformulation, and fused finite-difference approaches [2]–[6].These methods alter derivative evaluation or reformulate higher-order PDEs to reduce reliance on repeated spatial AD.Source: §1 · p. 2
• Prior spectral approaches include pseudo-spectral PINNs [7], neural spectral elements [8], SINN, Fourier feature-grid methods [10], and Fourier-filter differentiation in PhyCRNet.They use different neural representations or operator constructions to incorporate spectral or Fourier differentiation into physics-informed learning.Source: §1 · pp. 2–3
• Across five equation–framework settings, Fourier differentiation achieved mean paired speedups of 2.90×–18.52× and reduced peak allocated GPU memory by 68.7%–94.1%, with no consistent accuracy advantage.The comparison concerns one-dimensional periodic PINNs and requires a uniform structured spatial grid.Source: §1 · p. 3

2. Spatial Differentiation Procedures for Physical-Space PINNs

• Both differentiation procedures use the same physical-space neural representation and temporal differentiation; only the spatial derivatives entering the PDE residual differ.The paper frames this as a comparison of spatial automatic differentiation and Fourier spectral differentiation.Source: §2 · p. 4

2.1. Standard and Causal PINN Formulations

• Standard PINNs weight temporal residual contributions uniformly, whereas Causal PINNs use Wang et al.’s temporal causal weighting; the causal parameter is fixed at ϵ = 0.1.The initial-condition contribution is evaluated on the same spatial grid as the residual.Source: §2.1 · pp. 4–5

2.2. Spatial Automatic Differentiation

• Spatial AD obtains the required PDE derivatives by repeated, nested Jacobian–vector product differentiation, while temporal differentiation follows the same JVP-based path in both cases.Only the spatial derivative orders appearing in each PDE residual are evaluated.Source: §2.2 · p. 5

2.3. Fourier Spectral Differentiation

• Fourier differentiation samples network outputs on an endpoint-excluded uniform periodic grid, transforms them to Fourier space, and applies spectral multipliers to obtain spatial derivatives.A single transform is reused across derivative orders; the results return to physical space for the PDE residual, with gradients propagated through the transforms.Source: §2.3 · pp. 5–6

3.1. Benchmark Problems

• The benchmarks are one-dimensional periodic Allen–Cahn, KdV, and KS equations, with spatial derivative sets uxx, (ux, uxxx), and (ux, uxx, uxxxx), respectively.Standard PINNs cover Allen–Cahn and KdV; Causal PINNs cover all three equations.Source: §3.1 · pp. 6–7

3.2. Experimental Setup

• Each spatial domain uses 256 uniform periodic grid points and the same eight-layer gated network, with matched temporal sampling schedules for paired runs.The network has width 128 and 122,625 trainable parameters; periodic Fourier features mean no separate spatial boundary-condition loss is used.Source: §3.2 · p. 7
• Paired AD and Fourier runs match initialization, sampling, network configuration, optimizer settings, reference data, and training length, changing only the spatial differentiation procedure.Each setting uses three seeds and 50,000 optimization steps; causal pairs also share causal weighting.Source: §3.2 · p. 7

3.3. Reference Solutions and Evaluation

• Allen–Cahn and KdV references use a refined Fourier pseudospectral solver, while KS uses the frozen trajectory accompanying Wang et al..Accuracy is measured by relative L2 error; for KS, the final error is evaluated over the complete 26 × 256 space–time array, not only the final time slice.Source: §3.3 · p. 8
• Training time covers the 50,000-step loop, and peak allocated GPU memory is measured immediately before that loop.The timed loop includes scheduled evaluation and logging but excludes initialization, warm-up, and post-training artifact generation; memory reduction is defined relative to AD.Source: §3.3 · pp. 8–9

4. Results

• Each equation–framework setting used three paired random seeds, and aggregate values were reported as mean ± sample standard deviation.Seed 1234 supplied the solution-field visualizations; Table 1 summarizes error, speedup, and peak-memory reduction.Source: §4 · p. 9, Table 1 · p. 11

4.1. Standard PINN Results; 4.2. Causal PINN Results

• Across standard and Causal PINN experiments, neither spatial differentiation procedure showed a consistent final relative L2 error advantage.Errors remained of the same order for both procedures; paired rankings varied across random seeds.Source: §4.1 · p. 9, §4.2 · p. 11

4.1. Standard PINN Results

• Mean speedups were 3.19 ± 0.28× for standard Allen–Cahn and 7.85 ± 0.21× for standard KdV; Fourier also reduced peak GPU memory by 68.7% and 86.2%, respectively.Fourier reduced end-to-end training time for every paired seed.Source: §4.1 · p. 9, Figure 1 · p. 10
• For seed 1234, both procedures reproduced the displayed reference space–time structures for standard Allen–Cahn and KdV solutions.Figure 2 presents reference solutions, predictions, and pointwise absolute errors.Source: §4.1 · p. 9, Figure 2 · p. 10

4.2. Causal PINN Results

• Causal-PINN mean speedups were 2.90 ± 0.09×, 7.13 ± 0.16×, and 18.52 ± 2.65× for Allen–Cahn, KdV, and KS; peak-memory reductions were 68.7%, 86.2%, and 94.1%.Fourier reduced training time in all nine paired runs, and its peak memory remained near 0.285 GiB.Source: §4.2 · p. 11, Figure 3 · p. 12
• For seed 1234, both procedures reproduced the principal space–time structures of the displayed Causal-PINN Allen–Cahn, KdV, and KS solutions.Figure 4 presents the reference solutions, predictions, and pointwise absolute errors.Source: §4.2 · p. 11, Figure 4 · p. 13

5.1. Computational Interpretation

• Fourier reuses one transformed representation across derivative orders, but end-to-end gains also depend on how much training cost comes from derivative evaluation.Source: §5.1 · p. 14
• Across Causal PINN benchmarks, derivative count and maximum order rise alongside speedups and AD memory use, so the cross-equation trend is not a controlled scaling law.Source: §5.1 · p. 14

5.2. Relation to Existing Derivative-Evaluation Methods

• CAN-PINN and SK-PINN use numerical differentiation, SINN predicts spectral coefficients, and Fourier-filter PhyCRNet provides a precedent for spectral differentiation of network outputs [4].Source: §5.2 · pp. 14–15
• Fourier differentiation reduces end-to-end training time and peak allocated memory, while final relative L2 accuracy remains seed-dependent with no consistent advantage for either procedure.Source: §5.2 · p. 15

5.3. Scope and Limitations

• The comparison covers one-dimensional PDEs on uniform periodic grids; its speedups are hardware- and configuration-specific, and Causal PINNs use fixed ϵ = 0.1 without annealing.Source: §5.3 · p. 15
• Varying derivative workload within one fixed PDE and testing higher-dimensional periodic PDEs are proposed to examine scaling and whether runtime and memory gains persist.Source: §5.3 · p. 15

• The paired comparison holds other training components fixed and changes only spatial differentiation between automatic differentiation and Fourier spectral differentiation.Source: §6 · p. 15
• Across five equation–framework settings, Fourier spectral differentiation yields mean paired speedups of 2.90×–18.52× and reduces peak allocated GPU memory by 68.7%–94.1%, with no consistent accuracy advantage.Source: §6 · p. 16