Source-linked AI summary
A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks
Suchuan Dong, Naxian Ni
TL;DR
DNN solvers need a way to impose periodic boundary conditions without approximate penalty enforcement. The paper introduces periodic layers that compose arbitrary DNN functions with adjustable periodic functions, and experiments show exact periodicity to machine accuracy. The method’s approximation power remains theoretically unresolved.
Problem
Penalty methods impose periodic boundary conditions only approximately, and enforcing C^k conditions becomes difficult for moderate or large k, especially for C^∞ conditions.
Method
The method places a C^∞ or C^k periodic layer as the DNN’s second layer, composing an arbitrary DNN function with independent adjustable periodic functions.
Results
Extensive ordinary- and partial-differential-equation experiments demonstrate that the method enforces C^∞ and C^k periodic boundary conditions exactly, to machine accuracy.
Takeaways & Limitations
The resulting DNN is constructed to automatically satisfy the selected periodic conditions for the function and the prescribed derivatives.
Takeaways & Limitations
Whether the periodic DNNs can represent arbitrary periodic functions of a given regularity to arbitrary accuracy remains an open question.
Abstract
from arXiv · showhide
We present a simple and effective method for representing periodic functions and enforcing exactly the periodic boundary conditions for solving differential equations with deep neural networks (DNN). The method stems from some simple properties about function compositions involving periodic functions. It essentially composes a DNN-represented arbitrary function with a set of independent periodic functions with adjustable (training) parameters. We distinguish two types of periodic conditions: those imposing the periodicity requirement on the function and all its derivatives (to infinite order), and those imposing periodicity on the function and its derivatives up to a finite order $k$ ($k\geqslant 0$). The former will be referred to as $C^{\infty}$ periodic conditions, and the latter $C^{k}$ periodic conditions. We define operations that constitute a $C^{\infty}$ periodic layer and a $C^k$ periodic layer (for any $k\geqslant 0$). A deep neural network with a $C^{\infty}$ (or $C^k$) periodic layer incorporated as the second layer automatically and exactly satisfies the $C^{\infty}$ (or $C^k$) periodic conditions. We present extensive numerical experiments on ordinary and partial differential equations with $C^{\infty}$ and $C^k$ periodic boundary conditions to verify and demonstrate that the proposed method indeed enforces exactly, to the machine accuracy, the periodicity for the DNN solution and its derivatives.
1 Introduction
The paper addresses the difficulty of enforcing periodic boundary conditions exactly in DNN-based differential-equation solvers. It introduces periodic layers that guarantee either infinite-order or finite-order periodicity by construction.
- Motivation: Penalty methods enforce periodic boundary conditions only approximately and require penalty-coefficient tuning that can affect training and convergence.For moderate or large k, enforcing C^k conditions with penalties becomes difficult, while C^∞ conditions may be impractical.
- Motivation: Periodic conditions may require matching the solution and all derivatives to infinite order, or matching the function and derivatives through a finite order k.Fourier-based methods naturally seek smooth periodic functions, whereas non-Fourier methods must impose periodicity explicitly at cell boundaries.
- Method: The proposed method composes a DNN-represented arbitrary function with independent periodic functions having adjustable parameters.This construction leverages DNN universal approximation while encoding periodicity through a dedicated network layer.
- Method: A periodic layer placed as the DNN’s second layer automatically and exactly enforces C^∞ or C^k periodic boundary conditions for a prescribed period or order.The C^∞ layer uses sinusoidal functions, affine mappings, and nonlinear activations; the C^k layer uses generalized Hermite interpolation polynomials with affine mappings and nonlinear activations.
- Validation: The periodic-layer operations can be implemented as user-defined TensorFlow/Keras layers, and numerical experiments verify periodicity to machine accuracy.The experiments cover ordinary and partial differential equations with C^∞ and C^k periodic boundary conditions.
- Contributions: The paper contributes methods for representing smooth periodic functions and exactly enforcing C^∞ and C^k periodic boundary conditions for any k ≥ 0.The approach is defined for one and higher dimensions and is evaluated on periodic approximations and several ordinary and partial differential equations.
2 Enforcing Exact Periodic Conditions with DNN
The section defines C∞ periodicity as matching the function and all derivatives across a period, and Ck periodicity as matching derivatives through finite order k. It then constructs periodic DNN layers whose composition with arbitrary DNN functions enforces these conditions exactly, including in higher dimensions and selected directions.
- 2.1 C∞and Ck Periodic Conditions: C∞ periodic conditions require f(a)=f(b) and equality of derivatives at every order when b−a equals the period.The paper calls a smooth function satisfying these relations a C∞ periodic function on [a,b].
- 2.1 C∞and Ck Periodic Conditions: Ck periodic conditions require equality of the function and derivatives through finite order k, including order zero by convention.These comprise k+1 boundary conditions and apply even when the function is not smooth to infinite order.
- 2.2 Enforcing Exact C∞Periodic Conditions with DNN: The core construction composes an arbitrary DNN-represented function with independent periodic functions having adjustable training parameters.The periodic layer is incorporated as the network’s second layer to implement these independent functions.
- 2.2 Enforcing Exact C∞Periodic Conditions with DNN: For C∞ periodicity, a nonlinear transformation of cosine inputs supplies common-period, higher-frequency components before the arbitrary DNN function is applied.The resulting composition satisfies all C∞ periodic conditions exactly by the stated composition lemma.
- 2.2 Enforcing Exact C∞Periodic Conditions with DNN: The C∞ periodic layer extends to multiple dimensions and can impose periodicity in selected directions while leaving other directions nonperiodic.The 2D construction uses separate periodic functions for each coordinate; a modified layer enforces conditions only in x1.
- 2.3 Enforcing Exact Ck Periodic Conditions with DNN: For any k≥0, the Ck periodic layer is placed second in the DNN so the output automatically and exactly satisfies the corresponding Ck conditions.The same construction extends to two, three, and higher dimensions, with a modified 2D layer supporting periodicity in one direction.
3 Numerical Examples
Numerical examples show that the proposed periodic DNNs approximate periodic targets and PDE solutions accurately while enforcing the prescribed boundary periodicity exactly. The enforcement order determines which derivatives match: C∞ covers all tested derivatives, whereas C0 and C1 cover only the function, or the function and first derivative, respectively.
- Function approximation: The loss histories for C∞, C0, and C1 periodic DNN approximations show slow initial reduction, a sharp decrease, and a long tail during training.These characteristics are reported as common to the DNN training cases considered.
- Function approximation: All three periodic DNNs approximate the periodic function u1(x) well, with approximation curves overlapping the exact function curve.The corresponding absolute-error distributions are also compared.
- Function approximation: C∞ enforces periodicity for the function and its derivatives to machine accuracy, while C0 enforces only function-value periodicity and C1 adds first-derivative periodicity.The boundary checks use function values and derivatives through second order.
- Function approximation: For the C0 periodic function u2(x), the C0 periodic DNN is considerably more accurate, whereas C∞ and C1 produce larger errors near boundaries by bending the curve to match derivative periodicity.The three methods remain accurate in the bulk of the domain.
- Function approximation: For non-periodic u3(x), all methods agree with the exact function in the bulk, but large boundary errors and Gibbs-phenomenon fluctuations appear near both boundaries.The tests use C∞, C0, and C1 periodic DNNs despite the target being non-periodic.
- Differential equations: The exact C∞ periodic condition treats the Helmholtz problem as seeking a smooth periodic solution on the infinite domain, while C0 alone does not yield uniqueness for this second-order equation.This scope distinction explains why derivative conditions matter for the formulation.
- Differential equations: Against penalty enforcement, the proposed method matches solution, first-, and second-derivative boundary values exactly under C∞, while the penalty method matches none exactly.The comparison uses 14-significant-digit boundary values.
- Differential equations: For Helmholtz and wave-equation problems, DNN profiles agree well with exact solutions, while boundary tables verify the prescribed derivative periodicity for C∞, C1, and C2 cases.The 2D Helmholtz maximum absolute errors are less than 5 × 10^-3 for C∞ and less than 10^-2 for C1 periodic BCs.
4 Concluding Remarks
The paper presents periodic layers that make DNN outputs satisfy C∞ or C^k periodic boundary conditions exactly, with implementations available as user-defined Keras layers. Numerical evidence supports the method, while universal approximation for the resulting periodic DNNs remains an open question.
- The method composes an arbitrary DNN-represented function with independent known periodic functions having adjustable training parameters.Its construction follows from properties of compositions involving periodic functions and leverages DNN universal approximation.
- The C∞ periodic layer uses sinusoidal functions, affine mappings, and nonlinear activations, while the C^k layer uses generalized Hermite interpolation polynomials and related mappings.Both layers are incorporated into the network so the output automatically satisfies the corresponding periodic boundary conditions.
- The periodic layers can be implemented as user-defined Keras layers and used like built-in core Keras layers.The numerical examples were implemented with TensorFlow and Keras.
- The method is intended to provide a deep-learning tool for representing periodic functions and enforcing exact periodic boundary conditions in computational science applications.The paper anticipates use in classes of applications that are unexplored or scarcely explored by DNN-based techniques.
- Whether the periodic DNNs can represent arbitrary periodic functions of a given regularity to arbitrary accuracy remains an open theoretical question.The numerical experiments suggest a positive answer, but an analogous universal approximation theorem has not yet been established.