Source-linked AI summary
Multi-scale Deep Neural Network (MscaleDNN) Methods for Oscillatory Stokes Flows in Complex Domains
Bo Wang, Wenzhong Zhang, Wei Cai
TL;DR
Normal fully connected DNNs can struggle to learn high-frequency solution content, while MscaleDNN applies radial scaling and residual-based training to Stokes flows. In the reported tests, it captures fine flow structures where same-size normal networks fail to converge, including highly oscillatory cases.
Problem
Normal fully connected DNNs can be ineffective at learning high-frequency solution content, motivating methods for oscillatory Stokes flows.
Source: §1 · p. 2Method
Results
MscaleDNN captures fine flow structures where same-size normal fully connected networks fail to converge, and shows faster convergence across the reported Kovasznay-flow loss comparisons.
Source: §4 · p. 8, §6 · p. 18Takeaways & Limitations
The results support MscaleDNN as a meshless, high-resolution numerical method and a potential practical alternative to traditional finite element methods.
Source: §6 · pp. 17–18Convergence properties remain unresolved, and adaptive penalty selection and application to 3-D time-dependent incompressible flows are future work.
Source: §6 · p. 18
Abstract
from arXiv · showhide
In this paper, we study a multi-scale deep neural network (MscaleDNN) as a meshless numerical method for computing oscillatory Stokes flows in complex domains. The MscaleDNN employs a multi-scale structure in the design of its DNN using radial scalings to convert the approximation of high frequency components of the highly oscillatory Stokes solution to one of lower frequencies. The MscaleDNN solution to the Stokes problem is obtained by minimizing a loss function in terms of L2 normof the residual of the Stokes equation. Three forms of loss functions are investigated based on vorticity-velocity-pressure, velocity-stress-pressure, and velocity-gradient of velocity-pressure formulations of the Stokes equation. We first conduct a systematic study of the MscaleDNN methods with various loss functions on the Kovasznay flow in comparison with normal fully connected DNNs. Then, Stokes flows with highly oscillatory solutions in a 2-D domain with six randomly placed holes are simulated by the MscaleDNN. The results show that MscaleDNN has faster convergence and consistent error decays in the simulation of Kovasznay flow for all four tested loss functions. More importantly, the MscaleDNN is capable of learning highly oscillatory solutions when the normal DNNs fail to converge.
1 Introduction
The introduction motivates MscaleDNN as a response to normal DNNs’ difficulty learning high-frequency solution content. It extends radial scaling to Stokes flows, with the broader aim of developing DNN methods for time-dependent incompressible Navier–Stokes equations.
1 Introduction
- DNN-based PDE methods can be meshless because they optimize losses at randomly sampled domain points rather than fitting an unstructured mesh to complex geometry.The introduction also notes that boundary conditions can be enforced with loss penalties.Source: §1 · p. 2
- Normal fully connected DNNs can be ineffective at learning high-frequency solution content, motivating architectures that can represent widely differing scales.The introduction identifies high-Reynolds-number flows as containing many scales.Source: §1 · p. 2
- MscaleDNN uses radial scaling to shift higher-frequency solution content into a lower-frequency range before learning, then scales it back in physical space.The paper extends this approach to the Stokes problem as a first step toward DNN methods for time-dependent incompressible Navier–Stokes equations.Source: §1 · p. 3
2 Multi-scale DNN (MscaleDNN)
MscaleDNN combines parallel subnetworks that receive scaled inputs, using scale separation to make high-frequency solution content easier to learn. The paper also specifies activation, scale, and comparison-network choices for its experiments.
2 Multi-scale DNN (MscaleDNN)
- Parallel normal subnetworks receive scaled inputs and combine their outputs to approximate segments of a target function’s frequency content.Scaling converts a targeted high-frequency component to a lower-frequency range for faster learning.Source: §2 · p. 3
- The method uses localized-frequency activation functions, motivated by compact mother scaling and wavelet functions, rather than standard activations such as ReLU or tanh.The paper uses a plane wave activation function.Source: §2 · p. 3
- Input scales may be chosen as i or 2i−1 for the i-th subnetwork, with α_i denoting that subnetwork’s selected scale.The paper refers readers to [11] for further design details.Source: §2 · p. 4
- The comparison “normal” network is fully connected and has the same total number of neurons as MscaleDNN, but lacks its multi-scale features.All DNN models are trained with Adam.Source: §2 · p. 4
3 Loss functions and the MscaleDNN for Stokes problem
The Stokes solution is obtained by minimizing a residual-based loss after rewriting the equations as first-order systems. The paper compares three such formulations and adds a pressure-Poisson residual to address pressure convergence.
3 Loss functions and the MscaleDNN for Stokes problem
- The stated problem is a two-dimensional Stokes problem on an open bounded domain, with boundary data satisfying a compatibility condition.The passage states the setting and condition without specifying further scope.Source: §3 · pp. 4–5
- The loss construction uses three first-order Stokes formulations: vorticity-velocity-pressure, velocity-stress-pressure, and velocity-gradient of velocity-pressure.The MscaleDNN solution is obtained by minimizing a loss in terms of the Stokes residual.Source: §3 · p. 5
- The methods add a pressure-Poisson residual to the loss, with a tunable weight, because this residual is important for pressure convergence.The Poisson equation is obtained by taking the divergence of the Stokes equation.Source: §3 · p. 6
- Each formulation uses four MscaleDNNs for velocity, pressure, pressure gradient, and its formulation-specific auxiliary variable.The auxiliary field is vorticity, stress, or the gradient of velocity, respectively.Source: §3 · p. 6
- The named losses are ωVP-loss, VSP-loss, and VgVP-loss, alongside VP-loss from the original Stokes equation.VP-loss uses two MscaleDNNs, for velocity and pressure.Source: §3 · p. 6
4 Kovasznay flow in a square domain
The Kovasznay-flow benchmark compares normal fully connected DNNs and MscaleDNNs across Stokes loss functions. MscaleDNNs converge faster and achieve better accuracy across the three compared loss functions.
4 Kovasznay flow in a square domain
- The benchmark uses the Kovasznay analytical solution on [0,2]×[−0.5,1.5], with viscosity ν=0.1 and equal total neuron counts across network types.MscaleDNNs use six input scales; 50,000 interior and 10,000 boundary points are sampled for learning.Source: §4 · p. 7
- Both network types converge within 300 epochs for the tested losses except the simple VP-loss, which produces no reasonable results.The loss-function comparisons are shown for normal DNNs and MscaleDNNs, respectively.Source: §4 · p. 7, Figure 2 · p. 8, Figure 3 · p. 8
- MscaleDNNs converge much faster and achieve better accuracy than normal DNNs across the three compared loss functions.With the ωVP-loss, MscaleDNN solutions also match the exact solution along y=0.7.Source: §4 · p. 8, Figure 4 · p. 8, Figure 5 · p. 9
5 Oscillatory Kovasznay flows in a domain with multiple cylindrical voids
Across two highly oscillatory Stokes tests in a domain with six cylindrical holes, MscaleDNN converges where equal-sized fully connected DNNs do not. The ωVP and VgVP losses reach accuracy sooner than VSP, while adding a Poisson-equation residual significantly improves pressure error.
5 Oscillatory Kovasznay flows in a domain with multiple cylindrical voids
- The tests use two exact solutions with highly oscillatory velocity fields in a domain containing six cylindrical holes.The domain is Ω=[0,2]×[−0.5,1.5].Source: §5 · p. 9, Figure 8 · p. 10
5.1 Two frequency solution
- For the two-frequency solution, MscaleDNN with ωVP or VgVP produces very accurate results within 1,500 epochs, while VSP needs more learning and fully connected DNNs learn nothing.The fully connected and multi-scale networks have the same total number of neurons.Source: §5.1 · p. 11, Figure 12 · p. 13, Figure 13 · p. 13
5.2 Multiple frequency solution
- In the multiple-frequency case, MscaleDNN converges within 1,500 epochs while equal-sized fully connected DNNs fail; ωVP/VgVP reach accuracy sooner than VSP.The reported comparison uses the same total neuron count for both network types.Source: §5.2 · pp. 14–15, Figure 19 · p. 16, Figure 20 · p. 16
5.3 Error in pressure p and Poisson equation
- With Section 5.2 settings held fixed, adding the Poisson-equation residual leaves Err(u) compatible but significantly improves Err(p).The comparison is against the ωVP loss without that residual.Source: Figure 23 · p. 17, §5.3 · p. 17
6 Conclusion and future work
The conclusion presents MscaleDNN as a meshless, high-resolution approach for oscillatory Stokes flows in complex domains. It reports improved resolution over same-sized fully connected networks and identifies convergence, penalty selection, scale adaptation, and 3-D time-dependent flows as future work.
6 Conclusion and future work
- MscaleDNN captures fine flow structures in complex domains where same-sized normal fully connected networks fail to converge.The paper reports this as evidence of increased resolution power for the Stokes simulations.Source: §6 · pp. 17–18
- The DNN-based approach avoids mesh generation and matrix solvers, as well as delicate pressure-boundary and incompressibility treatments.These are identified as advantages over traditional mesh-based numerical methods.Source: §6 · p. 18
- Open issues include understanding MscaleDNN convergence, dynamically selecting penalty constants, adapting scales during learning, and extending to 3-D time-dependent incompressible flows.The authors identify these as topics for future work.Source: §6 · p. 18