Source-linked AI summary
A Review of Physics-based Machine Learning in Civil Engineering
Shashank Reddy Vadyala, Sai Nethra Betgeri1, John C. Matthews, Elizabeth Matthews
TL;DR
The paper addresses physics consistency, generalizability, and data limitations in machine-learning applications. It reviews physics-based approaches including Galerkin projection, parameter identification, Gaussian processes, and PINNs, highlighting applications and training constraints.
Problem
Machine-learning methods face difficulty providing physically consistent findings and lack generalizability to out-of-sample settings, while users often criticize them as black boxes.
Method
The paper examines physics-based methods including Galerkin projection, parameter identification, Gaussian processes, and physics-informed neural networks.
Results
Physics-based machine learning shows applicability to laminar flows with strong pressure gradients, while Gaussian-process approaches can generate accurate predictions without training data.
Takeaways & Limitations
The paper highlights geometry-based structural design methods as an important civil-engineering application of physics-based machine learning.
Takeaways & Limitations
PINNs with fully connected architectures frequently fail to train stably and provide correct solutions when underlying PDEs contain high-frequency or multiscale features.
Abstract
from arXiv · showhide
The recent development of machine learning (ML) and Deep Learning (DL) increases the opportunities in all the sectors. ML is a significant tool that can be applied across many disciplines, but its direct application to civil engineering problems can be challenging. ML for civil engineering applications that are simulated in the lab often fail in real-world tests. This is usually attributed to a data mismatch between the data used to train and test the ML model and the data it encounters in the real world, a phenomenon known as data shift. However, a physics-based ML model integrates data, partial differential equations (PDEs), and mathematical models to solve data shift problems. Physics-based ML models are trained to solve supervised learning tasks while respecting any given laws of physics described by general nonlinear equations. Physics-based ML, which takes center stage across many science disciplines, plays an important role in fluid dynamics, quantum mechanics, computational resources, and data storage. This paper reviews the history of physics-based ML and its application in civil engineering.
1. Introduction
Civil engineering applications expose limits of data-driven ML and computational simulation, motivating physics-based ML as a way to connect scientific knowledge with data and guide future research.
- Civil engineering ML models require large datasets, can produce physically inconsistent findings, and may not generalize to out-of-sample scenarios.
- Real-world civil engineering categories evolve, challenging methods that assume a largely stable world and depend on curated labeled datasets.
- High-fidelity numerical simulations support civil engineering applications but are often too time-consuming for iterative design and remain concentrated in final validation.
- Lower simulation costs could support broader numerical-tool use, construction-productivity improvements, and uncertainty quantification.
- Neither ML-only nor scientific-knowledge-only methods are considered sufficient for complicated scientific and technical applications.
- This review examines physics-based ML connections with physics models, synthesizes progress and gaps, and outlines opportunities for civil engineering research.
Basics of Neural Network and Physics-Based Machine Learning
Neural networks learn input-output relationships, while physics-based ML incorporates physical knowledge into training and can unroll optimization procedures into network layers.
- Neural networks express a form's input-output relationship through layered nonlinear functions and weight matrices.
- Physics-based ML combines known physics-based forward models with optimization algorithms, datasets, optimizers, and automatic differentiation.
- Physics-based ML models minimize a cost function comprising equation residuals and data across space and time.
- Inverse-problem formulations use measurements and signal priors to estimate sought information while enforcing data consistency.
- A physics-based network unrolls N optimization iterations into network layers, taking measurements and an initialization as inputs and producing an estimate after N iterations.
2. Reduced-order models
Reduced-order models address the computational burden of detailed civil engineering simulations by approximating their behavior for repeated design, optimization, and prediction tasks.
- Computational mechanics requires significant processing power, while mesh coarseness affects simulation convergence time.
- Reduced-order methods aim to replace heavier simulation models when repeated execution is otherwise limited by available computer resources.
- Reduced-order models capture source-model behavior so civil engineers can study dominant system effects using minimal computational resources.
- Civil engineers use reduced-order models to simplify full 3D simulations for structure-design optimization and broader structural analysis.
3. Proper Orthogonal Decomposition
Proper Orthogonal Decomposition (POD) extracts dominant spatial subspaces from high-fidelity snapshots to construct reduced-order models (ROMs). POD-ROM workflows combine offline basis extraction with online coefficient prediction, while Galerkin and data-driven variants trade physical structure, computational cost, and approximation challenges.
- 3.1. Reduced basis through Proper Orthogonal Decomposition: POD applies SVD or eigenvalue decomposition to snapshot matrices to extract dominant spatial subspaces and coherent system-evolution modes.These modes provide a low-rank representation of complicated spatiotemporal systems.
- 3.1. Reduced basis through Proper Orthogonal Decomposition: Reduced-basis construction uses high-fidelity snapshots in an offline stage, then retrieves expansion coefficients and reconstructs the field during online prediction.The reduced basis approximates the original field in a smaller space, with normalized optimal basis functions selected through an inner-product formulation.
- 3.1. Reduced basis through Proper Orthogonal Decomposition: Truncating the POD expansion to a finite length L yields a low-rank reduced basis, with SVD converting continuous expressions into a practical approximation.The approach is comparable to Principal Component Analysis and can support reduced-order representations of complex systems.
- 3.2. Intrusive reduced-order methods using the Galerkin procedure: Intrusive approaches use problem-specific formulations and are more computationally intensive, whereas nonintrusive approaches statistically infer snapshot mappings and can enable real-time predictions.Data-driven ROMs can also learn residuals between observational data and an existing ROM or emulate dimensionality-reduction mappings.
- 3.2. Intrusive reduced-order methods using the Galerkin procedure: Galerkin projection reinserts the truncated POD approximation into the governing equations and multiplies by POD modes to generate solvable equations for expansion coefficients.The resulting residual formulation can involve a generic nonlinear operator.
- 3.2. Intrusive reduced-order methods using the Galerkin procedure: Galerkin ROMs face unresolved speedup and parameter-dependence challenges, while nonintrusive interpolation can struggle when only a few samples are available.Parameter-dependent problems may require multiple reduced bases, whose combination can be difficult; broader settings can also expose curse-of-dimensionality effects.
- 3.4. Data driven methods ROMs: Data-driven methods broaden ROM design by supporting cheaper processing and high-dimensional simulation, while physics-based and data-driven models can increase ROM capabilities through rapid forward execution.The reviewed history traces ROM development from Galerkin methods and low-dimensional modeling through POD, snapshot methods, reduced bases, and data-driven approaches.
4. Development Physics-based machine learning
Physics-based machine learning developed to combine limited or noisy data with governing physics, addressing challenges that data-driven methods face in scientific and engineering applications.
- Physics-based machine learning emerged alongside developments in backpropagation, automatic differentiation, and deep learning, enabling applications in computational fluid dynamics and vibrations.
- Applications include CFD data recovery, aerodynamic design optimization, coarse-mesh error correction, bi-material flow modeling, and environmental science, especially hydrology.
- Physics-based learning combines noisy data with mathematical models and implements them using neural networks or other kernel-based regression networks.
- Data-driven methods can struggle when training data are insufficient or test points require extrapolation, whereas physics-based models represent underlying physical mechanisms.
- Physics-based models are designed around system mechanisms and can generate accurate predictions without training data when those physical representations are available.
A) Example problem Setup for linear PDEs
The linear-PDE example places a Gaussian-process prior after forward-Euler temporal discretization, producing a multi-output Gaussian process that captures the Euler rule.
- Forward Euler provides the simplest temporal discretization for the time-dependent linear PDE example.
- The example then places a Gaussian-process prior on the discretized formulation.
- The resulting multi-output Gaussian process captures the Euler rule.
- The paper presents pseudo-code for implementing PINNs with modern tools.
B) Example problem setup for nonlinearity PDEs
For nonlinear PDEs, PINNs approximate solutions with deep neural networks and train them by minimizing PDE residuals alongside initial, boundary, or observation mismatches.
- B) Example problem setup for nonlinearity PDEs: Backward Euler is applied to Burgers' equation, but directly assuming the current solution is a Gaussian process does not work because of the nonlinear term.
- B) Example problem setup for nonlinearity PDEs: The nonlinear formulation uses the preceding step's posterior mean in place of the unavailable direct Gaussian-process assumption.
- 4.3. Physics-Informed Neural Networks: PINNs approximate u(t, x) with deep neural networks and define a residual network through automated differentiation.
- 4.3. Physics-Informed Neural Networks: PINNs show applicability to laminar flows with strong pressure gradients in a Navier–Stokes boundary-layer example.
- 4.3. Physics-Informed Neural Networks: Operator-learning methods require many independent simulations spanning the space of interest, although evaluation becomes computationally efficient after training.
- 4.3. Physics-Informed Neural Networks: Training minimizes PDE residuals at collocation points, with additional penalties for initial and boundary conditions or inverse-problem observations.
- 4.3. Physics-Informed Neural Networks: PINNs can address forward and inverse time-dependent PDE problems, but high-frequency or multiscale solutions can produce unstable training and incorrect predictions.
5. Application of Physics-based ML to Civil Engineering
Physics-based ML has been applied in civil engineering across BIM, structural health monitoring, and structural design and analysis. The reviewed literature reflects growing publication activity and combines physical models with sensor, signal, and other application-specific data.
- Paper publication activity increased from 2014 to early 2021, alongside rising interest in physics-based ML beginning in 2019.
- Researchers mainly use sensor and signal data, selecting and reconstructing algorithms and network structures for different civil engineering problems.
- Physics-based ML applications in civil engineering center on 3D BIM, structural health monitoring, and structural design and analysis.
- 3D BIM: For BIM, physics-based ML supports 3D infrastructure modeling workflows, including terrain, alignments, tunnels, roads, and visualization.
- Structural health monitoring: For structural health monitoring, methods establish finite-element physical models and compare them with measured structural data, often using vibration-based updating.
- Structural design and analysis: In structural design, physics-based ML supports geometry-based methods and enables complex force diagrams to be constructed in milliseconds.
6. Future directions
Future directions emphasize automation, stronger public data infrastructure, and broader use of physics-based ML in civil engineering and construction. These directions respond to labor, cost, productivity, and workforce constraints.
- Civil engineering and construction face aging workers, higher labor costs, productivity losses, and insufficient onsite workers, affecting industry profits.
- Physics-based ML is expected to support automation of some civil engineering and construction processes.
- The paper identifies public civil-engineering datasets as essential for advancing physics-based ML applications.
- A construction-related dataset with shared annotation rules could promote construction-automation research and let researchers focus more on physics-based ML models.
7. Conclusions
The conclusion identifies data quality, physical consistency, generalizability, and site-environment effects as major barriers to black-box ML in civil engineering. It recommends coordinated databases and domain-specific physics-based ML development.
- Black-box ML has achieved limited success in civil engineering because of large data requirements, physically inconsistent outputs, and weak out-of-sample generalizability.
- Quality-data acquisition and the impact of site environments remain central challenges for civil-engineering ML applications.
- Multiple teams could jointly establish an extensive database using consistent annotation rules to ease data-acquisition difficulties.
- The paper envisions combining ML models with physics principles for scientific modeling and civil-engineering physical-modeling problems.
- Future research should develop dedicated physics-based ML models by integrating the approach with specific civil-engineering knowledge domains.