Source-linked AI summary
Multiscale modeling meets machine learning: What can we learn?
Grace C. Y. Peng, Mark Alber, Adrian Buganza Tepole, William Cannon, Suvranu De, Salvador Dura-Bernal, Krishna Garikipati, George Karniadakis, William W. Lytton, Paris Perdikaris, Linda Petzold, Ellen Kuhl
TL;DR
Biomedical machine learning is highly successful in data-rich diagnosis but can produce ill-posed or non-physical predictions when data are sparse or biased. This review examines how machine learning can be combined with physics-based and multiscale modeling. It identifies physics-informed learning, surrogate models, system and parameter identification, sensitivity analysis, and uncertainty quantification as complementary approaches for biological systems.
Problem
Sparse or biased biomedical data can make naive machine-learning applications ill-posed and generate non-physical predictions.
Method
The review synthesizes approaches that combine machine learning with physics-based and multiscale modeling, including physics-informed neural networks and multi-fidelity data fusion.
Results
The review identifies applications of the combined framework in surrogate modeling, system and parameter identification, sensitivity analysis, uncertainty quantification, and individualized clinical prediction.
Takeaways & Limitations
Combining machine learning with multiscale modeling is presented as a route toward robust and efficient models of biological systems.
Takeaways & Limitations
Data-driven predictions may violate fundamental physical laws, particularly when the model’s functional form cannot be determined explicitly.
Abstract
from arXiv · showhide
Machine learning is increasingly recognized as a promising technology in the biological, biomedical, and behavioral sciences. There can be no argument that this technique is incredibly successful in image recognition with immediate applications in diagnostics including electrophysiology, radiology, or pathology, where we have access to massive amounts of annotated data. However, machine learning often performs poorly in prognosis, especially when dealing with sparse data. This is a field where classical physics-based simulation seems to remain irreplaceable. In this review, we identify areas in the biomedical sciences where machine learning and multiscale modeling can mutually benefit from one another: Machine learning can integrate physics-based knowledge in the form of governing equations, boundary conditions, or constraints to manage ill-posted problems and robustly handle sparse and noisy data; multiscale modeling can integrate machine learning to create surrogate models, identify system dynamics and parameters, analyze sensitivities, and quantify uncertainty to bridge the scales and understand the emergence of function. With a view towards applications in the life sciences, we discuss the state of the art of combining machine learning and multiscale modeling, identify applications and opportunities, raise open questions, and address potential challenges and limitations. We anticipate that it will stimulate discussion within the community of computational mechanics and reach out to other disciplines including mathematics, statistics, computer science, artificial intelligence, biomedicine, systems biology, and precision medicine to join forces towards creating robust and efficient models for biological systems.
1. Motivation
Machine learning is highly effective for data-rich biomedical diagnosis, but sparse or biased data can produce ill-posed problems and non-physical predictions. The article examines how physics-based knowledge and multiscale modeling can constrain and improve learning while supporting surrogate modeling, parameter learning, sensitivity analysis, and uncertainty quantification.
- Data-rich diagnosis: Machine learning has achieved major successes in biomedical image-based diagnosis, supported by large amounts of annotated data.Examples include classification tasks in radiology, pathology, electrophysiology, and skin-cancer diagnosis.
- Sparse-data limitations: Sparse or biased data can make naive machine-learning applications ill-posed and produce non-physical predictions.The article asks whether known physics can constrain the admissible solution space.
- Physics-informed learning: Physics-informed learning incorporates governing knowledge to constrain machine-learning models and handle small, noisy, or otherwise difficult datasets.Early biomedical applications include cardiovascular-flow modeling and cardiac activation mapping.
- Multiscale modeling: Physics-based simulation integrates multiscale, multiphysics information to uncover mechanisms underlying the emergence of biological function.Multiscale modeling combines knowledge from molecular, cellular, and tissue levels to build organ models.
- Review scope: The article surveys machine-learning applications in multiscale modeling, including parameter learning, surrogate modeling, sensitivity analysis, and uncertainty quantification.Examples span cardiac electrophysiology, vascular hemodynamics, skin growth, and deformation prediction from pressure and topology.
2. Ordinary differential equations
Ordinary differential equations provide accessible representations of biological dynamics across molecular, cellular, organ, and population scales. Machine learning is used to identify governing dynamics, fit models, analyze uncertainty, and integrate diverse data, while sparse and limited-resolution observations remain central challenges.
- Ordinary differential equations: Ordinary differential equations model biological dynamics across molecular, cellular, organ, and population scales, often with more accessible observations than partial differential equations.Partial differential equations encode spatial variations that are often harder to measure.
- System identification: System identification infers governing equations or system dynamics from data using statistical methods such as regression, LASSO, ridge, and stepwise regression.These methods learn coefficients for combinations of algebraic and rate terms that best fit observations.
- Regression: Regression learns continuous input-output relationships and supports biomedical predictions including life expectancy, chemotherapy dose, and arrhythmogenic risk.The approach uses correctly identified training observations to evaluate new observations.
- Uncertainty quantification: Uncertainty quantification is needed in system identification because measurement and model errors affect inferred dynamics and biological outputs.Bayesian formulations provide a formal framework, while nonlinear processes can propagate input noise through the system.
- Applications: Machine learning and multiscale modeling have been applied to metabolic, cancer, neuroscience, and biomechanics problems represented by coupled nonlinear dynamics.Applications include drug side-effect prediction, pathway dynamics, genotype-to-phenotype mapping, neural dynamics, and heart-failure growth and remodeling.
- Open questions: Optimal experimental design can maximize information gain when biological data are sparse, using criteria such as Kullback-Leibler divergence or the Akaike Information Criterion.This approach can be combined with sparse identification and iterative model refinement.
- Challenges and limitations: Limited temporal resolution and sparse, incomplete, or heterogeneous data constrain the training and calibration of ordinary differential equation models.Simulations can generate missing data but may themselves be limited by poorly calibrated parameters, motivating robust inverse methods.
3. Partial differential equations
Partial differential-equation models describe biological systems across coupled spatial and temporal scales, but unknown parameters, incomplete physics, sparse data, and complex geometries make them difficult to solve and personalize. Machine learning can incorporate physical constraints, fuse heterogeneous data, build surrogates, and support inverse problems, while training, validation, and data-quality limitations remain important challenges.
- Partial differential equations: Biological PDE models couple multiple spatial and temporal scales with physical and biological processes, creating high-dimensional uncertainty-quantification challenges.Unknown parameters further complicate modeling in multidimensional parametric spaces.
- Partial differential equations: The most general modeling case combines partially known physics with scattered measurements to infer missing functional terms, parameters, and the PDE solution.This setting can also represent stochastic solutions caused by stochastic excitation or uncertain material properties.
- Physics-informed learning: Physics-informed learning embeds PDE information and constraints into machine-learning models, enabling learning from small and noisy data and supporting ill-posed inverse problems.PDEs can be encoded as informative priors for Gaussian processes or incorporated into deep neural networks.
- Applications and opportunities: Machine learning combines experimental and computational data across modalities and fidelities to create predictive surrogates, estimate parameters, identify systems, and discover functions.These capabilities support personalized computational models such as digital twins and forward applications such as treatment planning.
- Challenges and limitations: Personalized physics-based models remain difficult because complex three-dimensional domains require repeated grid generation, expert labor, and multiple spatial and temporal scales.Clinical prediction is also constrained when available data sources differ substantially in representativeness and human-specific response quality.
- Challenges and limitations: Physics-informed neural networks may be difficult to train for stiff multiscale systems, while rigorous benchmarks, validation guidelines, and reproducibility metrics remain needed.Non-convex optimization in high-dimensional spaces leads to long training times, and stochastic optimization can produce different results.
4. Data-driven approaches
Data-driven approaches use machine learning to analyze multiscale-model outputs, integrate experimental data, and support parameter identification, comparison, surrogate construction, and training-data augmentation. They also use biological learning principles to inspire new architectures and algorithms.
- Machine learning can tune multiscale-model parameters to reproduce higher-level dynamics and identify unknown constitutive relations.Genetic and evolutionary algorithms have been used for parameter tuning, while recurrent neural networks can identify constitutive relations.
- Machine learning analyzes large datasets from multiscale simulations and experiments using methods including clustering, regression, dimensionality reduction, reinforcement learning, deep learning, and parameter identification.These methods can process molecular-, cellular-, and organ-level data.
- Machine learning can compare simulated and experimental datasets by clustering and classifying model predictions, highlighting discrepancies for iterative model refinement.Differences between simulation and experiment can reveal higher-order features relevant to improving the multiscale model.
- Multiscale models can supplement insufficient experimental or clinical training data by generating simulations across parameter spaces for physics-informed machine learning.Machine learning can also help verify simulation results before datasets are expanded for artificial-intelligence applications.
- Biological learning and brain models may inspire new low-energy machine-learning architectures and algorithms incorporating spiking or oscillatory dynamics.The proposed inspiration extends beyond deep learning’s established success in image recognition.
5. Theory-driven approaches
Theory-driven machine learning combines physical laws, mechanistic models, and multiscale data to improve data efficiency, prediction, and model refinement. The review highlights applications from parameter learning and uncertainty quantification to personalized medicine and biological discovery, while emphasizing challenges involving data scarcity, bias, and physical validity.
- 5. Theory-driven approaches: Theory-driven approaches use structured physical laws and mechanistic models as prior information for learning and forecasting from imperfect, irregularly sampled data.The proposed framework integrates theory-driven machine learning with multiscale modeling in a closed loop for model- and data-driven discovery.
- 5.1. State of the art: Probabilistic and multi-fidelity methods combine information across scales, accelerate expensive computations, quantify predictive uncertainty, and guide acquisition of informative new data.Examples include combining coarse measurements with reduced-order models and selecting meso-scale simulations to recover constitutive laws.
- 5.1. State of the art: Neural differential equations identify latent dynamics from noisy, irregular time series and back-propagate through differential-equation solvers for calibration and forecasting.Automatic differentiation supports efficient derivative evaluation and model calibration, including forecasting with quantified uncertainty.
- 5.1. State of the art: Theory-driven models can enforce conservation, symmetry, or invariance, supporting robust prediction when observed data are very limited.One example uses reaction-conservation laws to model cell metabolism despite unknown exact rate-law forms.
- 5.2. Applications and opportunities: Applications span personalized medicine, cell-scale physics, biomolecular design, uncertainty-guided experimentation, and extrapolation beyond well-specified interpolation regimes.These approaches may integrate patient data with computational models, learn feasible solution spaces, explore large design spaces, and steer predictions toward physically consistent solutions.
- 5.4. Potential challenges and limitations: Key limitations include insufficient data, mismatched data resolution, training-data bias, and difficulty determining whether flexible models produce physically valid predictions.The review also identifies transparency, rigor, reproducibility, overfitting, and data bias as ongoing challenges.
6. Conclusion
The interface between machine learning and multiscale modeling offers applications in system and parameter identification, sensitivity analysis, uncertainty quantification, and physics-informed neural networks. Realizing broader impact in the life sciences requires attention to overfitting, data bias, transparency, rigor, and reproducibility.
- 6. Conclusion: Immediate applications include system identification, parameter identification, sensitivity analysis, uncertainty quantification, and physics-informed neural networks.
- 6. Conclusion: Broader progress requires increasingly sophisticated methods that address overfitting and data bias while improving transparency, rigor, and reproducibility.