PaperlayerView exact revision on arXiv

Source-linked AI summary

Review of multi-fidelity models

M. Giselle Fernández-Godino

arXiv:1609.07196v6stat.AP

TL;DR

High-fidelity modeling is often too costly, motivating methods that combine models of differing complexity for accurate, resource-efficient prediction. This review classifies multi-fidelity research and its integration strategies, finding surrogate-based combination to be predominant while identifying reporting, scalability, interpretability, and generalization challenges.

  • Problem

    High-fidelity data and analyses can be prohibitively costly, while comprehensive surveys of multi-fidelity models remain scarce.

  • Method

    The review classifies multi-fidelity literature by applications, fidelity types, surrogate models, combination strategies, and publication period, while examining surrogate and hierarchical approaches.

  • Results

    Approximately 70% of reviewed literature integrates fidelities into surrogate models, while optimization and uncertainty quantification account for approximately 90% of reviewed publications.

  • Takeaways & Limitations

    The review calls for standardized reporting metrics and presents multi-fidelity modeling as a way to pursue accuracy–cost trade-offs across scientific and engineering applications.

  • Takeaways & Limitations

    Calibration for a specific scenario may compromise broader predictive generalization, and scalable multi-fidelity frameworks remain an open research need.

Abstract

from arXiv · show

Multi-fidelity models provide a framework for integrating computational models of varying complexity, allowing for accurate predictions while optimizing computational resources. These models are especially beneficial when acquiring high-accuracy data is costly or computationally intensive. This review offers a comprehensive analysis of multi-fidelity models, focusing on their applications in scientific and engineering fields, particularly in optimization and uncertainty quantification. It classifies publications on multi-fidelity modeling according to several criteria, including application area, surrogate model selection, types of fidelity, combination methods and year of publication. The study investigates techniques for combining different fidelity levels, with an emphasis on multi-fidelity surrogate models. This work discusses reproducibility, open-sourcing methodologies and benchmarking procedures to promote transparency. The manuscript also includes educational toy problems to enhance understanding. Additionally, this paper outlines best practices for presenting multi-fidelity-related savings in a standardized, succinct and yet thorough manner. The review concludes by examining current trends in multi-fidelity modeling, including emerging techniques, recent advancements, and promising research directions.

1. Introduction.

Multi-fidelity modeling combines physical models of different complexity to balance predictive accuracy and computational cost. This review surveys applications, fidelity types, integration strategies, surrogate models, and emerging research directions.

  • Motivation: High-fidelity models offer desired accuracy but can be prohibitively expensive, whereas low-fidelity models reduce cost through simplifying assumptions.Examples include dimensionality reduction, linearization, simpler physics, coarser domains, and partial convergence.
  • Scope and definitions: Multi-fidelity models combine at least two physical models to bridge rapid computation and high accuracy at lower cost.The survey excludes simplified-surrogate distinctions and multilevel methods that replace high-fidelity models with occasional checks.
  • Integration strategies: Multi-fidelity surrogate models explicitly fuse fidelity information into one predictor, while hierarchical models select or combine fidelities without an explicit surrogate architecture.The review emphasizes surrogate-based methods while also discussing hierarchical approaches.
  • Findings and contributions: Optimization and uncertainty quantification are the most prevalent reviewed applications, while fluid and solid mechanics are dominant application fields.The review also addresses reproducibility, open-source methods, benchmarking, educational examples, and standardized reporting of savings.

2. Understanding model fidelity.

Model fidelity is relative and depends on how models differ in representation, numerical accuracy, data source, and their relationship across the design space. Effective multi-fidelity modeling therefore requires characterizing discrepancies and information transfer between levels.

  • Fidelity relationships: The relationship between fidelity levels influences how well a multi-fidelity model can approximate high-fidelity behavior.Consistent output differences across the design space may support a linear discrepancy model.
  • Fidelity relationships: Cross-validation and correlation measures help assess the reliability of low-fidelity models as predictors of high-fidelity outputs.Strong positive correlation can support information transfer and computational efficiency without sacrificing accuracy.
  • Fidelity hierarchy: The literature commonly uses two fidelities, although many multi-fidelity models can incorporate or extend to more than two.Fidelity differences may involve model complexity, numerical accuracy, or source, including experiments combined with simulations.
  • Relationship between fidelities: Fidelity is relative: a model is considered higher or lower fidelity only in comparison with another model.Grid refinement provides a straightforward comparison when otherwise identical simulations differ only in resolution.
  • Low-fidelity construction: Low-fidelity models can arise through dimensionality reduction, grid coarsening, linearization, partial convergence, reduced geometry, or simplified physics.These changes reduce complexity but may also reduce accuracy relative to a high-fidelity counterpart.

3. Domains of application.

The reviewed multi-fidelity literature is concentrated in fluid and solid mechanics, with fidelity distinctions reflecting domain-specific modeling choices. Fluid mechanics spans analytical, empirical, numerical, simulation, and experimental variations, while solid mechanics emphasizes mesh, material, temperature, dimensionality, and boundary conditions.

  • Fluid mechanics: Fluid mechanics studies span analytical expressions, empirical relations, numerical linear and Euler approximations, simulations, experiments, and coarse-to-refined analyses.These categories illustrate several ways fidelity can differ within one application domain.
  • Solid mechanics: Solid mechanics fidelity distinctions center on mesh density, material models, and temperature, with additional variations in dimensionality and boundary conditions.The review refers readers to appendix tables for more exhaustive classifications.

4. Combining fidelities.

The review contrasts multi-fidelity surrogate and hierarchical models, then surveys correction, adaptation, and inference strategies for combining fidelities. It emphasizes how these approaches balance predictive accuracy, computational efficiency, and uncertainty treatment.

  • Fusion vs. hierarchy: Multi-fidelity surrogate models integrate low- and high-fidelity data into unified predictors, while hierarchical models selectively employ fidelities according to optimization criteria.MFSMs historically dominated the surveyed literature through 2016, whereas MFHMs represented a substantial minority and were increasingly enabled by computational and algorithmic advances.
  • Hierarchical strategies: Hierarchical approaches include switching from low- to high-fidelity models when criteria are met, escalating resolution as needed, and selecting high-fidelity points using low-fidelity guidance.Examples include Markov Chain Monte Carlo sampling, adaptive complexity and resolution, stochastic collocation, and importance sampling.
  • Surrogate corrections: MFSMs use additive, multiplicative, comprehensive, or space-mapping corrections to improve low-fidelity predictions using high-fidelity information.Additive corrections model discrepancies, multiplicative corrections use ratios, comprehensive corrections combine both, and space mapping aligns models in parameter space.
  • Surrogate corrections: Space mapping iteratively calibrates the low-fidelity model in parameter space so optimization can use it more efficiently while retaining high-fidelity accuracy.The mapping is typically derived through data fitting or model calibration.
  • Inference methods: Deterministic methods produce fixed outputs for given inputs, whereas non-deterministic methods represent uncertainty and can produce different outputs for the same inputs.For uncertainty quantification, non-deterministic methods use statistical inference to treat parameter uncertainties and avoid costly standard Monte Carlo sampling.
  • Inference methods: Calibration aligns simulation or physical-model parameters with observed real-world data, but tuning for one scenario may compromise broader predictive capability.This scope limitation is explicitly noted for engineering calibration, including Bayesian calibration.
  • Inference methods: 55% of surveyed MFSM constructions used deterministic methods, compared with 45% using non-deterministic methods; comprehensive corrections were increasingly popular among non-deterministic methods.The review also identifies growing adoption of non-deterministic methods, including Kriging, co-Kriging, and Bayesian calibration models.

5. Reporting.

The review emphasizes transparent reporting and standardized evaluation of multi-fidelity models, including code availability, benchmark functions, cost, savings, accuracy, and limitations. It recommends organizing these details so readers can compare approaches and assess when multi-fidelity methods provide advantages.

  • Open sourcing: Open-sourcing implementations supports transparency, reproducibility, verification of claims, and community collaboration.The review identifies GitHub, GitLab, and Bitbucket as common hosting platforms and cites several multi-fidelity publications that shared code.
  • Benchmarking: Benchmark functions provide a standardized basis for assessing and comparing multi-fidelity methodologies.Frequently used examples include Forrester, Branin, Bohachevsky, Borehole, Currin, and Park91 functions, with implementations available in MATLAB/R and Python packages.
  • Cost savings and accuracy: Up to 86% cost savings and up to 51% accuracy improvement were reported in one study, but savings depend on the problem structure and surrogate model.The review cautions that straightforward application of multi-fidelity models is rarely universally adequate.
  • Cost savings and accuracy: Cross-validation error and RMSE are common evaluation metrics, although cross-validation may mainly help eliminate poorly performing models.RMSE can assess numerical accuracy, while evaluations should compare multi-fidelity models with low- and high-fidelity counterparts at fixed cost or accuracy.
  • Cost savings and accuracy: No significant correlation was found between single-analysis LFA/HFA cost ratios and complete-optimization MFO/HFO cost ratios.The analysis covered 18 studies that explicitly reported both ratios, and points below the 45-degree benchmark did not indicate a multi-fidelity cost advantage.
  • Reporting recommendations: Researchers should report detailed cost, savings, and accuracy breakdowns for low-, high-, and multi-fidelity models in an organized table.The proposed reporting includes model distinctions, surrogate and combination methods, and comparisons at equivalent costs or accuracy levels; an airfoil-optimization study is offered as an example.

6. Current trends.

Recent multi-fidelity modeling trends combine active learning, deep learning, transfer learning, Bayesian methods and physics-informed approaches, while interpretability, scalability and computational efficiency remain open challenges.

  • Active learning: Active learning selects high-fidelity data points strategically to update low-fidelity models and improve design-space exploration.The process iteratively refines low-fidelity predictions using data from high-fidelity domains.
  • Deep learning: Deep neural architectures help extract intricate features from detailed data, bridging fidelity levels and enhancing low-fidelity accuracy.CNNs and RNNs are highlighted as architectures for handling large datasets without the earlier need for aggressive dimensionality reduction.
  • Transfer learning: Transfer learning uses pre-trained high-fidelity representations on related tasks to share knowledge across fidelity levels while saving computational effort.The approach is presented as a way to achieve better performance through efficient knowledge sharing.
  • Physics-informed modeling: Physics-informed neural networks learn continuous, differentiable solutions to partial differential equations while respecting physical laws such as conservation of mass and momentum.Their loss functions can incorporate physics-based constraints alongside data from multiple fidelity levels.
  • Open challenges: Interpretability, scalability and computational efficiency remain unresolved concerns for deep learning-based multi-fidelity models, especially in large-scale optimization.The review points to parallel analysis, GPU and TPU usage, distributed architectures and exascale computing as areas for further research.
  • Future directions: Future directions include integrating active learning, neural operators, transfer learning, Bayesian methods and uncertainty quantification into unified frameworks.Neural operators may generalize across infinite-dimensional function spaces but often require extensive high-fidelity data and computational resources.

7. Conclusion.

The review classifies multi-fidelity applications and management strategies, finding that optimization and uncertainty quantification dominate the literature and surrogate-based integration is most prevalent. It also reports case-specific cost relationships, advocates standardized reporting, and identifies interpretability and scalability as adoption challenges.

  • Applications: Optimization and uncertainty quantification account for approximately 90% of the reviewed multi-fidelity publications.The classification scheme identifies these as the most prevalent application areas.
  • Fidelity management: Surrogate-model integration and hierarchical model use are the two predominant fidelity-management strategies.The former constructs a combined surrogate predictor, while the latter selects fidelity hierarchically according to criteria.
  • Fidelity management: Approximately 70% of the reviewed literature uses surrogate-model integration, although hierarchical contributions have increased in recent years.The review also notes a two-decade shift toward non-deterministic methods because they offer uncertainty estimates.
  • Cost reporting: No direct correlation links low- versus high-fidelity cost ratios with multi-fidelity versus high-fidelity optimization cost ratios.The finding indicates that cost relationships are case-specific rather than determined directly by analysis cost ratios.
  • Research practice: The review calls for standardized reporting metrics, open-source dissemination and benchmarking to improve evaluation, transparency and reproducibility.These practices are presented as ways to support community engagement and clearer assessment of multi-fidelity methods.
  • Open challenges: Interpretability and scalability concerns remain important challenges for broader adoption of neural-network-based multi-fidelity methods.The review presents these issues as critical for complex optimization and uncertainty quantification tasks.

Appendix B. Design of experiments.

Designing experiments for surrogate models requires sampling techniques that generate representative data, because the sampling methodology strongly affects surrogate accuracy.

  • Sampling rationale: Sampling techniques construct representative data sets for surrogate-model training, and their choice is crucial to surrogate accuracy.The review introduces experimental-design methods as mechanisms for generating suitable training points.

B.1. Traditional sampling methods.

Traditional sampling methods include grid-based, optimality-criterion and space-filling designs, with their suitability depending on dimensionality, noise and desired point distribution.

  • Management strategies: Multi-fidelity surrogate models and hierarchical models represent distinct fidelity-management strategies: fusion into a surrogate versus criterion-based fidelity selection.The review uses these categories to distinguish MFSMs from MFHMs.
  • Sampling overview: Figure 10 presents examples of sampling strategies.Use it to compare the illustrated sampling patterns rather than infer an outcome not stated in the caption.
  • Optimality-criterion methods: D-optimal designs select grid subsets by minimizing the Fisher information determinant, reducing polynomial-fit noise and often placing many points near domain boundaries.The approach can accommodate any number of samples and any domain shape.
  • Space-filling methods: Monte Carlo and Latin hypercube sampling are space-filling methods commonly used when noise is not a concern and uniformly distributed points are desired.A popular Latin hypercube variant maximizes minimum interpoint distance through the minimax criterion.

B.3. Space-filling methods.

Space-filling methods organize low- and high-fidelity samples to support multi-fidelity surrogate construction while reducing estimation difficulty and simulation expense. The review covers nested, nearest-neighbor, adaptive, and iterative placement strategies, alongside surrogate-model choices.

  • Nested and nearest-neighbor designs: Nested designs make high-fidelity points a subset of low-fidelity points, supporting space-filling augmentation and multi-fidelity surrogate construction.Examples include selecting a high-fidelity subset from an existing low-fidelity design using an optimality criterion.
  • Nested and nearest-neighbor designs: When high-fidelity points are nested within low-fidelity points, discrepancy-function parameters can be estimated independently for each Gaussian-process model.Without nesting, discrepancy estimation relies on low-fidelity surrogate parameter determination.
  • Nested and nearest-neighbor designs: Nearest-neighbor sampling independently generates both fidelity designs, then moves each nearest low-fidelity point onto its corresponding high-fidelity point.This procedure is presented as an alternative to nested sampling.
  • Adaptive and iterative placement: Adaptive sampling minimizes simulations needed to reach a specified accuracy through efficient interpolation and sampling, and compared Kriging and radial-basis strategies outperformed traditional space-filling methods.Other iterative methods place low-fidelity points optimally for local searches.
  • Surrogate-model choices: Surrogate models replace costly simulations or experiments, while multi-fidelity approaches may either combine fidelity data in one predictor or use separate surrogates hierarchically.The review discusses basis-function regression, polynomial chaos, Kriging, co-Kriging, moving least squares, POD, SVMs, and neural networks as relevant surrogate-model choices.

D.1. Example 1: Additive and multiplicative corrections.

This example uses a low-fidelity function and limited high-fidelity samples to estimate high-fidelity outputs through additive or multiplicative corrections. It also introduces comprehensive correction and a worked multi-fidelity construction using shared and low-fidelity-only data.

  • Additive and multiplicative corrections: With only three high-fidelity samples at x = [0.1, 0.5, 0.9], correction models use high-/low-fidelity ratios or differences to estimate high-fidelity outputs.Multiplicative correction fits ρ(x) to ratios, while additive correction fits δ to differences using second-order polynomial basis functions.
  • Additive and multiplicative corrections: The example compares additive and multiplicative corrections for a one-dimensional analytic problem under specified low-fidelity constants A = 0.7, B = 10, and C = 5.The low- and high-fidelity functions are assumed accessible, but high-fidelity evaluations are limited by cost.
  • Additive and multiplicative corrections: The two correction approaches perform similarly in this case, although more complex systems may show substantial performance differences favoring one approach.The review directs readers to an example notebook for replication.
  • Comprehensive correction: Comprehensive correction constructs a composite model that combines low- and high-fidelity information to improve predictive accuracy.The example applies the approach proposed by Zhang et al. and examines its coefficients.
  • Worked multi-fidelity construction: Given 20 high-fidelity and 200 low-fidelity points, one option builds a low-fidelity surrogate and integrates a discrepancy or ratio model, while another estimates high-fidelity values for 180 low-fidelity-only points before fitting the final surrogate.Both options use the 20 shared points to construct the discrepancy or ratio model.

D.4. Example 4: Non-deterministic problem.

The non-deterministic example applies co-Kriging to model uncertainty in a stochastic system. Its mean prediction is strong, but variability is under-predicted near the domain limits despite increasing uncertainty estimates there.

  • Non-deterministic problem: The extended Forrester-function example reports outstanding mean prediction performance, with uncertainty estimates increasing near the domain limits.The multi-fidelity model nevertheless under-predicts the function’s variability in those regions.

D.5. Example 5: Benchmark functions.

The benchmark examples evaluate multi-fidelity corrections on the Branin function and situate them within broader fidelity and methodology classifications. In the Branin case, additive and multiplicative corrections show comparable accuracy under MAPE.

  • Branin benchmark: Additive and multiplicative corrections applied to the Branin functions have comparable accuracy according to mean absolute percentage error.The comparison is presented in Figure 20.
  • Literature classifications: The review classifies fluid- and solid-mechanics studies by application, fidelity origin, model type, dimensionality, resolution, experimental status, flow state, convergence, and linearity.Tables 2–5 organize these categories across the reviewed literature.
  • Literature classifications: Additional tables identify papers using deterministic and non-deterministic multi-fidelity surrogate methods and report model-to-high-fidelity cost ratios across fields.The cost-ratio grouping includes fluid mechanics, solid mechanics, and other applications.
Loading 1609.07196v6…