Source-linked AI summary
Energy-aware frugal Bayesian optimization
Gaston Plat, Paul Saves, Nathalie Bartoli, Thierry Lefebvre, Joseph Morlier
TL;DR
Engineering design optimization must balance predictive performance with the computational resources required by costly black-box evaluations. This paper adds computational energy footprint to Bayesian optimization and uses Pareto parameter settings to manage that tradeoff. Experiments report tradeoffs between convergence accuracy and energy use, with the metric helping identify frugal trajectories.
Problem
Engineering design optimization seeks strong designs using costly black-box evaluations, while frugal Bayesian optimization must balance convergence with limited computational resources.
Source: §1 · p. 1Method
Results
The benchmark results show Pareto tradeoffs between accurate optima and computational energy, and the metric helps identify frugal convergence trajectories.
Source: §4 · pp. 8–9Takeaways & Limitations
Model parameter settings can balance convergence performance and electrical energy consumption in the tested benchmark problems.
Source: §4 · p. 8Evaluating each parameter configuration requires running the entire optimization problem, making hyperparameter optimization computationally costly; engineering results may also remain unconverged.
Source: §4 · p. 9
Abstract
from arXiv · showhide
Modern design optimization frameworks aim first and foremost for models with the most accurate predictions without balancing computational overhead. It remains a reason why scaled architecture and multidisciplinary design optimization problems are difficult to address, even with sample-efficient Bayesian optimizers. In this paper, a metric quantifying the computational energy footprint is introduced within a Bayesian optimization framework to guide the parameter setting of a model towards configurations that balance both performance and frugality. The computer experiments highlighted existing tradeoffs between optimum convergence and the underlying energy footprint, and sometimes resulted in both a better-found optimum and lower energy consumption.
1 Introduction
Engineering design optimization must find strong designs while managing the computational burden of costly black-box experiments. This paper introduces an energy-footprint metric and Pareto parameter selection within Bayesian optimization to balance computational cost and optimization performance.
1 Introduction
- Engineering optimization uses costly, derivative-free black-box experiments, and architecture optimization can require hierarchically searching heterogeneous, high-dimensional design spaces with coupled subproblems.These properties make computational burden an inherent challenge in engineering design.Source: §1 · p. 1
- Frugal Bayesian optimization must choose a resource-effective convergence trajectory under a budget while accounting for data and sample informativeness, prediction quality, model costs, and computational resources.The passage also identifies grid supply capabilities and remaining budget as considerations.Source: §1 · p. 1
- The paper addresses the lack of a formal framework bridging optimal design and efficient computation by integrating energy measurement into Bayesian optimization for automated model-parameter selection.Its stated aim is to save energy while maintaining accurate convergence.Source: §1 · p. 2
- The paper defines an electrical energy-footprint metric and identifies Pareto-optimal parameter settings to manage the tradeoff between computational cost and optimization performance.The framework is validated on a toy problem and a 10-bar truss mass-minimization benchmark.Source: §1 · p. 2
2 Methodology
The methodology combines space-filling initialization and Gaussian-process surrogates with Bayesian optimization to select designs across multiple objectives. It adapts the GP distance for hierarchical variables and uses acquisition functions to choose subsequent evaluations.
2.1 Initial design of experiments
- The initial design uses Latin hypercube sampling and the ϕp criterion with p = 10, with ESE used to obtain a uniform, subprojection-satisfying design.The criterion is applied by optimizing permutations of the initial design matrix’s columns.Source: §2.1 · p. 2
2.2.1 Regular Gaussian processes
- A regular GP models uncertainty with a mean and covariance, while its squared exponential kernel encodes assumptions about smoothness and correlation lengthscales.The GP predictive distribution is multivariate Gaussian, with the output variance and lengthscales selected by maximizing the logarithmic marginal likelihood.Source: §2.2.1 · pp. 2–3
2.2.2 GP handling hierarchical relationships between variables
- For hierarchical variables, the GP kernel uses an algebraic distance that accounts for inclusion and exclusion relationships while retaining the usual hyperparameter selection and prediction steps.The included-excluded distance distinguishes variables that are both included, both excluded, or have only one excluded.Source: §2.2.2 · p. 3
2.3 Unconstrained multi-objective Bayesian optimization
- For multiple objectives, Bayesian optimization selects subsequent queries using acquisition functions, including probability of improvement and hypervolume improvement.EHVI compares objectives together using hypervolume improvement; its stated formulation implicitly assumes independence between objective models.Source: §2.3 · pp. 4–5
3 Numerical results
The experiments apply multi-objective Bayesian optimization to MLP hyperparameter tuning and 10-bar truss mass minimization, evaluating configurations across energy and performance objectives. Both applications identify Pareto tradeoffs; some truss settings achieve lower energy consumption and a better feasible mass minimum than the empirically tuned configuration.
3.1 Optimization setup
- Multi-objective BO with regular GP surrogates identified Pareto-optimal hyperparameter configurations, with energy measured using CodeCarbon to combine hardware-consumption libraries.The reported computations ran on a CPU-oriented architecture with 32 GB DDR5 RAM and a 48-core Sapphire Rapids CPU.Source: §3.1 · p. 5
3.2 Toy model: parameter configuration for an accurate MLP classifier
- Up to 96% computational energy was saved as test error increased from 4.34% to 7.81%, revealing Pareto tradeoffs among MLP configurations.The optimization used 200 expensive evaluations, with 10 initial LHS points and 190 BO iterations; Figure 3 presents the energy-versus-error Pareto front.Source: §3.2 · pp. 5–6, Figure 3 · p. 7
3.3 Engineering application: parameter settings for efficient 10-bar truss mass minimization
- The truss HPO minimized optimum mass alongside computational energy, using 20 EHVI- or PI-based BO iterations after 10 initial LHS points.The setup compared hypervolume convergence and included an example of objective-value gains from a Pareto-optimal configuration.Source: §3.3 · p. 7, Table 3 · p. 8
- 91% lower computational energy consumption and a 39% lower feasible mass minimum were achieved by some truss configurations than by the empirically tuned configuration.The comparison baseline values came from, with the epoch value reduced from 10000 to 1000 for time-consuming reasons.Source: §3.3 · p. 8
4 Conclusions and discussions
Benchmark results show a Pareto tradeoff between convergence accuracy and computational energy, and the proposed energy-footprint metric helps identify frugal optimization trajectories. The method carries high computational cost because evaluating each parameter configuration requires running the entire optimization problem.
4 Conclusions and discussions
- Tables 4 and 5 present convergence comparisons and Pareto-optimal output examples across five repetitions, with best values bolded.Source: Table 4 · p. 8, Table 5 · p. 8
- The computational energy-footprint metric is integrated into Bayesian optimization as an objective to guide searches toward frugal trajectories.The study identifies Pareto-optimal parameter settings that manage computational cost and optimization performance.Source: §4 · pp. 8–9
- The benchmark results reveal a Pareto tradeoff between finding accurate optima and the computational energy required.Parameter configurations balance convergence performance against electrical energy consumption.Source: §4 · pp. 8–9
- Evaluating each hyperparameter configuration requires running the entire optimization problem, making the method computationally costly.The engineering application may not have fully converged, although configurations better than the empirical one were found.Source: §4 · p. 9