Source-linked AI summary
Simulation-Based Optimization over Discrete Spaces using Projection to Continuous Latent Spaces
Gabriel Hernández-Morales, Brenda Cansino-Loeza, Arturo Jiménez-Gutiérrez, Victor M. Zavala
TL;DR
Discrete choices make simulation-based design spaces difficult to search, and prior BO work in process design has focused on continuous decisions. This paper projects decisions into a VAE latent space for BO and reports high-quality separation-system designs using a few simulations.
Problem
Discrete decisions complicate black-box simulation-based design, while process-design BO studies have focused on continuous decisions.
Source: §1 · p. 1Method
Results
The authors report that the approach can identify high-quality designs with a few simulations in separation-system case studies.
Source: §1 · p. 3Takeaways & Limitations
The authors present VAE-based transformation as a practical alternative for navigating complex decision spaces in simulation-based optimization.
Source: §1 · p. 3The VAE architecture and hyperparameters are a critical challenge, and further work is needed to justify performance more rigorously and assess complex constrained geometries.
Source: §5 · p. 20
Abstract
from arXiv · showhide
Simulation-based optimization of complex systems over discrete decision spaces is a challenging computational problem. Specifically, discrete decision spaces lead to a combinatorial explosion of possible alternatives, making it computationally prohibitive to perform simulations for all possible combinations. In this work, we present a new approach to handle these issues by transforming/projecting the discrete decision space into a continuous latent space using a probabilistic model know as Variational AutoEncoders. The transformation of the decision space facilitates the implementation of Bayesian optimization (BO), which is an efficient approach that strategically navigates the space to reduce the number of expensive simulations. Here, the key observation is that points in the latent space correspond to decisions in the original mixed-discrete space, but the latent space is much easier to navigate using BO. We illustrate the benefits of our approach through a couple of case studies that aim to design complex distillation systems: the recovery of caprylic acid from water by liquid-liquid extraction and the separation of an azeotropic mixture using a thermally couple column know as extractive dividing wall column.
1 Introduction
Discrete decisions make black-box simulation-based design difficult, while existing optimization approaches can require many simulations or struggle to model discrete spaces. This work investigates using VAEs to project process-design decisions into a continuous latent space for Bayesian optimization.
1 Introduction
- Discrete process-design choices complicate optimization with black-box simulators, which provide no explicit access to algebraic equations.Examples include process topology, solvent selection, and equipment selection.Source: §1 · p. 1
- Evolutionary and other metaheuristics handle discrete spaces but may require many simulations and offer no clear stopping guidelines.Similar limitations are reported for other metaheuristics (Javaloyes-Ant´on et al., 2022).Source: §1 · p. 2
- Bayesian optimization uses probabilistic surrogates and uncertainty to explore costly design spaces, but process-design studies have focused on continuous decisions.Rounding discrete inputs can lead to local optima or infeasible solutions; fitting smooth surrogates directly over discrete spaces is generally difficult.Source: §1 · p. 2
- VAE latent spaces have supported molecular and protein optimization, including BO methods designed for discrete sequences.CoRel also reformulates discrete protein-sequence BO as a continuous problem using a VAE latent space.Source: §1 · pp. 2–3
- The paper applies VAE projection and BO to separation-system design and reports identifying high-quality designs with a few simulations.The authors describe the evidence as computational and empirical and present the framework as a practical alternative for navigating complex decision spaces.Source: §1 · p. 3
2 Projection of Discrete into Continuous Spaces
A VAE learns an encoder-decoder mapping from discrete inputs to a continuous latent space and back, with a probabilistic structure intended to make that space smooth. The section explains its loss and architecture, then illustrates projection on a reactor-design problem and reports that composite loss stabilizes at latent dimension eight.
2 Projection of Discrete into Continuous Spaces
- A VAE encodes discrete inputs into a continuous latent representation and decodes it to reconstruct the original data; the discussion assumes discrete inputs but notes extension to mixed-discrete spaces.The model is trained unsupervised to minimize reconstruction information loss.Source: §2 · p. 3
- Probabilistic encoding and sampling structure the latent space so nearby points correspond to similar original configurations, supporting exploration and surrogate modeling.The VAE encodes each input as a probability distribution and trains the decoder on sampled, perturbed latent vectors.Source: §2 · pp. 3–4
- The encoder embeds and concatenates discrete variables, then produces latent parameters; the decoder maps latent points to logits and softmax probabilities for reconstructing the inputs.The architecture uses an embedding layer and multilayer perceptron, with stochastic reparameterization in the encoder.Source: Figure 1 · p. 4, §2 · pp. 4–5
- VAE training combines cross-entropy reconstruction loss with KL-divergence regularization, and β weights the trade-off between loss components.The reconstruction term measures recovery of the original space, while the KL term regularizes the learned latent distribution.Source: §2 · p. 5
- The reactor example treats pathway choice as discrete and reaction time and temperature as discretized continuous decisions, then projects designs to a latent space for feasible exploration.The design objective is to maximize component C production across competing pathways, and the projected representation is described as capturing essential design-space features.Source: §2 · p. 6
- At latent dimension eight, the composite loss stabilizes; reconstruction quality is presented as a prerequisite for using the latent space in BO.Figure 3 presents the effect of latent dimension on total, reconstruction, and KL-divergence losses, and training-loss progression at dimension eight.Source: §2 · p. 7, Figure 3 · p. 7
3 Bayesian Optimization (BO) in VAE Latent Space
The framework replaces direct search over an expensive discrete design space with Bayesian optimization in a continuous VAE latent space. A Gaussian Process surrogate guides the search, with acquisition rules for single- and bi-criteria optimization.
3 Bayesian Optimization (BO) in VAE Latent Space
- The GP surrogate approximates the decoded latent-space cost function and supplies cost predictions and uncertainty to guide search.The framework maps designs x to latent points z, then decodes selected points for simulation and updates the cumulative dataset.Source: §3 · pp. 7–8, Figure 4 · p. 8
- The framework has VAE reconstruction and surrogate-model approximation errors; consistent search requires reconstruction error to be sufficiently small.The authors expect GP refinement to correct discrepancies caused by reconstruction errors.Source: §3 · pp. 8–9
- Expected Improvement balances exploration driven by predictive uncertainty with exploitation of the predicted mean cost.The acquisition function selects the next latent-space decision.Source: §3 · p. 9
- For bi-criteria optimization, the procedure predicts the Pareto frontier and selects its highest-uncertainty point for the next simulation.It uses a GP for each cost function and identifies predicted non-dominated points from predictive means.Source: §3 · p. 9
4 Case Studies
The case studies use AspenPlus simulations to optimize caprylic-acid recovery and dividing-wall-column separation while limiting expensive simulations. They examine purity and energy trade-offs in extraction and economy-safety trade-offs in the column design.
4.1 Liquid-liquid extraction of caprylic acid from water
- The VAE latent space mixes solvent, feed-position, and stage variables; multiple latent regions correspond to high purity and low reboiler duty.The t-SNE visualization shows the space as ellipsoidal.Source: §4.1 · pp. 10–11, Figure 6 · p. 12
- Solvents form different Pareto frontiers, with n-butyl acetate producing the best frontier and its Pareto points clustering in one latent region.Points for other solvents occupy other latent-space regions.Source: §4.1 · p. 11, Figure 7 · p. 13
- 95.14% purity with 24 kW reboiler duty is the caprylic-acid compromise solution, using n-butyl acetate.Across identified designs, average purity was 93.6%, ranging from 50.6% to 95.2%.Source: §4.1 · pp. 11, 14, Table 2 · p. 14
4.2 Dividing Wall Column
- FEDI ranges from 294-350 for equimolar feed and 310-370 for azeotropic feed; both compositions remain classified as hazardous.The reported ranges indicate that the safety index is sensitive to composition variation.Source: §4.2 · p. 16, Figure 10 · p. 18
- With 50 initial data points, GP predictions were restricted to 0.9–1.0 mol fraction and 370–348 FEDI; feasible designs expanded the explored space mainly in FEDI, and equimolar search intensified from iteration 30 onward.Latent points were distributed well throughout the space, with no particular Pareto-solution cluster, which the authors characterize as a complicated search.Source: §4.2 · p. 16
- Economy and safety conflict: the cost-minimizing design uses S/F 0.48, while FEDI minimization cuts it to 0.21 and raises reflux.The cost-minimizing design also uses a 0.25 main-column reflux ratio, a 22-tray dividing wall, and 31 total trays.Source: §4.2 · pp. 16, 19, Table 4 · p. 19
5 Conclusions and Future Work
The paper applies VAE-based transformations of discrete design spaces to simulation-based optimization and reports that the approach can navigate those spaces and quickly discover optimal solutions. It identifies VAE design choices as performance-sensitive and calls for further evaluation across design spaces.
5 Conclusions and Future Work
- Computational experiments show the approach can navigate design spaces and quickly discover optimal solutions.The paper evaluates the framework on a discretized CSTR reactor and conventional and intensified distillation processes.Source: §5 · p. 20
- The framework transforms discrete design spaces into continuous ones using VAEs and was applied to CSTR and distillation design.The paper notes possible application to more complex distillation columns, including multi-product dividing wall and thermally coupled reactive columns.Source: §5 · p. 20
- Further studies are needed to test transformations across design-space types and provide a more rigorous justification of performance.VAE architecture, hyperparameters, and latent-space dimension affect reconstruction loss and performance; poor reconstruction can produce infeasible points.Source: §5 · p. 20