PaperlayerView exact revision on arXiv

Source-linked AI summary

An Online Non-Stationary Simulation Optimization Approach Based on Regime Switching

Jianglin Xia, Haowei Wang, Songhao Wang, Szu Hui Ng

arXiv:2508.12634v1math.OC

TL;DR

The paper addresses online simulation optimization when input distributions change across regimes, creating uncertainty about both future regimes and input parameters. It proposes a Bayesian MSM-based objective and an online metamodel algorithm; the objective and its optimal solutions have consistency and asymptotic-normality guarantees, and numerical experiments report strong empirical performance.

  • Problem

    Online simulation optimization must handle non-stationary, regime-switching input distributions, with uncertainty from finite-data parameter estimation and prediction of the next regime.

    Source: §1 · p. 2

  • Method

    A Bayesian MSM-based framework approximates the objective, while a joint metamodel reuses simulation results across stages and uses regime-aware Expected Improvement to select experiments.

    Source: §1 · p. 3, §5.2 · p. 9

  • Results

    The approximated objective and optimal solutions are consistent and asymptotically normal, and numerical experiments report strong empirical performance on inventory and portfolio tasks.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The asymptotic normality results support hypothesis testing and confidence intervals for the objective and optimal solutions.

    Source: §1 · p. 3

    The main formulation assumes the number of regimes is known, with unknown-regime scenarios treated separately in Section 7.

    Source: §1 · p. 2

Abstract

from arXiv · show

Dynamic and evolving operational and economic environments present significant challenges for decision-making. We explore a simulation optimization problem characterized by non-stationary input distributions with regime-switching dynamics across sequential decision stages. This problem encompasses both prediction uncertainty, arising from the regime-switching behavior of input distributions, and input uncertainty, resulting from parameter estimation for these distributions and their dynamics using finite data streams. To address these uncertainties, we develop a Bayesian framework that approximates the true objective function using a Markov Switching Model (MSM). We rigorously validate this approximation by establishing the consistency and asymptotic normality of the objective functions and optimal solutions. To solve the problem in an online fashion, we propose a metamodel-based algorithm that leverages simulation results from previous stages to enhance decision-making. Furthermore, we tackle scenarios with an unknown number of regimes through a Bayesian nonparametric method. Numerical experiments demonstrate that our algorithm achieves superior performance and robust adaptability.

1 Introduction

The paper studies online simulation optimization when input distributions change through regime switches, making decisions difficult in dynamic environments. It combines Bayesian modeling of regime and parameter uncertainty with theoretical validation and an online metamodel-based algorithm.

Problem setting

  • Regime-switching creates both finite-data input uncertainty and persistent uncertainty about which regime governs the upcoming decision stage.The paper assumes regime count is known except in Section 7, while distribution parameters and switching points must be inferred from historical data.Source: §1 · p. 2

Bayesian approximation

  • The Bayesian objective uses a Markov Switching Model to predict the next regime and accounts jointly for regime-prediction and input-parameter uncertainty.Predictive regime probabilities weight each possible future regime, while Bayesian parameter estimation represents uncertainty in the input distribution and MSM dynamics.Source: §1 · p. 2

Theoretical contributions

  • As the amount of stage and input data grows, the approximated objective and its optimal solutions converge to their true-objective counterparts, with asymptotic normality supporting statistical inference.The stated convergence rate for estimated optimal values and solutions is O_p(1/√t), where t counts observations of ξ.Source: §1 · p. 3

Online algorithm

  • A unified metamodel jointly models decision variables and input parameters, reusing prior-stage simulation results to guide future decisions.A regime-aware Expected Improvement function selects the next design point and input distribution to simulate.Source: §1 · p. 3

Numerical experiments

  • The paper develops a metamodel-based online optimization algorithm with regime-aware Expected Improvement, establishes theoretical convergence, and demonstrates empirical performance on inventory and portfolio tasks using real-world 2008–2009 Great Recession regime and return data.Source: §1 · p. 3

2 Related Works

The related work spans input uncertainty, streaming input data, and regime-switching models. The paper positions its problem at the intersection of these areas.

2 Related Works

  • The review organizes relevant work around input uncertainty, streaming input data, and regime-switching streaming-data modeling.Source: §2 · p. 3

2.1 Simulation Optimization Under Input Uncertainty

  • Prior work on simulation optimization under input uncertainty addresses discrete and continuous optimization, Bayesian treatment of parametric uncertainty, and nonparametric models when the input family is unknown.Neglecting input uncertainty can lead to suboptimal designs and system risks.Source: §2.1 · p. 3

2.2 Simulation Optimization with Streaming Input Data

  • Streaming-data studies address evolving problems, ranking and selection, multiple input streams, and decision-dependent distributions, but not externally driven regime-switching input distributions.Source: §2.2 · p. 3

2.3 Regime-Switching Streaming Data Modeling

  • Existing portfolio studies often use estimated MSM parameters without accounting for their estimation uncertainty; a cited robust-optimization approach can be overly conservative and does not apply to simulation optimization.Source: §2.3 · p. 4

3 Problem Formulation

The formulation represents regime changes with an MSM and defines a Bayesian approximation to the next-stage objective. The approximation accounts for uncertainty in both future regimes and unknown input-model parameters.

3.1 The Preliminaries of MSM

  • An MSM pairs a finite-state Markov chain of latent regimes with regime-specific emission distributions for the observed input.The input parameter vector includes each regime’s distribution parameters and the transition-matrix parameters.Source: §3.1 · p. 4
  • The next-stage regime distribution is calculated from the current filtered regime probabilities and transition dynamics.Filtering the current regime given observations is performed recursively using Bayes’ rule and the law of total probability.Source: §3.1 · p. 4

3.2 Objective Formulation

  • The stage t+1 objective incorporates uncertainty about which regime will govern the input, with the true objective defined under known regime-process parameters.The true objective uses the predictive probability of the next regime given the observed input history.Source: §3.2 · p. 4
  • For exposition, the formulation assumes one new input observation arrives at each stage, while the approach can extend to varying numbers of observations.Source: §3.2 · pp. 4–5
  • The Bayesian approximation takes expectations over parameter uncertainty, next-regime uncertainty, and the input distribution conditional on that regime.Its three expectation layers represent posterior uncertainty in input parameters, predictive uncertainty over regimes, and the regime-specific input distribution.Source: §3.2 · p. 5
  • The posterior P(ϑ|ξ_t) is formed from the observed input sequence and integrates input-parameter uncertainty into the decision objective.Source: §3.2 · p. 5
  • Using the MSM predictive probabilities, the formulation can be simplified to a weighted combination of regime-specific stochastic optimization objectives.Each z(x, λ_l) is the ordinary stochastic optimization objective under the input distribution parameterized by λ_l.Source: §3.2 · p. 5

4 Theoretical Validation of the Bayesian Approximation

As input data accumulates online, the posterior is updated and the Bayesian approximation to the objective improves. The section validates this approximation through consistency and asymptotic normality of objective values and optimal solutions.

4.1 Consistency of Objectives

  • The posterior distribution for MSM parameters is consistent under the stated assumptions, supporting the subsequent consistency results for the Bayesian objective.The section derives posterior consistency first, then uses it to establish consistency of the objective function.Source: §4.1 · p. 6
  • As data accumulates, the proposed objective converges to the true objective, and the optimal solutions and values are consistent under the stated conditions.The results establish pointwise objective consistency and consistency of the sets of optimal solutions and their values.Source: §4.1 · p. 6

4.2 Asymptotic Normality of Objectives

  • The asymptotic results support confidence intervals for optimal values or solutions and estimating the real-world data needed to meet a target interval width.The section also describes using asymptotic variance estimates to assess convergence and determine data requirements for a chosen tolerance.Source: §4.2 · pp. 6–7
  • As t grows, the estimated objective and optimal values or solutions have asymptotically normal deviations from their counterparts under perfect distribution knowledge.The section establishes a Bayesian total-variation CLT for MSM and asymptotic normality results for objective values and optimal values; optimal-solution normality is also described.Source: §4.2 · p. 7
  • Unlike De Gunst and Shcherbakova (2008), the posterior CLT centers directly at the true parameter, yielding a more direct asymptotic characterization.The paper identifies this refinement as central to establishing asymptotic normality of the objective functions.Source: §4.2 · p. 7

5 Regime-Switching Online Bayesian Simulation Optimization Algorithm

RSOBSO uses a joint metamodel of decisions and input parameters to reuse simulations from earlier stages. A regime-aware expected-improvement search selects experiments, with convergence established under stated assumptions.

5.1 The GP Model for the Objective Function

  • A GP over decision x and input parameter λ lets runs from changing input distributions inform the current objective, integrated over regime predictions and the input posterior.Source: §5.1 · pp. 7–8
  • The GP can be computationally restrictive online at O(n^3); sparse approximations or moving windows can keep computation manageable.Source: §5.1 · p. 8

5.2 The GP-Based Optimization Algorithm

  • Regime-aware expected improvement selects the next design and input parameter by balancing exploration and exploitation.Source: §5.2 · p. 9

5.3 Algorithm Convergence Analysis

  • As t, u, m, and NMC tend to infinity, RSOBSO’s returned decision converges in probability to the true stage optimum, assuming compact X and known GP hyperparameters.Source: §5.3 · p. 10

6 Numerical Experiment

Across synthetic tests, regime-driven inventory, and portfolio optimization, RSOBSO performs robustly under changing regimes and uncertainty. The experiments compare it with methods that omit regime dynamics or input uncertainty and assess cumulative GAP or returns.

6 Numerical Experiment

  • The experiments compare four baselines that omit input uncertainty, regime prediction, or both, and measure each method using GAP under the next day's realized regime.All methods use the same evaluation budget after an initial history period.Source: §6 · p. 10

6.1 Synthetic Test Problems

  • The synthetic tests include four-regime exponential and three-regime Gaussian emissions, while inventory experiments examine bull–bear demand and four economic regimes.Inventory demand distributions vary by regime; the four-regime case uses CPI and GDP trend combinations.Source: §6.1 · p. 10, §6.1.1 · p. 10, §6.1.2 · p. 11

6.1.3 Analysis of Synthetic Test Problems

  • Single-regime methods can have high GAP in some regimes, while plug-in methods are unreliable and KDE estimates are unstable; RSOBSO adapts across regime shifts.The analysis attributes plug-in volatility to ignored input uncertainty and reports that KDE produces wide confidence intervals.Source: §6.1.3 · p. 11
  • RSOBSO outperforms benchmark methods on synthetic tests, remains robust across inventory regimes, and achieves the highest cumulative returns in both portfolio settings.In the inventory study, it also achieves lower GAP during the Great Recession.Source: §6.1.3 · p. 11, §6.2.3 · p. 12, §6.3.1 · p. 13

6.2.3 Analysis of the Regime-Driven Inventory Experiment

  • In the two-regime inventory case, regime-switching methods recover separated parameters and yield smaller cumulative GAP, with the lowest GAPs after February 2009.Frequent transitions before then hinder predictive accuracy.Source: §6.2.3 · p. 12
  • In the four-regime inventory case, RSOBSO estimates three small and one large rate, yielding lower GAP in Regime 1 and improved performance in Regimes 2 and 3.RSOPSO performs weakest; the overall GAP difference remains modest because Regime 1 is infrequent and Regimes 2 and 3 dominate for 19 months.Source: §6.2.3 · p. 12

6.3 Real-World Portfolio Optimization Problems

  • The portfolio study uses two-asset monthly return data and estimates certainty-equivalent returns from sample means and standard deviations.The two settings are Mkt/SMB and Mkt/HML.Source: §6.3 · p. 13

6.3.1 Analysis of Portfolio Optimization Experiments

  • RSOBSO earns the highest cumulative returns in both Mkt/SMB and Mkt/HML portfolios, with allocations responding to crisis losses and the May 2009 recovery.It allocates more to SMB during October 2008–February 2009 and shifts toward Mkt after identifying the recovery.Source: §6.3.1 · p. 13

7 Extensions to Unknown Number of Regimes

The paper extends RSOBSO to an unknown number of regimes using an HDP-HMM and combines this inference with the online algorithm. In the inventory experiments, the extension estimates regime counts and outperforms NOBSO.

7 Extensions to Unknown Number of Regimes

  • HDP-HMM RSOBSO uses a Bayesian nonparametric transition prior to infer regime number and updates its maximum regime limit as new observations arrive.The online limit is set to one above the maximum regime count inferred at the prior stage.Source: §7 · pp. 13–14
  • In the two-regime inventory experiment, inferred counts range from two to four and closely align with the true count.Figure 5 compares HDP-HMM RSOBSO with NOBSO.Source: §7 · p. 14, Figure 5 · p. 14
  • HDP-HMM RSOBSO consistently outperforms NOBSO, with input-parameter estimates remaining distinctly separated in magnitude in the three- and four-regime cases.The three-regime estimates typically include two values below 0.1 and one near 1; four-regime estimates show two small and two large values, or three small and one large.Source: §7 · p. 14

8 Summary

The paper develops and validates RSOBSO for online simulation optimization with regime-switching inputs, including an extension to unknown regime counts. Experiments report performance advantages over benchmarks that omit non-stationarity, input uncertainty, or joint GP modeling.

8 Summary

  • RSOBSO combines Bayesian objective approximation for regime and input uncertainty with a joint GP and regime-aware expected improvement, and experiments show robust performance even when regime count is unknown.The paper establishes consistency and asymptotic normality for the approximated objective function.Source: §8 · p. 14

A Algorithm for Hidden Regime Number Inference

The appendix describes HDP-HMM priors and an inference procedure for estimating the number of hidden regimes. It uses sampled regime counts to produce an inferred count and an upper limit.

A Algorithm for Hidden Regime Number Inference

  • The HDP-HMM shares statistical strength across regime transitions through global stick-breaking weights, while each regime has an emission parameter drawn from prior H.Regime-specific transition distributions use the shared global weights as their base measure.Source: A Algorithm for Hidden Regime Number… · pp. 14–15
  • Inference uses a blocked Gibbs sampler with a weak-limit Dirichlet approximation to the transition prior and a threshold on stick-breaking weights.The mode of sampled regime counts estimates the number of regimes.Source: A Algorithm for Hidden Regime Number… · p. 15
  • Algorithm 2 resamples model parameters across sampling steps and outputs the maximum inferred regime count and the mode-based inferred count.Its inputs include an upper regime limit, observations, sampling steps, and a threshold.Source: A Algorithm for Hidden Regime Number… · p. 15

B HDPHMM-RSOBSO Algorithm

Algorithm 3 initializes the metamodel from streaming data and simulation results, then updates its regime estimate, posterior samples, and Gaussian-process model as new data arrive.

B HDPHMM-RSOBSO Algorithm

  • Algorithm 3 initializes the regime estimate and design using streaming data, prior posterior samples, and Latin hypercube sampling.It runs replicated simulations at initial points and fits a Gaussian-process model to the observed output means.Source: B HDPHMM-RSOBSO Algorithm · p. 15
  • The online loop selects evaluation points, runs simulations, and updates the Gaussian-process model from observed output means.The loop repeats optimization steps across sequential stages.Source: B HDPHMM-RSOBSO Algorithm · p. 15
  • At each stage, the algorithm updates the regime count and estimate from the expanded input stream using Algorithm 2.It generates posterior samples using the latest updated regime assignments.Source: B HDPHMM-RSOBSO Algorithm · p. 15

C Proofs for Section 4

The proofs establish convergence properties for the Markov switching model and its Bayesian objective estimates, including a Gaussian limit and uniform convergence results.

C Proofs for Section 4

  • Under the stated assumptions, the MSM maximum-likelihood estimator is strongly consistent, and the score function satisfies a central limit theorem.These results support the subsequent posterior and objective-function convergence arguments.Source: C Proofs for Section 4 · pp. 15–16
  • The Bayesian objective estimate converges uniformly to the true objective under the stated continuity and compactness conditions.The argument uses pointwise convergence and a common Lipschitz bound.Source: C Proofs for Section 4 · pp. 16–17
  • The posterior distribution converges in total variation to a Gaussian distribution with mean Δt and covariance J^-1.The proof derives this from the score-function central limit theorem.Source: C Proofs for Section 4 · pp. 16–17
  • For any finite collection of decisions, the scaled objective process has finite-dimensional distributions converging weakly to those of its limit.The proof extends the Taylor-expansion argument coordinate by coordinate.Source: C Proofs for Section 4 · p. 17

D Experiment Settings in Section 6 and 7

The appendix specifies priors, decision spaces, and simulation settings for the experiments, then describes the objective functions used by the benchmark methods.

D Experiment Settings in Section 6 and 7

  • Throughout all numerical experiments, each transition-matrix row uses the symmetric Dirichlet prior Dir(1, · · ·, 1), as mentioned in Section 5.1.This section presents detailed settings for all numerical experiments.Source: D Experiment Settings in Section 6 and 7 · p. 18

D.1 4-Regime Exponential Emissions Case

  • The candidate space is x ∈ [0, 50]; the conjugate weakly informative prior for λ is Gamma(1, 0.1), and h = 100, u = 30, tmax = 25, m = 100, NMC = 100.Source: §D.1 · p. 18

D.2 3-Regime Gaussian Emissions Case

  • The candidate space is x = (x1, x2), with x1 ∈ [−20, 20] and x2 ∈ [−40, 40]; unknown µ has prior U(0, 50), with h = 50, u = 30, tmax = 25, m = 100, and NMC = 100.Source: §D.2 · p. 18

D.3 2-Regime Inventory Problem

  • The 2-regime inventory case uses a Gamma(1, 1) prior for λ, h = 48, m = 10, and regime-specific optima (63.8, 127.0) at cost 147 and (1, 70) at cost 38.The candidate decisions are (s, S) with s ∈ [1, 69] and S ∈ [70, 250].Source: §D.3 · p. 18

D.4 4-Regime Inventory Problem

  • The 4-regime inventory case uses a Gamma(1, 0.1) prior for λ and reports optima from (69, 191) at cost 222 to (1, 70) at cost 38 across its regimes.The settings include h = 96, m = 10, u = 30, tmax = 24, and NMC = 100.Source: §D.4 · p. 19

D.5 Portfolio Optimization Problems

  • The portfolio cases use uniform priors for unknown µ and σ, with h = 48, u = 30, tmax = 24, m = 1000, and NMC = 100.The prior ranges differ between the Mkt/SMB and Mkt/HML cases.Source: §D.5 · p. 19

D.6 Unknown Number of Regimes

  • The unknown-regime experiment uses Gamma priors for emission rates and concentration parameters, threshold τt = 1/(h + t), truncation Nmax = 10, and M = 100 posterior samples per stage.Source: §D.6 · p. 19

E Objective Functions of Benchmark Methods

  • The benchmarks differ in objective construction: RSOPSO uses a weighted plug-in estimator, NOBSO Bayesian-averages without switching, NOPSO uses a single plug-in estimator, and NOKSO directly evaluates z(x).Source: E Objective Functions of Benchmark… · p. 19

F Proof of Theorem 5

The proof of Theorem 5 establishes dense design coverage, convergence of the Gaussian-process posterior mean to the objective, and convergence of the algorithm’s optimal value.

F Proof of Theorem 5

  • Theorem 5’s proof proceeds from dense design points to posterior-mean convergence and then convergence of the returned solution’s objective value.Source: F Proof of Theorem 5 · p. 19

F.1 Density of the Design Points

  • The expected-improvement selection rule makes design points everywhere dense by favoring unexplored regions over neighborhoods of existing points.Near an existing point, predictive variance and expected improvement shrink; an unexplored region retains a lower bound on expected improvement.Source: §F.1 · p. 20

F.2 Convergence of the Posterior Mean

  • With dense evaluation points, the posterior mean converges in probability uniformly in x to the stage objective.The proof combines convergence of the Gaussian-process prediction to z(x, λ) with integration over λ.Source: §F.2 · p. 21

F.3 Convergence of the Optimal Value

  • The best solution identified along a dense sequence of evaluations converges toward the stage’s optimal value.Source: §F.3 · p. 21

G Proof of Theorem 6

  • For any fixed stage, Theorem 5 supplies an iteration threshold after which the stated approximation error is below the chosen bound.Source: G Proof of Theorem 6 · p. 21
Loading 2508.12634v1…