PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Moment Maps for Continuous-Time Markov Chains under Monte Carlo Noise

Madison Pratt, Olivia Prosper-Feldman

arXiv:2606.18409v1stat.ME

TL;DR

CTMC means and covariances can be difficult to obtain across parameter values because Monte Carlo simulation is costly and moment equations may not close. This paper learns parameter-to-moment mappings from noisy Monte Carlo targets using neural surrogates. In an SIR model, the surrogates recover both moments under fixed simulation budgets, with different allocation strategies for mean and covariance.

  • Problem

    CTMC moment equations generally do not close, while Monte Carlo estimation across many parameter values is costly and requires trading parameter coverage against replication.

    Source: §1 · p. 2

  • Method

    The paper trains separate neural network surrogates on Monte Carlo-derived targets to learn parameter-to-mean and parameter-to-covariance mappings, representing covariance through a Cholesky factor.

    Source: §1 · p. 3, §2.3 · p. 5, §2.3.2 · p. 6

  • Results

    In the SIR model, the surrogates accurately recover mean and covariance across the parameter space under fixed computational budgets, and the learned moments perform effectively in whitening.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Under the studied architectures, mean learning benefits more from parameter-space coverage, whereas covariance learning requires replication to control variance and transformation-induced bias.

    Source: §6 · p. 15

    The study examines one stochastic SIR model in depth, so extending the findings to additional and more complex CTMC models remains future work.

    Source: §6 · p. 15

Abstract

from arXiv · show

Continuous-time Markov Chains are widely used to model stochastic dynamical systems, but key summary quantities such as means and covariances are often intractable. While Monte Carlo sampling provides asymptotically exact estimates, it becomes computationally prohibitive when moments must be evaluated across many parameter values. We develop a simulation-based surrogate modeling framework that learns parameter-to-moment mappings from Monte Carlo-derived, noise-corrupted training targets, enabling efficient and accurate approximation across the parameter space. We show that Monte Carlo noise affects mean estimation primarily through additive variance, whereas covariance estimation is additionally impacted by bias arising from nonlinear transformations of empirical estimates. Using a stochastic Susceptible-Infected-Recovered model, we demonstrate that neural networks accurately learn both mean and covariance under fixed simulation budgets allocated to constructing the noisy training labels. We further characterize how to allocate computational resources between parameter-space coverage and Monte Carlo replication, showing that covariance estimation requires a balanced allocation to control both variance and bias, while mean estimation benefits more from increased parameter space coverage. Finally, we show that the learned moment mappings produce valid population-level quantities and perform well in downstream tasks such as whitening. These results highlight the importance of accounting for Monte Carlo noise in surrogate modeling and provide practical guidance for simulation-based learning in stochastic systems.

1 Introduction

CTMC moments are often unavailable in closed form, and estimating them across parameter values creates a costly trade-off between parameter-space coverage and Monte Carlo accuracy. This work learns parameter-to-moment mappings from noisy simulation targets, with distinct noise effects for mean and covariance, and evaluates the approach under fixed budgets.

1 Introduction

  • Moment equations form an infinite hierarchy that generally prevents closed-form solutions, while moment-closure approximations can be biased or non-physical in strongly nonlinear or highly stochastic regimes.Monte Carlo estimates converge to true moments as simulations increase, but finite-sample estimates can have high variance.Source: §1 · p. 2
  • Estimating moments over many parameter values is costly because simulation effort must be divided between dense parameter coverage and accurate estimates at each parameter.More simulations per parameter improve statistical accuracy but reduce coverage; multilevel Monte Carlo does not remove the need for many trajectories across the parameter space.Source: §1 · p. 2
  • Unlike fixed-function noisy-output models, this setting has training-target noise induced by finite simulation and controlled by how computational resources are allocated.The framework learns parameter-to-moment mappings from Monte Carlo targets; mean noise acts as additive noise, whereas covariance transformations introduce variance and bias.Source: §1 · pp. 2–3
  • Under fixed computational budgets, neural surrogates accurately recover mean and covariance across the parameter space in the stochastic SIR model.The study also reports effective use of the learned moments in whitening, indicating joint consistency.Source: §1 · p. 3

2 Methods

The methods sample CTMC parameters and trajectories, form moment estimates, and train separate neural surrogates to predict mean and covariance mappings. The covariance learner uses a Cholesky representation to produce valid covariance matrices.

2.1 Problem formulation

  • For each parameter, the method targets the population mean and covariance of a finite-dimensional trajectory vector, whose moments are estimated by Monte Carlo simulation.The process is considered under a fixed initial condition and evaluated at a fixed collection of discrete times.Source: §2.1 · p. 3

2.2.1 Parameter sampling

  • Latin Hypercube Sampling draws parameter values so each coordinate projection is uniformly distributed across the domain, providing space-filling coverage without clustering.Each parameter dimension is partitioned into N equal intervals, with one draw per interval before dimensions are randomly paired.Source: §2.2.1 · p. 4

2.2.2 CTMC simulation

  • At each sampled parameter, the method generates M independent CTMC sample paths using the Gillespie stochastic simulation algorithm.Events occur at rates determined by the current state; waiting times are exponential with the total event rate, and events are selected proportional to their rates.Source: §2.2.2 · pp. 4–5

2.2.3 Trajectory extraction

  • The method extracts one process component and evaluates each trajectory at fixed discrete times, representing it as a T-dimensional vector.Left limits are used at sampling times to avoid ambiguity if a jump occurs exactly then; the fixed initial-time value is excluded because it is deterministic.Source: §2.2.3 · p. 5

2.2.4 Monte Carlo estimators

  • Standard Monte Carlo estimators computed across the M trajectories provide estimates of the population mean and covariance at each parameter.The sample mean and sample covariance are described as unbiased estimates of their respective population quantities.Source: §2.2.4 · p. 5

2.3 Learning parameter-to-moment mappings

  • Separate fully connected multilayer perceptrons map parameter vectors to mean vectors or a covariance parameterization, using Monte Carlo estimates as noisy targets.Inputs and outputs are standardized using statistics computed from the training data.Source: §2.3 · p. 5

2.3.1 Mean mapping

  • The mean learner trains a neural network to approximate the parameter-to-mean mapping by minimizing empirical risk against Monte Carlo mean estimates.The network input is the parameter vector and the target is the Monte Carlo estimator.Source: §2.3.1 · p. 5

2.3.2 Covariance mapping

  • For covariance learning, the method preprocesses empirical covariances, factors them by Cholesky decomposition, and trains on vectorized lower-triangular factor entries.At inference, the predicted factor is reconstructed into a covariance as L_pred(θ)L_pred(θ)^⊤, yielding symmetric positive-definite matrices by construction.Source: §2.3.2 · p. 6

3 Effect of Monte Carlo Noise on Neural Network Targets

With a fixed simulation budget, Monte Carlo noise affects mean and covariance targets differently, so their resource-allocation strategies differ. Mean targets retain the true regression function, while nonlinear covariance-target construction introduces bias.

3.1 Mean mapping

  • Mean targets are unbiased, so Monte Carlo noise adds variance scaling as 1/M without changing the optimal predictor; under a fixed budget, more parameter locations may improve generalization.The mean mapping is learned from noisy sample-mean targets, and increasing M reduces their variance while increasing N improves population-risk approximation.Source: §3.1 · p. 6

3.2 Covariance mapping

  • Covariance-target preprocessing, Cholesky factorization, and vectorization introduce bias, making the effective regression function depend on M and converge to the population target as M → ∞ under suitable regularity conditions.Increasing M reduces both empirical covariance variance and transformation-induced bias, making replication more critical for covariance learning than for mean learning.Source: §3.2 · pp. 6–7

3 Effect of Monte Carlo Noise on Neural Network Targets

  • Monte Carlo noise contributes only variance for mean learning but both variance and bias for covariance learning, yielding different resource-allocation strategies under a fixed budget.The section’s comparison motivates empirical investigation of these trade-offs in the Results section.Source: §3.2 · p. 7

4 Experimental Setup

The experiments use a stochastic SIR model to evaluate neural-network moment mappings across training configurations that vary parameter coverage and Monte Carlo replication. Performance is assessed against high-fidelity reference moments using mean- and covariance-specific error measures.

4 Experimental Setup

  • The experiment targets the infected population’s mean and covariance over 13 time points, using the fixed initial state (S, I, R) = (763, 3, 0) and a biologically relevant transmission- and recovery-rate region.The covariance target is 13 × 13, making second-order dependence moderately high-dimensional and nontrivial to learn.Source: §4 · p. 7
  • Fully connected MLPs approximate both mappings, and predictions are evaluated on 1,000 test parameters against high-fidelity reference moments.The networks use two inputs, three hidden layers of width 128, ReLU activations, and dropout during training.Source: §4 · p. 7
  • Mean predictions are assessed with trajectory Relative Root Mean Squared Error, while reconstructed covariance predictions use Relative Frobenius Error against high-fidelity references.Additional measures assess whether predicted moments reflect population moments.Source: §4 · pp. 7–8

4.1 Impact of Computational Budget Experiment Details

  • The budget experiment varies parameter count N and simulations per parameter M across separate ranges for mean and covariance training configurations.Covariance configurations use N from 25 to 4,500 and M from 25 to 5,000; mean configurations use N from 100 to 50,000 and M from 2 to 200.Source: §4.1 · p. 8
  • Performance for each (M, N) configuration is averaged across independent training replicates, with more replicates used for smaller, noisier budgets.The configuration-level summaries are replicate-averaged RRMSE for the mean model and replicate-averaged RFE for the covariance model.Source: §4.1 · p. 8

5 Results

The results assess learned CTMC mean and covariance mappings under finite simulation budgets, including how accuracy and resource allocation vary by target moment. They also examine reuse of the mappings and their effectiveness in downstream whitening to decorrelate stochastic trajectories.

5.1 Neural networks accurately learn the mean and covariance

  • Across test parameters, median trajectory RRMSE for the mean was 2.0% (mean 3.2%, 95th percentile 8.5%), while median covariance RFE was 7.8% (mean 8.8%, 95th percentile 15.2%).Approximately 29.5% of parameters had relative error exceeding 10%, while 2.4% exceeded 20%.Source: §5.1 · pp. 8–9

5.2 Increasing simulation budget reduces error

  • Increasing simulation budget reduced evaluation error 7.8× for the mean and 18.1× for covariance across the tested budgets.Improvements slowed at higher budgets, and configurations with similar budgets could still differ substantially in error, especially for covariance.Source: §5.2 · p. 10, Figure 3 · p. 11

5.3 Allocation of simulation budget impacts performance

  • Covariance error has an interior minimum across allocations, with fitted optimal scaling M(B) ≈ 1.15 B^0.48 and N(B) ≈ 0.87 B^0.52.The near-balanced exponents reflect the need to balance replication, which reduces Monte Carlo noise and bias, with parameter coverage for generalization.Source: §5.3 · pp. 11, 13
  • Mean error also has an interior minimum, but optimal allocation favors parameter coverage, with fitted scaling M(B) ≈ 0.21 B^0.42 and N(B) ≈ 4.86 B^0.58.The optimal mean allocation shifts toward larger N as budget grows; modest replication suffices, and further gains come primarily from broader parameter-space coverage.Source: §5.3 · pp. 12–13

5.4 Amortization of simulation cost

  • After its upfront training cost, the neural network provides predictions at negligible additional cost, whereas brute-force Monte Carlo cost scales as M × K.As the number of target parameters grows, the network reuses its accuracy at fixed cost while brute-force estimation requires proportionally more computation.Source: §5.4 · pp. 13–14

5.5 Learned moments in downstream tasks

  • Whitening with learned moments largely removed temporal correlation, leaving a correlation matrix close to diagonal with maximum absolute residual correlation of approximately 0.22.The result indicates that the learned covariance captures dominant temporal dependencies and the learned mean centers trajectories.Source: §5.5 · p. 15
  • The downstream goal is to remove second-order dependence rather than achieve full independence, and this is accomplished with high accuracy.Source: §5.5 · p. 15
  • The learned mean and covariance jointly enable effective decorrelation of stochastic trajectories, functionally validating their use together in downstream tasks.The results show the moments are not only individually accurate but also jointly consistent.Source: §5.5 · p. 15
  • The learned moments support downstream tasks that remove dependence structure or standardize model outputs.Source: §5.5 · p. 15

6 Discussion

The framework learns CTMC parameter-to-moment mappings from Monte Carlo outputs, offering a more efficient alternative to repeatedly simulating at each parameter value. The discussion emphasizes moment-specific simulation allocation, reports a whitening illustration, and identifies the single-model scope and future directions.

6 Discussion

  • The learned surrogate models approximate CTMC moments across parameter space without repeatedly simulating at every parameter value.The discussion describes this as a more efficient approach under a fixed computational budget.Source: §6 · p. 15
  • Mean estimation benefits more from parameter-space coverage, while covariance estimation needs replication to control variance and nonlinear-transformation bias.The discussion recommends tailoring simulation effort to the moment being learned rather than allocating it uniformly.Source: §6 · p. 15
  • The empirical study focuses on one model, the SIR system; extension to other CTMCs, higher-dimensional parameter spaces, and longer time horizons remains future work.The authors also identify more expressive architectures and adaptive simulation-allocation or stopping strategies as future directions.Source: §6 · pp. 15–16
  • After whitening with learned moments, the correlation matrix of 300 SIR trajectories at (α, β) = (0.55, 0.55) is approximately identity.The raw trajectories show strong temporal dependence, which the whitening transformation largely removes.Source: Figure 7 · p. 16

Supplementary information.

Declarations

The supplied declarations report funding, no competing interests, and access to reproducibility materials. The appendix passages describe numerical preprocessing used to maintain covariance-matrix structure and support stable factorization.

Funding.

  • The authors report National Science Foundation support under Grant No. DMS-2045843, including support for MP and OPF.Source: Funding · p. 16

Competing interests.

Data availability.

B.1 Symmetrization of empirical covariance matrices

  • The empirical covariance is symmetric in exact arithmetic; symmetrization removes numerical antisymmetry without changing its symmetric component.The text attributes small asymmetries in numerical representations to floating-point rounding and accumulation order.Source: §B.1 · p. 17

B.2 Diagonal loading and positive definiteness

  • Diagonal loading shifts covariance-matrix eigenvalues and is increased until Cholesky factorization succeeds, adding only a small perturbation.The procedure addresses failure to be strictly positive definite due to finite sample size or numerical error.Source: §B.2 · p. 18

C.0.1 Effect of Monte Carlo noise on the mean learner

Monte Carlo noise affects the mean and covariance learners differently: it adds variance to mean regression targets, but nonlinear covariance preprocessing also changes the finite-sample target. Consequently, the two moment learners have different replication and parameter-coverage considerations.

C.0.1 Effect of Monte Carlo noise on the mean learner

  • For the mean learner, Monte Carlo sampling adds a variance term scaling as 1/M without changing the underlying regression function, so the optimal predictor remains the true mean mapping.M primarily controls training-target variance, while N governs generalization performance.Source: §C.0.1 · p. 18
  • The mean learner's generalization error is controlled by the deviation between empirical and population risk.Source: §C.0.1 · p. 18

C.0.2 Effect of Monte Carlo noise on the covariance learner

  • For covariance learning, applying a nonlinear Cholesky transformation to an unbiased empirical covariance generally yields a biased target, so the finite-M learner targets gM(θ) rather than the population target ℓ(θ).Source: §C.0.2 · pp. 18–19
  • Under positive definiteness, almost-sure covariance-estimator convergence, asymptotically negligible preprocessing, and uniform integrability, gM(θ) converges to ℓ(θ) as M increases.Source: §C.0.2 · p. 19
  • Increasing M reduces covariance-target variability and the bias between the finite-M target and population Cholesky target, making replication more critical for covariance than mean learning.Source: §C.0.2 · p. 19

Appendix D Theoretical analysis for the mean learner

The appendix proof of the mean-learner population-risk result uses unbiasedness to eliminate a cross term and the variance of an average of independent samples. Its finite-sample ERM analysis compares empirical and population risks.

D.1 Population risk decomposition

  • The population-risk decomposition uses unbiasedness of the Monte Carlo estimator to make the cross term vanish.Source: §D.1 · p. 19
  • Because the Monte Carlo mean averages M independent samples, its variance term is derived from that sample average.Source: §D.1 · p. 19

D.2 Finite-sample ERM bound

  • The finite-sample ERM proof adds and subtracts empirical risks, uses the minimizer property to make the middle term non-positive, and bounds the remaining terms.Source: §D.2 · p. 19

Appendix E Theoretical analysis for the covariance learner

The covariance-learner proof shows that a cross term vanishes and identifies a unique minimizer. The appendix also defines joint moments for the CTMC SIR model.

Decomposition and Minimizer

Appendix F Moment Equations for the CTMC SIR Model

  • The appendix defines joint moments for the CTMC SIR model, with p_s,i(t) denoting the probability of observing s susceptible and i infected individuals at time t.Source: Appendix F · p. 20

F.1 Derivation of the Moment Equation

The appendix derives SIR moment equations from the forward Kolmogorov equation and illustrates first- and second-order cases. It explains that covariance dynamics inherit dependence on higher-order moments, leaving the hierarchy unclosed.

F.1 Derivation of the Moment Equation

  • The general SIR moment equation follows by multiplying the forward Kolmogorov equation by s^a i^b and summing over states.Source: §F.1 · pp. 20–21

F.2 Example: First Moment of the Infected Population; F.3 Example: Second-Order Moment

F.4 Covariance

  • Covariance evolution depends on higher-order moments because the derivatives of E[S] and E[I] depend on E[SI], whose derivative depends on higher moments.Source: §F.4 · p. 21

F.5 Lack of Closure

  • The moment equations form an infinite hierarchy: lower-order moments depend on higher-order moments, so the equations are not closed without additional approximation.Source: §F.5 · p. 21

G.1 M, N Configurations and Replicates

  • Mean and covariance networks are trained using all pairings of parameter-location count N and replicate size M; Table G1 covers mean-learner configurations and replicates per N.Source: §G.1 · p. 22, Table G1 · p. 22

G.2 Additional Plotting Metrics

The supplement describes additional error metrics and visualizations for mean and covariance networks across test parameters and training budgets.

G.2 Additional Plotting Metrics

  • Additional metrics include absolute error across time points and entrywise relative error for the mean, plus entrywise relative error for covariance.Source: §G.2 · p. 22

Supplementary Figures

  • Figure 1 maps mean-network RRMSE by test parameter and shows its distribution, clipping the color scale at the 95th percentile and marking higher values in blue.Source: Figure 1 · p. 25
  • Figure 2 maps covariance-network RFE by test parameter and shows its distribution, clipping the color scale at the 95th percentile and marking higher values in red.Source: Figure 2 · p. 25
  • Figure 3 plots test-space RRMSE and RFE across three training-budget sizes.Source: Figure 3 · p. 26
Loading 2606.18409v1…