Learn-Then-Differentiate Gradient Estimation

• Gradient information supports simulation optimization, sensitivity analysis, uncertainty quantification, and financial risk management, among other applications.Examples include service-system performance, supply-chain policies, and financial sensitivities (Greeks) (Ho and Cao 1991, Glasserman 2004, Fu 2006).Source: §1 · p. 1
• Classical simulation-gradient methods can be organized by whether they differentiate the sample performance, the probability measure, or both.Representative methods include pathwise and finite-difference methods, likelihood-ratio and weak-derivative methods, and generalized likelihood-ratio and Malliavin-calculus-based estimators.Source: §1 · p. 2
• LTD has been studied through separate analyses for specific learning methods or surrogate models, motivating a common framework.Source: §1 · p. 2
• LTD learns an approximation of expected performance from simulation data and then differentiates that surrogate.Surrogates include higher-order kernel regression (Hansen 2008), local polynomial regression (Neumeyer and Van Keilegom 2010, Racine 2016), and kernel ridge regression (Fischer and Steinwart 2020, Liu and Li 2023).Source: §1 · p. 2
• LTD needs little structural information about the simulator, provides gradients across a parameter-space region, and separates offline learning from online evaluation.This supports black-box simulation and can make gradient evaluation rapid once the surrogate is learned.Source: §1 · p. 2
• The framework gives a common weighted representation for LTD estimators, with learning methods differing in how they construct the weights.It connects the learning method and its approximation guarantees to the corresponding estimator and statistical properties, and extends to methods meeting its conditions.Source: §1 · p. 3

• A common weighted representation covers a broad class of LTD estimators and provides an algorithm-independent foundation for their interpretation and analysis.Source: §1 · p. 3

• Every weighted LTD estimator induces a learned measure representation from simulation observations, and differentiating the surrogate is equivalent to differentiating that representation.LTD therefore fits the measure-differentiation class through a measure represented nonparametrically by statistical learning rather than analytically by an explicit probabilistic model.Source: §2 · p. 3

• Avoiding state-space enlargement can yield substantially lower variance than likelihood-ratio estimators in the reported numerical experiments.Source: §3 · p. 4

4. A general convergence theory.

• Under sufficiently smooth probabilistic models, LTD convergence rates can approach the canonical n^-1/2 rate.The result depends jointly on the smoothness of the expected-performance function and the approximation capability of the learning algorithm.Source: §4 · p. 4

5. Applications across learning methods.

• For kernel regression, local polynomial regression, and kernel ridge regression, the framework recovers established derivative-rate results through a common argument.For multiple kernel learning and a particular smooth neural-network method, it gives new convergence guarantees for gradients and higher-order derivatives under stated regularity conditions.Source: §5 · p. 4
• The five applications show how the framework unifies existing results and provides a systematic route to new derivative-estimation guarantees.The paper develops the applications after introducing the methods in Section 3 and presents numerical experiments in Section 6.Source: §5 · p. 4

2. Foundations

• The objective is to estimate gradients or higher-order derivatives of expected performance from simulation information, allowing a d-dimensional parameter.The parameter may enter through the performance function, the probability measure, or both.Source: §2 · pp. 4–5

2.1. Glasserman’s Framework

• Glasserman’s framework organizes gradient estimation by whether it differentiates the performance function, the probability measure, or both.The pathwise estimator follows when parameter dependence is through the performance function and differentiation can be interchanged with integration; the likelihood-ratio estimator uses a density under corresponding regularity conditions.Source: §2.1 · p. 5

2.2.1. Pathwise Methods and Nonsmooth Performance Functions.

• For the Asian digital option, the discontinuous payoff makes the pathwise estimator identically zero even though the expected option price depends on the initial asset price.The payoff discontinuity invalidates the interchange of differentiation and expectation, although the expected performance is smooth.Source: §2.2.1 · p. 6

2.2.2. Likelihood-Ratio Methods and State-Space Enlargement.

• The likelihood-ratio method requires an explicit likelihood ratio; the arithmetic-average distribution has no known tractable closed form, so the example instead enlarges the state space to the discretized asset-price path.The enlarged state has an available joint density from geometric Brownian motion transition densities.Source: §2.2.2 · p. 7
• As the time step shrinks, the enlarged-path likelihood-ratio estimator’s variance explodes, a finite-dimensional manifestation of approaching singularity between path distributions.The performance-relevant random-variable distributions remain mutually absolutely continuous, whereas continuous paths from different deterministic initial prices are mutually singular.Source: §2.2.2 · p. 7
• In black-box simulations, neither the performance-relevant distribution nor an enlarged-state distribution may be available, limiting classical likelihood-ratio methods.Enlargement can also reduce statistical efficiency by retaining likelihood-ratio variation beyond that summarized by the performance-relevant random variable.Source: §2.2.2 · p. 8

3. Learn-Then-Differentiate Gradient Estimation

• LTD constructs a surrogate for the expectation function and differentiates it, making derivatives available from a sufficiently smooth fit without additional simulation experiments.The framework is motivated in part by settings where simulation is performed offline and derivatives are needed later, including contextual optimization and financial engineering.Source: §3 · pp. 8–9
• A weighted representation accommodates response-independent and response-adaptive learning methods, including multiple kernel learning and certain neural-network estimators.Not every weighted surrogate is suitable for gradient estimation: k-nearest-neighbor methods and regression trees generally produce nondifferentiable approximations.Source: §3 · pp. 8–9

3.1. Representative Weighted Surrogate Models

• Kernel regression, local polynomial regression, and kernel ridge regression represent local averaging, local approximation, and global regularization, respectively.Their surrogate differentiability is governed primarily by kernel smoothness, while their approximation and derivative convergence rates depend on the learning method.Source: §3.1 · pp. 9–10

Question 1. What mathematical object does an LTD estimator differentiate?

• The first central question is what mathematical object LTD differentiates, since differentiating a learned surrogate does not fit directly into the classical performance-function or probability-measure framework.The paper addresses this question in Section 4.Source: §3.2 · p. 11, Question 1. What mathematical object… · p. 11

Question 2. How efficient are LTD estimators?

• The paper asks whether LTD gradient estimators can inherit near-optimal efficiency, given canonical Monte Carlo’s n^-1/2 benchmark (Glasserman 2004, Fu 2006) and surrogate expectation-function convergence close to n^-1/2 under suitable smoothness assumptions.Source: Question 2. How efficient are LTD… · p. 11

4.1. Probability Measures as Differentiable Objects

• At the population level, a differentiable representation of the parameterized probability measure lets derivatives act on that representation while the performance function remains fixed.This parallels likelihood-ratio differentiation, where a likelihood ratio represents the measure relative to a fixed reference measure.Source: §4.1 · p. 12, §4.1.1 · p. 13

4.1.1. Kernel Mean Representation.

• A kernel mean representation expresses expectations without a probability density, provided the performance function belongs to the RKHS and the representation satisfies the stated differentiability conditions.The representation determines expectations of RKHS performance functions through an inner product.Source: §4.1.1 · pp. 12–13

4.1.2. Piecewise Smooth Performance Functions.

• For piecewise-smooth performance functions, a parameter-independent partition and RKHS-representable branches yield derivative formulas using subprobability measures while keeping branch functions fixed.Although the theoretical representation uses a partition, it disappears from the final estimator, which depends on observed performance values and differentiated surrogate weights.Source: §4.1.2 · p. 14

4.2. Learning Probability Measure Representations

• Using fitted surrogate weights to combine RKHS-valued simulation responses learns measure representations whose derivatives yield the weighted LTD estimator, without requiring the partition in implementation.The same fitted weights are used across regions, and the training data are held fixed when differentiating.Source: §4.2 · pp. 15–16

4. A Unified Interpretation of Learn-Then-Differentiate

• Weighted LTD can be interpreted as differentiating a learned representation of the underlying probability measure, not the performance function itself.This interpretation places the estimator within a framework for probability-measure differentiation.Source: §4 · p. 11, §4.3 · p. 16

4.3. Relationship with Likelihood-Ratio Methods

• Unlike likelihood-ratio methods, LTD learns a kernel mean measure representation from simulation data, requires no analytical distribution, and avoids state-enlargement singularity issues.Both approaches differentiate the probability measure, but use different representations.Source: §4.3 · pp. 16–17

5. Statistical Efficiency of Learn-Then-Differentiate Estimators

• The section asks whether LTD can attain or approach the canonical n^-1/2 Monte Carlo convergence rate as simulation budget grows.The rate is presented as a benchmark achieved by classical pathwise and likelihood-ratio estimators when applicable.Source: §5 · p. 17

5.1. Smoothness of the Expectation Function

• Even when sample performance is nonsmooth, the expectation can be β-times continuously differentiable if the density derivatives satisfy the stated smoothness and integrable-bound conditions.If the conditions hold for every integer β≥1, the expectation is infinitely differentiable.Source: §5.1 · pp. 18–19

5.2. A General Convergence Principle

• Interpolation inequalities connect fitting error to derivative error, while L2 measures integrated error and L∞ controls it uniformly across the parameter domain.The transfer result requires a fitted-derivative norm bound; it applies when the fitted surface and expectation have the required smoothness.Source: §5.2.1 · pp. 19–20, §5.2 · p. 19, §5.2.2 · p. 20

5.3. Representative LTD Estimators

• The framework recovers established derivative-rate results for kernel regression, local polynomial regression, and kernel ridge regression, and yields new guarantees for multiple kernel learning and smooth neural networks.For the new guarantees, the common transfer argument uses each method’s surface-fitting rate and suitable smoothness control.Source: §5.2.2 · p. 20, §5.3 · p. 21

5.2.2. Transferring Surface Rates to Derivative Rates.

• The transfer theorem separates learning from differentiation: the learning method supplies the surface rate, while a fitted-derivative norm bound determines the derivative rate.The paper notes that the results apply beyond weighted representations when the stated conditions hold.Source: §5.2.2 · pp. 20–21
• Differentiation can weaken a surface-fitting rate: the guaranteed exponent loss is (β−r)/β, with greater derivative order increasing the loss and greater smoothness reducing it.The transferred rate also holds at every fixed parameter value under the stated conditions.Source: §5.2.2 · pp. 20–21

5.3.1. Recovering Existing Convergence Results.

• Kernel regression’s L2 and L∞ derivative rates match Stone’s minimax rates, and its uniform rate agrees with Hansen (2008).The section’s transfer analysis applies under the stated method-specific conditions.Source: §5.3.1 · p. 22

Appendix EC.3.1). Applying Theorem 2 gives

• Both the L2 and L∞ derivative rates attain Stone’s minimax rates.The uniform derivative rate also agrees with Hansen (2008).Source: Appendix EC.3.1). Applying Theorem 2… · p. 22
• Kernel regression’s uniform derivative rate agrees with the result established by Hansen (2008).Source: Appendix EC.3.1). Applying Theorem 2… · p. 22
• The supplied result identifies Stone’s rates as the benchmark for both derivative-error criteria.Source: Appendix EC.3.1). Applying Theorem 2… · p. 22

Appendix EC.3.2). Applying Theorem 2 gives

• Both L2 and L∞ derivative rates for local polynomial regression attain Stone’s minimax rates.The uniform rate agrees with Neumeyer and Van Keilegom (2010, Lemma A.1).Source: Appendix EC.3.2). Applying Theorem 2… · p. 22
• LTD differentiates the fitted local-polynomial intercept as a function of the query point, which need not equal the conventional derivative estimator from local-polynomial coefficients.Source: Appendix EC.3.2). Applying Theorem 2… · p. 22
• For kernel ridge regression, the passage considers a Matérn kernel when β>d/2 and says suitable kernel smoothness and regularization yield an L2 surface-fitting rate.The supplied passage does not state that rate’s value.Source: Appendix EC.3.2). Applying Theorem 2… · p. 22

Appendix EC.1.3:

• The common argument recovers established convergence-rate exponents across learning methods, including higher-order derivatives within applicable smoothness ranges.For kernel ridge regression, the L2 derivative rate agrees with Fischer and Steinwart (2020); for d=1, Liu and Li (2023) obtain sharper uniform rates under specific assumptions.Source: Appendix EC.1.3 · p. 23

5.3.2. Deriving Further Convergence Guarantees.

• Multiple kernel learning gains derivative guarantees from its surface-fitting rate; when its largest component dimension d* is below d, the component structure mitigates dimensionality, though the uniform bound still involves d.Source: §5.3.2 · p. 23
• For smooth neural networks, a Park approximation result combined with a least-squares oracle inequality supports a surface-fitting guarantee, which yields gradient and higher-order derivative guarantees through the framework.The guarantee uses the stated fitting and smoothness conditions.Source: §5.3.2 · p. 24

5.3.3. Smoothness, Dimensionality, and Statistical Efficiency.

• Across the five methods, the L2 derivative-rate exponent is (β−r)/(2β+d); increasing dimension slows convergence, while increasing smoothness improves it and increasing derivative order reduces it.Source: §5.3.3 · p. 24
• With sufficient smoothness and corresponding approximation and regularity conditions, the rate exponent approaches 1/2 for fixed r and d, matching the canonical Monte Carlo rate.The paper cautions that its tuning choices describe asymptotic regimes, not implementation prescriptions; finite-sample variance, stability, and computational cost also matter.Source: §5.3.3 · pp. 24–25

6.2. Wireless Communication Network

• The wireless example estimates gradients for transmit powers and antenna angles under black-box access, where sample-path derivatives and likelihood-ratio scores are unavailable.Finite differences serve as the classical comparator among the methods considered.Source: §6 · p. 25, §6.2 · p. 28

6. Numerical Examples

• The examples use five surrogate methods, 50 independent replications, and 100 fixed evaluation points; reference derivatives assess accuracy only.The Asian-option example provides closed-form prices and first- and second-order derivatives, while the wireless example uses only simulation outputs for estimation.Source: §6 · p. 25

6.1. Geometric Asian Option

• As monitoring frequency increases, LR errors rise substantially, especially for Gamma, while LTD errors are generally less sensitive; Gamma errors exceed Delta errors across methods.Among LTD methods, MKL generally performs best in one dimension, while NN has the lowest Delta and Gamma errors for d=2, 3, and 4.Source: §6.1 · pp. 26–27, Table 1 · p. 27, Table 2 · p. 28
• In the Asian-option comparison, larger offline budgets generally improve LTD accuracy, while pathwise estimators are more accurate than all five LTD methods at the smallest LTD budget.A fitted LTD surface supplies both Delta and Gamma estimates at new initial prices without further simulation.Source: §6.1 · pp. 26–27

6.2. Wireless Communication Network

• Wireless LTD errors increase across d=2, 4, and 6, and the most accurate LTD method varies with application, dimension, and simulation budget.KR has the largest overall error; KRR and MKL are most accurate overall at d=4 and d=6.Source: §6.2 · p. 29, Table 3 · p. 29

7. Conclusion

• For surrogates with weighted representations, LTD differentiates a learned representation of the probability measure while holding the performance function fixed, and surface-fitting accuracy yields gradient and higher-order derivative guarantees.The framework recovers results for kernel regression, local polynomial regression, and kernel ridge regression, and establishes additional guarantees for multiple kernel learning and smooth neural networks.Source: §7 · p. 30

EC.1. Representative Surrogate Learning Methods

• The appendix analyzes kernel regression, local polynomial regression, kernel ridge regression, multiple kernel learning, and smooth neural networks, including a neural-network surface-fitting rate derived from approximation and a least-squares oracle inequality.It combines surface-fitting rates with higher-order derivative bounds to obtain LTD convergence rates.Source: EC.1. Representative Surrogate… · p. 34
• With tuning parameters fixed in advance, the first three methods have response-independent weights; response-selected tuning, multiple kernel learning, and qualifying jointly trained networks can yield response-adaptive weights.When differentiating with respect to the query point, the training data and tuning parameters are held fixed; MKL likewise holds its learned kernel combination fixed, while the neural-network representation requires stated output-layer conditions.Source: EC.1. Representative Surrogate… · p. 34, EC.1.4. Multiple Kernel Learning · p. 38, EC.1.5. Smooth Neural Networks · p. 39

EC.2. Gagliardo–Nirenberg Interpolation Inequalities

• The appendix presents a bounded-domain Gagliardo–Nirenberg inequality that allows different norms for a function and its derivatives, then derives four special cases used in the main analysis.The four cases are related to the interpolation inequalities used in Lemma 1.Source: EC.2.2. A General Interpolation… · p. 41, EC.2.3. Derivation of the Four Special… · p. 42

EC.3. Derivative Norm Bounds for Representative Surrogate Learning Methods

• The derivative-bound section targets O_p(1) bounds on fitted-surrogate derivatives through order β for kernel regression, local polynomial regression, kernel ridge regression, multiple kernel learning, and smooth neural networks.Its stated conditions include a bounded design density bounded away from zero, with additional moment, kernel-regularity, and bandwidth conditions for uniform bounds on the local methods.Source: EC.3. Derivative Norm Bounds for… · p. 42

EC.3.1. Kernel regression

• For kernel regression, integration by parts and a denominator bounded away from zero control population derivatives, while stochastic derivatives incur a factor of h^-1 per derivative.Integrated second moments and, for uniform bounds, a maximal inequality with additional tail assumptions are used through order β.Source: EC.3.1. Kernel regression · p. 43

EC.3.2. Local polynomial regression

• Under regular local designs and rapid kernel decay, local polynomial regression attains O_p(1) derivative norms at the stated bandwidth rates.The rates are h_2 ≍ n^-1/(2β+d) and h_∞ ≍ (log n/n)^(1/(2β+d)).Source: EC.3.2. Local polynomial regression · pp. 43–44

EC.3.3. Kernel ridge regression

• For the Matérn kernel under the stated design conditions, the effective-dimension bound scales as λ^-d/(2β).The bound uses population covariance eigenvalues μ_j ≲ j^-2β/d.Source: EC.3.3. Kernel ridge regression · pp. 44–45

Appendix EC.1.4, with 𝑑∗:= max𝑚𝑑𝑚and 𝛽> 𝑑∗/2. Assume a fixed decomposition 𝐽= Í

• For a fixed smooth component decomposition with β > d*/2, the sum-kernel effective dimension is bounded by the sum of component effective dimensions.The proof uses component norm bounds and an operator-concavity argument for KRR.Source: Appendix EC.1.4, with 𝑑∗:=… · pp. 45–46

EC.4.1. Benchmark methods

• The geometric Asian option’s exact price and derivatives provide evaluation benchmarks, while likelihood-ratio weights from the monitored path extend to arithmetic Asian payoffs.The LR construction uses the full monitored asset path as the state; its factors involving Δt help explain why variance can increase with monitoring frequency.Source: EC.4.1. Benchmark methods · pp. 47–48

EC.4.2. LTD estimator

• The LTD comparison fits portfolio price from simulated responses without giving learners its additive structure, and tunes on each replication’s responses before differentiating with parameters fixed.KR and LPR fit locally, whereas KRR, MKL, and neural networks fit global models.Source: EC.4.2. LTD estimator · p. 48

EC.4.2.1. KR and LPR.

• KR and LPR differentiate their fitted local response surfaces; LPR differentiates the intercept while accounting for changes in local weights and fitted coefficients.This surface derivative can differ from estimates obtained directly from polynomial coefficients; bandwidth safeguards and a small ridge adjustment help stabilize derivatives and fits.Source: EC.4.2.1. KR and LPR · pp. 48–50

EC.4.2.2. KRR and MKL.

• MKL combines three Gaussian scales, constrains each weight to at least 1/6, and normalizes kernels for function and derivative scales before fitting and differentiation.Response-based GCV selects parameters; normalization is included in both the regression fit and derivative calculation.Source: EC.4.2.2. KRR and MKL · pp. 50–52
• For KRR and MKL, GCV includes a penalty for fitted-value sensitivity to responses; their sensitivity matrices are respectively the response smoother and a local Jacobian accounting for mixture-weight changes.Both matrices include the fitted mean.Source: EC.4.2.2. KRR and MKL · p. 52

EC.4.2.3. Smooth NN.

• The smooth neural network minimizes response error subject to bounds on its fitted function and derivatives, which are computed from the network without derivative observations.GCV chooses initialization parameters, while held-out response error selects the number of training rounds; the final fit is repeated on all training sites.Source: EC.4.2.3. Smooth NN · p. 54
• Finite-point checks and local optimization mean the neural-network fit is only a numerical approximation and does not certify the global Sobolev bound assumed by the theory.The optimization may not reach the global least-squares minimum, and constraint checks cover only finitely many points.Source: EC.4.2.3. Smooth NN · p. 54

EC.4.3. Parameter settings

• The parameter settings balance spatial coverage and simulation noise by dividing each budget as evenly as possible among sites, with global methods using Halton designs and KR/LPR using Cartesian midpoint observations.All methods use a training box extending beyond the test region.Source: EC.4.3. Parameter settings · p. 55

EC.5.1. Simulation Model Building

• The wireless example is a semi-analytical black-box simulator combining distance-based path loss and shadowing with directional antenna gains to generate a user’s signal-to-interference-plus-noise response.The gradient target is the expected response over user locations and shadowing.Source: EC.5.1. Simulation Model Building · pp. 55–56

EC.5.2. Benchmark method

• The central finite-difference benchmark uses specified perturbation half-steps, while reference gradients use quarter-sized half-steps and more observations but retain Monte Carlo and finite-difference error.Common random numbers reduce noise within each reference pair.Source: EC.5.2. Benchmark method · p. 57

EC.5.3. LTD estimator

• All five LTD estimators fit the full active inputs and simulated responses without internal simulator knowledge, with inputs normalized to [-1,1] and neural-network derivatives constrained through order three or four.The neural-network constraints and local optimization retain the limitations described for that numerical approximation.Source: EC.4.2.3. Smooth NN · p. 54, EC.5.3. LTD estimator · p. 57

EC.5.4. Parameter settings

• Wireless parameter choices define an illustrative six-variable test problem, with lower-dimensional experiments varying powers or angles and test ranges inset so finite-difference perturbations remain within the training box.The fixed settings produce directional variation and meaningful output noise; training-site designs and counts vary by estimator family.Source: EC.5.4. Parameter settings · p. 58, Table E.3 · p. 58