Offline Deep Q* Estimation with Diffusion Models

1 Introduction

• Offline RL shifts the central challenge from exploration to reliable value or policy estimation using limited data generated by an unknown behavior policy.Online interaction may be costly, unsafe, or infeasible.Source: §1 · p. 2
• Existing direct value-learning methods can propagate recursive approximation errors, induce overestimation bias, require costly repeated optimization, and often rely on Bellman completeness.These limitations motivate separating environment modeling from value estimation.Source: §1 · p. 3
• The proposed framework estimates conditional reward laws and transition kernels with diffusion models, constructs a data-driven optimal Bellman operator, and minimizes empirical Bellman residuals over neural networks.This decouples operator estimation from value learning and enables end-to-end analysis of generative and value-estimation errors.Source: §1 · pp. 4–5
• The analysis quantifies how conditional reward and transition estimation errors propagate through the optimal Bellman operator to Q* estimation.The framework first derives diffusion-estimator rates, then analyzes their effect on value learning.Source: §1 · pp. 4–5
• The excess Bellman residual risk is O~(n^-2β/(d_x+d_a+2β)), while under concentrability the Q* estimator has L2 rate O~(n^-β/(d_x+d_a+2β)) without Bellman completeness.Here d_x and d_a are state and action dimensions, and β is the Hölder smoothness index of the target value function.Source: §1 · p. 5
• Experiments on the MIMIC-III clinical dataset compare the method with MOPO, DFQI, and MABO using training and evaluation rewards, supporting the proposed separation.The supplied introduction identifies empirical benefit but provides no numerical comparison values.Source: §1 · p. 5

1.1 Contribution

• Conditional diffusion models estimate reward laws and transition kernels, which define a data-driven optimal Bellman operator for neural Q* estimation.The estimator minimizes empirical Bellman residuals over a DNN class.Source: §1.1 · p. 6
• Theoretical analysis derives sharp total-variation diffusion-estimation rates and oracle value-stage guarantees without Bellman completeness assumptions.It explicitly characterizes propagation of generative estimation error through the Bellman operator and its interaction with value-estimation error.Source: §1.1 · p. 6
• n^(-2β/(d_x+d_a+2β)) Bellman-residual convergence and n^(-β/(d_x+d_a+2β)) L2(Q*) convergence follow under regularity and concentrability conditions.The Bellman residual rate depends on operator-learning error, while the L2 guarantee requires concentrability.Source: §1.1 · p. 6

1.2 Related Work

• Prior offline RL methods use recursive fitted targets, model-based rollouts, or minimax Bellman objectives, whereas this work separates environment-law estimation from value learning and analyzes excess Bellman loss.The framework targets Bellman-operator estimation from reward laws and transition kernels rather than policy optimization with learned rollouts.Source: §1.2 · pp. 7–8
• Unlike prior diffusion methods focused on policy learning, trajectory planning, or alternative statistical targets, this work estimates one-step conditional reward laws and transition kernels for Q* estimation.The closest related diffusion world model jointly generates multistep future states and rewards, whereas this work uses one-step environment-law estimates in the optimal Bellman operator.Source: §1.2 · pp. 9–10

1.3 Outlines

• The paper is organized around preliminaries, the proposed method, theoretical results, numerical experiments, and a conclusion, with proofs deferred to the Appendix.Sections 2–6 respectively cover these components.Source: §1.3 · p. 10

2.1 Markov Decision Process

• The paper models a discounted MDP as (X, A, P, R, γ), with continuous state-action spaces, reward distributions, transition kernels, and optimal Bellman equation Q*=T*Q*.The setting assumes X × A = [0, 1]^dx × [0, 1]^da and uses three independent offline datasets for operator and value learning.Source: §2.1 · pp. 10–11

2.2 Diffusion Models

• Conditional diffusion models learn covariate-dependent score functions so reverse-time SDE sampling generates samples from target conditional distributions.The forward process transforms data toward a tractable prior, while score matching estimates the conditional score for reverse generation.Source: §2.2 · pp. 12–13

2.3 Deep Neural Networks

• ReLU DNNs are specified by affine layers, ReLU activations, depth, width, parameter size, weight bounds, and uniform function bounds.The section also introduces Hölder classes, total variation distance, covering numbers, and admissibility concepts used in the theoretical analysis.Source: §2.3 · pp. 13–14

2.4 Notations

• The notation section defines asymptotic order symbols, logarithm-suppressed rates, integer rounding, extrema, vector norms, and Lq norms.These conventions support the rate statements and functional analysis used later in the paper.Source: §2.4 · p. 15

3 Method

• Conditional diffusion models estimate the unavailable reward law and transition kernel from offline data, inducing a data-driven approximation of the optimal Bellman operator.The framework uses separate batch datasets Sr and Sx for reward and transition learning, respectively.Source: §3 · p. 16

3.1 Optimal Bellman Operator Learning

• The diffusion learner trains a conditional score estimator and applies explicit Euler–Maruyama reverse-time discretization to generate approximate conditional samples, with Gaussian initialization and domain clipping.The unknown score makes direct reverse-process simulation intractable; score matching estimates it, while the terminal conditional law is replaced by N(0, I_dx) for practical sampling.Source: §3.1 · pp. 17–18

3.2 Value Function Learning

• The value stage samples m rewards and next states from the learned conditional models, constructs an empirical γ-contraction Bellman operator, and fits a bounded ReLU DNN estimator by minimizing its Bellman residual.The same generated reward and next-state samples are reused across candidate Q-functions, and the maximized next-state values are averaged before forming the Bellman backup.Source: §3.2 · pp. 19–20
• For continuous actions, numerical action optimization may use discretization or gradient-based methods, while the analysis assumes exact maximization and excludes the resulting optimization error.Source: §3.2 · p. 21
• The proposed method uses learned reward and transition laws directly to construct the optimal Bellman operator, unlike model-based methods that use models for rollouts or conservative optimization.Source: §3.2 · p. 21
• After training conditional diffusion models, Q* is estimated by minimizing the empirical Bellman residual with respect to the learned operator.Source: §3.2 · p. 21
• Conditional diffusion models flexibly represent complex reward and transition laws, including continuous and potentially multimodal conditional distributions.Source: §3.2 · p. 21
• The approach does not add model-generated rollout trajectories after fitting the conditional diffusion models.Source: §3.2 · p. 21

4 Theoretical Analysis

• The value-function error is controlled by the Bellman residual under a randomized comparison kernel, with the bound depending on distribution mismatch and δ-greedy comparison sets.The analysis reduces value estimation to controlling the Bellman residual while accounting for the mismatch quantified by C_δ.Source: §4 · p. 22
• The error decomposition separates conditional diffusion estimation errors in the reward and transition models from statistical and approximation errors in value learning.Parts I and II capture discrepancies between true and learned Bellman operators, while Part III contains value-learning statistical and approximation errors.Source: §4 · p. 22

4.1 Error Bound for Optimal Bellman Operator Learning

• The operator-learning analysis decomposes total-variation error into score estimation and EM discretization, initialization replacement, and time-truncation components under density regularity assumptions.Coordinatewise clipping does not enlarge the total-variation bounds, and the analysis treats the resulting components separately.Source: §4.1 · p. 23

4.1.1 Bound ESx,T,Z,W

• The score-estimation theorem accounts for time discretization in the discrete-time regime and achieves the nonparametric minimax-optimal rate up to logarithmic terms.The result contrasts with analyses confined to continuous time and explicitly incorporates discretization effects.Source: §4.1.1 · p. 26

4.1.2 Bound ESx,T,ZEW

• The initialization-replacement error is controlled through data processing, KL divergence, and Pinsker’s inequality, while time-truncation error is controlled using bounded support and Hölder continuity assumptions.The time-truncation analysis concerns the discrepancy between the intermediate and target conditional distributions at the stopping time.Source: §4.1.2 · p. 27, §4.1.3 · p. 28

4.1.3 Bound EW

• Theorem 4.2 and Corollary 4.1 provide end-to-end error guarantees for diffusion-based transition-kernel and reward-law estimation, explicitly accounting for reverse-process numerical discretization.These bounds combine initialization, time-truncation, and diffusion-estimation errors before deriving the optimal Bellman-operator bound.Source: §4.1.3 · pp. 28–29

4.2 Error Bound for Value Function Learning

• Value-learning error is decomposed into statistical and approximation terms, with offset Rademacher complexity yielding a fast statistical rate for ReLU DNNs.The analysis assumes i.i.d. batch data and approximates Q* through its DNN class under a Hölder regularity assumption.Source: §4.2 · pp. 29–31
• The approximation analysis avoids Bellman completeness because its error reduces to approximating Q* directly within the DNN class rather than approximating Bellman updates.This contrasts with recursive value-learning methods whose approximation guarantees typically require Bellman updates to remain in, or be well approximated by, the function class.Source: §4.2 · p. 31

4.3 Main Results

• Theorem 4.5 combines operator-estimation, approximation, and statistical errors to bound the excess Bellman residual of Q̂ under the sampling distribution μ.The analysis then introduces a concentrability condition to compare μ with the target distribution ρ and translate residual risk into direct estimation error.Source: §4.3 · p. 32
• Under concentrability, Theorem 4.6 converts excess Bellman residual control into an L2 value-function error bound, yielding the oracle rate n^-β/(d_x+d_a+2β).The bound follows from the residual inequality and the choice δ = n^-β/(d_x+d_a+2β).Source: §4.3 · p. 33
• The method avoids Bellman completeness by controlling approximation error through direct approximation of the Hölder-continuous fixed point Q*, rather than requiring the whole function class to be Bellman complete.This replaces containment or approximation of T*F with approximation of Q* itself.Source: §4.3 · pp. 33–34
• With total offline budget en, Theorem 4.7 gives an achievable rate governed by min{2α/(2d_x+d_a+2α), 2β/(d_x+d_a+2β)} up to logarithmic factors and constants.The allocation sets n = ω_n en, n_r = ω_r en, and n_x = ω_x en for fixed positive proportions.Source: §4.3 · p. 34
• Theorem 4.8 establishes the information-theoretic lower bound en^-max{α/(d_x+d_a+2α), β/(d_x+d_a+2β)} for direct Q* estimation over the considered MDP class.Unlike Theorem 4.7, this lower bound permits arbitrary estimators and does not require recovering conditional environment distributions.Source: §4.3 · pp. 34–35
• The achievable rate reflects the statistical cost of first estimating reward and transition distributions, including transition estimation over the joint space (x, a, x′) of dimension 2d_x+d_a.The lower bound instead characterizes intrinsic value-estimation difficulty without the intermediate environment-modeling requirement.Source: §4.3 · p. 35

5 Numerical Experiments

• The experiments use Pendulum and Swimmer benchmarks plus MIMIC-III sepsis treatment, comparing the method with MOPO, DFQI, and MABO.Gymnasium data use random-policy rollouts, while MIMIC-III provides clinician-action outcomes and a fixed offline training set evaluated with a personalized simulator.Source: §5 · pp. 35–36

5.1 OpenAI Gymnasium Environments

• 71.90 versus 39.33 for MABO was the evaluation reward on Swimmer, giving the proposed method the best final performance among the compared methods.Swimmer tests coordinated multi-joint control, and the method maintained a relatively high reward during training.Source: §5.1 · p. 38, Table 1 · p. 39
• In Pendulum, all methods reached stable training rewards, but the proposed method achieved the best evaluation reward while DFQI’s evaluation performance was substantially lower.MABO and MOPO also improved during training but obtained lower final evaluation rewards.Source: §5.1 · p. 37, Table 1 · p. 39

5.2 Real-World Data

• The final processed cohort contains approximately 12,600 patients, with an 11-dimensional state representation, 9 discrete treatment actions, and reward rt = −SOFAt+1, where larger rewards indicate lower subsequent organ dysfunction.The random policy obtains an average reward of −8.27.Source: §5.2 · pp. 40–41
• −5.33 was the proposed method’s highest final reward on MIMIC-III, outperforming the clinician policy at −5.59 and also MOPO, MABO, and DFQI.Because reward is defined as negative future SOFA, this indicates lower predicted organ dysfunction; the method also achieved the best simulated return among learned policies.Source: §5.2 · p. 41
• Evaluation rewards are summarized over 150 returns using a box plot, with whiskers spanning the min–max range.Source: §5.2 · p. 41
• Evaluation results are reported for the MIMIC-III sepsis treatment task.Source: Table 2 · p. 41

6 Conclusion

• The framework learns reward laws and transition kernels with conditional diffusion models, then estimates Q* by minimizing the resulting empirical Bellman residual.This separation decouples conditional generative modeling from value learning and enables analysis of Bellman-operator error propagation.Source: §6 · p. 42
• The method avoids the Bellman completeness assumption commonly used in existing deep reinforcement-learning theory.The separation between operator estimation and value learning is identified as enabling this theoretical simplification.Source: §6 · p. 42
• O(n^-2β/(d_x+d_a+2β)) is the excess Bellman residual risk rate, while Q-hat attains O(n^-β/(d_x+d_a+2β)), matching the oracle value-stage rate.These rates ignore logarithmic factors and concern the residual risk and value-function estimation error, respectively.Source: §6 · p. 42
• Numerical experiments support the theoretical findings and demonstrate the proposed method’s practical effectiveness.Source: §6 · p. 42

Appendix

• The appendices organize the technical foundations into score-function bounds, optimal Bellman-operator learning bounds, value-function results, and auxiliary approximation lemmas.Appendix A develops the score analysis, Appendix B treats operator learning, Appendix C proves value-learning results, Appendix D proves the main theorem, and Appendix E collects auxiliary results.Source: Appendix · p. 42

A Bounds on the Score Function

• Score-function approximation localizes the integration domain, uses Taylor expansions for densities and Gaussian kernels, and approximates the resulting polynomial terms with ReLU networks.The appendix also establishes high-probability bounds for the conditional density and score function, while local-polynomial density approximation achieves error O(N^-α).Source: A Bounds on the Score Function · p. 44

A.3 Approximating of p(x|w) via Local Polynomials

• Under the stated assumptions, local-polynomial integrals approximate p_t(x|w) and σ_t∇p_t(x|w), extending the density approximation construction to diffusion quantities.Source: §A.3 · pp. 53, 56, 57, §A.4 · p. 57

A.4 Approximating of pt(x|w) and σt∇pt(x|w) via Local Polynomial Integrals

• Local polynomial expansions replace the exponential integrals in g2 with tractable approximations, while bounding the gaps from pt and g1 through intermediate functions g2 and g3.The argument uses Taylor approximation with k = O(log ϵ−1) and derives successive error bounds before approximating g3.Source: §A.4 · pp. 59–60

A.5 Approximating of Local Polynomial Integrals via ReLU Neural Networks

• ReLU networks are introduced to approximate the derived local polynomial integrals on the bounded domain ∥x∥∞≤C0.The construction first approximates the OU coefficients and then combines the resulting functional components into networks for the local integrals.Source: §A.5 · pp. 62–63

Its parameters satisfy

• The resulting ReLU constructions approximate ψ uniformly with error ≲ϵ0 and extend to Ψ through product and local-weight subnetworks, with local weights represented exactly.The bounds control component errors using clipped factors and preserve the approximation target after combining state- and w-dependent terms.Source: Its parameters satisfy · pp. 64, 66, 72, 73
• ReLU networks approximate the function in (30), with the resulting error bounded as stated in the construction.The construction combines local networks and accounts for the OU schedule approximation.Source: Its parameters satisfy · pp. 73–74
• Parallelization of ReLU networks yields a vector network h_R(t, x, w) with components h_R,1 through h_R,d_x that approximate the target vector function.The construction applies the componentwise approximation lemma and uses ReLU parallelization.Source: Its parameters satisfy · p. 76
• The vector-valued network h_R approximates (31), while increasing the antiderivative exponent by one does not change network-complexity bounds.The exact representation of η_n,N(w) adds no approximation term.Source: Its parameters satisfy · pp. 75–76

ral Networks

• The section constructs two ReLU neural networks to approximate the true score ∇log p_t(x|w), with one stated to work for all t ∈[2N −1, T].Source: ral Networks · pp. 76, 84
• The first interval’s approximation proof decomposes the error into three terms and establishes Term III ≲ N^-2α log N, completing Lemma A.12.The construction uses input clipping and separate bounds over regions determined by the density tail behavior.Source: ral Networks · pp. 76, 78, 82, 84
• The two interval-specific networks are assembled by switching weights into a piecewise ReLU network without changing the stated complexity orders.The switching and product networks add only O(log N) depth and size per coordinate, which remain dominated by the original complexities.Source: ral Networks · pp. 88–89

Error decomposition

• The auxiliary-loss analysis separates the statistical error from the approximation error before deriving an upper bound for the statistical component.The decomposition is obtained after taking expectations and infima over the neural-network class G.Source: §A.7 · pp. 89–90

Lipschitz continuity

• Lipschitz continuity of the loss is established for restricted ReLU networks in G by comparing arbitrary s1, s2 ∈ G through their construction structure.This regularity supports subsequent empirical-process control over the network class.Source: §A.7 · p. 91

Covering number evaluation

• The neural-network class is controlled through a covering-number evaluation with depth D = O(log^4 N), width W = O(N^(d_x+d_w) log^7 N), and sparsity S = O(N^(d_x+d_w) log^9 N).These architectural bounds determine the complexity terms entering the statistical-error estimate.Source: §A.7 · p. 92

Concentration analysis

• Bernstein’s inequality controls the centered loss process using logarithmic envelope and variance bounds, yielding a high-probability estimate for the empirical score-function error.The proof introduces a variance-dependent threshold and selects δ and ϵ0 in terms of the sample size and network complexity.Source: §A.7 · pp. 93–95

Three-term bound

• The remaining error is decomposed into three terms, which are bounded separately using approximation arguments and Hoeffding’s inequality before combining the resulting estimates.The final displayed bound includes a logarithmic complexity factor and dependence on m_x through the term 13/2 N(log^3 N + log^2 N log m_x + log^3/2 m_x)√m_x.Source: §A.7 · pp. 95–98

A.8 Error Bound for the Score Function

• Lemmas 4.2 and 4.3 are combined to prove Theorem 4.1.Source: §A.8 · p. 98
• The score-function approximation error is bounded in terms of d_x, d_w, α, n_x, and a logarithmic factor.Source: §A.8 · p. 98
• The displayed bounds distinguish approximation error from the final statistical error obtained through the theorem-proof sequence.Source: §A.8 · pp. 98–99
• The final statistical error bound is derived after applying the stated parameter substitutions to equations (40)–(41).The substitutions involve C_µ = 2^α/(α∧1) and C_λ = α.Source: §A.8 · pp. 98–99
• The resulting statistical error rate depends on d_x, d_w, α, n_x, and log^1/8 n_x.Source: §A.8 · p. 99
• The derivation concludes with the proof declared complete.Source: §A.8 · p. 99

B.1 Proof of Lemma 4.4

• Lemma 4.4 compares the true reverse-process path law with continuous-time EM interpolation using common diffusion coefficients and stopped drift differences.The proof invokes Lemma E.3 up to a stopping time, then uses lower semicontinuity of relative entropy and monotone convergence.Source: §B.1 · pp. 99–100
• The path-law comparison yields total-variation control through the data-processing and Pinsker inequalities.Both processes share initialization and diffusion coefficient, enabling the terminal-distribution comparison.Source: §B.1 · p. 100
• For interior times, Gaussian convolution identities, Jensen’s inequality, OU representations, and score regularity control drift differences along the discretized path.The argument uses positive noise scales away from the endpoints and bounded-support properties.Source: §B.1 · p. 100
• The proof bounds the terminal-distribution discrepancy between pT1 and epT1 using the shared initialization and diffusion structure.The resulting terminal distributions are explicitly identified after applying Pinsker’s inequality.Source: §B.1 · p. 100
• Moment bounds for OU paths and score terms are established using Lipschitz estimates, Jensen’s inequality, Cauchy–Schwarz, and Burkholder–Davis–Gundy inequalities.These estimates cover q ∈ {2, 4, 6} and support the preceding discretization bounds.Source: §B.1 · pp. 101–104
• The discretization analysis decomposes score residuals into integrated score error and within-step variation, then controls higher-order mesh terms under a deterministic threshold.For sufficiently small Δ, terms involving Δ^2, Δ^3, and Δ^8 are absorbed into the leading term.Source: §B.1 · p. 102

B.2 Proof of Lemma 4.5

• Lemma 4.5 begins from convexity of KL divergence together with Lemma E.2.The proof then converts the resulting divergence bound into a total-variation bound using Pinsker’s inequality.Source: §B.2 · p. 105

• Assumption 4.2 provides the bound used for the first term involving µ_T1.Source: §B.3 · p. 106
• The proof bounds Term II from the fact that p_T1(·|w) is a probability density and factors 1 − µ_T1^d_x through 1 − µ_T1.Source: §B.3 · p. 107
• Assumption 4.1 controls the boundary Term III and Term B; combining the preceding inequalities establishes the result for every w.Source: §B.3 · pp. 107–108

B.4 Proofs of Theorem 4.2 and Corollary 4.1

• Theorem 4.2 follows by applying Lemmas 4.4–4.6 to the three terms and replacing the estimated transition law with the true kernel.This substitution yields the stated bound.Source: §B.4 · p. 108
• The reward-law estimation error is obtained by the same argument as Theorem 4.2, with the next-state variable replaced by the reward variable.The proof explicitly identifies this variable replacement as the key adaptation.Source: §B.4 · pp. 108–109

B.5 Proof of Theorem 4.3

• Theorem 4.3 uses an epsilon-cover of the function class, Hoeffding’s lemma, and integration of the resulting tail bound to control Monte Carlo fluctuations.The proof bounds each function by a covered representative before applying concentration.Source: §B.5 · p. 110
• For the ReLU DNN class, the proof substitutes its size, depth, width, and weight bounds into the preceding estimates to obtain equations (12), (13), (53), and (54).The resulting estimates are then combined with Theorem 4.2 and Corollary 4.1 to complete the proof.Source: §B.5 · pp. 110–112

C.1 Proof of Lemma 4.7

• The proof of Lemma 4.7 bounds statistical error with offset Rademacher complexity after establishing pointwise control from Lemma C.1.The loss is bounded using the reward bound R_max and the discount factor.Source: §C.1 · pp. 112–113
• The Lemma 4.7 argument replaces expectations with a ghost-sample empirical mean and approximates the class using an infinity-norm covering set.The covering number is denoted N(Q, δ, ∥· ∥∞), with representatives chosen within the cover.Source: §C.1 · pp. 113–114
• Conditional on the data, independent Rademacher variables and Hoeffding’s inequality yield a uniform concentration bound over covered functions.The proof then sets A = log N and applies Lemma E.5 with η = 1/(3C) to derive the statistical-error bound.Source: §C.1 · pp. 114–115

Contraction argument

• Lemma C.1 shows that the empirical optimal Bellman operator bT* is a γ-contraction with respect to the supremum norm.The argument considers arbitrary measurable Q1 and Q2 bounded between 0 and Rmax and uses shared samples at each (x, a).Source: §C.2 · p. 116

Theorem conclusion

• Theorem 4.4 follows by incorporating the statistical error bound (19) with the approximation error bound (55).The proof concludes after combining these two error contributions.Source: §C.2 · p. 116

Theorem 4.5

• The main theorem follows by combining the Lemma 4.1 error decomposition with Theorems 4.3–4.4, yielding the stated estimator guarantee.The proof conditions on the learned operator and uses the independent value sample to control the empirical value stage.Source: D Main Results · pp. 117–118

Theorem 4.6

• Under the positive-measure and density-ratio assumptions, the L2 error is bounded by √Cδ/(1−γ) times Bellman error plus γδ/(1−γ).The result is obtained by comparing Q and Q* through policy regions and applying the density-ratio condition.Source: D Main Results · pp. 118–119

Theorem 4.7

• Choosing δ = δ_n := n^−β/(d_x+d_a+2β) and substituting the operator and value-stage bounds proves the final upper-rate theorem.The preceding choices also yield m ≲ n^−2β/(d_x+d_a+2β), while the bound includes a term proportional to n^−2α/(2d_x+d_a+2α) log n.Source: D Main Results · p. 120

Theorem 4.8

• The lower-bound construction uses disjoint smooth bumps to build MDPs whose reward densities and optimal values remain in the prescribed Hölder classes.The constructed reward density satisfies fixed positive bounds, and the transition density is uniform.Source: D Main Results · pp. 121–122
• Theorem 4.8 establishes minimax difficulty by packing these MDPs with separated value functions while controlling information through independent reward, transition, and value samples.The construction uses Jh ≍ h^−(d_x+d_a) disjoint bumps and sets h proportional to nbar^−1/(d_x+d_a+2s).Source: D Main Results · pp. 123–125

E Auxiliary Results

• The auxiliary results state concentration, Gaussian-divergence, diffusion-measure, neural-network approximation, and information-theoretic lemmas supporting the theoretical analysis.They include results attributed to Boucheron et al. (2013), Jiao et al. (2025a, 2025b), Chen et al. (2022c), Feng et al. (2024), and Duchi (2026).Source: E Auxiliary Results · pp. 125–127
• The auxiliary lemmas also supply Hoeffding and Bernstein inequalities together with product-measure, testing, packing, and randomized-estimator results for lower-bound and risk arguments.These results include independent-observation divergence relations and separated parameter constructions.Source: E Auxiliary Results · p. 127

E.1 Construction of a Large ReLU Network

• ReLU networks can be concatenated to realize compositions and constructed to implement identity mappings, with additional parallelization for combining multiple subnetworks.The identity construction uses width W = 2d, size S = 2dD, and parameter bound B = 1.Source: §E.1 · p. 128

E.2 Approximation of Basic Functions with ReLU Networks

• The cited lemmas give neural-network approximators for products, reciprocals, and square roots, with product depth O(log d(log d + log ϵ^-1 + d log C)) and reciprocal and square-root depth O(log^2 ϵ^-1) and width O(log^3 ϵ^-1).Source: §E.2 · p. 129

E.3 Clipping and Switching Functions

• A ReLU clipping network maps each coordinate x_i to min{b_i, max{x_i, a_i}}, with scalar lower and upper bounds available as a special case.The construction is given for vectors a and b satisfying a_i ≤ b_i.Source: §E.3 · pp. 129–130
• Under ordered time intervals and separate approximation guarantees, switching networks combine two approximators while preserving their respective interval-wise accuracy.The switching-function construction applies coordinate-wise to vector-valued functions and uses networks with depth D = 3, width W = 2, and size S = 8.Source: §E.3 · p. 130