Generative Neural Networks for Sinkhorn Distributionally Robust Hypothesis Testing

Fenglin Zhang; Teyan Liu; Jie Wang

Source-linked AI summary

Generative Neural Networks for Sinkhorn Distributionally Robust Hypothesis Testing

Fenglin Zhang, Teyan Liu, Jie Wang

arXiv:2608.22746v1Aug 24, 2026stat.MLcs.LGmath.OC

TL;DR

SDRHT seeks robust detectors under Sinkhorn-based distributional ambiguity, but existing conic-program approaches do not scale well. The paper learns least-favorable distributions through convex potentials parameterized by HyCNNs, and reports superior accuracy and robustness across sample sizes and dimensions. The resulting framework provides a generative approach to robust hypothesis testing while retaining theoretical approximation and transport-map guarantees.

Problem
SDRHT must construct robust detectors under Sinkhorn-based ambiguity, while existing sample-average approaches produce large-scale conic programs whose cost grows rapidly with sample size.
Method
The paper reformulates SDRHT using conditional-KL representations, strong duality, and Brenier transport, then parameterizes convex potentials with HyCNNs for generative optimization and sampling.
Results
Numerical results demonstrate superior performance and robustness compared with existing methods across tasks with different sample sizes and dimensions.
Takeaways & Limitations
The framework offers continuous least-favorable distributions and a more tractable generative route for solving Sinkhorn distributionally robust hypothesis testing.
Takeaways & Limitations
The transport reformulation remains infinite-dimensional, and induced densities, inverse maps, and log-determinant terms are generally unavailable in closed form.

Abstract

from arXiv · show

This paper studies the Sinkhorn distributionally robust hypothesis testing (SDRHT) problem, seeking a robust detector against least-favorable distributions in Sinkhorn discrepancy-based ambiguity sets centered at the empirical distributions. Existing approaches solve this problem by solving large-scale conic programs, which are not scalable. To overcome this, we propose a generative framework that learns least-favorable distributions and supports efficient training and end-to-end sampling. For the Sinkhorn discrepancy-based ambiguity sets, we first derive an equivalent conditional-KL-divergence representation with respect to kernel-smoothed reference distributions. This property allows us to prove strong duality for both constrained and unconstrained minimax SDRHT formulations. Based on the closed-form optimal detector and Brenier's theorem, we reformulate the max-min dual formulation as a maximization problem over convex potentials whose gradients characterize invertible transport maps between kernel-smoothed distributions and their least-favorable counterparts. We efficiently approximate these potentials using Hyper Input Convex Neural Networks (HyCNNs) equipped with stochastic gradient estimators and prove the representation power of HyCNNs and the distributional universality of their induced transport maps. Numerical results show that the proposed method achieves superior accuracy and robustness across different sample sizes and dimensions, while avoiding the scalability limitations of classical SDRHT methods.

1 Introduction

The paper addresses distributional misspecification and scalability challenges in robust hypothesis testing by learning continuous least-favorable distributions through convex-potential transport maps.

Finite-sample tests face misspecification from noisy, limited data and distribution shifts, motivating reliable and computationally efficient data-driven tests.
Sinkhorn ambiguity sets encourage continuous estimates and permit robust testing beyond the empirical support, but existing sample-average methods yield costly large-scale conic programs.
The proposed framework learns continuous least-favorable distributions for SDRHT by optimizing convex potentials and their induced transport maps.
Strong duality and Brenier-based reformulation convert the infinite-dimensional minimax problem into optimization over convex potentials.
HyCNNs and stochastic gradient estimators make potential optimization finite-dimensional, while numerical studies report superior accuracy and robustness across sample sizes and dimensions.
On a toy Moons dataset, the learned least-favorable distributions induce a nonlinear robust decision boundary with high accuracy.

2 Sinkhorn Distributionally Robust Hypothesis Testing

This section formulates SDRHT over Sinkhorn-discrepancy ambiguity sets, establishes strong duality, and reformulates the problem through convex-gradient transport maps. The resulting robust detector uses least-favorable distributions but remains infinite-dimensional before generative approximation.

Problem formulation: The Sinkhorn discrepancy is an entropy-regularized Wasserstein variant defined through transport costs, couplings, and relative entropy.Its ambiguity-set construction is used to model distributional uncertainty around empirical distributions.
Problem formulation: SDRHT minimizes worst-case surrogate risk over detectors against distributions in Sinkhorn-discrepancy ambiguity sets.The ambiguity sets are centered on empirical distributions and provide a non-parametric characterization of least-favorable distributions.
Conditional-KL representation: Proposition 1 represents each Sinkhorn ambiguity set equivalently through a conditional KL divergence relative to kernel-smoothed reference distributions.The representation connects Sinkhorn-based ambiguity sets with KL-divergence ambiguity sets and establishes that the associated reference distributions are probability measures.
Strong duality: Theorem 1 proves strong duality for constrained and soft-constrained SDRHT under the stated transport-cost and loss-function assumptions.The proof uses weak compactness of the ambiguity sets and weak upper semicontinuity of expected loss rather than compactness of the detector space.
Strong duality: With exponential loss, the optimal detector retains a likelihood-ratio form using the densities of the least-favorable distributions.Strong duality eliminates detector minimization, but optimizing over least-favorable distributions remains infinite-dimensional.
Convex-gradient transport maps: Brenier’s theorem converts the dual problem into optimization over convex potentials whose gradients define transport maps between continuous reference and least-favorable distributions.The reformulation exposes the transport structure, while induced densities, inverse maps, and log-determinants are generally unavailable in closed form.

3 A HyCNN-Based Generative Method

This section parameterizes the convex potentials governing least-favorable distributions with HyCNNs and trains their gradients as invertible transport maps. Stochastic estimators make the resulting optimization computationally tractable, including log-determinant terms.

Generative framework: HyCNNs parameterize convex potentials, converting the infinite-dimensional optimization into finite-dimensional training and sampling.The paper also establishes HyCNN representation power and distributional universality of the induced gradient maps.
The Architecture of HyCNN: Passthrough connections and nonnegative weights preserve convexity when the activation is convex and component-wise non-decreasing.The architecture directly connects inputs to hidden units, and the convexity conditions are formalized in Proposition 2.
The Architecture of HyCNN: HyCNN generalizes ICNNs and uses a smooth log-sum-exp approximation of maxout to obtain differentiability.Softplus operators enforce nonnegative weights, while the smooth maxout activation avoids nondifferentiability at ties.
Training HyCNNs: A positive quadratic term makes each approximated potential strongly convex, ensuring that its gradient is invertible.The method learns one convex-gradient map per hypothesis because compositions of such maps need not remain gradients of convex potentials.
Training HyCNNs: Training draws samples from kernel-smoothed reference distributions, transports them with potential gradients, and updates network parameters by stochastic gradient ascent.Risk and Sinkhorn-discrepancy gradients are approximated with N-sample estimators within the training algorithm.
Stochastic gradient estimators: The risk-gradient estimator avoids direct differentiation through inverse maps and log-determinant Hessians using matrix-free stochastic estimation.Hutchinson trace estimation and conjugate-gradient solves provide the log-determinant-gradient estimator; log-determinants are exact for d ≤32 and estimated by SLQ when d > 32.

4 Representation Power of HyCNNs

This section establishes that HyCNNs can approximate convex potentials and their gradients, with quantitative hyperparameter guidance. Their gradient maps are then shown to be distributionally universal for approximating candidate least-favorable distributions.

HyCNNs are analyzed for representing convex functions, their gradients, and induced transport maps.The section focuses on representation capacity and distributional universality.
For any Lipschitz convex function on a compact convex set, a maxout HyCNN can uniformly approximate it within ε.The construction has depth-width product L · h = O((D/ε)^d).
Smooth HyCNNs extend the representation result and provide quantitative guidelines for selecting their hyperparameters.The extension concerns smooth approximations with sufficiently small smoothing parameter τ.
Gradient convergence requires separate analysis because uniform convergence of potentials does not generally imply convergence of gradient fields.Theorem 4 addresses this gap by proving approximation of gradients for smooth HyCNNs.
Under finite-second-moment and absolute-continuity assumptions, smooth HyCNN gradient maps are distributionally universal for approximating candidate least-favorable distributions.The result uses Brenier’s theorem and weak convergence of the transported distributions.

5 Experiments

Experiments across synthetic, MNIST, Higgs, adversarial, and image-generation settings evaluate accuracy, robustness, computation, and qualitative transport behavior. SDRO-H generally achieves strong accuracy and robustness, while SDRO remains computationally feasible where WDRO becomes prohibitive.

5.1 Synthetic Gaussian Mixture Dataset: Across dimensions d ∈{4,10,30,50,70,100}, SDRO-H achieves the lowest classification error rate in almost all GMM settings.SDRO-C remains competitive in Figures 3b–3f, while HyCNN is reported as more suitable than standard ICNN for approximating convex potentials.
5.1 Synthetic Gaussian Mixture Dataset: At n0 = n1 = 10 and d = 100, the best WDRO accuracy is inferior to SDRO-H under varied hyperparameter combinations.Moderately increasing λ and decreasing ϵ may improve SDRO-H accuracy; the passage attributes this to reduced conservatism and weaker smoothing.
5.1 Synthetic Gaussian Mixture Dataset: Under both L2 and L∞ PGD attacks, proposed-method accuracies remain higher and more stable as attack intensity ∆ increases, with SDRO-H best across all ∆ values.The comparison uses GMM data with n0 = n1 = 10 and d = 50, averaging accuracy over m ∈[10] and five trials.
5.2 MNIST Handwritten Digits Classification: On MNIST, SDRO has lower error across training-set sizes and highest accuracy across all non-zero PGD attack intensities under both norms.The experiments use d = 784 and compare against well-tuned DRO methods and common classifiers, with computation time reported as acceptable and stable as n′ increases.
5.3 High-energy Physics Higgs Dataset: On Higgs, SDRO achieves the highest accuracy in relatively low computation time, while WDRO fails to produce an acceptable solution within 1,800 seconds at large sample sizes.The passage attributes WDRO’s scalability problem to a linear program with O(n0n1) decision variables.
5.4 Human Face Generation: In the gender-generation task, convex-gradient maps gradually modify images toward visual characteristics associated with the opposite dataset label after a small number of training epochs.Figure 7 shows original images followed by generated images across 30 epochs.

6 Conclusions and Discussions

The paper concludes that its generative SDRHT framework combines theoretical guarantees with strong empirical accuracy and robustness across sample sizes and dimensions. It identifies multi-class testing, direct gradient parameterization, and efficient transport for extremely high-dimensional distributions as future directions.

Conclusions: The framework provides approximation guarantees for convex potentials and their gradients, together with distributional universality of the induced transport maps.These results support the paper’s theoretical characterization of its generative construction.
Conclusions: Numerical results report superior performance and robustness compared with existing methods across tasks with different sample sizes and dimensions.The conclusion states the empirical comparison at the paper level without restricting it to a single dataset.
Future Work: Future work includes extending the method to multi-class hypothesis testing and directly parameterizing gradients to improve transport-map approximation.The authors also identify efficient convex transport-map methods for extremely high-dimensional distributions as an open direction.

B.2 Proof of Theorem 1

The proof of Theorem 1 establishes the compactness of Sinkhorn ambiguity sets and the weak upper semicontinuity of expected loss, providing the ingredients for the theoretical result.

Proof Strategy: The proof first establishes weak compactness of Sinkhorn ambiguity sets and weak upper semicontinuity of expected loss.These properties are stated as the two preliminary components of the proof.
Weak Compactness: Lemma 1 states that the ambiguity set is weakly compact under Assumption 1 for closed Ω, each k ∈K, and any ρ̄k ≥ 0.The proof is presented in Appendix B.2.1.
Expected Loss: Lemma 2 states that Eω∼Q[Lk(φ,ω)] is weakly upper semicontinuous in Q under Assumption 2(I).The proof of this lemma is deferred to Appendix B.2.2.

B.2.1 Proof of Lemma 1

The proof of Lemma 1 connects Sinkhorn ambiguity sets to KL-divergence representations and establishes weak lower semicontinuity, yielding weak closedness and compactness.

Compactness: Because the ambiguity set is weakly closed within a weakly compact set, it is weakly compact.This completes the proof of Lemma 1.
Weak Closedness: For each k ∈K, fk(Q) is shown weakly lower semicontinuous in Q, so its sublevel sets and the ambiguity set are weakly closed.The proof considers weakly convergent sequences of distributions and couplings, using weak lower semicontinuity of KL divergence.

B.2.2 Proof of Lemma 2

The proof establishes uniform integrability and weak upper semicontinuity of the loss expectations, then verifies minimax conditions for strong duality and attainment.

Lemma 2: Donsker–Varadhan’s variational representation shows the inner problem is bounded and supports uniform integrability of L_k.Exponential integrability under Assumption 2(I), dominated convergence, and β_k →∞ complete the argument.
Lemma 2: For continuous detectors and suitable generating functions, L_0 and L_1 are non-negative and upper semicontinuous, while the ambiguity set is weakly closed.These properties are used to establish weak upper semicontinuity of E_ω∼Q[L_k(φ,ω)] under weak convergence of Q.
Lemma 2: Truncation, upper semicontinuity, and monotone convergence establish weak upper semicontinuity of the loss expectation.The proof controls the truncated losses and then lets the threshold M tend to infinity.
Theorem 1: The ambiguity sets are weakly compact, and the objective satisfies the convexity, semicontinuity, and level-compactness conditions required by the lopsided minimax theorem.The proof also establishes a finite lower bound in the detector variable and bounded upper-level sets in the distribution variables.
Theorem 1: Strong duality holds for soft-SDRHT, and the supremum in the dual representation is attained.Attainment follows from weak upper semicontinuity and weakly compact upper-level sets via Weierstrass’ theorem.

B.3 Proof of Theorem 3

Theorem 3 proves that HyCNNs can approximate convex functions and their smooth versions, with architecture complexity controlled by approximation accuracy and dimension.

Theorem 3(I): A finite υ-net discretizes a compact domain with J ≤ O((D/υ)^d) points, enabling approximation by supporting hyperplanes.For an L_f-Lipschitz convex function, the maximum of these hyperplanes controls the discretization error.
Theorem 3(I): HyCNNs with maxout activations exactly represent maxima of J affine functions using width h = 1 and depth L = O(J).The construction can be converted to nonnegative inter-layer weights by augmenting the input with its negation.
Theorem 3(I): Theorem 3(I) follows by combining supporting-hyperplane discretization with the exact HyCNN representation of affine maxima.This yields a HyCNN approximation whose architecture depends on the net size and therefore on dimension and accuracy.
Theorem 3(II): Smooth maxout activations approximate standard maxout activations, with the smoothing error controlled by τ and the network constant C.Choosing sufficiently small τ preserves the same order of architecture complexity as the standard HyCNN.
Theorem 3(II–III): The smooth HyCNN approximation achieves architecture scaling L·h = O((ε)^d) as stated in the proof, and a sequence of such networks converges uniformly on compact domains.The compact-domain sequence uses ε_n = 1/n and concludes uniform convergence to the target function.

B.4 Proof of Theorem 4

Theorem 4 establishes convergence of HyCNN gradients when the target convex function is Lipschitz smooth, linking function approximation to transport-map approximation.

Theorem 4(II): For an M-Lipschitz smooth function, the proof constructs HyCNN approximants with prescribed accuracy and architecture scaling.The argument evaluates the approximant along a displacement in the gradient direction.
Theorem 4(II): The gradient sequence ∇[HyCNN_n] converges uniformly to ∇f at the rate established in the theorem.The rate follows from the smooth HyCNN approximation bound and the relation L_n·h_n = O(ε_n^-d) stated in the proof.

B.5 Proof of Theorem 5

Theorem 5 uses Brenier’s theorem and HyCNN approximation to show that induced transport maps converge in distribution to the Brenier map.

Theorem 5: Because μ is absolutely continuous and μ and ν have finite second moments, Brenier’s theorem provides a finite convex potential f whose gradient defines the transport map.The proof then approximates f on expanding compact neighborhoods.
Theorem 5: Smooth HyCNNs converge locally uniformly to the Brenier potential f on R^d.For every fixed compact set, sufficiently large domains contain the set, allowing the local convergence argument.
Theorem 5: The induced gradients converge almost surely under X ∼ μ to ∇f(X), yielding weak convergence of the pushforward measures.Absolute continuity of μ transfers almost-everywhere gradient convergence into convergence of the transported distributions.

C Technical Note for Training and Testing HyCNN

The section develops gradient computation for HyCNN training, efficient evaluation of Hessian-related terms, and end-to-end testing with generated least-favorable distributions. It also describes the WDRO baseline's kernel smoothing and dependence on sample size.

Efficient derivatives: Strong convexity makes the Hessian determinant positive and enables Hessian-inverse vector products through quadratic optimization rather than explicit matrix inversion.The log-determinant Hessian is computed efficiently, while vector-Jacobian products support automatic differentiation.
Gradient computation: HyCNN training computes gradients with respect to network parameters using chain-rule, total-derivative, envelope-theorem, and automatic-differentiation procedures.The derivation treats gradients with respect to both θ0 and θ1 and uses the optimized Sinkhorn discrepancy estimator.
Testing procedure: During testing, samples are traced through kernel-smoothed distributions, least-favorable log-densities are evaluated, and decisions use the selected optimal detector.The procedure evaluates transformed locations and log-density corrections involving log det∇2 e ϕk.
WDRO baseline: The WDRO baseline uses a penalty parameter and transport variables, assumes testing samples share empirical support, and applies kernel smoothing to extend distributions over the entire space.The smoothing step convolves the discrete optimal distribution with a bandwidth-parameterized kernel.
WDRO baseline: O(n0 · n1) decision variables make the WDRO linear program sensitive to training-sample count, although it is not sensitive to dimension d.The baseline specifies a Gaussian kernel Gh(x) = exp(−∥x∥2/2h2).