PaperlayerView exact revision on arXiv

Source-linked AI summary

Brenier Meets Adversarial Training: Optimal Transport Geometry for Robust Learning

Alireza Abdollahpoorrostam, Ehsan Sharifian, Buse Şen, Marco Cuturi, Daniel Kuhn

arXiv:2609.31363v1cs.LGmath.OCstat.ML

TL;DR

Penalized Wasserstein DRO offers a framework for learning under distribution shift, but evaluating its nonconvex worst-case risk is difficult. The paper reformulates the adversary over transport maps and proposes MPA and ICNN-based maps. Across regression, adversarial training, and robust control, both methods improve robustness and out-of-distribution generalization over standard baselines.

  • Problem

    Penalized Wasserstein DRO requires evaluating a worst-case risk that becomes nonconvex for practical models, making the adversary’s optimization difficult.

    Source: §1 · p. 1

  • Method

    The paper reformulates the adversary’s problem over transport maps and proposes MPA with reassignment and ICNN maps parameterized to enforce cyclical monotonicity.

    Source: §1 · p. 2, §5 · p. 7

  • Results

    Across regression, adversarial training, and robust control, both approaches improve robustness and out-of-distribution generalization over standard baselines; ICNN-based training has the strongest overall robustness, while MPA is competitive with DRO baselines.

    Source: §1 · p. 2, §7 · p. 10

  • Takeaways & Limitations

    Cyclical monotonicity is the structural property missing from standard per-sample adversarial training, and the paper’s two remedies improve robustness across the reported settings.

    Source: §7 · p. 10

    MPA’s O(B^2) reassignment and the ICNN auxiliary network make each iteration more expensive than standard single-step adversarial updates.

    Source: §7 · p. 10

Abstract

from arXiv · show

Distributionally robust optimization (DRO) provides a principled framework for learning under distribution shift, but its practical use is hindered by the difficulty of evaluating worst-case risks for nonconvex loss functions. We study a penalized DRO formulation in which the adversary may choose any distribution but incurs a Wasserstein penalty for deviating from the empirical distribution. We show that the adversary's problem can be reformulated as an optimization problem over transport maps that push empirical samples to adversarial ones, and we prove that optimal maps are cyclically monotone. We also show that standard adversarial training---based on per-sample local optimization---violates cyclical monotonicity and wastes transport costs unless the adversary is severely restricted. We propose two remedies. First, we introduce multi-start particle ascent, which alternates parallel gradient ascent with reassignment to enforce cyclical monotonicity across samples. Second, we parameterize adversarial maps as gradients of input-convex neural networks, which guarantees cyclical monotonicity by construction. Experiments on robust regression, image classification, and robust control show that our methods consistently outperform standard adversarial training and state-of-the-art baselines, achieving improved robustness and better generalization under distribution shift.

1 Introduction

Robust learning must address distribution shifts beyond prescribed attacks, but practical Wasserstein DRO is difficult when its penalized worst-case problem is nonconvex. The paper reformulates the adversary over transport maps and proposes methods that preserve their optimal transport structure.

1 Introduction

  • Robustness to diverse threat models, common corruptions, and distribution shifts remains challenging even for well-trained models and certified defenses.The paper frames robust learning as generalization under broader distributional variation, not only resistance to a prescribed attack.Source: §1 · p. 1
  • Penalized Wasserstein DRO lets the adversary choose any distribution while charging for deviation from the empirical distribution, reducing risk evaluation to finite-dimensional optimization over transported samples.The nonconvex problem is commonly approximated using independent sample-wise updates or entropic surrogates.Source: §1 · pp. 1–2
  • The paper recasts the adversary’s problem as optimization over transport maps and identifies cyclical monotonicity as a central structure for robust learning.It proposes Multi-start Particle Ascent (MPA) and input-convex neural network (ICNN) maps; ICNN maps enforce cyclical monotonicity by design.Source: §1 · p. 2
  • Across CIFAR-10 adversarial training and robust control, ICNN-based training delivers the strongest overall robustness, while MPA remains competitive with DRO baselines.The paper describes both methods as complementary to additional regularization strategies used in robust training pipelines.Source: §1 · p. 2

2 Background

The background introduces Wasserstein distance and transport maps, then relates optimal transport to cyclical monotonicity. It also sets out the paper’s robust-learning and penalized DRO setting.

2 Background

  • The Wasserstein distance of order 2 is defined by minimizing expected squared transport displacement over couplings between two distributions.A coupling is a joint distribution with the specified distributions as marginals.Source: §2 · p. 2
  • When the reference distribution is absolutely continuous, Brenier’s theorem gives a unique cyclically monotone transport map inducing the optimal coupling.The map pushes the reference distribution to the target distribution.Source: §2 · p. 2
  • A map is cyclically monotone when its graph satisfies the stated permutation inequalities, equivalently allowing representation through a convex potential.Rockafellar’s theorem characterizes cyclically monotone maps as subgradients of convex functions.Source: §2 · p. 2
  • The Monge gap is nonnegative and equals zero exactly when the map is cyclically monotone on the reference distribution’s support.The paper assumes the loss is continuous in model parameters and differentiable and Lipschitz in data; penalized Wasserstein DRO charges the adversary in proportion to its Wasserstein deviation.Source: §2 · pp. 2–3

3 Reparameterizations of the Adversarial Risk

The section recasts penalized Wasserstein adversarial risk using transport maps and establishes how map quality relates to DRO optimization. It also distinguishes implicit and explicit map constructions.

3 Reparameterizations of the Adversarial Risk

  • Wasserstein DRO risk can be evaluated as an empirical expectation of adversarial loss, or as a functional optimization over transport maps.The transport-map formulation simultaneously addresses the global maximization problems associated with all empirical samples, and its pushforward gives a worst-case distribution.Source: §3 · p. 3
  • Pointwise formulations yield implicit adversarial maps, while global formulations motivate explicit parametric maps such as neural networks.The two formulations provide complementary ways to construct adversarial maps.Source: §3 · p. 3
  • For neural losses, the penalty condition λ > Lzz/2 needed for pointwise strong concavity can be difficult to establish and can weaken robustness by pulling perturbations close to training samples.At practical penalty values below that threshold, the pointwise maximization is nonconvex and maps are approximate.Source: §3 · p. 4
  • A near-optimal adversarial oracle supports high-accuracy projected gradient descent for DRO when the loss is convex in model parameters and its gradients are uniformly bounded.The section notes the convergence proposition relies on convexity in the model parameters, while suggesting its conclusions may extend near local minima for nonconvex losses.Source: §3 · p. 4
  • An adversarial map’s error is at least λ times its Monge gap, so inefficient transport necessarily makes the map suboptimal.A map with positive Monge gap transports the reference distribution inefficiently and weakens the stated convergence guarantees.Source: §3 · p. 4
  • The optimal adversarial map is cyclically monotone, even for a discrete reference distribution and a loss that is not concave in the data.The result makes cyclical monotonicity a necessary optimality condition for the Wasserstein DRO adversary.Source: §3 · p. 4

4 Implicit Adversarial Maps

The section contrasts per-sample particle ascent with batch-wide reassignment and develops MPA to improve implicit adversarial maps. It also gives conditions and guarantees for these methods.

4 Implicit Adversarial Maps

  • Standard particle ascent uses local gradient ascent initialized at each empirical sample, but its map is generally suboptimal unless λ > Lzz/2.The passage notes the method is computationally appealing because particles can be updated in parallel.Source: §4 · p. 5
  • Per-sample Particle Ascent is cyclically monotone for one-dimensional data under the stated Lipschitz and step-size conditions, but can violate cyclical monotonicity in two dimensions.The two-dimensional example reports a strict violation after the particles converge to different local maxima.Source: §4 · p. 5
  • Reassignment can improve a map by matching empirical samples to better candidate outputs while preserving the pushforward distribution; allowing many-to-one assignments can reduce adversarial error by more than the Monge gap.Assignment-stationarity prevents a sample from improving through a switch to another sample’s candidate.Source: §4 · pp. 5–6
  • Assignment-stationarity means no sample can improve its objective by switching to another sample’s candidate solution, and it implies cyclical monotonicity.The converse does not hold: a cyclically monotone map need not be assignment-stationary because cyclical monotonicity ignores the loss.Source: §4 · p. 6
  • Multi-start Particle Ascent alternates parallel local optimization of adversarial samples with batch-wide reassignment, then uses a final reassignment to enforce assignment-stationarity and cyclical monotonicity on the batch.MPA extends multi-start search across coupled sample-wise optimization problems by sharing locally found maxima as candidate solutions.Source: §4 · p. 6
  • With infinitely many local-optimization steps, MPA’s adversarial samples stop changing after finitely many rounds; with sufficiently large batches, it is ε-optimal with probability tending to one as batch size grows.The global-optimality guarantee assumes independent samples from a distribution with everywhere positive density and holds for every λ > 0.Source: §4 · pp. 6–7

5 Explicit Adversarial Maps

The section introduces explicit adversarial maps to address MPA’s restriction to a given training batch, and enforces cyclical monotonicity by representing maps as gradients of convex potentials.

5 Explicit Adversarial Maps

  • Explicit parameterized maps address MPA’s transductive restriction to the training batch, which prevents generalization to unseen data.The section contrasts these maps with MPA’s implicit batch-specific map.Source: §5 · p. 7
  • Training alternates K gradient-ascent steps on the adversarial-map parameters with E projected gradient-descent steps on the model parameters.The outer gradient is approximated using Danskin’s theorem and the current map iterate as an approximate maximizer.Source: §5 · p. 7
  • A cyclically monotone transport map is representable as a subgradient of a convex potential, or as its gradient where the potential is differentiable.The section uses this convex-potential representation to enforce cyclical monotonicity in explicit maps.Source: §5 · p. 7
  • ICNNs model convex potentials whose gradients are cyclically monotone, and can approximate any Lipschitz convex potential on a compact set to arbitrary precision.The paper names maps induced by these potentials ICNN maps.Source: §5 · p. 7
  • The architecture models a scalar potential and evaluates its gradient map by automatic differentiation, restricting quadratic terms to the final readout layer.The paper notes that quadratic forms at every layer are computationally expensive.Source: §5 · pp. 7–8
  • Readout rank controls architecture capacity; the single-quadratic design also uses moment-matched initialization, an identity-map warm start, and adaptive inner step sizes.The paper reports that increasing readout rank with a last-layer quadratic achieves the best robust performance and reduces runtime relative to quadratic terms at every layer.Source: §5 · p. 8

6 Experiments

The experiments test the methods across feature-space classification, image-space attacks and shifts, and robust control. ICNN-DRO leads key image-space comparisons and robust control, with the section also reporting Monge gaps and runtime.

6 Experiments

  • The experiments cover feature-space logistic regression, image-space attacks and shifts, robust control, and comparisons with six baselines.Additional experiments address uncertain least squares and cross-model universality.Source: §6 · p. 8

6.1 Adversarial Multi-class Logistic Regression

  • In feature-space logistic regression, ICNN-DRO slightly beats NN-DRO across most perturbation budgets, ties at extreme budgets, and preserves clean accuracy.NN-DRO degrades significantly on clean samples.Source: Figure 2 · p. 8, §6.1 · p. 9

6.2 Robustness to Adversarial and Natural Distribution Shifts

  • ICNN-DRO outperforms PA and WFR on attacks, leads on unseen-domain benchmarks, and tops mean corruption accuracy.The comparison spans adversarial attacks, CIFAR-10.1 and CIFAR-10.2, and CIFAR-10-C.Source: Table 1 · p. 9, §6.2 · p. 9
  • ICNN-DRO reaches 58.11% against PGD and 57.71% against AA, with a 0.4% PGD–AA gap versus PA’s 3.55%.Its clean accuracy is 91.41%, within 1.71% of ERM.Source: Table 1 · p. 9, §6.2 · p. 9
  • ICNN-DRO’s 1.01e-7 Monge gap is nearly four orders below every baseline; its 4,037s runtime is faster than every DRO baseline.Only first-order RO is faster, at a cost of 7.45 AA points.Source: Table 1 · p. 9, §6.2 · p. 10

6.3 Robust Control

  • At λ=1, ICNN-DRO achieves the best episode length on every control variant, especially extrapolative shifts; MPA is the runner-up.The harder variants lie outside the training uncertainty box.Source: Figure 3 · p. 10, §6.3 · p. 10

7 Conclusions

The paper concludes that cyclical monotonicity is the structural property missing from standard per-sample adversarial training, and that MPA and ICNN-based adversaries restore that structure in different ways. It reports improved robustness and out-of-distribution generalization over standard baselines, while noting added computational cost and positioning the methods as complementary to other robust-training strategies.

7 Conclusions

  • Across regression, adversarial training, and robust control, both approaches improve robustness and out-of-distribution generalization over standard baselines.The conclusion also identifies cyclical monotonicity as the structural property missing from standard per-sample adversarial training.Source: §7 · p. 10
  • MPA restores transport structure implicitly through ascent and reassignment, while ICNN-based adversaries enforce it by parameterizing a Brenier map.The paper characterizes cyclical monotonicity as the key structural property missing from standard per-sample adversarial training.Source: §7 · p. 10
  • MPA's O(B^2) reassignment step and the ICNN auxiliary network make each training iteration more expensive than standard single-step adversarial updates.The conclusion says this increased cost is offset by significant robustness benefits.Source: §7 · p. 10

A Related Work

  • Unlike prior ICNN optimal-transport methods, this approach addresses a nonconvex primal optimization over transport maps and does not require an absolutely continuous reference distribution.Here cyclical monotonicity is necessary but not sufficient for optimality, and restricting maps to convex-potential gradients acts as a safeguard against poor local optima.Source: A Related Work · pp. 16–17
  • Earlier DRO approaches generate adversarial samples independently, whereas MPA reassigns particles across the batch to enforce cyclical monotonicity.The cited approaches include independent gradient ascent and sampling from an entropic Gibbs measure.Source: A Related Work · p. 17
  • Prior adversarial-map approaches use unconstrained neural networks and require a reference distribution with a density; the ICNN approach constrains maps to be cyclically monotone without that density assumption.The paper also notes that prior approaches do not identify cyclical-monotonicity enforcement as a way to steer away from local maxima.Source: A Related Work · p. 17

B Technical Background Results

The appendix establishes that the penalized adversarial risk is finite and attained by a measurable transport map. It also justifies relaxing the coupling space to include measures with infinite second moments, since such couplings have objective value minus infinity.

B Technical Background Results

  • A Borel-measurable selector T* exists that chooses a maximizing adversarial point for every reference point, and the induced map attains the inner maximization.The appendix obtains the map through measurable selection from the nonempty, closed-valued set of pointwise maximizers.Source: B Technical Background Results · pp. 17–19
  • The adversarial risk F(θ) is finite when the reference distribution has finite second moments and the loss is Lipschitz in the data point.The quadratic Wasserstein penalty bounds the objective above, while the identity coupling gives a finite feasible value.Source: B Technical Background Results · pp. 17–18
  • Couplings with infinite transport cost have objective value −∞, so enlarging the coupling space to all Borel probability measures does not increase the supremum.The appendix notes that infinite transport cost is equivalent to infinite second moments for these couplings.Source: B Technical Background Results · p. 18
  • For an empirical reference distribution, the disintegration reduces to choosing a maximizer separately for each training sample, yielding the adversarial distribution from those selected points.The appendix says no measurable selection is needed in this finite-sample case.Source: B Technical Background Results · p. 19

C Proofs

The proofs connect transport-map structure to adversarial-risk approximation and analyze when particle ascent preserves order. The appendix also states finite-termination and high-probability near-optimality properties for MPA under specified conditions.

C Proofs

  • Under the stated convexity and bounded-gradient conditions, the projected-gradient proof bounds average risk using subgradients whose error is ε(θt, Tt).The proof derives the subgradient property and applies the projection update to obtain the bound.Source: C Proofs · p. 20
  • Any lack of cyclical monotonicity on the reference distribution's support gives a strictly positive lower bound on the adversarial error.For an empirical distribution, the Monge gap has a permutation-based representation.Source: C Proofs · p. 20
  • In one dimension, particle ascent maps are shown to be non-decreasing in both continuous and discrete time.The continuous-time argument rules out an order reversal; the discrete-time induction uses the stated step-size condition η ≤ (Lzz + 2λ)^−1.Source: C Proofs · p. 21

D Properties of MPA

  • The MPA inner loop is assignment-stationary and therefore cyclically monotone on the batch; with K = ∞, it terminates in finitely many rounds.The appendix also states that sufficiently large batch size guarantees a near-optimal adversarial map with high probability.Source: D Properties of MPA · p. 22
  • The outer loop may remain at its initial iterate unless the current adversarial map is cyclically monotone.This is stated as a demonstrated property of the outer loop in the MPA appendix discussion.Source: D Properties of MPA · p. 22

D.1 Finite-time Termination of MPA

Under the stated smoothness and analyticity assumptions, exact MPA terminates after finitely many rounds with an output that is both stationary and assignment-stationary. Its proof uses ascent and reassignment steps that are monotone, with strict improvement whenever they change the iterate.

D.1 Finite-time Termination of MPA

  • Ascent converges from any initialization to a stationary point when 0 < η < 2/(L_zz + 2λ), provided the smoothness and analyticity assumptions hold.Analyticity ensures convergence of the bounded ascent sequence; the step-size condition is required.Source: §D.1 · p. 23
  • Each ascent or reassignment step is monotone in the batch objectives, and is strictly improving in at least one coordinate whenever it changes the batch.Both operators are idempotent under the stated assumptions and step-size condition.Source: §D.1 · p. 24
  • With K = ∞, full-batch MPA terminates within 1 + Σ_i(|f_i(M_i)| − 1) rounds and returns a stationary, assignment-stationary output.This guarantee assumes Assumptions D.1 and D.2 and 0 < η < 2/(L_zz + 2λ).Source: §D.1 · p. 25
  • If an epoch stops before reaching a fixed point, its last iterate may not be assignment-stationary; a final reassignment enforces cyclical monotonicity only on the mini-batch.Ascent can break assignment-stationarity, even when K = ∞.Source: §D.1 · p. 25

D.2 Global Optimality Guarantees for MPA

MPA’s implicit adversarial map is near-optimal with high probability when the empirical batch is sufficiently large and satisfies the stated distributional assumptions. The guarantee is not dimension-free: its sample complexity can grow exponentially with ambient dimension.

D.2 Global Optimality Guarantees for MPA

  • For any λ > 0, MPA’s inner loop computes adversarial risk F(θ) to absolute error at most ε with high probability when the batch is sufficiently large, unlike PA’s guarantee requiring λ > L_zz/2.The comparison is with the state-of-the-art PA guarantee described for Sinha et al. [60].Source: §D.2 · p. 25
  • Under Assumptions D.1 and D.3, with 0 < η < 2/(Lzz + 2λ), the MPA map Tmpa is near-optimal in problem (2) with high probability, for any K∈N∪{∞}, R∈N, and tolerance ε>0.Assumption D.3 states that the empirical samples are drawn independently from P0, which has an everywhere positive density with respect to Lebesgue measure on Rm.Source: §D.2 · pp. 26, 28
  • The global guarantee relies on a sufficiently large, diverse batch: outliers and uncovered inliers are each unlikely to exceed O(ε) of the batch.Covered inliers have a near-optimal adversarial sample among the empirical samples.Source: §D.2 · pp. 26–27
  • The sample-complexity bound grows exponentially with ambient dimension m, even for standard normal data, because the covering mass scales as O(ε^(m/2)).The bound is governed primarily by the inverse covering mass 1/p(ε).Source: §D.2 · p. 29
  • With an insufficiently diverse batch, MPA may be suboptimal and can have higher adversarial error than corresponding PA; an ascent-first variant avoids that comparison but loses a global optimality guarantee.The authors report that original MPA consistently attains lower adversarial error than the ascent-first variant in their experiments.Source: §D.2 · pp. 29–30

D.3 An Illustrative Example where PA Stalls and MPA Progresses

In the constructed example, PA’s failure to satisfy cyclical monotonicity produces a vanishing outer gradient and stalls optimization at the initial iterate. MPA’s reassignment step escapes that fixed point and permits progress.

D.3 An Illustrative Example where PA Stalls and MPA Progresses

  • PA’s inexact gradient vanishes at θ₁ = 1, trapping outer optimization there even though θ₁ is neither a minimizer nor a stationary point of F(θ).The example embeds a two-atom instance where discrete PA fails cyclical monotonicity and assignment-stationarity.Source: §D.3 · p. 30
  • MPA escapes the spurious fixed point θ₁ = 1 and produces a strictly better iterate θ₂ < 1.The example uses an inexact-gradient outer optimization scheme.Source: §D.3 · pp. 30–31
  • Reassignment warm-starts the next ascent phase at the best adversarial sample from the previous phase, allowing MPA to exploit information about the optimization landscape inaccessible to PA alone.This can let outer minimization progress when PA yields a vanishing gradient.Source: §D.3 · p. 31

E ICNN Training Details

The appendix describes how ICNN convexity constraints and initialization are implemented, then explains adaptive inner step sizes and reports experimental settings.

E ICNN Training Details

  • Exponential reparameterization keeps ICNN hidden-to-hidden and output weights non-negative, while a passthrough absorbs positive-mean offsets and adapts them to the input.The experiments use the passthrough design; the initialization also corrects for the positive mean introduced by exponentiation.Source: E ICNN Training Details · pp. 31–32

E.1 Adaptive Inner Step Size Selection

  • BB step sizes estimate local inverse curvature, while Armijo backtracking reduces overly aggressive steps until they provide sufficient ascent or reach the backtracking limit.When the BB curvature estimate is invalid, the procedure falls back to the default initial step size.Source: §E.1 · pp. 32–33

F Supplementary Details for Section 6

  • Across experiments, the authors searched constant and adaptive BB-plus-Armijo inner step sizes and report the best-performing configuration for each method.The comparisons used the same full training-loop setup across methods, including fixed hardware and distributed-training settings.Source: F Supplementary Details for Section 6 · p. 33

F.1 Supplementary Details for Section 6.1

  • For the logistic-regression experiment, all methods used 10 epochs, batch size 128, and penalty λ = 10, while MPA used five rounds of 20 ascent iterations.Other settings included method-specific inner-iteration budgets and a shared BB-plus-Armijo parameter configuration.Source: §F.1 · p. 34

F.2 Supplementary Details for Section 6.2

The supplementary sections detail the CIFAR-10 evaluation, robust-control implementation, and additional experiments. They report complementary checks against gradient masking and robust-control results across penalty levels.

F.2 Supplementary Details for Section 6.2

  • ICNN-DRO’s PGD–AutoAttack gap is 0.4%, versus 3.55% for PA, and its loss rises smoothly from clean inputs to PGD endpoints in two inspected examples.The appendix presents the attack comparison and loss-landscape inspection as complementary evidence against gradient masking.Source: §F.2 · p. 35

F.3 Supplementary Details for Section 6.3

  • For robust control, ICNN-DRO has the highest mean episode length on every evaluation variant at each tested λ, with MPA second throughout.The variants include harder extrapolative shifts, where WFR falls substantially behind ICNN-DRO; performance is non-monotonic in λ for several methods.Source: §F.3 · p. 37, Table 2 · p. 37
  • The CartPole differentiable-rollout surrogate smooths discrete actions and episode termination, but its pathwise gradient estimator is generally biased relative to the original discrete-MDP objective.A bounded sigmoid-composed adversarial map preserves physical parameter bounds while remaining differentiable end-to-end.Source: §F.3 · pp. 38–39

G Additional Experiments

  • The additional studies cover uncertain least squares, cross-model perturbation transfer, empirical Monge gaps, ICNN adversarial capacity, and penalty annealing.The appendix overview identifies the quadratic output-layer term’s rank, rather than ICNN depth or width, as the binding constraint on adversarial capacity.Source: G Additional Experiments · p. 39

G.1 Robust Least Squares

In uncertain least squares, DRO methods outperform ERM under increasing distribution shift, with ICNN-DRO best and MPA close behind. In cross-model tests, ICNN perturbations transfer more strongly within the source model’s architecture family.

G.1 Robust Least Squares

  • As perturbation level ∆ increases, all DRO variants substantially outperform ERM; ICNN-DRO performs best, followed closely by MPA.The experiment tests on distributions whose uncertainty interval expands beyond the nominal training range.Source: §G.1 · pp. 39–40, Figure 5 · p. 39
  • In one dimension, PA is already cyclically monotone, yet MPA still outperforms it because reassignment lets particles inherit stronger neighboring points and escape weak local maxima.The experiment’s worst-case samples tend toward the uncertainty-set boundaries, leaving limited room for a structured map-based advantage.Source: §G.1 · p. 40

G.2 Cross-Model Universality

  • The transfer test applies one R1-trained, input-dependent perturbation unchanged to each target model on the same 9,085 CIFAR-10 test points.The source model participates in adversary training; target models are used only for evaluation.Source: §G.2 · p. 40
  • Transferred attacks reduce held-out ResNet accuracy by 39.26–55.80 points and recover 82%–91% of source errors, compared with 24.68–39.98 points and 73%–81% for ViTs.The perturbations transfer more strongly within the ResNet family than across architectures.Source: §G.2 · pp. 40–41, Table 3 · p. 40

G.3 An Empirical Monge Gap

Across these sections, empirical Monge gaps track some—but not all—method rankings, readout rank dominates ICNN capacity, and penalty annealing improves CIFAR-10 results with fewer inner ascent steps.

G.3 An Empirical Monge Gap

  • The Sinkhorn-based estimate upper-bounds the actual Monge gap, which declines as λ rises because the penalty pulls maps toward identity and the inner problem closer to ERM.The computation uses Sinkhorn divergence because directly evaluating the Wasserstein distance is costly for high-dimensional empirical measures.Source: §G.3 · p. 41
  • In CartPole, ICNN-DRO has the smallest estimated Monge gap and best control performance, with MPA second; in logistic regression, the gap does not fully determine empirical ranking.In CartPole, MPA improves over PA by roughly three orders of magnitude in estimated gap.Source: §G.3 · pp. 41–42

G.4 Design Axes of the ICNN Adversary

  • Across ICNN designs, a simple network with high readout rank outperforms larger low-rank models at lower computation; rank governs directional capacity because the quadratic readout term dominates.The network-gradient contribution is empirically dominated by the readout term.Source: §G.4 · pp. 42–43

G.5 Penalty Annealing for the Global Map

  • Penalty annealing decreases λ without resetting the classifier or global ICNN map, gradually strengthening the adversary while keeping the map cyclically monotone.The schedule also incrementally increases map displacement, avoiding unstable jumps associated with a cold solve.Source: §G.5 · pp. 43–44
  • 59.94% vs. 59.83% PGD accuracy with 2.5× fewer inner steps (K=20 vs. 50); annealing also yields 91.48% clean accuracy and improves all but CIFAR-10.2 (−0.02 percentage points).The matched fixed-λ comparison uses K=20; the PGD comparison is against fixed-λ K=50.Source: Table 7 · p. 44, §G.5 · p. 44
Loading 2609.31363v1…