PaperlayerView exact revision on arXiv

Source-linked AI summary

Which Optimizer Leaks Most? Optimizer Implicit Bias as a Training-Data Privacy Controller

Sagi Habani, Yehonathan Refael, Ofir Lindenbaum

arXiv:2610.18294v1cs.LGcs.AI

TL;DR

Released checkpoints can expose training data, and existing reconstruction attacks primarily rely on gradient-descent training. This paper extends stationarity-based reconstruction to optimizer-specific margin geometries and analyzes their weight-specificity and reachability. It finds stronger reconstructions for Muon than SGD in the main experiments, while Adam is most resistant to the evaluated attacks.

  • Problem

    Prior stationarity-based reconstruction attacks focus on gradient-descent training, leaving open whether optimizer choice changes how much training data can be recovered from a checkpoint.

    Source: §1 · p. 1

  • Method

    The paper adapts stationarity-based reconstruction to optimizer-specific margin norms and analyzes the weight-specificity and reachability of their stationarity targets.

    Source: §1 · pp. 2–3

  • Results

    Muon yields the strongest reconstructions in the main experiments, including SSIM@30 of 0.431 on MNIST versus 0.346 for SGD, while Adam remains most resistant to the evaluated attacks.

    Source: §1 · p. 2, §5.1 · p. 7

  • Takeaways & Limitations

    Optimizer choice is a practical training-data privacy control, so checkpoint privacy assessments that ignore the training algorithm may miss a source of leakage.

    Source: §6 · p. 9

    The Muon–SGD ordering is a reachability-based prediction rather than a theorem, and the comparison depends on training budget, sample size, and architecture.

    Source: §1 · p. 2, §4.3 · p. 7

Abstract

from arXiv · show

Releasing a neural network checkpoint can leak information about its training data, and existing reconstruction results rely on gradient-descent training. We ask whether the choice of optimizer affects the ability to recover the training data. Extending work connecting Karush--Kuhn--Tucker conditions to training-data reconstruction, we show that the implicit biases of popular optimizers (SGD, Adam, Muon) leave very different fingerprints of the training data. We argue that the key distinction is the \emph{weight-specificity} of these implicit biases, namely how much continuous information about the final weights the optimizer-dependent target of the stationarity conditions encodes. The limiting stationarity conditions for models trained with SGD and Muon retain enough weight-specific information to reconstruct training examples in practice. For Adam in the $\ell_\infty$ implicit-bias regime, the optimizer-dependent part of these conditions encodes only a discrete pattern of maximal coordinates and their signs, and at nondegenerate KKT points the conditions are too few to identify the training set to first order. Experiments on MNIST and CIFAR-10 support these claims: models trained with Muon can yield stronger reconstructions at much lower training cost than models trained with SGD, although this advantage depends on the training regime and architecture, and models trained with Adam appear consistently more resistant to the evaluated attacks.

1 INTRODUCTION

The paper asks whether optimizer choice changes how much training data can be recovered from a released checkpoint, extending stationarity-based reconstruction beyond gradient descent. Its analysis and experiments find distinct leakage patterns across SGD, Muon, and Adam, with the Muon–SGD comparison depending on training regime and architecture.

1 INTRODUCTION

  • Checkpoint-only reconstruction matters because prior attacks recovered training images from classifier parameters without query access or other data.Those attacks optimize candidate inputs to satisfy a stationarity condition relating the weights to training data (Haim et al., 2022).Source: §1 · p. 1
  • At KKT directions, weight-specificity is p−1 for SGD, sum_k(m_k+n_k−2) for Muon with simple top singular values, and zero for Adam’s ℓ∞ margin.Adam’s target depends only on the locations and signs of maximal coordinates; at nondegenerate KKT points its stationarity constraints are too few for first-order training-set identifiability.Source: §1 · p. 2
  • Muon yields the strongest reconstructions, SGD recovers coarser structure, and Adam is most resistant in the main experiments.Muon’s advantage depends on training budget, sample size, and architecture; SGD can be comparable at small sample sizes or match or exceed Muon in some architectures.Source: §1 · pp. 2–3, Figure 1 · p. 2
  • The attack extends Haim et al. (2022) by adapting its stationarity objective to the optimizer’s margin geometry and correcting failure modes of a naive extension.The scale-invariant penalty enforces dual-norm equality without requiring cross-block coupling.Source: §1 · pp. 2–3

2 RELATED WORK

Prior reconstruction attacks invert the ℓ2-margin KKT conditions associated with gradient descent; this paper extends that formulation to other optimizer-induced margin geometries. The preliminaries specify the optimizer norms and the checkpoint-only threat model used in the analysis.

2 RELATED WORK

  • Earlier attacks reconstruct samples by inverting gradient descent’s ℓ2-margin KKT conditions, with extensions to multiclass settings, transfer learning, and membership inference.The cited work also examines reliance on image priors; this paper extends the formulation to other margin geometries (Haim et al., 2022; Golbari et al., 2025; Refael et al., 2026b).Source: §2 · p. 3

2.1 Preliminaries: notation and threat model

  • The paper defines a gauge as a transformation group that preserves the constraint system and maps solutions to solutions.The objective is made invariant along such directions because the constraints cannot distinguish those solutions.Source: §2.1 · p. 3
  • Under suitable assumptions, homogeneous networks converge toward KKT directions, with Adam and Muon associated with optimizer-dependent margin norms.The analysis uses a global ℓ∞ norm for Adam and a max-spectral norm for Muon; the margin norm is the only optimizer property that distinguishes the three attacks in the analysis.Source: §2.1 · p. 4
  • The adversary observes the final parameters, architecture, and optimizer family, but not training inputs, labels, sample count, or intermediate iterates.The attacker fixes the observed parameters and uses candidate inputs with fixed balanced labels.Source: §2.1 · p. 4

3 A NORM-PARAMETERIZED RECONSTRUCTION OBJECTIVE

The reconstruction objective is built around stationarity for a general optimizer-dependent margin norm. It discards only uninformative multiplier scale and enforces alignment with the stationarity target, yielding separate Muon and Adam losses.

3 A NORM-PARAMETERIZED RECONSTRUCTION OBJECTIVE

  • The general-norm attack fixes the trained parameters and optimizes candidate inputs and multipliers to satisfy the optimizer-specific stationarity condition.It instantiates the formulation for Muon and Adam while retaining the existing multiplier and prior terms.Source: §3 · p. 4, §3.1 · p. 4

3.1 Stationarity for a general norm

  • For a general margin norm, stationarity becomes a subdifferential set-membership condition that can be enforced through magnitude and alignment inequalities.The paper states that the candidate-input information lies in the alignment requirement, since the multipliers in the magnitude bound are free unknowns.Source: §3.1 · p. 4, §3.2 · p. 4

3.2 Two scale freedoms, and why only one may be discarded

  • Multiplier rescaling is a gauge, so the loss discards its uninformative magnitude while preserving the direction of the residual in the target cone.Blockwise alignment is exact under the stated assumptions, requires no cross-block coupling term, and recovers Haim et al. (2022)’s Euclidean objective up to multiplier scale.Source: §3.2 · p. 5

3.3 The Muon (spectral) objective

  • Muon’s spectral loss avoids a single singular-vector target, which can fail with repeated top singular values or mismatched relative block scales.The resulting condition is exact at KKT directions for arbitrary per-block weights, with every layer carrying information.Source: §3.3 · p. 5

3.4 The Adam (ℓ∞) objective

  • Adam’s ℓ∞ loss requires the candidate gradient combination to be supported on maximal-magnitude weights with matching signs.The target is global across parameter coordinates and imposes no further condition on the combination.Source: §3.4 · pp. 5–6

4 HOW MUCH DO THE OPTIMIZERS LEAK? A SPECIFICITY ANALYSIS

The analysis compares how much each optimizer’s stationarity target encodes about the weights, then combines that specificity with reachability to explain practical reconstruction leakage. Adam’s ℓ∞ target carries only a discrete pattern, while the Muon-versus-SGD comparison depends partly on training cost and regime.

4.1 Weight-specificity and target-cone variability

  • Weight-specificity measures the independent continuous weight information encoded by the optimizer-dependent stationarity target, beyond merely counting equations.It varies with the geometry and stratum, not the dataset, and is not itself an information-theoretic bound on input identifiability.Source: §4.1 · p. 6, §4 · p. 6

4.2 Consequences, and why Adam resists

  • Adam’s ℓ∞ target depends only on the locations and signs of maximal-magnitude coordinates, giving it zero continuous weight-specificity.At nondegenerate KKT points, its stationarity system has at most n_I equations for n_I(d + 1) unknowns and fails regular first-order identifiability when n_I d ≥ 2.Source: §4.1 · p. 6, §4.2 · p. 6
  • SGD’s target encodes the full weight direction, while Muon’s encodes top singular structure across active blocks, preserving a width-dependent data fingerprint.For Muon, the template contributes at least Θ(m_k + n_k) functionals per block when the top singular value is simple.Source: §4.2 · p. 6, §4.1 · p. 6

4.3 Two axes, and the vulnerability ordering

  • Practical leakage reflects both specificity and reachability: Muon is strong on both, SGD is highly specific but slow to converge, and Adam is fast but nonspecific.The Muon-over-SGD ordering is a reachability-based prediction, not a theorem, and may narrow when SGD has enough training budget.Source: §4.3 · p. 7

5 EXPERIMENTS

Across the evaluated tasks, Muon yields the strongest reconstructions and Adam the weakest; Muon’s advantage over SGD depends on training regime and architecture. Reconstructions are concentrated among near-margin samples, and image priors can improve scores without changing the leading optimizer.

5.1 Main Results

  • Muon has the highest SSIM@30 on MNIST (0.431), binary CIFAR-10 (0.551), and 10-class CIFAR-10 (0.534), versus SGD’s 0.346, 0.229, and 0.383.Muon leads on every metric and task in the reported no-prior comparison, with two orders of magnitude fewer training iterations than SGD.Source: §5.1 · p. 7, Table 1 · p. 8
  • Adam has the lowest SSIM@30 on every task without a prior, at most 0.249, below the 0.4 recognizability level.This is consistent with the ℓ∞ target carrying no continuous information about weight magnitudes and the stationarity system being too weakly constrained to identify individual samples.Source: §5.1 · p. 7
  • Muon remains best under every prior, with SSIM@30 between 0.431 and 0.660, while Adam never exceeds 0.344.On MNIST, adding all priors raises SSIM@30 by 0.229 for Muon and 0.105 for SGD, but only 0.017 for Adam.Source: §5.1 · pp. 7–8, Table 1 · p. 8
  • After convergence on binary CIFAR-10, SGD is competitive at small sample size: at n = 100 its SSIM@30 is 0.623 against Muon’s 0.698, versus 0.234 against 0.613 at n = 500.Across sample sizes Muon remains best on every metric and Adam has the lowest SSIM@30; the Muon gap is partly a training-cost gap.Source: §5.1 · p. 8
  • Adam is worst on every metric for every tested architecture, while the Muon comparison with SGD varies with architecture and training budget.The stated interpretation is that Adam’s weakness follows from its ℓ∞ target cone, whereas the ℓ2-versus-spectral comparison depends on the residual map and budget.Source: §5.1 · p. 8
  • Recognizable reconstructions occur only for samples at or near the minimal margin; Muon recovers many, SGD a handful, and Adam essentially none.Samples well inside the margin are never recovered in the reported settings.Source: §5.1 · pp. 8–9, Figure 2 · p. 8

6 DISCUSSION

The discussion frames optimizer implicit bias as a consequential source of checkpoint leakage and argues that privacy evaluation should account for the training algorithm. The study focuses on standard SGD, Adam, and Muon, while identifying analysis of newer optimizer families as future work.

6 DISCUSSION

  • With architecture, data, and interpolation behavior held identical, the optimizers leave distinct geometric fingerprints: Muon reconstructs most strongly, SGD less, and Adam is most resistant.The authors characterize optimizer choice as a practical privacy control, while emphasizing that their checkpoint-only stationarity analysis provides no formal privacy guarantee.Source: §6 · p. 9
  • The analysis covers standard SGD, Adam, and Muon; extending implicit-bias analysis to memory-efficient, low-rank, and orthogonalized optimizers remains future work.The discussion names AdaRankGrad, SUMO, and other methods as targets for studying how adaptive rank reduction and subspace orthogonalization relate to leakage.Source: §6 · p. 9

AI use statement

  • Generative AI tools were used for text polishing and organizational suggestions, with authors reviewing and refining the assisted work.The authors report no AI use for research ideas, code, mathematical derivations, or experimental data generation or analysis.Source: AI use statement · p. 9

Supplementary Materials

The appendix establishes scale and block-activity properties used in the optimizer-specific stationarity analysis, then clarifies how equation counts and target cones constrain reconstruction. It distinguishes cone sufficiency for linear-in-parameter models from Adam’s ℓ∞ stratum invariance.

A.1 Scale covariance in the trained weights (from §3.2)

  • Rescaling θ along its ray preserves homogeneous stationarity after rescaling the multipliers, but does not preserve feasibility or complementary slackness under the fixed margin constraint.This justifies disregarding the norm of θ in the homogeneous stationarity equation.Source: §A.1 · p. 11

A.2 Every block is active (from §3.2)

  • At a KKT direction, every Muon block attains the maximum, and every Adam block contains a globally maximal coordinate.These block-activity properties hold under Assumption 2.1.Source: §A.2 · p. 11

A.3 Why per-block alignment loses nothing (from §3.2)

  • Per-block alignment remains exact at the encountered KKT weights, so a separate coupling term cannot distinguish aligned candidates and is unnecessary.The Euler identity establishes the exactness of the per-block approach.Source: §A.3 · p. 11

A.4 Equation counting at a generic weight vector does not separate the geometries

  • At generic weights, the three geometries have Θ(p) constraints, but at Adam’s KKT points the codimension can be as small as zero and is at most the on-margin example count, below the necessary first-order identifiability bound.The ℓ2 and spectral geometries meet that bound in the over-parameterized regime; the bound is necessary, not sufficient.Source: §A.4 · pp. 11–12

A.5 Target-cone sufficiency for linear-in-parameter models (from §4.1)

  • For linear-in-parameter models, equal target cones yield identical stationarity-feasible sets and global-alignment-loss zero sets, making the cone sufficient for this formulation.This does not alone imply input non-identifiability; that also depends on the residual map and cone codimension.Source: §A.5 · p. 12

A.6 Stratum invariance under ℓ∞(from §4.2)

  • Adam’s ℓ∞ target depends only on the locations and signs of maximal coordinates, transmitting no continuous information about weight magnitudes within a fixed stratum.The resulting stationarity equations may still constrain inputs through the feature map.Source: §A.6 · p. 12

A.7 Discussion of Theorem A.10 (from §4.2)

The discussion distinguishes Adam’s underdetermined stationarity system from Muon’s set-valued target, and explains why fixed-target losses can collapse without aligning candidates to the data. These results depend on stated nondegeneracy and local regularity assumptions.

A.7 Discussion of Theorem A.10 (from §4.2)

  • Under Assumption A.9, Adam’s stationarity system has at most nI normal equations for nI(d+1) unknowns and fails regular first-order identifiability.With the additional constant-rank and relative-interior hypotheses, nearby stationarity-consistent candidates form a positive-dimensional family whose input projection has dimension at least nI(d−1).Source: §A.7 · p. 12
  • The Adam result concerns idealized ℓ∞ bias and this stationarity-only attack; actual local ambiguity additionally requires Theorem A.10(iii)’s hypotheses.The network claim rests on Assumption A.9, and the result is not a differential-privacy-style guarantee.Source: §A.7 · p. 13

A.8 Set-valued target and collapse (from §3.3)

  • A fixed-target loss can minimize at zero multipliers when the residual points oppositely to its target, whereas the alignment loss is constant along multiplier rays.Thus the fixed-target loss can switch candidates off instead of aligning them with the data; this ray argument is a mechanism, not proof of failure over the full domain.Source: §A.8 · p. 13

A.9 Remarks on Lemma 4.2 (from §4.1)

  • Muon’s specificity template persists at higher singular-value multiplicity, while its scale-free target cone omits singular-value magnitudes.A loss cannot recover continuous target information absent from the constraint, although the fixed cone can still constrain inputs.Source: §A.9 · p. 13

A.11 Per-tensor form of the Adam objective (from §3.4)

  • For the per-tensor Adam objective, the relaxation is strict at general weights, but its zero set matches the global form at a KKT direction.At a KKT direction every tensor contains a globally maximal coordinate.Source: §A.11 · p. 14

B.3 Scale statements and the Euler identity

  • Rescaling nonnegative multipliers scales the residual without changing feasibility, allowing a nonzero residual to be normalized to the checkpoint norm in the dual norm.This establishes an equivalence between membership in the subdifferential target and membership in its positive cone for the alignment formulation.Source: §B.3 · p. 15

A.8 Set-valued target and collapse (from §3.3)

  • Muon’s KKT target can be set-valued: when a block’s top singular value has multiplicity above one, stationarity does not select a unique point target.Even with simple top singular values, the relative block weights are determined through the Euler identity and depend on block weights and degrees.Source: §A.8 · p. 13, §B.5 · p. 16

B.6 Per-block versus global tightness for Muon

  • Muon’s per-block alignment condition is exact at a KKT direction because every block attains the maximum, making the global tightness condition vacuous there.The blockwise condition enforces tightness within each block; the additional global condition would matter only if some block were inactive.Source: §B.6 · p. 16

B.7 Specificity and collapse

The optimizer-dependent cone has different continuous specificity across geometries: ℓ2 tracks the checkpoint direction, spectral geometry tracks blockwise partial isometries, and ℓ∞ depends only on maximal-coordinate signs. At nondegenerate ℓ∞ KKT points, the resulting stationarity constraints are insufficient for regular first-order identification.

B.7 Specificity and collapse

  • For ℓ2, the target cone is determined by the checkpoint direction, giving specificity S = p−1.The direction uniquely determines the cone.Source: §B.7 · p. 17
  • For spectral geometry, the target cone determines blockwise partial isometries, whose continuous variability has dimension Sk(ρk).For simple top singular values, Sk = mk + nk − 2.Source: §B.7 · p. 17
  • For ℓ∞, the target cone depends only on the maximal-coordinate locations and signs, so its continuous specificity is zero for every maximum multiplicity.The cone is discrete data rather than a continuous functional of the checkpoint.Source: §B.7 · p. 17
  • In linear-in-parameter models, a unique ℓ∞ max-margin LP solution is a vertex and gives the span condition used for nondegeneracy.The active-constraint normals span the parameter space under strict complementarity.Source: §B.7 · p. 17

C LOCAL IDENTIFIABILITY

The paper formalizes regular first-order identifiability as a rank condition that permits only the multiplier-scaling ambiguity. It proves this condition fails for ℓ∞ at nondegenerate KKT points, while the positive ℓ2 and spectral cases remain only partially established.

C LOCAL IDENTIFIABILITY

  • Regular first-order identifiability requires Jacobian rank nI(d+1)−1, excluding infinitesimal ambiguities beyond the one-dimensional multiplier gauge.Maximal rank is sufficient for local isolation modulo that gauge, but rank deficiency alone does not rule out an isolated nonlinear solution.Source: C LOCAL IDENTIFIABILITY · p. 18
  • Regular first-order identifiability requires codim C(θ) ≥ nI(d+1)−1, a necessary condition that ℓ2 and spectral geometry satisfy in the stated over-parameterized regime.The condition fails for ℓ∞ at nondegenerate KKT points; under Theorem A.10(iii)’s additional hypotheses, those points belong to a positive-dimensional local family.Source: C LOCAL IDENTIFIABILITY · p. 18
  • The paper proves Adam’s failure of regular first-order identifiability, but proves only a necessary condition for the positive ℓ2 and spectral cases.Sufficiency of high specificity and high codimension for regular first-order well-posedness remains conjectured.Source: C LOCAL IDENTIFIABILITY · p. 18
  • On binary MNIST with n = 500, Muon recovers sharp digit structure even without an image prior.Figure 3 compares reconstructions without a prior and with the range prior; odd rows show reconstructions and the rows below show matched training images.Source: Figure 3 · p. 19

D EXPERIMENTAL DETAILS

The experiments use MNIST and CIFAR-10 classifiers trained with SGD, Adam, or a hybrid Muon setup, then reconstruct inputs by optimizing the KKT stationarity conditions. Reconstruction quality is evaluated after matching recovered samples to training examples.

D EXPERIMENTAL DETAILS

  • The 10-class CIFAR-10 evaluation uses a class-balanced 1,000-image test subset (100 per class); trained configurations use N = 500 samples, with 50 per class for the 10-class setup, and interpolate the training set.Source: D EXPERIMENTAL DETAILS · p. 19, Table 4 · p. 20
  • Muon uses five-step Newton–Schulz updates on hidden matrices and SGD on the input and output layers, yielding a mixed spectral- and Euclidean-geometry bias.The first layer uses variance 10^-4 initialization and later layers use Kaiming normal initialization.Source: D EXPERIMENTAL DETAILS · p. 19
  • The reconstruction attack optimizes dual variables and input samples against optimizer-specific KKT stationarity losses.It uses SGD with momentum 0.9 for 100,000 iterations; losses follow the respective stationarity equations for SGD, Adam, and Muon.Source: D EXPERIMENTAL DETAILS · pp. 19–20
  • A modified ReLU backward pass supports gradient-based reconstruction while leaving the forward pass as exact ReLU.Its parameter α interpolates between a scaled linear backward pass as α approaches zero and the standard ReLU derivative as α approaches infinity.Source: D EXPERIMENTAL DETAILS · p. 20
  • Reconstruction settings are selected by random search on SSIM@30, and recovered samples are matched one-to-one to training examples using centered, normalized ℓ2 distance.The best-matched instances, such as the top 30 pairs, are used for reporting.Source: D EXPERIMENTAL DETAILS · p. 20

E ADDITIONAL EXPERIMENTAL RESULTS

Across the reported datasets and prior regimes, Muon achieves the strongest reconstruction metrics, including without prior constraints. The section also reports that recoverability concentrates near the training margin and declines at larger sample sizes.

E ADDITIONAL EXPERIMENTAL RESULTS

  • Muon achieves the superior value in every reported metric across datasets and prior configurations, including substantially higher structural fidelity without prior constraints.The reported metrics are PSNR, SSIM, L1 distance, and L2 distance; Tables 5–7 cover binary and multiclass MNIST and CIFAR-10 under multiple priors.Source: E ADDITIONAL EXPERIMENTAL RESULTS · pp. 20–21
  • In multiclass CIFAR-10, recognizable reconstructions occur only near the minimal training margin; Muon produces the most and Adam essentially none.Figure 4 plots per-sample SSIM against margin for 500 range-prior samples, with class colors and an SSIM = 0.4 reference line.Source: Figure 4 · p. 21, E ADDITIONAL EXPERIMENTAL RESULTS · p. 21
  • SSIM@30 peaks at n = 100 and drops substantially for n ≥ 500, indicating a harder inverse problem as dataset size increases.The comparison selects the top 30 instances from a candidate pool of m = 2n.Source: E ADDITIONAL EXPERIMENTAL RESULTS · p. 21
  • Each attack iteration costs q differentiable parameter-gradient evaluations plus O(p) residual aggregation; Muon additionally computes blockwise singular values.Here q is the candidate count, p the parameter count, and d the input dimension; explicit attack-state memory is O(qd+p), excluding activations and autodifferentiation workspace.Source: E ADDITIONAL EXPERIMENTAL RESULTS · p. 21

F.1 Notation details (from §2.1)

The appendix defines the geometries and regularity assumptions used in the analysis, places KKT reconstruction among other privacy attacks, and reviews the implicit-bias basis for the reconstruction approach.

F.1 Notation details (from §2.1)

  • In the ℓ2, spectral, max-spectral, and ℓ∞ geometries, alignment is characterized by positive scaling, top singular subspaces, active blocks, or largest-magnitude coordinates with matching signs.The generic stratum excludes ties and other degeneracies; Assumption 2.1 absorbs a first-layer bias into the input block, and differentiability holds almost everywhere for ReLU networks.Source: §F.1 · pp. 21–22

F.2 Other privacy attacks (from §2)

  • KKT-based reconstruction uses only final model weights, unlike membership inference, model inversion, or gradient leakage, while optimizer choice does not provide differential privacy’s formal guarantee.Differentially private training limits individual examples’ influence and may incur a utility cost.Source: §F.2 · p. 22

F.3 Implicit bias of gradient flow (from §2.1)

  • For homogeneous ReLU networks trained by gradient flow with logistic or exponential loss, the stated assumptions yield convergence in direction to a first-order stationary point of the maximum-margin problem.The KKT conditions include nonnegative multipliers and complementary slackness, so only examples on the margin can have nonzero multipliers.Source: §F.3 · pp. 22–23

F.4 Implicit-bias background (from §2)

  • The reconstruction rationale links optimizer-selected margin solutions to stationarity equations; on CIFAR-10 and MNIST without priors, Muon yields higher SSIM and more images above 0.4 than SGD and Adam.The ordering reported for those reconstructions is Muon > SGD > Adam.Source: §F.4 · p. 23, Figure 6 · p. 25

Related papers

Loading 2610.18294v1…