PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR

Chongyu Fan, Gaowen Liu, Mingyi Hong, Ramana Rao Kompella, Sijia Liu

arXiv:2605.19282v1cs.LG

TL;DR

Muon’s suitability beyond pretraining is underexplored in VLA and RLVR, where its spectral whitening has limitations. Pion replaces uniform whitening with a high-pass NS iteration and consistently outperforms AdamW and Muon across the evaluated VLA and RLVR settings, including cases where Muon collapses.

  • Problem

    Muon’s core matrix-sign inductive bias remains underexplored in non-pretraining regimes such as VLA and RLVR.

  • Method

    Pion uses a two-stage Promotion+Suppression high-pass NS iteration that preserves leading singular directions, suppresses noisy tails, and supports per-head updates at Muon’s per-step cost.

  • Results

    Pion consistently outperforms AdamW and Muon across VLA training on LIBERO/LIBERO-Plus and RLVR post-training on Qwen3-1.7B/4B over MATH and GSM8K, including settings where Muon collapses.

  • Takeaways & Limitations

    The results support Pion as a computationally efficient Muon replacement for the evaluated VLA and RLVR training settings.

  • Takeaways & Limitations

    Conventional Muon lacks rank adaptiveness across heterogeneous modules, while explicit low-rank projection adds computational overhead and limits scalability.

Abstract

from arXiv · show

Muon is a matrix-aware optimizer that leverages Newton-Schulz (NS) iterations to enforce spectral gradient orthogonalization by driving all singular values of the momentum matrix toward 1. While this uniform spectral whitening enhances exploration and outperforms AdamW in LLM pretraining, we show it could lead to fundamental limitations beyond pretraining in two regimes: (i) cross-modality vision-language-action (VLA) training, where inherently low-rank action-module gradients cause amplification of noisy tail directions, and (ii) reinforcement learning with verifiable rewards (RLVR), where low-SNR gradients and the need to preserve per-head specialization from prior training make whitening unstable. To address these challenges, we propose Pion, a drop-in replacement for Muon that preserves its computational efficiency while replacing uniform spectral whitening with a two-stage Promotion+Suppression mechanism, which we call the high-pass NS iteration. This design induces a sharp spectral high-pass effect, anchoring dominant singular values at 1 while suppressing noisy tail components toward 0, with controllable filter strength. To preserve pretrained per-head heterogeneity, Pion also supports a per-head mode that applies updates independently across attention heads via a simple reshape, at no extra cost. In VLA training on LIBERO and LIBERO-Plus, Pion consistently outperforms both baselines across l_1-regression (VLA-Adapter) and flow-matching (VLANeXt) architectures, e.g., reaching 100% success rate on LIBERO Object after 1,500 training steps with VLA-Adapter, vs. 97.0% for Muon and only 32.2% for AdamW. The advantage of Pion further extends to a real Franka Research 3 robot with a pi_0.5 backbone under the DROID setup on three grasp-and-place tasks. In RLVR post-training on Qwen3-1.7B/4B with GRPO and GMPO, Pion also outperforms AdamW on MATH and GSM8K while Muon collapses to zero.

1 Introduction

The paper identifies a shared spectral mismatch behind Muon’s limitations in VLA and RLVR beyond pretraining: uniform whitening amplifies noisy directions when informative gradients are concentrated in leading singular values. It proposes Pion, a computationally equivalent high-pass NS alternative that preserves dominant directions, suppresses noise, and supports cost-free per-head updates.

  • Motivation: Muon’s desirability beyond pretraining is questioned because its matrix-sign operation drives all singular values toward 1.The paper frames this as an underexplored inductive-bias question in non-pretraining regimes.
  • Spectral mismatch: In VLA, low-rank action gradients and, in RLVR, low-SNR policy gradients concentrate information in leading singular values while noisy tails dominate.Muon’s uniform whitening elevates those noisy tails to the magnitude of informative directions.
  • Method: Pion replaces Muon’s uniform whitening with a two-stage Promotion+Suppression polynomial iteration called high-pass NS.The method redesigns the scalar polynomial map applied during Newton–Schulz iterations.
  • Method: Pion preserves leading singular directions, suppresses noise, and matches Muon’s per-step computational cost without SVD or sketching.Its per-head mode applies the iteration independently across attention heads through a simple reshape at no additional cost.
  • Results: Pion consistently outperforms AdamW and Muon across VLA and RLVR evaluations while matching Muon’s computational efficiency.Evaluations cover ℓ1-regression and flow-matching heads on LIBERO and LIBERO-Plus, a real Franka Research 3 robot, and Qwen3-1.7B/4B RLVR with GRPO and GMPO on MATH and GSM8K.

2 Related Work

Prior work develops matrix-aware optimizers and diverse VLA architectures, while RLVR builds on policy-gradient and RLHF methods. This paper identifies cross-modal VLA optimization as overlooked, emphasizing low-rank action-module gradients and the need for rank-adaptive optimization.

  • Muon and matrix-aware optimizers: Matrix-aware optimizers exploit weight spectral geometry, with Shampoo and SOAP using memory-intensive Kronecker-factored preconditioners and Muon orthogonalizing momentum through Newton-Schulz iterations.Muon variants target per-parameter learning rates, noise robustness, Newton-Schulz coefficients, and distributed orthonormalization.
  • Vision-language-action models: VLA models convert pretrained vision-language models into closed-loop robot policies, differing mainly in action heads, tokenization, and decoder designs.The cited action-head families include ℓ1-regression and flow-matching, alongside discrete and diffusion decoders.
  • Vision-language-action models: Cross-modal VLA optimization is overlooked because action-module gradients are low-rank and call for a rank-adaptive optimizer.This motivates treating optimization as distinct from the architectural choices that otherwise differentiate VLA models.
  • RLVR and policy optimization for LLM reasoning: RLVR turns programmatic verifiers into post-training rewards, building on classical policy gradients and reinforcement learning from human feedback.Subsequent work primarily refines the GRPO objective through changes such as importance-ratio normalization.

3 Muon and Two Underexplored Training Regimes: VLA and RLVR

Muon orthogonalizes momentum spectrally by mapping positive singular values to 1, with Newton–Schulz iterations providing a decomposition-free approximation. The paper examines Muon beyond LLM pretraining in VLA multimodal policy learning and RLVR verifiable-reward post-training, where its limitations motivate further study.

  • Muon as spectral optimization: Muon updates weight matrices using momentum transformed by a matrix sign operator that preserves singular vectors while mapping every positive singular value to 1.This produces spectrally isotropic updates with equal magnitude across singular directions.
  • Newton–Schulz iterations in Muon: Newton–Schulz iterations approximate Muon’s matrix sign operation through low-order matrix polynomials, avoiding the significant overhead of explicit SVD computation.The input is Frobenius-normalized so its singular values lie within [0, 1].
  • Underexplored regimes for Muon beyond LLM pretraining: The paper focuses on two underexplored regimes beyond LLM pretraining: multimodal vision-language-action training and reinforcement learning with verifiable rewards.These settings expose limitations of the conventional Muon design that can hinder broader adoption.
  • VLA: VLA trains policies mapping visual observations and language instructions to continuous robot actions through a VLM backbone and action head.The action head may use either ℓ1 regression or flow matching.
  • RLVR: RLVR further updates an SFT-initialized policy with rule-based verifiable rewards through rollout, scoring, and policy-update stages using GRPO or GMPO.Unlike SFT, RLVR uses policy gradients rather than matching token-level teacher signals on offline demonstrations.

4 Rethinking Muon in Heterogeneous and Noisy Training Regimes

The section identifies two limitations of conventional Muon beyond pretraining: it cannot adapt to rank heterogeneity in VLA modules and its uniform whitening destabilizes low-SNR RLVR gradients. These failures motivate more rank- and noise-adaptive spectral optimization.

  • Rank adaptiveness in cross-modality VLA training: Conventional Muon lacks rank adaptiveness across heterogeneous VLA modules, causing suboptimal performance, while explicit low-rank projection adds substantial computational overhead.LRMuon adapts action-module updates through SVD or Gaussian sketching, but both are substantially slower than Newton-Schulz iterations.
  • SNR tolerance for RLVR post-training: GRPO exhibits consistently lower gradient SNR than SFT, reflecting additional noise in RLVR post-training.The comparison uses AdamW with Qwen3-1.7B trained on MATH levels 3–5 and evaluated on MATH500.
  • SNR tolerance for RLVR post-training: Near-zero MATH500 accuracy under Muon contrasts with steadily improving AdamW accuracy, as uniform whitening amplifies noisy directions and corrupts the policy.Muon accuracy drops from the initial checkpoint and converges to near zero during GRPO training.
  • SNR tolerance for RLVR post-training: Muon’s uniform spectral whitening is ill-suited to low-SNR post-training because it amplifies noisy gradient directions to the same magnitude as informative ones.Both VLA rank failures and RLVR noise failures are attributed to inappropriate spectral exploration induced by the msign operator and Newton-Schulz iterations.

5 Pion: sPectral hIgh-pass Optimization on momeNtum

Pion replaces Muon’s uniform spectral whitening with a two-stage high-pass Newton–Schulz iteration that promotes dominant singular values and suppresses noisy tails. It preserves Muon’s per-step cost while offering a per-head mode to retain heterogeneous attention-head updates, especially for RLVR.

  • Pion optimizer: Pion inherits Muon’s control flow and per-step cost while changing only the Newton–Schulz iteration coefficients.It chains Promotion and Suppression stages to create a high-pass spectral filter.
  • High-pass NS mechanism: The high-pass iteration uses kp Promotion steps followed by ks = 5 − kp Suppression steps, with k = 5 by default and kp controlling the spectral cutoff.Promotion reinforces dominant singular values, while Suppression attenuates smaller components.
  • High-pass NS mechanism: Promotion uses fp(σ) = 1.875σ − 1.25σ^3 + 0.375σ^5, monotonically amplifying singular values while preserving their relative ordering.The selected ap = 1.875 maximizes promotion within the derived admissible range ap ∈ [0, 1.875].
  • High-pass NS mechanism: Suppression uses fs(σ) = 2.5σ^3 − 1.5σ^5, pinning large singular values at 1 and contracting small singular values toward 0.Its zero linear term near the origin drives small components toward zero through higher-order terms.
  • Per-head mode: Per-head mode reshapes attention projections into independent head sub-matrices, enabling heterogeneous, layer-dependent updates that default-mode Pion fails to provide.The paper uses default mode for VLA training and per-head mode for RLVR post-training because pretrained attention heads have functionally meaningful norm heterogeneity.

6 Experiments

Across VLA and RLVR experiments, Pion consistently outperforms AdamW and Muon, while improving training efficiency and avoiding Muon’s collapse under low-SNR RLVR gradients. Its advantage extends from benchmark VLA tasks to real-robot evaluation and is specifically tied to high-pass filtering.

  • VLA benchmark results: Pion improves over Muon on every VLA-Adapter LIBERO suite and reaches high success-rate regimes in substantially fewer training steps than both baselines.VLA-Adapter is evaluated on four LIBERO suites under fixed budgets of 1,500 steps for Object and 15,000 steps for the others.
  • Real-robot evaluation: 85.6% average success rate: Pion outperforms AdamW at 31.1% and Muon at 38.9% on three real-robot grasp-and-place tasks.All optimizers use the same DROID dataset and 20,000 training steps, with success measured over 30 randomized trials per task.
  • Additional VLA studies: Pion beats LRMuon across all top-k ranks, while modality-wise sweeps prefer Muon for vision/language and Pion for action.Per-head Pion on the action head also beats Muon and AdamW but underperforms the default mode.
  • RLVR results: Muon’s accuracy remains near zero across eight RLVR settings, whereas Pion succeeds under GRPO and GMPO on Qwen3-1.7B/4B with MATH and GSM8K.The observed failure is attributed to Muon’s uniform whitening amplifying noisy directions under low-SNR RLVR gradients.
  • Filter-direction ablation: LPMuon fails to train and remains at the initial checkpoint, showing that reversing Pion’s high-pass filter into a low-pass mapping removes the observed RLVR benefit.LPMuon retains Muon’s NS structure and per-step cost while contracting large singular values and amplifying small ones.

7 Conclusion · Appendix

The paper identifies rank- and noise-adaptiveness limitations in Muon beyond LLM pretraining and proposes Pion, a same-cost high-pass NS replacement. Pion preserves leading singular directions, suppresses noisy tails, and consistently outperforms AdamW and Muon across the reported VLA and RLVR settings.

  • 7 Conclusion: Muon lacks rank adaptiveness in cross-modality VLA training beyond LLM pretraining.This is identified as one of two limitations of Muon.
  • 7 Conclusion: Muon lacks noise adaptiveness in RLVR post-training beyond LLM pretraining.This is identified as the second limitation of Muon.
  • 7 Conclusion: Pion is proposed as a drop-in replacement for Muon’s Newton-Schulz iteration.The replacement directly targets Muon’s identified limitations.
  • 7 Conclusion: Pion’s high-pass Newton-Schulz iteration preserves leading singular directions while suppressing the noisy tail.This mechanism provides rank- and noise-adaptive spectral behavior.
  • 7 Conclusion: Pion operates at the same per-step cost as Muon.The proposed replacement retains Muon’s computational cost per optimization step.
  • 7 Conclusion: Pion consistently outperforms AdamW and Muon across VLA training on LIBERO/LIBERO-Plus and RLVR post-training.The passage reports consistent gains across both application regimes.

A Additional Preliminaries: VLA Training and RLVR Training … C.4 On-policy SNR comparison

The appendix defines the VLA and RLVR training setups, specifies LRMuon, and analyzes why on-policy GRPO can have lower SNR than SFT. The comparison identifies degenerate reward groups, extreme task difficulty, and low score distinctiveness as key sources of weak RLVR signal.

  • A.1 VLA action heads and training objectives: VLA experiments use deterministic ℓ1-regression and flow-matching action heads to model actions conditioned on multimodal inputs.VLA-Adapter instantiates the ℓ1-regression head, while VLANeXt instantiates the flow-matching head.
  • A.2 RLVR training: GRPO and GMPO: RLVR alternates rollout, verifier scoring with group-relative advantages, and clipped policy updates, comparing GRPO’s arithmetic ratio aggregation with GMPO’s geometric aggregation.GRPO aggregates importance ratios at token level, whereas GMPO uses a sequence-level geometric mean.
  • B Low-rank Muon (LRMuon) Algorithm: LRMuon replaces Muon’s Newton-Schulz polar-factor approximation with an exact SVD-based top-k partial-isometry update.Given Mt = UΣV⊤, LRMuon truncates to the top-k singular subspace before forming the update.
  • C SNR Analysis for SFT and RLVR: The SNR appendix derives matched-batch, per-step SNR expressions for SFT and RLVR and accounts for noise sources unique to RLVR.The variance definition is the total scalar variance and matches the main-text Frobenius-based SNR denominator after vectorization.
  • C.1 Setup and gradient estimators: The analysis models RLVR with binary verifier rewards, group-relative advantages, token log-probability scores, and on-policy GRPO gradients under regularity assumptions.Assumptions include constant per-token score variance, temporal uncorrelatedness, i.i.d. Bernoulli rewards, and conditionally independent trajectories.
  • C.2 SFT variance and SNR: SFT uses a deterministic coefficient −1 on every token, while its mean and variance are combined to obtain the SFT SNR.The displayed SFT signal includes g, T, and the mean score direction ¯s.
  • C.3 GRPO variance and SNR (on-policy): Degenerate GRPO groups with K ∈{0, g} have zero advantage and contribute no signal, while non-degenerate groups provide the reward-dependent signal.The non-degenerate probability is qnd = 1 − p^g − (1 − p)^g, and the large-g coefficient approaches p(1 − p).
  • C.4 On-policy SNR comparison: On-policy RLVR-to-SFT SNR becomes large when p approaches 0 or 1, or when successful and failed rollouts have nearly identical score directions.In the large-g approximation, the ratio is ∥¯s∥2/(p(1 − p)∥∆∥2).

C.5 Additional SNR degradation in GRPO · C.6 Combined bound · D SVD Factorization of Newton–Schulz Polynomial Iteration

The analysis identifies three GRPO-specific SNR degradations—off-policy importance sampling, clipping, and degenerate reward groups—and combines them into an off-policy variance and SNR bound. It also shows that Pion’s matrix-level Newton–Schulz high-pass design reduces through the SVD to scalar polynomial filtering of singular values while preserving singular vectors.

  • C.5 Additional SNR degradation in GRPO: Off-policy GRPO updates multiply the variance term in (A19) by (1 + χ2), where χ2 is zero on-policy and grows with each inner gradient step.χ2 is the per-token chi-squared divergence between πΘ and πold.
  • C.5 Additional SNR degradation in GRPO: Clipping attenuates GRPO SNR by (1 −α), because signal-squared scales by (1 −α)2 while variance scales by (1 −α).The clipping mask is modeled as a Bernoulli(1 −α) mask independent of the per-token score in the mean-field sense.
  • C.5 Additional SNR degradation in GRPO: As p →0 or p →1, degenerate reward groups reduce effective batch size because many groups have zero reward variance, yielding zero normalized advantage and no learning signal.This weakens GRPO signal beyond the large-g approximation p(1 −p).
  • C.6 Combined bound: The on-policy variance combined with importance sampling and clipping yields an off-policy variance and SNR ratio bound.Clipping is handled as a direct SNR effect rather than absorbed into the variance bound because it scales signal-squared and variance simultaneously.
  • D SVD Factorization of Newton–Schulz Polynomial Iteration: For an odd quintic Newton–Schulz matrix polynomial, SVD factorization converts the matrix map into a scalar polynomial f(σ; a, b, c) = aσ +bσ3 +cσ5 applied entrywise to singular values.This reduces matrix-filter design to choosing three scalar coefficients on [0, 1].
  • D SVD Factorization of Newton–Schulz Polynomial Iteration: The SVD factorization preserves the left and right singular vectors U and V unchanged while independently reshaping each singular value σi.The compact SVD setup uses X = UΣV⊤ with Σ containing the strictly positive singular values.
  • D SVD Factorization of Newton–Schulz Polynomial Iteration: Composing multiple Newton–Schulz steps composes their scalar polynomials, enabling Pion to chain Promotion for kp steps and Suppression for ks steps into a composite high-pass filter.The composite acts entrywise on Σ while leaving (U, V) untouched throughout.

E Derivation of the Promotion and Suppression Polynomials … G Per-Head Norm Heterogeneity Affects Forward and Backward Computation

The paper derives Pion’s high-pass Newton–Schulz filter by chaining Promotion and Suppression polynomials with complementary spectral constraints. It specifies whole-matrix and per-head implementations, motivated by how heterogeneous head norms affect attention behavior and gradient scales.

  • E Derivation of the Promotion and Suppression Polynomials: Pion chains two odd quintic maps, selecting Promotion and Suppression coefficients so the composite iteration realizes a high-pass filter on normalized singular values.The scalar map is f(σ; a, b, c) = aσ + bσ^3 + cσ^5, and the chained iteration is f_s^k_s ◦ f_p^k_p.
  • E.1 Promotion polynomial fp: Promotion fixes σ = 1, makes it first-order stationary, imposes boundary concavity, and preserves singular-value ordering while remaining within the spectral budget.Within the constrained family, these conditions support global monotonicity on [0, 1] and ensure f_p(σ) ≤ f_p(1) = 1.
  • E.1 Promotion polynomial fp: 1.875 is the chosen maximal feasible Promotion slope at the origin, yielding coefficients (a_p, b_p, c_p) = (1.875, −1.25, 0.375) for k = 5 iterations.The origin slope controls amplification of small singular values, motivating the maximal feasible choice under the fixed iteration budget.
  • E.2 Suppression polynomial fs: Suppression fixes and stabilizes σ = 1 while setting f′_s(0) = 0, uniquely producing a_s = 0, b_s = 2.5, and c_s = −1.5.Removing the linear term drives small singular values toward 0 through the higher-order terms, while monotonicity preserves their relative ordering.
  • F The Pion Optimizer: Full Algorithmic Description: Pion is a drop-in Muon replacement that swaps per-step orthogonalization for five high-pass NS iterations, split into k_p Promotion and k_s = 5 − k_p Suppression steps.The default mode processes each weight matrix as a whole, while the per-head mode applies the same filtering independently after reshaping attention projections.
  • F The Pion Optimizer: Full Algorithmic Description: Per-head filtering adds only a single reshape and independently applies high-pass NS across attention heads, enabling head-specific normalization without extra computational cost.The algorithm splits the attention projection along the head dimension, processes heads in batched Promotion and Suppression loops, then reshapes back.
  • G Per-Head Norm Heterogeneity Affects Forward and Backward Computation: Per-head norm heterogeneity modulates both attention behavior and gradient scale, so whole-matrix spectral optimization can equalize update scales across heads and mix their inherited specialization.Q/K norms affect attention sharpness, V/O norms affect output magnitude, and trained heads exhibit substantial norm variation.

H Detailed Training Setups for VLA and RLVR Experiments … K Additional VLA Experiments

The appendix documents matched optimizer setups for VLA, RLVR, and real-robot evaluations, and extends VLA analysis with rollout comparisons and targeted ablations. Across these studies, optimizer assignment is isolated as the changed variable while Pion uses suppression-dominant high-pass schedules.

  • H Detailed Training Setups for VLA and RLVR Experiments: AdamW, Muon, and Pion share identical VLA training setups, hardware, and evaluation protocols; only optimizer assignment changes.Pion uses suppression-dominant high-pass NS schedules with k_s ≥3, equivalently k_p ≤2 under fixed total k = 5.
  • H Detailed Training Setups for VLA and RLVR Experiments: Table A1 specifies shared training hyperparameters for the three optimizer configurations across both VLA models on LIBERO.The configurations are applied identically to both models and share all other listed hyperparameters.
  • H Detailed Training Setups for VLA and RLVR Experiments: Table A2 reuses the three optimizer configurations across GRPO and GMPO and Qwen3-1.7B and Qwen3-4B RLVR experiments.Within each benchmark, the configurations share all other listed training and rollout hyperparameters.
  • J Visualization of Real-Robot Rollouts: AdamW failed to lift the cucumber, Muon dropped it during transport, and Pion grasped and placed it cleanly in the Cucumber →Plate rollout.The comparison uses one π0.5 rollout per optimizer and task, with six uniformly sampled frames.
  • J Visualization of Real-Robot Rollouts: On Cube →Plate, AdamW and Muon released the cube prematurely, whereas Pion grasped and placed it accurately.The premature releases occurred before reaching the plate, at frame 3 in either baseline row.
  • K Additional VLA Experiments: The additional VLA appendix studies Pion versus LRMuon, per-head versus default Pion, and modality-wise optimizer assignment across Vision, Language, and Action branches.All three studies use VLA-Adapter action-module training as the reported context.

K.1 Pion vs. LRMuon for VLA training … L Low-pass Muon (LPMuon): Coefficient Design via Constrained Polynomial Fitting

Across VLA ablations, Pion’s adaptive high-pass filtering and modality-aware assignment outperform low-rank or uniform-whitening alternatives, while LPMuon uses constrained quintic fitting to realize a tunable low-pass spectral profile. These results support matching optimizer spectral behavior to each module’s gradient structure without sacrificing Muon-like computational efficiency.

  • K.1 Pion vs. LRMuon for VLA training: Pion outperforms LRMuon at every tested top-k rank while retaining near-Muon computational cost.LRMuon uses exact per-step SVD and a fixed top-k polar factor; Pion instead applies an adaptive soft spectral filter through high-pass NS.
  • K.2 Per-head vs. default Pion on VLA: 100.0% versus 99.6% on Object, 99.4% versus 98.8% on Spatial, 92.4% versus 91.6% on Long, and 97.2% versus 97.4% on Goal compare default and per-head Pion.The default mode leads on three of four LIBERO suites and has a +0.4 four-suite average gap.
  • K.3 Modality-wise optimizer assignment on VLA: 100.0% success for Muon on V/L plus Pion on action exceeds 97.0% for all-Muon and every single-module configuration.The sweep evaluates nine optimizer assignments on VLA-Adapter/LIBERO Object at 1,500 steps.
  • K.3 Modality-wise optimizer assignment on VLA: 73.6% with Pion versus 40.0% with Muon on the action head, compared with 43.6% for AdamW, supports Pion for low-rank action gradients.The action branch is evaluated with Vision and Language fixed at AdamW.
  • K.3 Modality-wise optimizer assignment on VLA: 94.6% with Muon versus 73.8% with Pion for Language, and 96.8% versus 17.8% for Vision, support Muon for high-rank V/L modules.These comparisons hold the other two branches fixed at AdamW.
  • L Low-pass Muon (LPMuon): Coefficient Design via Constrained Polynomial Fitting: LPMuon fits a five-step composition of odd quintic polynomials to approximate a sharp band-indicator target profile.Unlike Pion’s analytically constrained polynomials, LPMuon coefficients are optimized because composition quality depends on the full filter and has parameter redundancies.
  • L Low-pass Muon (LPMuon): Coefficient Design via Constrained Polynomial Fitting: LPMuon applies the fitted scalar filter to each singular value at matrix level, preserving Muon’s per-step 5-matmul cost without explicit SVD.Oddness gives an antisymmetric extension, while the matrix polynomial acts entry-wise on the momentum’s singular values.
  • L Low-pass Muon (LPMuon): Coefficient Design via Constrained Polynomial Fitting: Across τ ∈ {0.1, 0.2, . . . , 0.9}, fitted LPMuon filters anchor |σ| ≤τ at ±1 and drive |σ| >τ toward 0.The constrained objective weights pass-band, stop-band, overshoot, and non-negativity terms as (λp, λs, λo, λnn) = (3, 8, 30, 30).

M Limitations · N Broader Impact

Pion is expected to underperform Muon on LLM pretraining because high-pass filtering discards potentially useful tail directions. Its broader impact includes lower post-training costs alongside dual-use risks from more capable robotic and language-based agents.

  • M Limitations: Pion is expected to underperform Muon on LLM pretraining.The paper leaves adapting Pion to pretraining as future work.
  • M Limitations: Muon benefits LLM pretraining by uniformly lifting every singular value to 1.This uniform whitening maximizes spectral exploration.
  • M Limitations: Pion’s high-pass NS attenuates tail singular values and can discard useful directions.Pion is designed for regimes where informative descent concentrates in a few leading singular values.
  • M Limitations: The paper leaves adapting Pion for LLM pretraining to future work.Its stated limitation is that pretraining does not match the low-rank regime Pion targets.
  • N Broader Impact: Pion lowers training costs for capable VLA policies and RLVR-tuned reasoning LLMs.It stabilizes post-training under the same compute budget as Muon, potentially broadening access to embodied agents and reasoning models.
  • N Broader Impact: Pion’s compute-efficient stabilization may broaden access to embodied agents and reasoning models.The passage specifically links this benefit to VLA policies and RLVR-tuned reasoning LLMs.
  • N Broader Impact: More capable VLA policies and reasoning LLMs carry dual-use risks.The passage identifies unsafe deployment and misuse for harmful content as examples.
  • N Broader Impact: These risks include unsafe deployment of robotic agents and harmful-content misuse by language-based agents.The paper frames these as standard risks associated with more capable VLA policies and reasoning LLMs.
Loading 2605.19282v1…