PaperlayerView exact revision on arXiv

Source-linked AI summary

ISO: An RLVR-Native Optimization Stack

Hanqing Zhu, Wenyan Cong, Zhizhou Sha, Sagnik Mukherjee, Xinyuan Song, David González-Martínez, Xiaoxia Wu, Yuandong Tian, Shiwei Liu, David Z. Pan, Zhangyang "Atlas" Wang

arXiv:2607.19331v1cs.LGcs.AI

TL;DR

RLVR’s weight-update optimization layer remains largely inherited from pre-training despite differing supervision. ISO studies fixed-spectrum optimization by adapting singular frames, achieving strong reasoning and coding gains while matching accuracy with fewer training steps.

  • Problem

    RLVR optimization remains largely inherited from pre-training despite its sparse outcome-level rewards differing from dense token-level supervision.

  • Method

    ISO reuses fixed base spectra while adapting singular frames through offline ISO-Merger and online ISO-Optimizer.

  • Results

    ISO preserves strong reasoning and coding gains, with ISO-Optimizer reaching matched accuracy in 2.7× fewer steps on Qwen3-8B-Base.

  • Takeaways & Limitations

    Spectral inheritance provides a reusable substrate for RLVR post-training: inherit the spectrum and learn how it acts through adaptable frames.

  • Takeaways & Limitations

    ISO adds SVD-based polar-retraction overhead, increasing optimizer-update time by approximately 86 seconds per step in one representative setting.

Abstract

from arXiv · show

Reinforcement learning with verifiable rewards (RLVR) is rapidly advancing the reasoning capabilities of language models, yet the optimization layer that converts reward feedback into weight-space updates remains poorly understood. Building on our prior analysis (Zhu et al., 2025), we study this missing layer through the singular structure of model weights and identify spectral inheritance: RLVR can reuse the base model's weight spectra while acquiring new behavior through changes in the associated input and output singular frames. We operationalize spectral inheritance as Isospectral Optimization (ISO), an RLVR-native, fixed-spectrum optimization framework with complementary offline and online instantiations. Offline, ISO-Merger combines the frame changes of shared-base specialists into a single fixed-spectrum model, requiring no post-merge data, rollouts, gradient updates, or on-policy distillation (OPD). It recovers complementary specialist capabilities and achieves the strongest aggregate performance among the compared data-free merging methods. Online, ISO-Optimizer applies a chosen base optimizer, including AdamW and Muon, to the frame variables while keeping the base spectra fixed. Across reasoning and coding tasks ranging from 1.5B to 8B parameters, ISO-Optimizer improves accuracy in the reported runs and reaches matched scores with substantially fewer training steps. On Qwen3-8B-Base, AdamW reaches an aggregate accuracy of 0.495 after 270 training steps. ISO-AdamW reaches the same accuracy after only 100 training steps and improves further to 0.509 after 210 training steps. Together, ISO offers a concrete answer to RLVR's missing optimization layer: rather than inheriting pre-training optimization wholesale, design post-training around the structure of reward-driven adaptation: inherit the spectrum, optimize the frames.

1. Introduction

RLVR can acquire new behavior while reusing the base model’s weight spectra through changes to both singular frames. ISO operationalizes this structure as a fixed-spectrum RLVR-native optimization stack with offline merging and online optimization.

  • RLVR can reuse the base model’s weight spectra while acquiring new behavior by changing the associated input and output singular frames.
  • Restoring base spectra while retaining learned frames preserves most acquired gains, and fixed-spectrum training that updates only frames still supports strong RLVR learning.
  • Both singular frames must remain adaptable because restricting adaptation to only incoming or outgoing singular subspaces leaves substantially more checkpoint change unexplained.
  • ISO: Isospectral Optimization for RLVR: ISO represents post-training change within fixed-spectrum families of base weights while keeping both singular frames adaptable.
  • ISO: Isospectral Optimization for RLVR: ISO has two complementary instantiations: ISO-Merger composes shared-base specialist checkpoints offline, while ISO-Optimizer performs online learning with optimizers such as AdamW or Muon.

2. Spectra Stay: From Near-Isospectrality to Spectral Inheritance

Unconstrained RLVR checkpoints remain close to the fixed-spectrum families of their base weights, a property termed near-isospectrality. Functional interventions further show that RLVR can reuse the base model’s spectral structure while acquiring behavior through changes in singular frames, establishing spectral inheritance.

  • Long-Horizon Evidence: 10−2%: the RLVR endpoint’s average spectral distance is approximately this value across analyzed layers, while its relative spectral residual averages approximately 3%.The closest fixed-spectrum checkpoint is only a few percent of the base-to-RL displacement.
  • Near-Isospectrality: RLVR checkpoints remain close to the fixed-spectrum families of their base weights across analyzed matrices and throughout the sampled Qwen3-8B-Base trajectory.This descriptive property is called near-isospectrality.
  • Dimensional Calibration: 1.02–1.35: RLVR’s mean κspec values across matrix types show no strong additional suppression of spectrum-changing directions after dimensional calibration.The same order-one pattern persists along the sampled Qwen3-8B-Base trajectory, contrasting with illustrative SFT values between 89 and 1364.
  • Functional Interventions: Restoring the base spectrum preserves most acquired performance, whereas retaining base frames while replacing the spectrum with the RL-trained spectrum does not improve the base model.These interventions separate the functional contribution of the spectrum from that of the associated frames.
  • Spectral Inheritance: Fixed-spectrum parameterization acquires strong reasoning gains and outperforms the AdamW baseline in this run, while spectrum-only training does not achieve comparable gains.Together, the interventions support spectral inheritance: RLVR can reuse the base model’s spectral structure rather than rewrite it.

3. Frames Move: What Changes Under Spectral Inheritance?

The results show that spectral inheritance requires fixing the incoming spectrum while keeping both singular frames adaptable. Freezing either frame leaves substantial endpoint updates unexplained, whereas adapting both frames yields low residuals and remains effective after an RL objective shift.

  • Both frames must remain adaptable: 87% of the cumulative W0 → W2 checkpoint update remains unexplained by remixing within both incoming singular subspaces.The corresponding one-sided restrictions leave 45% and 42% unexplained for the output and input subspaces, respectively.
  • Both frames must remain adaptable: 1.8% is the median residual when the incoming spectrum is retained while both singular frames adapt.This contrasts with the much larger residuals from freezing one or both incoming singular subspaces.
  • Both frames must remain adaptable: Both frame variables must remain adaptable, while the spectrum can remain fixed, among the tested structural restrictions.This finding directly motivates ISO’s parameterization W(U, V) = UΣ0V⊤.
  • Objective-shift stress test: The same frame-adaptability requirement reappears for W1 → W2 after re-anchoring at an already RL-trained checkpoint and changing the RL objective.Stage II starts from the Stage I RL endpoint and optimizes a distinct embodied-manipulation objective.

4. From Spectral Inheritance to Isospectral Optimization (ISO)

Isospectral Optimization (ISO) is an RLVR-native framework that reuses the base spectrum while optimizing both singular frames as adaptable coordinates. It supports offline checkpoint-only expert composition through ISO-Merger and online RLVR training through frame-based ISO-Optimizer updates.

  • Core ISO parameterization: ISO parameterizes weights as W(U, V) = UΣ_0V^⊤, keeping the base spectrum fixed while leaving both singular frames adaptable.In exact arithmetic, every represented matrix belongs to the base model’s fixed-spectrum family; implementation preserves this up to floating-point error.
  • ISO-Merger: ISO-Merger combines shared-base specialists’ learned frames while reusing their common base spectrum, requiring no post-merge data, rollouts, gradient updates, or on-policy distillation.It targets checkpoint-only consolidation of specialists trained from the same base checkpoint.
  • ISO-Merger: ISO-Merger masks unstable trailing modes, aggregates expert frame directions in shared coordinates, then projects, retracts, and reconstructs a merged fixed-spectrum model.Unless stated otherwise, the keep ratio is ρ_keep = 0.9; the procedure targets rather than guarantees unit self-retention after stabilization and feasibility operations.
  • ISO-Optimizer: ISO-Optimizer applies a conventional optimizer directly to frame variables under the fixed base spectrum, rather than updating weights and projecting afterward.ISO-AdamW and ISO-Muon inherit the update equations and optimizer-state dynamics of AdamW and Muon while sharing the same frame parameterization and retraction.
  • Implementation trade-offs: ISO’s trainable factor tensors and optimizer states require more nominal storage than a dense matrix, but sharding and optimizer-state offloading can reduce per-device memory.The fixed spectrum itself requires no optimizer state, and the paper describes these overheads as manageable given observed optimization and training-step savings.

5. Experiments

Experiments show that ISO-Merger provides strong data-free composition of RLVR specialists, while ISO-Optimizer improves RLVR accuracy and convergence across mathematical reasoning and coding tasks. These gains occur under fixed-spectrum constraints and extend across AdamW, Muon, and larger model scales.

  • ISO-Merger: 63.80 overall average on Qwen2.5-7B and 44.38 on DeepSeek-R1-Distill-Qwen-1.5B make ISO-Merger the strongest data-free composition method in both settings.The corresponding strongest training-free baselines score 62.88 and 43.52, respectively.
  • ISO-AdamW: 2.2× fewer training steps let ISO-AdamW match the strongest AdamW run’s final aggregate accuracy on Qwen3-4B-Base, while continuing to improve thereafter.All ISO iterates remain in the constrained fixed-spectrum family F (W0).
  • ISO-AdamW: 28.74 and 43.46 aggregate accuracy make ISO-AdamW strongest at Qwen3-1.7B-Base and Qwen3-4B-Base, respectively.At 1.7B, the best AdamW and Muon runs reach 28.35 and 27.70; at 4B, they reach 41.69 and 42.23.
  • Coding results: ISO-AdamW improves over AdamW on both LiveCodeBench v5 and v6 after 220 training steps on DS-1.5B.The validation curve had plateaued by the 220-step evaluation, and the corresponding training curves show ISO-AdamW leading tuned AdamW baselines.
  • ISO-Muon: 0.428 versus 0.422 is ISO-Muon’s final accuracy on Qwen3-4B-Base, and it matches the strongest Muon run’s final accuracy by step 220 rather than step 300.ISO-Muon uses a fixed-spectrum Stiefel-coordinate Muon update.
  • Larger-scale results: 2.7× fewer training steps let ISO-AdamW reach 0.495 accuracy by step 100, whereas the weight-space AdamW run reaches 0.495 only after 60 additional steps and then plateaus.This larger-scale comparison is obtained without additional tuning at 8B.

6. Related Work

Related work has explored structured matrix optimization and geometry-aware merging, but RLVR’s optimization layer remains comparatively underexplored and typically inherited from pre-training. ISO builds on RLVR’s spectral inheritance to provide a shared fixed-spectrum principle spanning online policy learning and offline expert consolidation.

  • RLVR dynamics and optimization: RLVR research has largely emphasized data, objectives, environments, and infrastructure, leaving reward-to-parameter optimization comparatively underexplored.The optimization layer is typically inherited from pre-training, including the optimizer and parameterization.
  • RLVR dynamics and optimization: Spectral inheritance posits that base-model singular-value spectra remain functionally reusable while associated input and output frames adapt.ISO uses this principle to guide RLVR optimization algorithm design.
  • Matrix-geometric optimization and constrained parameterizations: Matrix-geometric methods structure matrix-valued parameters or updates, including Muon’s orthogonalized momentum updates and DION’s scalable distributed approximations.These approaches act on different objects and impose different structural constraints.
  • Matrix-geometric optimization and constrained parameterizations: Pion independently preserves weight spectra through left and right orthogonal-equivalence transformations and adapts its optimizer to RLVR.Its overlap with ISO is most direct in online RLVR optimization, while the approaches differ in construction and scope.
  • From matrix-geometric priors to an RLVR-native post-training stack: ISO turns RLVR-derived spectral inheritance into a reusable fixed-spectrum interface spanning online reward-driven updates and offline consolidation of independently trained experts.ISO-Optimizer and ISO-Merger are complementary realizations of one post-training stack design, with consolidation requiring no additional rollouts or distillation.
  • Geometry-aware model merging: Geometry-aware merging includes Euclidean task-vector methods such as Task Arithmetic, TIES, and TSV-Merge, alongside OrthoMerge’s Lie-algebra merging of orthogonal transformations.ISO-Merger is specialized to shared-base RL experts.

7. Conclusion … B.1. The Fixed-Spectrum Family and Nearest Representatives

The paper concludes that RLVR can preserve and reuse base-model spectra while adapting singular frames, motivating ISO’s fixed-spectrum merging and optimization framework. The appendices validate this view through qualitative checks, fixed-spectrum theory, optimization details, and experimental protocols.

  • 7. Conclusion: RLVR preserves reusable base-model weight spectra while acquiring behavior through changes in associated singular frames.Restoring base spectra after training preserves most acquired performance, and fixing them during training still supports reasoning and coding gains.
  • 7. Conclusion: ISO-Merger combines shared-base RL experts without post-merge data, rollouts, gradient updates, or distillation, achieving the strongest aggregate performance among compared data-free methods.ISO-Optimizer applies conventional optimizers to frame variables under fixed spectra, reaching matched accuracy in fewer steps, including 2.7× fewer steps on Qwen3-8B-Base.
  • Appendix Outline: The appendices cover spectrum-restoration checks, fixed-spectrum theory, ISO gradients, a same-base SFT–RLVR case study, merging details, data-free experiments, numerical precision, and online RLVR training.Appendix B includes fixed-spectrum distance, nearest representatives, dimension-aware calibration, reconstruction classes, unexplained-update ratios, and rank sensitivity.
  • A. Qualitative Index-Alignment Check: RLVR rank-index singular directions rotate mildly and smoothly, whereas many SFT directions become nearly orthogonal, making rank-index tracking less stable under SFT.These plots are qualitative sanity checks; formal frame-adaptability conclusions use projector-based reconstructions and unexplained-update ratios.
  • B. Fixed-Spectrum Theory and Frame-Adaptability Diagnostics: A thin SVD represents each weight matrix as W = UΣV⊤ with orthonormal singular frames and Σ containing singular values in nonincreasing order, including zero multiplicities.The fixed-spectrum theory uses U ∈ St(dout, q), V ∈ St(din, q), and Σ = Diag(σ(W)).
  • B.1. The Fixed-Spectrum Family and Nearest Representatives: The fixed-spectrum appendix formalizes the family of matrices obtained from a source matrix through two-sided orthogonal transformations that preserve singular values.It also derives the exact distance to this family and nearest representatives using von Neumann’s trace inequality.
  • B.1. The Fixed-Spectrum Family and Nearest Representatives: Nearest fixed-spectrum representatives are basis-independent as a set, but individual representatives can be nonunique under repeated or nearly flat singular-value blocks.Near-degenerate distinct values can make numerical singular vectors ill-conditioned, so rebasing uses the numerical SVD representative and Appendix A checks qualitative stability.
  • B.1. The Fixed-Spectrum Family and Nearest Representatives: The appendix also derives a finite-horizon spectral-drift result for matrix trajectories represented by successive updates E_t = W_t+1 − W_t.This connects cumulative optimization updates with spectral movement over a finite training horizon.

B.2. Spectral-Distance Metrics and Dimension-Aware Calibration … C. First-Order Properties of the ISO Parameterization

The appendix characterizes spectrum-changing directions, compares reconstruction classes for parameterization sufficiency, and tests unexplained-update behavior across truncation ranks. These analyses distinguish fixed-spectrum structure from more permissive subspace-based explanations.

  • B.2. Spectral-Distance Metrics and Dimension-Aware Calibration: For a full-rank W0 with simple singular values, the first-order spectrum-changing subspace is the normal space of its fixed-spectrum family.Fixing q singular values imposes q independent first-order constraints.
  • B.2. Spectral-Distance Metrics and Dimension-Aware Calibration: The spectrum-changing basis consists of the q singular outer products, while isotropic perturbations allocate expected squared energy according to subspace dimension.The relevant dimensions are q for the spectrum-changing subspace and d_out d_in for the ambient space.
  • B.3. Reconstruction Classes and Unexplained-Update Ratios: Frobenius-optimal reconstructions are defined for remix, left-subspace, right-subspace, and isospectral classes, with the corresponding unexplained-update ratios providing endpoint residual measures.The isospectral residual has a closed form, and the reconstruction tests evaluate whether retained incoming spans permit low-residual endpoint descriptions.
  • B.3. Reconstruction Classes and Unexplained-Update Ratios: The remix class fixes incoming and outgoing spans but permits internal rotation, mixing, rescaling, and spectrum changes.Because the remix class contains the internal fixed-spectrum slice, failure of remix also rules out an explanation based solely on internal fixed-spectrum rotations.
  • B.3. Reconstruction Classes and Unexplained-Update Ratios: Reconstruction residuals are not complexity-normalized model-selection results because the classes have different dimensions; they instead test parameterization sufficiency.The more permissive remix and one-sided classes provide optimistic lower bounds relative to associated fixed-spectrum restrictions.
  • B.4. Rank Sensitivity of the Frame-Adaptability Test: Across truncation fractions, the ordering of unexplained-update ratios remains stable: u_iso stays below 2%, while subspace-retaining alternatives leave tens of percent unexplained.Figure 13 reports this pattern across representative language and vision modules and all three transitions.

C.1. Proof of Proposition 4.1 · C.2. Factor-Gradient Derivation

The proof establishes that first-order singular-value changes equal the diagonal of U⊤HV along fixed-spectrum curves. The factor-gradient derivation then shows how ISO updates Stiefel factors while polar retraction keeps represented iterates on the fixed-spectrum set.

  • C.1. Proof of Proposition 4.1: The proof analyzes a differentiable curve W(ε) in F(W0) through W = UΣ0V⊤.The construction assumes all q singular values are positive and simple, enabling locally differentiable Stiefel representatives U(ε) and V(ε).
  • C.1. Proof of Proposition 4.1: Differentiating the fixed-spectrum representation decomposes the curve’s first-order weight motion into changes of its Stiefel factors.
  • C.1. Proof of Proposition 4.1: Stiefel constraints imply that U⊤U̇ and V̇⊤V are skew-symmetric.This follows by differentiating U(ε)⊤U(ε) = I and V(ε)⊤V(ε) = I at ε = 0.
  • C.1. Proof of Proposition 4.1: Because Σ0 is diagonal, the factor-motion terms U⊤U̇Σ0 and Σ0V̇⊤V have zero diagonal.The proof uses this property to isolate the singular-value contribution in the first-order perturbation.
  • C.1. Proof of Proposition 4.1: For each singular value, differentiable singular vectors remain unit vectors, so their normalization constraints determine the relevant first-order identities.The argument applies this construction to k = 1, …, q.
  • C.1. Proof of Proposition 4.1: The diagonal of U⊤HV is exactly the vector of first-order singular-value changes, proving Proposition 4.1.
  • C.2. Factor-Gradient Derivation: For RLVR loss L(W) with gradient GW = ∇WL(W), ISO derives raw gradients for infinitesimal Stiefel-factor changes ξU and ξV.These changes lie in the tangent spaces TUSt(dout, q) and TVSt(din, q), respectively, and induce the corresponding fixed-spectrum weight motion.
  • C.2. Factor-Gradient Derivation: ISO uses Euclidean gradients in ambient factor coordinates rather than explicitly constructing a weight-space projected gradient.The fixed-spectrum parameterization and polar retraction keep every represented iterate in F(W0) up to numerical precision.

D. A Same-Base SFT–RLVR Case Study … E.4. Top-𝑘keep singular-mode masking

A same-base comparison shows that RLVR largely preserves the base spectrum while SFT strongly changes spectrum-related coordinates, and substituting spectra alone does not transfer behavior. ISO-Merger therefore combines specialist frame motion under a shared base spectrum, with sign alignment, tangent projection, and top-mode masking used to stabilize the merge.

  • D. A Same-Base SFT–RLVR Case Study: The controlled study compares distilled SFT and RLVR checkpoints derived from the same Qwen2.5-14B base using seven shared transformer projection matrices.The comparison controls starting backbone weights while retaining differences in training data, objectives, and optimization procedures.
  • D. A Same-Base SFT–RLVR Case Study: RLVR’s per-rank spectral changes are visually near zero, whereas distilled SFT has δΣ approximately 10−2 for most projection types and approximately 0.4 for attention output projection.Both endpoints are measured relative to their shared base checkpoint.
  • D. A Same-Base SFT–RLVR Case Study: RLVR’s mean κspec ranges from 1.4 to 3.2 with overall mean 1.9, while distilled SFT ranges from 117 to 4810 with overall mean 873.The SFT concentration is largest in the attention output projection, while RLVR’s largest deviations occur in the earliest layers.
  • D.1. Reverse Spectral Substitution: Replacing the SFT spectrum with an arbitrary proportion of the RL spectrum while fixing SFT singular frames leaves performance nearly unchanged.This indicates that substituting the RLVR spectrum alone does not transfer acquired behavior into SFT frames.
  • E. ISO-Merger Details: ISO-Merger uses the shared base spectrum Σ0 and merges expert-specific motion through Stiefel factors from experts fine-tuned from the same base.The method is defined for corresponding SVDs of the shared base and KRL experts.
  • E.2. Sign canonicalization: SVD sign ambiguity can create spurious displacement columns with norm ∼2, so ISO-Merger canonicalizes each frame column against the base using the same sign for both frames.Repeated singular values also permit joint in-block rotations, but this freedom is not explicitly canonicalized for generic matrices with simple spectra.
  • E.3. Stiefel tangent projection: ISO-Merger projects left- and right-frame displacements onto their corresponding Stiefel tangent spaces before aggregation.The right-frame tangent is computed analogously to the left-frame projection.
  • E.4. Top-kkeep singular-mode masking: Dkeep retains the leading kkeep columns because trailing small-singular-value modes are noisy, disagree across experts, and degrade the merged model; ρkeep is 0.9 by default.Masked displacements define local first-order effect proxies, with feasibility imposed after aggregation rather than during coordinate selection.

E.5. Retention Coefficients with a Unit-Retention Target … E.8. Full ISO-Merger algorithm

ISO-Merger targets unit self-retention through a stabilized and clipped coefficient system, then reconstructs merged factors whose spectrum matches the base spectrum up to numerical precision. The full procedure applies this construction across two-dimensional weights while averaging one-dimensional parameters with specified algorithmic safeguards.

  • E.5. Retention Coefficients with a Unit-Retention Target: The merger forms a Gram system to determine coefficients for nonzero proxies and omits zero-norm proxies by setting their coefficients to zero.Zero-norm proxies contribute no row or column to the Gram system.
  • E.5. Retention Coefficients with a Unit-Retention Target: The coefficient solve targets unit self-retention, uses ridge stabilization with λridge = 10^-12, and clips coefficients to prevent sign reversal or over-amplification.Unit self-retention is targeted rather than guaranteed after stabilization, clipping, tangent projection, and retraction.
  • E.6. Retraction and reconstruction: The method combines masked displacements and projects the result onto the tangent space at the anchor before retraction.The retraction uses a polar factor; for rank-deficient inputs, the SVD expression selects one valid nearest Stiefel factor, which need not be unique.
  • E.6. Retraction and reconstruction: The retraction arguments are full column rank after tangent projection, and the resulting merged factors and matrix carry the base spectrum Σ0 up to numerical precision.The full-column-rank condition is stated with an analogous identity for V0 + ξV,★.
  • E.7. Merged parameter scope: All per-layer 2D projection matrices and embedding/unembedding matrices use the ISO-Merger construction, while one-dimensional parameters use a standard task-vector average.The one-dimensional parameters include normalization scales and attention biases.
  • E.8. Full ISO-Merger algorithm: Algorithm 1 summarizes the full procedure with keep ratio ρkeep (0.9), ridge λridge (10^-12), and clip range [cmin, cmax] ([0, 1.5]).The algorithm’s stated invariant is a merged model θ★.
  • E.8. Full ISO-Merger algorithm: For each 2D weight matrix, the algorithm processes per-layer projections and embedding/unembedding matrices, then masks trailing singular modes using Dkeep.The algorithm also includes joint sign canonicalization for each singular mode.
  • E.8. Full ISO-Merger algorithm: One-dimensional parameters are merged by task-vector average, using w★ = w0 + 1/K as specified in the algorithm.This algorithmic operation corresponds to the broader one-dimensional parameter scope described in E.7.

F. Data-Free Merging Experimental Details

The data-free merging experiments use shared-base RLVR specialists across Qwen2.5 and DeepSeek-R1-Distill, comparing ISO-Merger with five established baselines. ISO-Merger uses two-sided fixed-spectrum frame changes and achieves the strongest reported aggregate results under both backbones.

  • Backbones and experts: The experiments merge three Qwen2.5-7B-Instruct specialists—CURE, ToolRL, and MemAgent—and two DeepSeek-R1-Distill-Qwen-1.5B specialists—Archer2.0 and JustRL.The specialists target coding, tool use, long-context memory, and math capabilities.
  • Merging baselines: The comparison includes Task Arithmetic, TIES, TSV-Merge, RAM, and OrthoMerge-G-TIES as five data-free baselines.The first four use Euclidean task-vector representations, whereas OrthoMerge-G-TIES adds a geometry-aware orthogonal component.
  • ISO-Merger versus OrthoMerge: ISO-Merger composes two-sided input and output frame changes under the shared base spectrum, unlike OrthoMerge’s one-sided input-space transformation with Euclidean residuals.This two-sided fixed-spectrum parameterization gives ISO-Merger a lower-residual description than retaining only the incoming subspace.
  • Baseline hyperparameters: TIES uses λ=1.0 and ρTIES=0.2, TSV-Merge uses α=1.0 with automatic rank reduction, and RAM uses ε=10^-5.ISO-Merger performs SVD operations in FP64, while OrthoMerge-G-TIES uses FP32 SVD-based Procrustes, 20% retained density, and λ=1.0 for TIES residuals.
  • Results: 54.85 is ISO-Merger’s overall average for worst@4, exceeding 53.23 for the strongest training-free baseline.On DeepSeek-R1-Distill-Qwen-1.5B, ISO-Merger reaches a total average of 34.98 versus 33.62 for the best baseline.

G. Numerical Precision of the SVD-Based Retraction

ISO’s spectral invariant holds in exact arithmetic but can weaken under finite-precision SVD errors. FP64 GPU SVD sharply reduces reconstruction error at a small measured runtime increase, motivating its use for polar retraction.

  • In exact arithmetic, ISO reconstruction preserves diag(Σ0) when U+ and V+ have orthonormal columns.Finite-precision SVD errors can weaken this invariant.
  • 7.7899 × 10−4 versus 2.8924 × 10−8: FP32 GPU SVD has substantially larger reconstruction error than FP64 GPU SVD.The comparison uses a representative q_proj matrix from layer 10 of Qwen3-1.7B-Base.
  • 2.7 × 10^4: FP64 reduces reconstruction error approximately while increasing runtime from 0.894 to 0.923 seconds.ISO therefore uses FP64 GPU SVD for polar retraction throughout.

H. Online RLVR Training Details

Online RLVR experiments use a fixed DAPO setup across weight-space and ISO variants, with task-specific training configurations for mathematical reasoning and competitive coding. ISO training updates frame variables with polar retraction while evaluations and rollout protocols are controlled across methods.

  • Common setup: DAPO experiments run on NVIDIA A100 80 GB GPUs with all non-optimizer settings held fixed between weight-space and ISO variants.Experiments are implemented using Verl.
  • Common setup: Online dynamic filtering removes prompt groups whose sampled rollouts are all correct or all incorrect, using temperature 1.0 and top-p=1.0 for training rollouts.The KL term is applied as loss shaping rather than as part of the rollout reward.
  • Mathematical reasoning: 400 training steps are used for Qwen3-1.7B-Base and Qwen3-4B-Base mathematical reasoning, with global prompt batch size 256 and 16 or 12 rollouts per prompt, respectively.Both models use maximum prompt and response lengths of 1,024 and 8,192 tokens.
  • Mathematical reasoning: 210 training steps compare ISO-AdamW at 7.5 × 10^-7 with weight-space AdamW at 2 × 10^-6 on Qwen3-8B-Base, followed by 60 additional baseline steps.After step 80, both methods use a 16,384-token maximum response length because ISO-AdamW produces longer responses.
Loading 2607.19331v1…