PaperlayerView exact revision on arXiv

Source-linked AI summary

MeRoTune: RoPE-Safe Merging with a Tunable Dial

Salman Faroz

arXiv:2609.07971v1cs.CL

TL;DR

Weight merging can combine models whose attention subspaces are misaligned, and unconstrained query/key corrections need not preserve attention behavior when RoPE intervenes. MeRoTune restricts corrections to matrices that commute with RoPE and trains them against a blend ratio. The paper reports alignment-tax and benchmark patterns for examined fine-tunes, while noting that its evaluation covers one architecture and one verified pair.

  • Problem

    Weight merging assumes fine-tuned models’ attention subspaces align, while query/key corrections that cancel without RoPE may not preserve behavior when RoPE intervenes.

    Source: §1 · pp. 1–2

  • Method

    MeRoTune trains each fine-tune’s RoPE-commuting correction against a blend ratio, producing a post-hoc merge dial.

    Source: §1 · p. 2

  • Results

    Among examined fine-tunes, some scored below base on every axis, while fine-tune A scored 0.536 vs. 0.536 on the harder subject-knowledge benchmark.

    Source: §7 · p. 10

  • Takeaways & Limitations

    The RoPE-commutant correction leaves each model’s outputs unchanged in isolation and affects behavior only when combined with another model’s correction.

    Source: §3.3 · p. 4

    The evaluation covers one architecture and one verified mutual-gap pair, so its specific numbers should not be assumed to transfer to unrelated pairs without re-running the check.

    Source: §8 · p. 10

Abstract

from arXiv · show

When you merge two fine-tuned models from the same base checkpoint by simply averaging their weights, you implicitly assume their attention subspaces are still aligned. Recent work attempts to fix misalignments by learning an invertible correction matrix, $M$, for each model's query and key projections. This correction cancels out---using $M$ on the query side and $M^{-T}$ on the key side---right before the dot product. However, this cancellation is only exact if nothing sits between the projection and the dot product. In reality, almost all modern open-weight language models put a rotary position embedding (RoPE) exactly there. In this paper, we show that this cancellation is exact under RoPE if and only if $M$ commutes with RoPE's per-position rotation. We derive the specific class of matrices where this holds: a scaled rotation acting independently within each RoPE frequency pair. This forms a strict, low-dimensional subset of the unconstrained matrices that current methods normally train. Building on this, we turn this constrained matrix class into a new merging method. While keeping the base weights entirely frozen, two fine-tunes each learn their own RoPE-compliant correction matrices. We optimize these corrections against a chosen blend ratio so the final result can be adjusted post-hoc like a dial, rather than locked into a single fixed merge. Our default approach trains at one fixed blend ratio, similar to how LoRA sets its scaling hyperparameter in advance. We also experiment with resampling the blend ratio randomly at every training step, and we report the results of both approaches.

1 Introduction

Weight-space merging assumes fine-tuned models’ internal coordinates agree, while learned query/key corrections can address misalignment. Because RoPE intervenes before the attention dot product, safe corrections must commute with its rotations; MeRoTune uses this constrained class for a blend-ratio dial and checks whether alignment is warranted.

1 Introduction

  • Raw weight merging methods combine tensors without checking whether the fine-tunes’ internal coordinate systems agree.The listed methods include weight averaging, task arithmetic [10], TIES [21], DARE [22], SLERP, and Model Stock [11].Source: §1 · p. 1
  • RoPE makes unconstrained query/key corrections unsafe: unless M commutes with its position rotations, the correction can distort relative-position behavior.The paper derives a specific two-parameter family within each RoPE frequency pair for which the cancellation remains exact.Source: §1 · p. 2
  • MeRoTune gives each fine-tune its own RoPE-legal correction, trained against a blend ratio so the merged result can be adjusted at runtime.The default fixes the ratio before training; the paper also tests random resampling at each step.Source: §1 · p. 2
  • The paper introduces a pre-training condition for identifying genuine bidirectional capability gaps and reports that it rejects three superficially plausible model pairs.The check was applied empirically to five candidate models.Source: §1 · p. 2
  • The evaluation compares official TIES and DARE-TIES implementations across blend ratios, fixed versus randomly resampled training ratios, and tied versus independent scaling factors.The paper distinguishes plain averaging as the identity-correction case, which assumes no misalignment rather than correcting one.Source: §1 · p. 2

2 Background

Model-merging methods combine fine-tuned checkpoints or task vectors, but generally treat weights as raw tensors rather than checking their internal bases. RoPE rotates query and key pairs by position, making the attention logit depend on relative position.

2.1 Model merging

  • Task arithmetic, TIES, DARE, SLERP, and related merging methods combine weights or task vectors without checking whether the tensors share an internal basis.TIES trims task vectors and elects coordinate signs; DARE drops and rescales coordinates, while SLERP interpolates along a weight-sphere geodesic.Source: §2.1 · p. 2

2.2 Rotary position embedding

  • RoPE divides each query and key head into frequency pairs and rotates each pair according to token position.The resulting attention logit depends on relative position because rotations compose as R((n−m)θ).Source: §2.2 · p. 2

3 RoPE-Compatible Query-Key Transforms

With RoPE between projection and dot product, the usual query/key correction cancels exactly only when it commutes with RoPE’s rotations. The paper derives the permitted class as independent scaled rotations within frequency pairs, yielding a low-dimensional, well-conditioned correction that preserves each model’s outputs in isolation.

3.1 The cancellation, and what breaks it

  • With RoPE inserted, the corrected attention logit matches the original for every relative position if and only if M commutes with every RoPE rotation.An unconstrained correction can silently distort relative-position dependence; the paper notes this is not visible to a training loss that does not check the identity.Source: §3.1 · pp. 2–3

3.2 The commutant of a single RoPE pair

  • For one non-degenerate RoPE pair, the commuting matrices are exactly M=aI+bJ, a scaled rotation parameterized by two real values.The paper identifies the algebra as isomorphic to the complex numbers and states its nonzero members are invertible and orientation-preserving.Source: §3.2 · p. 3

3.3 The commutant of a full RoPE head

  • For a full head, distinct RoPE frequencies rule out off-diagonal pair blocks, leaving d/2 independent 2×2 scaled rotations in a d-dimensional subspace.This is a d-dimensional subset of the d²-dimensional space searched by the unconstrained construction.Source: §3.3 · p. 3
  • A RoPE-commutant correction preserves a model’s outputs when applied alone, so its effect arises only when combined with another model’s correction.The learned transform can therefore address how the models’ bases are reconciled without changing either model in isolation.Source: §3.3 · p. 4
  • Every nonzero 2×2 commutant block has condition number 1, so these corrections cannot become ill-conditioned during training.The paper contrasts this guarantee with unconstrained invertible corrections, whose conditioning can degrade arbitrarily.Source: §3.3 · p. 4

4 MeRoTune

MeRoTune learns RoPE-compliant corrections for each fine-tune’s query/key projections and combines them at a chosen blend ratio. Its training and parameterization constrain the corrections, while a generalized form allows independent scaling of each fine-tune’s change from the shared base.

4.1 Parameterization

  • The RoPE-commutant parameterization uses 128× fewer parameters per head than unconstrained alternatives while providing RoPE safety and perfect conditioning.For Qwen2.5-1.5B-Instruct, it has 7,168 trainable scalars in total.Source: §4.1 · p. 5

4.2 Dual-sided merge operator

  • Each model learns its own RoPE-legal correction, and their transformed query/key weights are combined at α; plain averaging is the M=I case that assumes aligned frames.At α=0 or α=1, the merge reproduces fine-tune A or B exactly, regardless of the learned corrections.Source: §4.2 · p. 5
  • Alignment is targeted only when both fine-tunes improve on their own specialties and are no better on the other’s; otherwise correction can add parameters and training noise without a genuine conflict to resolve.The paper checks this condition for candidate pairs before training.Source: §4.2 · pp. 5–6

4.3 Training objective

  • Training minimizes the two models’ next-token losses on answer tokens across blend ratios, with the loss terms unweighted by α.The default fixes α=0.5 before training; a separate variant resamples α from [0.05, 0.95] each step, while exploratory extreme fixed ratios generalized poorly beyond their vicinity.Source: §4.3 · p. 6

4.4 Decoupling the blend ratio

  • The decoupled generalization replaces the tied convex blend with independent scales for each fine-tune’s change from the shared base, but increasing both scales does not guarantee simultaneous single-model performance.Setting sa=1−α and sb=α recovers the original blend exactly.Source: §4.4 · p. 6

5 Experimental Setup

The experiments use Qwen2.5-1.5B-Instruct as the shared base and select fine-tunes that satisfy the paper’s capability-gap condition. They compare against mergekit TIES and DARE-TIES and evaluate several benchmarks under fixed and randomized blend-ratio training.

5.1 Base model and fine-tunes

  • Among five candidate Qwen2.5-1.5B-Instruct fine-tunes, only the Indonesian/code and Japanese pair satisfied the capability-gap condition.Candidates were checked on native-language subject-knowledge benchmarks; the Japanese fine-tune was distilled from a Qwen2.5-32B-Instruct teacher.Source: §5.1 · p. 6

5.2 Training data

  • Fine-tune B’s retention set contains 220 filtered examples of general-purpose Japanese instructions, not its distillation data or code-specific examples.Fine-tune A uses a small Indonesian/code instruction set.Source: §5.2 · p. 7

5.3 Baselines

  • The serious merge baselines are official mergekit TIES and DARE-TIES; plain averaging is treated as a passive reference within MeRoTune’s parameter space.The TIES and DARE-TIES runs use density 0.5 and equal 0.5/0.5 weights for each side.Source: §5.3 · p. 7

5.4 Benchmarks and training configuration

  • Evaluation covers MMLU, ARC-Challenge, and full INCLUDE Indonesian and Japanese splits, with 500 training steps and α fixed at 0.5 by default or resampled from [0.05, 0.95].The INCLUDE splits contain 450 Indonesian and 299 Japanese examples; MMLU uses 5,700 examples and ARC-Challenge uses 100.Source: §5.4 · p. 7

6 Results

At fixed training α=0.5, MeRoTune does not lose to TIES on any Table 2 cell, though ARC-C is a noise-floor wash. Learned transforms mostly make small rotations, α trends vary by benchmark, and independent blend coefficients degrade results as both rise.

6.1 Main comparison

  • MeRoTune does not lose to TIES on any Table 2 cell, but its best ARC-C margin is within the reported noise floor.The randomized-α variant beats DARE-TIES at every tested blend ratio but still loses to TIES on Japanese at α=0.5 and ARC-C at α=0.9.Source: Table 2 · p. 7, §6.1 · pp. 7–8

6.2 What the learned transforms look like

  • Typical learned blocks rotate about 10° with scale near 1, while fewer than 1% of blocks rotate beyond 45° and shrink substantially.For blocks with |θ|>45°, mean scale is 0.55 for Ma and 0.60 for Mb; the authors cannot rule out unregularized optimizer capacity as an explanation for this tail.Source: §6.2 · p. 8, Figure 2 · p. 8, Table 3 · p. 8

6.3 Alpha as a dial

  • Across the nine-point α sweep, metrics broadly follow the stronger fine-tune, but trends are not uniformly monotonic; both retention losses fall and remain bounded during training.MMLU is monotonically non-increasing, while ARC-C has no consistent trend; a new α can be evaluated without retraining.Source: §6.3 · pp. 8–9, Table 4 · p. 9, Table 5 · p. 9

6.4 Decoupling the blend ratio

  • Every benchmark degrades as the independent coefficients rise together, so removing the tied convex constraint harms results rather than combining both fine-tunes’ strengths.The tested settings reuse the randomized-α run’s learned transforms without retraining.Source: §6.4 · pp. 9–10, Table 6 · p. 9

7 Discussion

The discussion distinguishes genuine capability trade-offs from broad fine-tuning regressions and limits learned corrections to query/key projections, leaving corrections elsewhere open for future work.

7 Discussion

  • Fine-tunes that score below base on every axis, including their stated specialty, show an alignment tax and are disqualified regardless of comparisons with other candidates.The authors attribute this pattern to general regression from aggressive full-parameter SFT rather than a genuine capability trade-off.Source: §7 · p. 10
  • Fine-tune A gains nothing on the harder subject-knowledge benchmark (0.536 vs. 0.536) despite improving on Belebele-style reading comprehension in an unpublished pre-check sweep.The authors argue the harder benchmark is needed to verify specialty gains rather than relying on a stated specialty or shallower benchmark.Source: §7 · p. 10
  • Learned corrections are restricted to query/key projections; whether unconstrained corrections help value, output, MLP, or embedding weights remains open.Those other tensors have no RoPE rotation to misalign, but adding learned corrections would give up the stated free safety guarantee and expand the search from d-dimensional to d2-dimensional.Source: §7 · p. 10

8 Limitations

The empirical conclusions are bounded by evaluation on one architecture and one verified pair, limited training runs, and benchmark-dependent statistical noise.

8 Limitations

  • All results use one architecture (Qwen2.5-1.5B) and one verified mutual-gap pair, so Section 6 numbers should not be assumed to transfer without rerunning the candidate check.The candidate check rejected superficially attractive pairs that failed its criteria.Source: §8 · p. 10
  • The fixed-versus-randomized blend comparison uses one run of each, so its outcome may reflect this pair or these runs rather than a general property of fixing the ratio.The independent-scaling ablation reuses transforms trained only under the randomized-α objective and comes from a single run; fixed-α transforms and multiple seeds remain unchecked.Source: §8 · p. 10
  • ARC-C at n=100 has SE ≈0.05, making several Section 6 margins suggestive rather than statistically established; MMLU’s 57 subtasks at n=100 each is the sole comparison considered conclusive alone.The reported standard errors are approximately 0.024 for Indonesian (n=450), 0.029 for Japanese (n=299), and 0.007 for MMLU (5,700 examples).Source: §8 · p. 10

9 Conclusion

The conclusion presents RoPE-safe corrections and a pre-training mutual-gap check as foundations for a blend-ratio dial. On the verified pair, the fixed-ratio setup matched or beat TIES and DARE-TIES across tested ratios, while ablations favored tied scaling.

9 Conclusion

  • The fixed-ratio setup never loses to official TIES or DARE-TIES at any tested blend ratio and clears the authors’ statistical noise floor on most benchmarks.The randomized-ratio variant beats DARE-TIES throughout, but does not beat TIES outright.Source: §9 · p. 10
  • The method derives RoPE-safe corrections in closed form and builds a dual-sided merging method that can be scanned as a post-hoc blend-ratio dial.The conclusion contrasts this with producing a single fixed-ratio checkpoint.Source: §9 · p. 10
  • A pre-training mathematical condition checks whether two fine-tunes disagree enough to be worth aligning.Source: §9 · p. 10
  • Decoupling the tied blend ratio into independently scaled factors degrades every benchmark rather than combining the best of both fine-tunes.Source: §9 · p. 10

Related papers

Loading 2609.07971v1…