PaperlayerView exact revision on arXiv

Source-linked AI summary

Geometric Calibration of Supervised Fine-Tuning

Jianzhu Bao, Jia-Chen Gu, Qingyuan Liu, Haozhen Zhang, Kai-Wei Chang, Nanyun Peng, Wenya Wang

arXiv:2610.18634v1cs.LGcs.AI

TL;DR

SFT can improve target-task performance while accompanying regression beyond the training domain, and the paper asks how its weight-geometry changes can inform refinement. It introduces SPACE, which calibrates SFT updates using the pre-SFT principal subspace. Across evaluated settings, SPACE improves both ID and OOD performance over SFT, including the reported mathematical-SFT gains.

  • Problem

    SFT can improve target-task performance but may also lead to capability regression beyond the training domain, motivating study of how SFT changes weight geometry.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    SPACE is a post-hoc method that projects SFT updates away from the pre-SFT principal input subspace, restoring the pre-SFT transformation there while retaining updates in the orthogonal complement.

    Source: §1 · p. 2, §4.1 · p. 4

  • Results

    SPACE improves both ID and OOD performance over SFT across evaluated settings; math-SFT gains are 2.85/2.26 points on Qwen2.5-7B and 2.41/1.52 points on Qwen3-14B-Base.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    The results support using pre-SFT weight geometry as a reference for refining completed SFT updates.

    Source: §6 · p. 10

    Because SPACE operates on existing language-model checkpoints, resulting models may retain the underlying LLMs' safety risks, requiring validation, monitoring, and task-specific safeguards downstream.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Recent studies investigate the weight geometry through its spectral structure, which characterizes how the weights transform different input directions. Within this view, supervised fine-tuning (SFT) is shown to induce dense updates that perturb the spectral geometry of pre-SFT weights along their principal directions, potentially disrupting the model's prior knowledge. These observations motivate using the geometry of pre-SFT weights as a reference for refining learned SFT updates. We study this perspective as Geometric Calibration and develop SPACE Subspace Projection for Adapted Checkpoint Enhancement), a lightweight post-hoc method that selectively reverts SFT update components within the principal subspace of the pre-SFT weights while preserving those in its orthogonal complement. SPACE is a data-free, training-free method that requires only the pre- and post-SFT checkpoints. We evaluate SPACE on Qwen2.5, Qwen3, Qwen3.5, Llama-3.1, and Olmo-3 models ranging from 2B to 14B, across mathematical, coding, and medical SFT, as well as several SFT variants. Overall, SPACE mitigates out-of-domain (OOD) capability regression while further improving in-domain (ID) performance over the uncalibrated SFT checkpoints. For math SFT, SPACE yields average ID/OOD gains of 2.85/2.26 points on Qwen2.5-7B, and 2.41/1.52 points on Qwen3-14B-Base, respectively. These results highlight how the geometric relationship between SFT updates and pre-SFT weights shapes the adapted model's capabilities and, more broadly, motivate geometry-aware parameter updates for continual model improvement.

1 INTRODUCTION

SFT can improve target-task performance while accompanying out-of-domain regression, and its effects on weight geometry motivate refining completed updates against pre-SFT weights. SPACE uses that geometry to calibrate checkpoints and improves both in-domain and out-of-domain results across the reported settings.

1 INTRODUCTION

  • SFT gains on target tasks can accompany regression in capabilities beyond the training domain, motivating analysis of how SFT changes weight geometry.The cited spectral perspective describes dense SFT updates that perturb principal directions associated with pre-existing or transferable capabilities.Source: §1 · p. 1
  • SPACE projects SFT updates onto the complement of the pre-SFT principal input subspace, restoring the pre-SFT transformation along principal directions while retaining other learned changes.It is a lightweight post-hoc method that requires only pre- and post-SFT checkpoints, without data or additional training.Source: §1 · p. 2, Figure 1 · p. 2
  • The paper frames pre-SFT weight geometry as a reference for refining completed SFT updates through Geometric Calibration.SPACE conditions refinement on the relationship between the update and pre-SFT weights, unlike approaches based on parameter interpolation or properties of the learned update.Source: §1 · pp. 1–2
  • SPACE improves average in-domain and out-of-domain performance over SFT across five model families, mathematical, coding, and medical SFT, and multiple SFT variants.For mathematical SFT, gains are 2.85/2.26 points on Qwen2.5-7B and 2.41/1.52 points on Qwen3-14B-Base.Source: §1 · p. 2

2 RELATED WORK

Prior work addresses SFT generalization and forgetting through training-time changes, while post-hoc refinement methods modify completed checkpoints. This work instead uses the geometric relationship between pre-SFT weights and learned updates to refine a completed SFT update.

Generalization and forgetting in SFT

  • Prior approaches to mitigating capability degradation mainly intervene during SFT by changing training-data selection or supervised objectives.The paper contrasts these methods with post-hoc refinement that requires neither data nor additional training.Source: §2 · pp. 2–3

Weight geometry in model adaptation

  • Research on pretrained-weight geometry links principal subspaces with prior or transferable capabilities and uses structured subspaces to manage adaptation or reduce interference.This work applies the pre-SFT principal subspace to refine one completed SFT update post hoc.Source: §2 · p. 3

Post-hoc model refinement

  • Existing post-hoc methods refine checkpoints through interpolation, pruning, model souping, or spectral filtering, but generally do not use the pre-SFT weights' geometric relationship with the update to select components for modification.SPACE studies that relationship directly when refining completed SFT checkpoints.Source: §2 · p. 3

3 PROBLEM FORMULATION

SFT updates can improve target-task performance yet coincide with capability regression beyond the training domain. The paper investigates calibrating update components and formalizes refinement using pre-SFT parameters as a geometric reference.

3 PROBLEM FORMULATION

  • SFT can induce non-trivial parameter updates that improve target-task performance but may also accompany capability regression beyond the training domain.The paper investigates whether identifying and calibrating update components can mitigate regression while preserving or further improving target-domain performance.Source: §3 · p. 3
  • Geometric Calibration treats the post-SFT checkpoint as a starting point for further refinement, using pre-SFT weight geometry as the reference.The calibrated update is defined as the output of an operator conditioned on the pre-SFT parameters.Source: §3 · p. 3
  • The calibration operator maps the learned SFT update to a calibrated update conditioned on the pre-SFT parameters.The paper seeks a final calibrated checkpoint formed by adding that update to the pre-SFT checkpoint, with improved overall capabilities.Source: §3 · p. 3

4 METHOD

SPACE refines completed SFT updates using pre-SFT weight geometry: it preserves the pre-SFT transformation on principal input directions while retaining SFT changes on the orthogonal complement.

4.1 MOTIVATION

  • The principal subspace is associated with important prior or transferable knowledge, while SFT's dense updates perturb pre-SFT geometry there.Source: §4.1 · p. 4
  • P0 is the rank-k orthogonal projector whose W0P0 best approximates W0 in Frobenius norm, and its solution uses leading right singular vectors.Source: §4.1 · p. 4

4.2 SPACE: SUBSPACE PROJECTION FOR ADAPTED CHECKPOINT ENHANCEMENT

  • SPACE removes SFT update components in the principal input subspace while retaining them on its orthogonal complement.Source: §4.2 · p. 4
  • The dimension k is the smallest number of leading singular components capturing at least fraction ρ of each matrix's total spectral energy.Source: §4.2 · p. 5
  • In practice, SPACE uses ρ = 0.5 and applies projection to all two-dimensional weight matrices by default; these choices work well across evaluated settings.Source: §4.2 · p. 5

5 EXPERIMENTS

Across mathematical, coding, and medical SFT, SPACE improves on the corresponding SFT checkpoints, with further gains across SFT variants and model families. Ablations favor a hard projection using the leading right singular subspace of pre-SFT weights.

5.1 MAIN EXPERIMENTS ON MATHEMATICAL REASONING SFT

  • SPACE achieves the highest average ID and OOD gains among baselines on both math models, improving every Qwen2.5-7B benchmark and exceeding Base on five of six OOD benchmarks.Qwen3-14B-Base also gains despite already strong mathematical reasoning after SFT.Source: §5.1 · p. 6

5.2 ABLATION STUDY

  • With ρ = 0.5 and α = 1, SPACE gives a good ID/OOD balance; larger subspaces favor OOD at the cost of ID gains, while performance generally rises with calibration strength.The results support hard projection with ρ as the sole trade-off hyperparameter.Source: §5.2 · p. 7
  • Calibrating all two-dimensional matrices performs best overall; MLP projections drive the gains, Up is strongest individually, and attention adds gains when combined with MLP calibration.Attention alone has little effect, and Down alone is ineffective.Source: §5.2 · p. 7
  • The leading right singular subspace of pre-SFT weights best balances ID and OOD gains; bottom right favors OOD with almost no ID gain, while subspaces from ΔW perform worse.Right subspaces also outperform left subspaces, consistent with SPACE’s input-side formulation.Source: §5.2 · p. 7

5.3 EXPERIMENTS ON DIFFERENT TARGET DOMAINS

  • SPACE improves on SFT across coding and medical tasks and outperforms DG-Hard and MonoSoup; coding adaptation is largely preserved as OOD performance improves.Medical ID gains are more pronounced on benchmarks where SFT regresses relative to Base.Source: §5.3 · pp. 7–8, Table 2 · p. 8

5.4 COMPATIBILITY WITH DIFFERENT SFT METHODS

  • SPACE improves overall ID and OOD performance across DFT, ASFT, and RPSFT, with larger aggregate gains than MonoSoup for each variant.Gains are especially pronounced for RPSFT, suggesting post-hoc calibration complements its training-time regularization; benefits also extend to LoRA.Source: Table 3 · p. 8, §5.4 · p. 9

5.5 GENERALIZATION ACROSS MODEL FAMILIES AND SCALES

  • Across evaluated model families and scales, SPACE yields positive average ID and OOD gains; on Llama-3.1-8B it improves every evaluated OOD benchmark while further strengthening ID performance.Gains remain evident at 2B scale, and Appendix C.2 reports larger average improvements than MonoSoup across evaluated models.Source: Table 4 · p. 9, §5.5 · p. 9

5.6 ADDITIONAL ANALYSES

  • Additional analyses find SPACE effective at intermediate SFT checkpoints, robust to training hyperparameters, and more effective post hoc than as a training-time projection.Source: §5.6 · p. 9

6 CONCLUSION

The conclusion presents SPACE as a lightweight post-hoc implementation of Geometric Calibration that uses pre-SFT weight geometry to refine completed SFT updates. Across the evaluated settings it improves both ID and OOD performance over SFT, while downstream models may retain safety risks inherited from their underlying LLMs.

6 CONCLUSION

  • SPACE removes update components in the pre-SFT principal input subspace while preserving the remaining adaptation, using only pre- and post-SFT checkpoints without data or additional training.Source: §6 · p. 10
  • Across model families, scales, domains, and SFT variants, SPACE improves ID and OOD performance over SFT and outperforms representative post-hoc baselines.The ablations support the principal right singular subspace of pre-SFT weights as an effective calibration reference.Source: §6 · p. 10
  • SPACE’s resulting models may retain underlying LLM risks, including biased, inaccurate, or potentially harmful outputs, so downstream applications should maintain validation, monitoring, and task-specific safeguards.Source: ETHICS STATEMENT · p. 10

A DERIVATION OF THE PRINCIPAL INPUT PROJECTOR

The derivation identifies the leading right singular vectors of the pre-SFT weight matrix as the optimal rank-k projector. The supplied training settings specify the main experiments’ common optimization recipe.

A DERIVATION OF THE PRINCIPAL INPUT PROJECTOR

B.1 DETAILED TRAINING SETTINGS OF THE MAIN EXPERIMENTS

  • Unless otherwise specified, training uses AdamW, global batch size 256, learning rate 5 × 10^-5 with cosine decay and 0.1 warm-up, for one epoch on four H200 GPUs.The SFT objective is standard next-token prediction over response tokens.Source: §B.1 · p. 17

B.2 DETAILED EVALUATION SETTINGS OF THE MAIN EXPERIMENTS

The appendix specifies evaluation settings and post-hoc baselines, then reports that SPACE improves both ID and OOD performance when applied to a LoRA checkpoint.

B.2 DETAILED EVALUATION SETTINGS OF THE MAIN EXPERIMENTS

  • Math and OOD benchmarks use distinct prompt, sampling, and generation settings, with evaluation metrics and test-set coverage varying between Qwen2.5-7B and Qwen3-14B-Base.Math evaluations use the Qwen2.5-Math pipeline at temperature 1.0 and top-p 1.0; OOD evaluations follow Ren et al. (2026) at temperature 0.6 and top-p 0.95.Source: §B.2 · p. 17

B.3 BASELINE DETAILS

  • SPACE calibrates updates against pre-SFT principal input subspaces, unlike baselines that scale or prune updates or derive spectral transforms from the learned update.WiSE-FT and LiNeS scale updates; TIES, DARE, and FAPM prune entries; POME, DG-Hard, and MonoSoup use spectral structure from the learned update.Source: §B.3 · p. 18

C.1 COMPATIBILITY WITH LORA FINE-TUNING

  • SPACE improves average ID and OOD performance over the original Qwen2.5-7B LoRA checkpoint and achieves larger aggregate gains than MonoSoup.The evaluated LoRA setup uses rank r = 64 and scaling factor α = 128.Source: §C.1 · p. 18, Table 5 · p. 18

C.2 ADDITIONAL RESULTS ON MODEL FAMILIES AND SCALES

Additional evaluations find that SPACE improves average ID and OOD performance across the reported model scales and families, with larger average gains than MonoSoup.

C.2 ADDITIONAL RESULTS ON MODEL FAMILIES AND SCALES

  • SPACE consistently improves average ID and OOD performance across all evaluated models, with gains extending across Qwen scales and to Olmo and Llama families.SPACE achieves larger average improvements than MonoSoup on both ID and OOD benchmarks for every evaluated model.Source: §C.2 · p. 19
  • SPACE achieves larger average improvements than MonoSoup on both ID and OOD benchmarks for every evaluated model.Source: §C.2 · p. 19
  • The reported generalization results include Qwen3-4B-Base in addition to the models in the main text.Source: §C.2 · p. 19

C.3 PERFORMANCE AT INTERMEDIATE SFT CHECKPOINTS

SPACE provides gains even before SFT is complete, and its average ID and OOD improvements persist across the tested training hyperparameters.

C.3 PERFORMANCE AT INTERMEDIATE SFT CHECKPOINTS

  • The intermediate-checkpoint evaluation applies SPACE independently at 100, 200, 300, and 390 steps and compares average results across six ID and six OOD benchmarks.The 390-step checkpoint corresponds to one SFT epoch; changes are reported relative to that final SFT checkpoint.Source: §C.3 · p. 19, Figure 3 · p. 20
  • At 100 steps, SPACE exceeds the final 390-step SFT checkpoint by 1.28 points on ID and 2.10 points on OOD benchmarks; at 390 steps, its ID improvement reaches 2.85 points.The OOD improvement remains relatively stable during training; the corresponding 100-step SFT checkpoint is 1.09 ID and 0.70 OOD points below the final checkpoint.Source: §C.3 · pp. 19–20, Figure 3 · p. 20

C.4 ROBUSTNESS TO SFT TRAINING HYPERPARAMETERS

  • SPACE yields positive average gains on both ID and OOD performance across all evaluated batch sizes, learning rates, and training epochs.Gain magnitudes vary across settings.Source: §C.4 · p. 20

C.5 TRAINING-TIME VS. POST-HOC PROJECTION

In the Qwen2.5-7B comparison, post-hoc SPACE achieves the largest average gains on ID and OOD benchmarks, ahead of training-time projection alternatives.

C.5 TRAINING-TIME VS. POST-HOC PROJECTION

  • The comparison includes a training-time projection loss, online projection at K = 1 and K = 100, default post-hoc SPACE, and RPSFT.RPSFT regularizes updates jointly within leading left and right singular subspaces; the projection loss penalizes updates along the leading right singular subspace.Source: §C.5 · p. 20
  • Post-hoc SPACE achieves the largest average gains on both ID and OOD benchmarks, while projection loss improves OOD but offers little average ID gain and online projection yields modest gains.The authors suggest that constraining or repeatedly projecting updates during optimization may interfere with learning.Source: §C.5 · p. 20, Table 11 · p. 22

Related papers

Loading 2610.18634v1…