PaperlayerView exact revision on arXiv

Source-linked AI summary

Shadow-FT: Tuning Instruct Model via Training on Paired Base Model

Taiqiang Wu, Runming Yang, Jiayi Li, Pengfei Hu, Yik-Chung Wu, Ngai Wong, Yujiu Yang

arXiv:2505.12716v3cs.CLcs.AI

TL;DR

Directly tuning Instruct models can bring marginal gains or degeneration, despite the need to adapt LLMs to tasks. Shadow-FT fine-tunes the paired Base model and grafts its updates onto the Instruct model; across evaluated model sizes and tasks, it consistently outperforms tuning baselines and vanilla Instruct.

  • Problem

    Directly tuning Instruct models can yield marginal improvement or performance degeneration, leaving effective adaptation of these models an open challenge.

    Source: §2.2 · p. 3

  • Method

    Shadow-FT fine-tunes a paired Base model and directly grafts its learned weight updates onto the Instruct model.

    Source: §1 · p. 2

  • Results

    Shadow-FT consistently outperforms tuning baselines and vanilla Instruct across model sizes and tasks; on Qwen-3-4B, its average score of 69.4 is 3.7 higher than conventional tuning methods.

    Source: §4.2 · p. 5

  • Takeaways & Limitations

    The conclusion reports that Shadow-FT needs no additional training cost or parameters and performs across math, coding, and reasoning benchmarks.

    Source: §7 · p. 10

    Shadow-FT cannot be applied when a paired Base model is unavailable, as for Qwen3-32B and Qwen3-Next.

    Source: F LIMITATIONS · p. 24

Abstract

from arXiv · show

Large language models (LLMs) consistently benefit from further fine-tuning on various tasks. However, we observe that directly tuning the Instruct (i.e., instruction-tuned) models often leads to marginal improvements and even performance degeneration. Notably, paired Base models, the foundation for these Instruct variants, contain highly similar weight values (i.e., less than 2% on average for Llama 3.1 8B). The Base model tends to be a good learner yet a weak backbone without post-training. Therefore, we propose a novel Shadow-FT framework to tune the Instruct models by leveraging the corresponding Base models. The key insight is to fine-tune the Base model, and then \textit{directly} graft the learned weight updates to the Instruct model. Our proposed Shadow-FT introduces no additional parameters, is easy to implement, and significantly improves performance. We conduct extensive experiments on tuning mainstream LLMs, such as Qwen 3 and Llama 3 series, and evaluate them across 19 benchmarks covering coding, reasoning, and mathematical tasks. Experimental results demonstrate that Shadow-FT consistently outperforms conventional full-parameter and parameter-efficient tuning approaches. Further analyses indicate that Shadow-FT can be applied to multimodal large language models (MLLMs) and combined with direct preference optimization~(DPO). Codes and weights are available at \href{https://github.com/wutaiqiang/Shadow-FT}{Github}.

1 INTRODUCTION

The introduction identifies a tension: direct tuning of instruction-tuned models can yield marginal gains or degeneration, while their paired Base models have highly similar weights. Shadow-FT learns updates on the Base model and grafts them onto the Instruct model, with experiments spanning multiple models and benchmarks.

1 INTRODUCTION

  • Direct tuning of Instruct models can produce marginal improvements or performance degeneration, motivating more effective ways to adapt them.The introduction frames instruction-following and domain-specific knowledge as real-world needs not covered by pretraining alone.Source: §1 · p. 1
  • The authors analyze paired BASE and INSTRUCT model weights using relative absolute difference σ and find them highly similar, with an average gap σ of 0.016.Source: §1 · p. 1
  • Shadow-FT fine-tunes the paired Base model and directly grafts its learned weight updates onto the Instruct model.The models share structures, so the proposed transfer requires no additional operations.Source: §1 · p. 2, Figure 1 · p. 2
  • Experiments cover 19 benchmarks across math, code, and reasoning, and the introduction reports consistent effectiveness and robustness across varied mainstream LLMs.The evaluated tuning settings include full-parameter and low-rank approaches; further analyses consider MLLMs and DPO.Source: §1 · p. 2
  • The authors present leveraging Base models to improve their Instruct counterparts as a direction for further research and broader applications.Source: §1 · p. 2

2 PRELIMINARIES AND MOTIVATION

The preliminaries contrast full-parameter and low-rank tuning, then describe why direct Instruct tuning can underperform. They motivate Shadow-FT with the high weight similarity of paired Base and Instruct models.

2.1 BACKGROUND

  • Full-parameter fine-tuning updates all parameters, whereas LoRA freezes the original weights and learns a low-rank branch (Hu et al., 2022).Source: §2.1 · p. 2
  • The Base model is the pretrained version, while its Instruct variant is post-trained for human preference alignment and reasoning and performs better at instruction following.Source: §2.1 · p. 3

2.2 DIRECTLY TUNING INSTRUCT

  • Conventional LoRA tuning of Qwen-3-4B on BAAI-2k drops Math-7 by 2.6 points, Code-3 by 6.8, and Knowledge-9 by 2.6 versus vanilla Instruct.The section describes direct Instruct tuning as producing marginal improvement or degeneration in more cases.Source: §2.2 · p. 3

2.3 SIMILAR WEIGHTS: BASE & INSTRUCT

  • Paired Base and Instruct models have relative weight gap σ < 0.03, with σ = 0.016 for the illustrated Llama-3.1-8B layer.Figure 2 compares the same layer in each model and their gap; the gap is negligible despite a 10× z-axis zoom.Source: §2.3 · p. 3, Figure 2 · p. 3

3 METHODOLOGY

Shadow-FT learns weight updates by tuning the paired Base model, then adds those updates to the Instruct model. The paper distinguishes this approach from Chat Vector by its tuning scope and motivation.

3.1 SHADOW-FT

  • Shadow-FT fine-tunes the Base model and grafts its learned weight updates onto the Instruct model, which shares the same structures.The authors say this introduces no extra training cost; the difference from vanilla fine-tuning is which starting weights learn the updates.Source: §3.1 · pp. 3–4

3.2 RELATION WITH TASK VECTORS

  • Unlike Chat Vector, which focuses on continual pretraining, Shadow-FT applies to full-parameter tuning, LoRA, and DPO and targets tuning-related degeneration (Huang et al., 2023).Source: §3.2 · p. 4

4 EXPERIMENTS

Experiments tune mainstream LLMs with full-parameter or LoRA methods and evaluate mathematical, coding, and commonsense reasoning. Shadow-FT consistently outperforms conventional tuning baselines, with gains reported across models and tuning strategies.

4.1 EXPERIMENTAL SETUP

  • Experiments use 2,000 high-reward, uniformly sampled BAAI-2k examples and full-parameter or LoRA tuning, evaluating Math-7, Code-3, and Knowledge-9 averages.The experiments cover Qwen 3 and Llama 3 series; Math-7, Code-3, and Knowledge-9 aggregate mathematical, coding, and commonsense reasoning benchmarks.Source: §4.1 · p. 4, Table 1 · p. 5

4.2 MAIN RESULTS

  • Shadow-FT consistently outperforms tuning baselines and vanilla INSTRUCT across model sizes and tasks, without extra training overhead.On Qwen-3-4B, it scores 69.4 on average, 3.7 above conventional tuning and 1.4 above vanilla INSTRUCT.Source: §4.2 · pp. 5–6, Table 1 · p. 5
  • Figure 3 summarizes Math-7, Code-3, and Knowledge-9 averages across LoRA ranks, while Figure 4 compares methods on Medical-o1-reasoning-SFT.The Figure 3 caption reports best performance across a search over four learning rates.Source: Figure 3 · p. 6, Figure 4 · p. 6
  • 73.7/67.4/64.9 on Math-7/Code-3/Knowledge-9 for Shadow-FT full-parameter tuning of Qwen-3-4B, versus 72.9/66.4/62.9 for conventional tuning.Shadow-FT also performs well with LoRA; on Qwen-3-4B, its Math-7/Code-3/Knowledge-9 scores are 75.9/69.7/65.0, respectively, 4.7/10.1/3.9 higher than conventional LoRA.Source: §4.2 · p. 6
  • Shadow-FT (LoRA) can outperform Shadow-FT (full), including 69.7 vs. 67.4 on Code-3 for Qwen-3-4B, while conventional full tuning can outperform conventional LoRA.The authors leave the difference between the methods for future investigation.Source: §4.2 · p. 6

5 EXTENSIVE ANALYSIS

Across tuning ranks and domain-specific tasks, Shadow-FT generally improves over conventional tuning. The analyses also examine optimization dynamics, DPO, multimodal models, weight scaling, and additional model families.

5.1 RANKS IN LORA

  • Across LoRA ranks 4–512, Shadow-FT outperformed conventional LoRA; conventional LoRA worsened as rank grew, while Shadow-FT benefited from larger ranks.The reported averages followed learning-rate searches over 5e-5, 1e-4, 2e-4, and 5e-4.Source: §5.1 · p. 6

5.2 TUNING ON DOMAIN DATA

  • Shadow-FT LoRA scored 77.4 on Math-7 for Qwen-3-8B, 6.2 above LoRA’s 71.2 and 2.9 above vanilla INSTRUCT.On medical data, conventional LoRA degraded performance while Shadow-FT improved it; Code-Z1 tuning also achieved 36.7 on AIME-24.Source: Table 2 · p. 7, §5.2 · p. 7

5.3 MECHANISTIC ANALYSIS OF OPTIMIZATION DYNAMICS

  • At initialization, INSTRUCT loss was 22.6% higher and gradient 3.25× larger; by step 61 its loss remained 2.4% higher despite similar gradients.INSTRUCT gradients fell precipitously, while BASE gradients decreased more moderately and followed a steadier trajectory.Source: §5.3 · pp. 7–8, Figure 5 · p. 8

5.6 WEIGHT DELTA SCALING

  • Shadow-FT outperformed vanilla INSTRUCT across tested scaling factors; factors slightly above 1 performed better, while identifying the best factor remains future work.The method grafts learned BASE weights onto INSTRUCT.Source: §5.6 · pp. 8–9

5.4 COMBINED WITH DPO

  • Shadow-DPO outperformed DPO at ranks 8 and 128; one reported comparison is 55.39 vs. 54.62.The experiment used LoRA on 1,000 paired Math-Step samples.Source: §5.4 · p. 8, Table 3 · p. 9

5.5 PERFORMANCE ON MLLM

  • On ChartQA, Shadow-FT scored 63.80 vs. LoRA’s 60.28 on Gemma-3-27B, and 80.6 vs. 79.92 on Llama-3.2-Vision-90B.Both methods adapted the multimodal models, with Shadow-FT’s advantage especially notable on larger models.Source: §5.5 · p. 8, Table 4 · p. 9

5.7 MODEL ZOO: MORE LLMS

  • Shadow-FT consistently outperformed conventional tuning across the Gemma-3, Yi, and Falcon model families.Source: §5.7 · p. 9

6 RELATED WORK

LLM tuning includes full-parameter, parameter-efficient, and preference-alignment approaches, while model-guidance methods transfer knowledge between models. Shadow-FT instead directly grafts full dynamic weight updates from BASE to INSTRUCT.

6.1 TUNING FOR LLMS

  • LLM tuning spans full-parameter updates, PEFT methods that reduce tuning costs through parameter selection or low-rank branches, and RLHF alignment methods.The paper distinguishes Shadow-FT by tuning INSTRUCT through BASE and says it can be combined with existing baselines.Source: §6.1 · p. 9

6.2 MODEL GUIDANCE IN TUNING

  • Shadow-FT directly grafts full dynamic weight updates, unlike knowledge-distillation and proxy-tuning approaches that transfer knowledge at feature or data level.Unlike RE-Adapt, it does not approximate static weight differences with a low-rank model.Source: §6.2 · p. 9
  • Shadow-FT differs from RE-Adapt’s low-rank approximation of static weight differences by transferring full dynamic updates without that assumption.The authors also distinguish their work from a concurrent transfer approach and report broader model, benchmark, MLLM, and DPO coverage.Source: §6.2 · p. 9

7 CONCLUSION

Shadow-FT tunes instruction models through their corresponding base models and consistently outperforms conventional full-parameter and parameter-efficient tuning across the reported tasks. The conclusion also reports generalization to multimodal models and compatibility with DPO.

  • Shadow-FT consistently outperforms conventional full-parameter and parameter-efficient tuning across diverse math, coding, and reasoning benchmarks.The paper reports experiments across Qwen, Llama, Gemma, and Falcon, with no additional training cost or parameters.Source: §7 · p. 10
  • Shadow-FT fine-tunes instruction models by leveraging their corresponding base models.The authors connect the approach to high similarity between paired model weights.Source: §7 · p. 10
  • The approach generalizes to multimodal large language models and can be combined with direct preference optimization.Source: §7 · p. 10

REPRODUCIBILITY STATEMENT

The paper reports reproducibility resources and describes the provenance and documentation for its experiments.

  • Reproduction materials include supplementary source code, documented hyperparameters and configurations, and publicly available benchmarks.MATH-7 results are averaged across three trials, and benchmark details appear in Appendix A.Source: REPRODUCIBILITY STATEMENT · p. 10

ETHICS STATEMENT

The authors state that the research uses publicly available models and benchmarks and involves no human subjects.

  • The work uses publicly available models and benchmarks, involves no human subjects, and commits to releasing code.Source: ETHICS STATEMENT · p. 10

LLM USAGE DISCLOSURE

The appendix documents broader-model tuning and similarity results alongside evaluation conventions, training settings, detailed score tables, and a disclosure of LLM assistance.

LLM USAGE DISCLOSURE

  • The authors used LLMs for code assistance, boilerplate, language and formatting refinement, and proofreading, while retaining responsibility for the research and analysis.Source: LLM USAGE DISCLOSURE · p. 16

A BENCHMARKS DETAILS

B MODEL ZOO: MORE LLMS

  • 52.55 average for Shadow-FT tuning Gemma-3-4B, 1.1 above vanilla INSTRUCT and 7.91 above conventional tuning methods.The additional model families include Gemma-3, Yi, and Falcon.Source: B MODEL ZOO: MORE LLMS · p. 16, Table 6 · p. 17

C SIMILARITY ON MORE LLMS

D.1 HYPER-PARAMETERS

  • Training used batch size 2, gradient accumulation steps 16, an input cutoff of 4096, and one epoch.Source: §D.1 · p. 18

D.2 DETAILED DATA OF TABLE 1

  • Detailed scores for the Table 1 results are listed in Tables 9, 10, and 11.Source: §D.2 · p. 18

D.3 DETAILED TABLE FOR MEDICAL BENCHMARKS

These appendix sections detail medical and Pass@k evaluations, rank and delta-scaling analyses, a Math-500 case study, and a boundary on Shadow-FT's applicability.

D.3 DETAILED TABLE FOR MEDICAL BENCHMARKS

  • Table 7 reports Llama-3.2-1B medical-benchmark results; Shadow-FTR denotes integrating fine-tuned INSTRUCT weights into BASE.Source: §D.3 · p. 18, Table 7 · p. 18

D.4 PERFORMANCE ON PASS@K

  • Pass@k is estimated from n rollouts with n ≥ k; x is a dataset prompt and c counts correct solutions.Source: §D.4 · p. 18
  • In thinking mode, Shadow-FT and INSTRUCT follow similarly steep, rapidly saturating curves while vanilla FT maintains a >30% gap across k; in non-thinking mode, Shadow-FT rises with k and surpasses INSTRUCT.Source: §D.4 · pp. 18–19, Figure 7 · p. 19

D.5 RANKS IN LORA ON LLAMA-3.1-8B

  • Figure 8 compares Llama-3.1-8B tuning performance across different LoRA ranks.Source: Figure 8 · p. 19

D.6 WEIGHT DELTA SCALING

  • Shadow-FT interpolates BASE-learned deltas into INSTRUCT with α; α = 1.0 gives the reported overall trade-off on MATH-7, while adaptive scaling remains future work.α = 0.0 recovers the original INSTRUCT model; Table 8 summarizes the math benchmark results.Source: §D.6 · pp. 19–20, Table 8 · p. 20

E CASE STUDY ON LLAMA-3.1-8B

F LIMITATIONS

  • Shadow-FT cannot be applied to models without paired BASE versions, including Qwen3-32B and Qwen3-Next; finding a suitable shadow remains future work.Source: F LIMITATIONS · p. 24

Related papers

Loading 2505.12716v3…