PaperlayerView exact revision on arXiv

Source-linked AI summary

NormTune: Norm-Guided Parameter-Efficient Fine-Tuning for Reasoning-Intensive Tasks

Zhangming Li, Qinghao Hu, Weifan Guan, Yifan Zhang, Peisong Wang, Jian Cheng

arXiv:2610.29134v1cs.AI

TL;DR

PEFT reduces adaptation costs but leaves a performance gap on reasoning-intensive tasks, where full fine-tuning is resource-intensive. NormTune strengthens under-adapted layers according to their weight-update norms. Across four benchmarks, it improves rank-8 LoRA, Bone, and MoRA, while rank-256 variants reach performance comparable to S1 full fine-tuning.

  • Problem

    Reasoning-intensive tasks expose a performance gap for PEFT, while full fine-tuning imposes substantial resource demands as model scale increases.

    Source: §1 · pp. 1–2

  • Method

    NormTune adaptively strengthens under-adapted layers according to weight-update norms, reducing earlier-layer gradient conflicts and reallocating optimization strength.

    Source: §1 · p. 2

  • Results

    Across AIME 2024, MATH 500, GPQA, and AIME 2025, rank-8 NormTune gains average accuracy of 24.24%, 11.33%, and 16.21% for LoRA, Bone, and MoRA.

    Source: §1 · p. 2

  • Takeaways & Limitations

    NormTune-enhanced rank-8 methods surpass 90% of Full-FT performance, while rank-256 variants achieve performance comparable to S1 full fine-tuning.

    Source: §1 · p. 2, §3.1 · p. 6

Abstract

from arXiv · show

Full fine-tuning has achieved remarkable success on reasoning-intensive tasks, but it often requires substantial GPU resource consumption. Parameter-efficient fine-tuning (PEFT) provides a promising alternative for reducing this cost, yet we observe a substantial performance gap on reasoning-intensive tasks. In this paper, we present three key observations showing that this gap is closely associated with norm-related under-adaptation, where PEFT produces markedly smaller weight-update norms than full fine-tuning. Based on these observations, we propose Norm-Guided Parameter-Efficient Fine-Tuning (NormTune), a simple yet effective strategy that adaptively strengthens insufficiently updated layers according to their weight-update norms. Extensive experiments show that NormTune consistently enhances multiple PEFT methods across challenging reasoning benchmarks. Specifically, when trained on S1K-1.1, integrating NormTune into rank-8 LoRA, Bone, and MoRA yields substantial average accuracy improvements of 24.24\%, 11.33\%, and 16.21\% , respectively, across AIME 2024, MATH 500, GPQA, and AIME 2025. Moreover, NormTune-enhanced rank-256 LoRA, Bone, and MoRA achieve performance comparable to S1 full fine-tuning, with respective average gains of 2.17\%, 0.93\%, and 3.00\% over this baseline.

1 INTRODUCTION

PEFT reduces the resource burden of adapting large models, but existing methods show a substantial performance gap on reasoning-intensive tasks. NormTune responds to norm-related under-adaptation, and experiments report gains across multiple PEFT methods and reasoning benchmarks.

1 INTRODUCTION

  • Full fine-tuning is costly as model scale grows, while PEFT updates only a small subset of parameters; the paper asks whether low-rank PEFT can retain comparable adaptation capability on reasoning tasks.Full fine-tuning of Qwen2.5-32B-Instruct at 32K context requires 562.07 GB even with gradient checkpointing.Source: §1 · p. 1
  • PEFT produces substantially smaller weight-update norms than Full-FT, especially in earlier layers; Figures 2 and 3 compare layer-wise norms across methods and task settings.The paper links these smaller updates with weaker effective adaptation and performance degradation.Source: §1 · p. 2, Figure 2 · p. 3, Figure 3 · p. 3
  • At earlier training, PEFT assigns greater loss contribution to reasoning tokens than Full-FT and reduces loss on normal tokens less than on reasoning tokens.The paper distinguishes transition, self-verification, template, and reasoningboundary tokens as reasoning tokens.Source: §1 · p. 2
  • Earlier-layer gradients from reasoning-token and normal-token losses conflict, while total token gradients remain aligned with normal-token optimization and are partially weakened.The paper describes this interference as weakening effective parameter updates in earlier layers.Source: §1 · p. 2
  • NormTune reduces earlier-layer gradient conflicts and reallocates optimization strength toward under-adapted layers; the paper presents it as the first approach enabling comparable PEFT performance on reasoning-intensive tasks.The first-approach claim is qualified by “to the best of our knowledge.”Source: §1 · p. 2
  • 24.24%, 11.33%, and 16.21% average accuracy gains for rank-8 LoRA, Bone, and MoRA, respectively, across AIME 2024, MATH 500, GPQA, and AIME 2025.All three methods are trained on S1K-1.1.Source: §1 · p. 2

2 METHODOLOGY

The observations link reasoning-task PEFT under-adaptation to token-loss imbalance, conflicting gradients, and smaller layer-wise weight updates, especially in earlier layers. NormTune uses update norms to strengthen weakly updated layers, with probing and bucket-wise learning rates guiding the adjustment.

2.1 KEY OBSERVATIONS ON REASONING-INTENSIVE TASKS

  • At earlier training, PEFT assigns a larger loss contribution to reasoning tokens than full fine-tuning; by step 310, it reduces normal-token loss less relative to reasoning-token loss.The analysis sampled 16 S1K-1.1 sequences and used the empirical 99th percentile of token losses to inspect the high-loss tail.Source: §2.1 · p. 3
  • Earlier-layer gradients from reasoning-token and normal-token losses conflict, but their combined gradient remains positively aligned with the normal-token gradient and reduces its effective update magnitude.In deeper layers, the two token-group optimization signals become more compatible.Source: §2.1 · pp. 3–4
  • PEFT produces substantially smaller weight-update norms than full fine-tuning, particularly in earlier layers, while other benchmarks do not show the same early-layer under-adaptation.Layer adaptation strength is measured by aggregating entry-wise L1 norms of effective weight updates across target modules in each layer.Source: §2.1 · p. 4

2.2 NORMTUNE

  • A 30-step PEFT probe estimates layer-wise update magnitudes, avoiding the cost of obtaining the statistics from a completed PEFT run.Completed PEFT and full fine-tuning runs show approximately 6× maximum-to-minimum ratios of layer-wise update norms.Source: §2.2 · p. 5
  • NormTune groups layers by update magnitude and maps each bucket’s relative adaptation sufficiency to a learning rate between ηmin and ηmax.The mapping gives the weakest-adapted bucket the largest rate, decreases monotonically with adaptation magnitude, and keeps rates near maximum for buckets near q = 0.Source: §2.2 · p. 5
  • NormTune consistently increases update norms in insufficiently adapted layers, aligning them more closely with full fine-tuning and improving performance.The method assigns larger learning rates to weakly updated layers based on their weight-update norms.Source: §2.2 · pp. 4, 6

3 EXPERIMENTAL RESULTS

NormTune-enhanced PEFT improves reasoning-task performance across model scales and task types, with strong results against full fine-tuning. Ablations favor adaptive layer-wise adjustment over fixed learning-rate changes, and performance is relatively insensitive to the bucket count tested.

3 EXPERIMENTAL RESULTS

  • Greedy decoding at temperature T = 0 is used for locally evaluated checkpoints, with repeated fixed-checkpoint evaluations and unweighted means for the four-benchmark Avg. column.MATH 500 and GPQA use four evaluation runs; AIME 2024 and AIME 2025 use 16 where applicable, rather than independent training runs.Source: §3 · p. 6

3.1 MAIN COMPARISON RESULTS

  • 24.24%, 11.33%, and 16.21% are the average accuracy gains for rank-8 NormTune+LoRA, Bone, and MoRA, respectively; rank-256 variants exceed S1 Full-FT by 2.17%, 0.93%, and 3.00%.Rank-8 variants surpass 90% of Full-FT performance and outperform several other rank-8 PEFT baselines.Source: §3.1 · p. 6, Table 1 · p. 7
  • All NormTune-enhanced variants outperform T1 on the same Qwen2.5-32B-Instruct backbone, and NormTune+MoRA surpasses SSFT-32B-GFPO by 0.58%.The paper reports these results with lower resource consumption.Source: §3.1 · p. 7

3.2 ABLATION STUDY

  • 68.6%, 67.36%, and 69.43% are the average scores NormTune achieves for LoRA, Bone, and MoRA, outperforming linear, stage, and exponential-decay adjustment strategies.Fixed learning-rate decay improves PEFT over its original versions, but its gains are smaller than NormTune’s.Source: Table 4 · p. 8, §3.2 · p. 9
  • 55.16% to 49.70%, 56.90% to 55.65%, and 59.66% to 49.47% are the average-accuracy changes for LoRA, Bone, and MoRA when the learning rate rises from 1 × 10^-5 to 1 × 10^-4.Full-FT also drops from 66.43% to 4.90%, motivating adaptive scaling of under-adapted layers rather than uniform learning-rate increases.Source: Table 5 · p. 8, §3.2 · pp. 8–9

3.1 MAIN COMPARISON RESULTS

  • NormTune+LoRA, Bone, and MoRA match Z1-7B Full-FT on code-related reasoning, while all three methods outperform Full-FT on OLMo-3-7B.NormTune+LoRA exceeds Z1-7B by over 0.6%; across 4B–32B models and math, code, and scientific question-answering tasks, NormTune consistently improves PEFT methods.Source: Table 2 · p. 7, §3.1 · p. 8, Table 3 · p. 8

3.2 ABLATION STUDY

  • 61.64% versus 37.40% is the average score for NormTune+LoRA versus LoRA at rank 8; NormTune+LoRA peaks at 68.6% at rank 256, then falls to 65.17% at rank 512.At rank 256, it slightly exceeds the Full-FT baseline of 66.43%.Source: Figure 5 · p. 9, §3.2 · p. 9

3.3 HYPERPARAMETER SENSITIVITY ANALYSIS

  • 62.12% to 64.14% is the GPQA accuracy range across bucket counts n ∈ {4, 8, 16, 32, 64}, with the best result at n = 16.The result indicates limited sensitivity to bucket count across the tested range.Source: Figure 5 · p. 9, §3.3 · p. 9

4 CONCLUSION

The conclusion reports that NormTune improves PEFT methods on reasoning-intensive tasks and can approach full fine-tuning performance with substantially lower resource consumption. The accompanying statements address AI use, ethics, and reproducibility.

4 CONCLUSION

  • NormTune consistently improves different PEFT methods and achieves performance comparable to full fine-tuning with substantially lower resource consumption.The conclusion reports experiments on three benchmarks.Source: §4 · p. 10

AI USE STATEMENT

  • Generative AI assisted only with organization and formatting of early drafts, not method conception, experimental implementation, or substantive writing.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The work involves no human subjects and introduces no new datasets; its primary foreseeable impact is improved PEFT performance on reasoning-intensive tasks.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The anonymized supplement is planned to include configurations, fixed splits and seeds, per-cell manifests, and fresh-process evaluation scripts.Source: REPRODUCIBILITY STATEMENT · p. 10

A APPENDIX

The appendix supplements the main paper with related work, extended results, and implementation details. It covers additional evaluations and analyses, experimental settings, hardware, and evaluation prompts.

A APPENDIX

  • The appendix supplements the main paper with related work, additional results, and implementation details.Source: A APPENDIX · p. 14
  • Appendix A.2 adds Qwen3-4B reasoning-intensive evaluations and analyses across ranks and task types, including GPQA and AIME results and detailed rank and task comparisons.Source: A APPENDIX · p. 14
  • Appendix A.3 describes additional benchmarks, experimental settings, and hardware configurations, while A.4 presents evaluation prompts.Source: A APPENDIX · p. 14
  • Appendix A.1 introduces the related work.Source: A APPENDIX · p. 14

A.1.1 TEST-TIME SCALING FOR REASONING

The related-work discussion contrasts test-time scaling approaches for reasoning with their computational demands. It distinguishes reinforcement learning from supervised fine-tuning and notes that SFT remains resource-intensive.

A.1.1 TEST-TIME SCALING FOR REASONING

  • Test-time scaling improves reasoning performance by allocating more computation at inference time, but reinforcement-learning approaches can require expensive post-training, be difficult to reproduce, or add inference-time search.Source: §A.1.1 · p. 14
  • Test-time scaling methods are broadly grouped into reinforcement learning and supervised fine-tuning, with cited work spanning both approaches.Source: §A.1.1 · p. 14
  • Supervised fine-tuning approaches are generally easier to reproduce and adopt than reinforcement-learning approaches, but still require substantial compute, especially for large models, and often high-memory GPUs.Source: §A.1.1 · p. 14

A.1.2 PARAMETER-EFFICIENT FINE-TUNING

PEFT adapts pretrained models without updating all backbone parameters, reducing GPU memory use. The discussion describes the trade-offs of early PEFT methods and low-rank adaptation, then introduces NormTune for reasoning-intensive tasks.

A.1.2 PARAMETER-EFFICIENT FINE-TUNING

A.2.1 REASONING-INTENSIVE EXPERIMENTS

On Qwen3-4B, NormTune improves PEFT performance across reasoning benchmarks and backbones; rank-8 methods gain substantially, while rank-256 LoRA reaches performance comparable to Full-FT.

A.2.1 REASONING-INTENSIVE EXPERIMENTS

  • 44.2% versus 37.7% average score for NormTune+LoRA versus LoRA on Qwen3-4B, surpassing Full-FT by 6%.The comparison covers GPQA, AIME’24, and AIME’25.Source: §A.2.1 · p. 14, Table 7 · p. 15
  • NormTune+MoRA raises MoRA’s average score from 33.9% to 42.9%, while NormTune+Bone achieves consistent gains and performance comparable to Full-FT.Source: §A.2.1 · p. 14, Table 7 · p. 15
  • TempBalance+LoRA, TempBalance+Bone, and TempBalance+MoRA reduce performance by 0.7%, 11%, and 17.3%, respectively, versus their corresponding PEFT methods.Source: §A.2.1 · p. 14, Table 7 · p. 15

A.2.2 OTHER EXPERIMENTS

Across the additional experiments, NormTune improves rank-256 LoRA and low-rank PEFT shows a larger gap from Full-FT on reasoning tasks than on commonsense and long-context tasks. The appendices also specify training settings, evaluation benchmarks, and prompt protocols.

A.2.2 OTHER EXPERIMENTS

  • 68.6% average accuracy for rank-256 NormTune+LoRA, slightly surpassing Full-FT on reasoning-intensive tasks.NormTune consistently improves LoRA across ranks, with especially pronounced gains at low ranks.Source: §A.2.2 · p. 15
  • Rank-8 PEFT is close to Full-FT on commonsense and long-context tasks but performs substantially worse on reasoning-intensive tasks.Source: §A.2.2 · p. 15, Table 11 · p. 17
  • Ten probing steps worsen performance; results are stable after 30 steps, while 50 steps delays NormTune activation by 20 updates and may slightly reduce performance.On Qwen3-4B GPQA, performance is relatively insensitive to bucket number n, with n = 16 remaining a strong default.Source: §A.2.2 · p. 15, Table 12 · p. 18, Table 13 · p. 18

A.3 ENVIRONMENT AND SETTINGS

  • Training uses eight NVIDIA A800 80GB GPUs; the default setup measures layer-wise norms after 30 included training updates, uses n = 16 buckets, and does not require a separate probing run.The additional probing-stage cost on Qwen2.5-32B-Instruct is 0.018 GPU-hours on eight A800 GPUs.Source: §A.3 · p. 16
  • Experiments use four base models and five training datasets, with the additional evaluation covering eight commonsense tasks and GovReport long-document summarization.The listed models include Qwen3-4B, Qwen-2.5-Coder-7B-Instruct, OLMo-3-7B, and Qwen2.5-32B-Instruct.Source: §A.3 · p. 16

A.4 PROMPTS FOR EVALUATION

  • Evaluation prompts vary by benchmark: S1K-1.1 uses lm-evaluation-harness prompts, Data-Repetition uses its official protocol, and commonsense tasks use an Alpaca-style template.For Z1, prompts come from the Qwen2.5-Coder repository for LiveCodeBench and BigCodeBench, with a specified step-by-step boxed-answer prompt for MATH-500 and GPQA Diamond.Source: §A.4 · p. 17
  • Repeated greedy evaluations average one completion per problem for S1K-1.1 and Z1; Data-Repetition and OLMo-3-7B use k sampled completions for Avg@k and Pass@k.Repeated runs use the same checkpoint and questions, rather than independent training runs or stochastic decoding samples.Source: §A.4 · p. 22
Loading 2610.29134v1…