PaperlayerView exact revision on arXiv

Source-linked AI summary

Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance

Ahmed Alajrami, Xingwei Tan, Nikolaos Aletras

arXiv:2510.03528v2cs.CL

TL;DR

Prior work found that models can be sensitive to prompt variation, while the effects of training on noisy instructions remained an open question. The paper fine-tunes models on instruction data with varied perturbations and evaluates them on standard benchmarks. It reports that structural perturbations can enhance performance, particularly under noisy evaluation, and that highly perturbed training can also help on standard prompts.

  • Problem

    Models have shown sensitivity to instruction phrasing, but how noisy instruction data affects models during training had not been explored.

    Source: §1 · p. 1

  • Method

    The authors fine-tune models on instruction datasets with varying proportions of perturbed instructions and evaluate them on MMLU, BBH, and GSM8K.

    Source: §1 · p. 2

  • Results

    Structural perturbations can enhance performance, particularly under noisy evaluation, and models trained on highly perturbed instructions tend to perform better with both noisy and standard prompts.

    Source: §8 · p. 9

  • Takeaways & Limitations

    Instruction perturbation is a simple, potentially effective strategy for improving model resilience when user instructions are inconsistent or ambiguous.

    Source: §8 · p. 9

    Experiments used only English instructions and tasks, so the effects of perturbation across languages remain untested.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Instruction-tuning plays a vital role in enhancing the task-solving abilities of large language models (LLMs), improving their usability in generating helpful responses on various tasks. However, previous work has demonstrated that they are sensitive to minor variations in instruction phrasing. In this paper, we explore whether introducing perturbations in instruction-tuning data can enhance LLMs' resistance against noisy instructions. We focus on how instruction-tuning with perturbations, such as removing stop words or shuffling words, affects LLMs' performance on the original and perturbed versions of widely-used benchmarks (MMLU, BBH, GSM8K). We further assess learning dynamics and potential shifts in model behavior. Surprisingly, our results suggest that instruction-tuning on perturbed instructions can, in some cases, improve downstream performance. These findings highlight the importance of including perturbed instructions in instruction-tuning, which can make LLMs more resilient to noisy user inputs.

1 Introduction

Earlier work examined sensitivity to prompt variation and noisy instructions at inference, but the effects of noisy instruction-tuning were unexplored. This paper asks whether training on perturbed instructions can improve resilience to noisy user inputs.

1 Introduction

  • Prior studies found sensitivity to prompt variability and examined inference-time noise, while the effects of noisy instruction-tuning remained unexplored.Instruction datasets often use high-quality paraphrases with minimal noise.Source: §1 · p. 1
  • The study asks whether fine-tuning base models on perturbed instructions can improve resilience to noisy user inputs.The question is motivated by prior work framing training noise as a form of regularization that can prevent overfitting and improve generalization.Source: §1 · p. 1
  • The experiments vary the proportion of perturbed instructions in combined training data and evaluate MMLU, BBH, and GSM8K.The datasets combine GPT4-Alpaca, Super-Natural, and Dolly; perturbation proportions range from 25% to 100%.Source: §1 · p. 2
  • The authors report a systematic study of syntactic and semantic instruction changes and suggest noisy fine-tuning may offer a simple approach to robustness.They question whether complete instruction comprehension is always necessary for effective task learning.Source: §1 · p. 2

2 Related Work

Related work examines how instruction-tuning changes model behavior and how instruction-tuned models respond to varied or noisy prompts. This study shifts attention to the effects of training on noisy instructions.

2.1 Analyzing Instruction-tuning

  • Instruction-tuning research has examined output-token shifts and found that knowledge content can originate from untuned models, with tuning aligning instructions to existing parametric knowledge.Lin et al. (2024) observed shifts concentrated in stylistic tokens; Ren et al. (2024) characterized tuning as self-aligning instructions with existing knowledge.Source: §2.1 · p. 2

2.2 Robustness of Instruction-tuned Models

  • Robustness studies report sensitivity to paraphrases, noisy speech-recognition or OCR inputs, and reasoning prompts, while also proposing alignment, augmentation, and denoising approaches.Gu et al. (2023) found greater sensitivity in zero-shot than few-shot settings; Sun et al. (2024) reported paraphrase-related inconsistency.Source: §2.2 · pp. 2–3
  • Unlike prior studies, this work investigates how training on noisy instructions affects models’ ability to adapt to instruction perturbations.The related-work section includes an example table of GPT4-Alpaca instructions and perturbations.Source: §2.2 · p. 3, Table 1 · p. 3

3 Instruction Perturbation Strategies

The paper perturbs instruction text in six ways to test how models handle changes to its wording, order, and spelling. The strategies range from removing words to inserting contextually plausible tokens or typos.

3 Instruction Perturbation Strategies

  • The six strategies are stop-word deletion, word shuffling, word deletion, word replacement, word insertion, and misspelling.They are applied to the instruction component, not the associated context.Source: §3 · p. 3
  • Word-order shuffling changes syntax, while deleting stop words or other words tests whether models can infer intent from incomplete instructions.Shuffling is capped at 25%; random deletion removes 25% of words, whereas stop-word deletion targets functional words.Source: §3 · p. 3
  • Replacement and insertion use BERT-predicted tokens, testing sensitivity to lexical variation and to added, potentially redundant or meaning-shifting words.The insertion strategy places masks at randomly selected positions covering approximately 25% of the instruction’s word count.Source: §3 · pp. 3–4
  • The misspelling strategy introduces character-level typos into randomly selected instruction words to test whether models can discern intended meaning.A typo is introduced into 25% of the words in each instruction.Source: §3 · p. 4

4 Experimental Setup

The experiments fine-tune four open-weight model sizes on mixtures of three instruction datasets, then assess performance across benchmark and safety evaluations. The setup varies the perturbation share in training and evaluation instructions.

4.1 Models

  • The study evaluates Qwen-2.5 models at 7B and 72B and Llama-3.1 models at 8B and 70B.These are the two open-weight base-model families used in the experiments.Source: §4.1 · p. 4

4.2 Instruction Datasets

  • Training mixtures combine GPT4-Alpaca, Super-Natural Instruction, and Dolly, with perturbed-instruction shares of 0%, 25%, 50%, 75%, or 100%.The datasets include synthetic GPT-4 responses, diverse tasks, and human-generated practical instruction-following examples.Source: §4.2 · p. 4

4.3 Implementation Details

  • The authors use LoRA for 7B and 8B models and QLoRA for 70B and 72B models, fine-tuning each for one epoch.Experiments ran on a single NVIDIA H100 GPU.Source: §4.3 · p. 4

4.4 Evaluation

  • Evaluation covers MMLU, BBH, and GSM8K at five instruction-perturbation levels, plus ToxiGen and TruthfulQA on original prompts.MMLU reports average accuracy, BBH average exact match, and GSM8K exact match; ToxiGen measures toxic outputs, while TruthfulQA assesses truthfulness and informativeness.Source: §4.4 · pp. 5–6

5 Results

Across the three benchmarks, instruction perturbations often coincide with stronger performance on noisy instructions and sometimes on original instructions, but the best perturbation level varies by task and model. CoT results favor noisy fine-tuning in some settings, while instruction-tuning gains are uneven across tasks.

Fine-tuning on perturbed instructions may enhance robustness under noisy prompts

  • On BBH (CoT), Llama-70B trained with 100% perturbed instructions scores highest at 25%, 50%, 75%, and 100% perturbed evaluation; Qwen-72B trained on original instructions leads at 25%.Qwen-7B trained with 50% perturbation has 0.5% higher accuracy than VAN on MMLU when evaluated with 75% perturbed instructions.Source: §5 · p. 6

Higher proportions of perturbed instructions appear to be beneficial in some contexts

  • For MMLU (5-shot) and BBH (CoT), Qwen-7B, Llama-8B, and Llama-70B peak with 50% or more noisy instructions, whereas smaller models favor less perturbation on GSM8K (CoT).The authors suggest incomplete or ambiguous instructions may be especially challenging and harmful for smaller models on multi-step math reasoning.Source: §5 · p. 6

Observed gains on standard unperturbed benchmarks from noisy fine-tuning

  • On unperturbed MMLU (5-shot), Llama-8B and Llama-70B trained with 100% perturbed instructions achieve best scores of 66.0% and 78.6%, respectively.On BBH (CoT), Qwen-7B trained with 75% perturbation scores 0.6% higher than its original-instruction-trained variant.Source: §5 · p. 6

CoT remains more effective than direct prompting

  • On BBH (CoT), Llama-70B trained with fully perturbed instructions scores 79.0% and 76.4% with 25% and 50% perturbed evaluation, respectively; CoT remains more effective than direct prompting.Qwen-72B performs best with less perturbed training instructions, yet its weaker direct-prompting performance is described as suggesting a perturbation benefit for generalization and reasoning robustness.Source: §5 · p. 6

Instruction-tuning yields uneven gains across tasks

  • Instruction-tuning yields modest MMLU gains regardless of perturbation but consistently benefits BBH, especially for Llama models; underrepresented or poorly learned tasks may gain more.The proposed explanation is that pretraining may already represent some tasks well, leaving less room for further gains.Source: §5 · pp. 6–7

6 Analysis

The analysis finds perturbation-associated gains in safety and benchmark performance, but effects vary across models and perturbation types. Stop-word removal and word shuffling sometimes improve results, while qualitative examples show that replacing or deleting key instruction words can impair correctness.

6.1 Safety and Bias

  • On ToxiGen, perturbation tuning is associated with lower toxicity for Qwen-7B and Llama-8B at 100% perturbation and Llama-70B at 75%; on TruthfulQA, three of four models improve with fully perturbed data, except Qwen-7B.Qwen-72B performs better on ToxiGen with original instructions.Source: §6.1 · p. 7

6.2 Individual Perturbation Strategies

  • STOP improves Llama-8B over original-instruction training on several benchmarks, while SHFL yields the highest TruthfulQA and ToxiGen scores despite shuffling 25% of instruction words.Table 4 covers STOP and SHFL on Dolly, plus varying word-shuffle levels on GPT4-Alpaca.Source: §6.2 · p. 7, Table 4 · p. 7

6.3 Perturbation Intensity Ablation

  • With Llama-8B on GPT4-Alpaca, shuffling 50% or more of instruction words often yields strong benchmark results, and fully shuffled instructions sometimes outperform unperturbed training.The authors suggest heavier noise may reduce reliance on particular wording or fixed prompt templates.Source: Table 4 · p. 7, §6.3 · p. 8

6.4 Qualitative Analysis

  • Llama-70B often answers correctly after stop-word removal or misspellings, but replacing key words or deleting “sentence” and “order” can hinder correctness.Table 5 presents generated answers for a perturbed BBH adjective-order question.Source: §6.4 · p. 8, Table 5 · p. 8

7 Theoretical Grounding of Fine-Tuning on Noisy Instructions

The authors frame noisy instruction-tuning as possible implicit regularization and data augmentation that may support robust, generalizable task representations. They also caution that effects are not uniform across models, with smaller models showing inconsistent gains and possible sensitivity to excessive perturbation.

Theoretical grounding

  • Noisy instruction-tuning may act as implicit regularization and data augmentation, broadening training formulations and discouraging overfitting to canonical phrasing.The authors suggest perturbations may provide useful inductive bias and improve generalization across prompt formats.Source: §7 · p. 8
  • The proposed mechanism is that exposure to syntactic or semantic anomalies may reduce reliance on superficial linguistic patterns and support more robust task representations.These are presented as plausible explanations rather than established causal findings.Source: §7 · p. 8

Model-specific boundary

  • Perturbation benefits are not uniform: larger Llama-70B and Qwen-72B models benefit substantially, while smaller Llama-8B and Qwen-7B models show inconsistent gains.The authors note a possible upper limit beyond which perturbations may hurt limited-capacity models or tasks requiring precise instruction-following.Source: §7 · pp. 8–9

8 Conclusion

The conclusion reports that training on highly perturbed instructions can improve performance under noisy and standard prompts, and identifies instruction perturbation as a potential resilience strategy.

8 Conclusion

  • Highly perturbed instruction training tends to improve performance under both noisy test conditions and standard prompts.The authors suggest this encourages more flexible task representations.Source: §8 · p. 9
  • Structural perturbations can enhance model performance, particularly when evaluation instructions are noisy.The study evaluates models of varying sizes across diverse benchmarks.Source: §8 · p. 9
  • Instruction perturbation is presented as a potentially effective resilience strategy, with adaptive or semantically aware techniques proposed for future study.Source: §8 · p. 9

Limitations

The experiments cover English instructions and downstream tasks only; the authors note that language-specific characteristics may affect perturbation outcomes.

Limitations

  • Experiments use English instructions and downstream tasks only.The authors cite the availability of diverse public instruction-tuning data as the reason for this scope.Source: Limitations · p. 9
  • The effects of perturbation across languages remain unexamined, despite differences in sensitivity to word order and stop words.Source: Limitations · p. 9
  • The authors call for studying perturbation in relation to each language's characteristics, noting Chinese has fewer stop words and less rigid syntax than English.Source: Limitations · p. 9

Appendix

The appendix identifies Open Instruct as the implementation adopted for instruction-tuning and evaluation, and points to Table 6 for fine-tuning hyperparameters.

Appendix

B Full Results

The appendix points to full MMLU, BBH, and GSM8K results in Tables 7–9, supplementing results presented in Table 3.

B Full Results

  • Tables 7–9 provide full results for MMLU, BBH, and GSM8K, alongside the results presented in Table 3.Source: B Full Results · p. 13
  • Table 7 reports MMLU accuracy for original and perturbed evaluation instructions.Bold values mark the best performance for each model.Source: Table 7 · p. 14
  • Table 8 reports average exact match (EM) on BBH, while Table 9 reports exact match (EM) on GSM8K, for original and perturbed evaluation instructions.Both tables mark the best performance for each model in bold.Source: Table 8 · p. 14, Table 9 · p. 15

Related papers

Loading 2510.03528v2…