PaperlayerView exact revision on arXiv

Source-linked AI summary

Prompt-Robust Language Models: Which Training Strategies Work?

Frederic Sadrieh, Michal Štefánik

arXiv:2609.01217v1cs.AI

TL;DR

LLMs remain highly sensitive to prompt formulation, so the paper systematically compares train-time data-construction and robustness-objective strategies under controlled conditions. Robustness fine-tuning improves over standard fine-tuning and in-context learning, but large prompt gaps remain, while COIN and PPCL often fail to beat one-template-per-batch training because their auxiliary objectives do not generalize beyond the quantity they penalize.

  • Problem

    LLM performance varies substantially with prompt formulation, and prior work leaves unclear which train-time construction strategies and robustness objectives provide meaningful gains.

  • Method

    The paper systematically compares multi-template data construction, consistency regularization, and contrastive methods under controlled train-time robustness experiments.

  • Results

    40–57% of performance remains between the best- and worst-performing prompts across every tested method, while COIN and PPCL often fail to outperform simple data construction.

  • Takeaways & Limitations

    ONE-AT-A-TIME training is the most reliable simple data-construction strategy, whereas mixed-template batches face gradient interference and auxiliary losses fail to generalize beyond their training target.

  • Takeaways & Limitations

    Experiments cover models up to 8B parameters across only two model families, so whether the observed patterns hold at larger scales remains open.

Abstract

from arXiv · show

Despite their strong performance, large language models remain highly sensitive to prompt formulation. Prior work addresses this through refined data construction or through dedicated robustness objectives. We reproduce and compare these strategies under controlled conditions, and measure how effective they are in addressing models' prompt sensitivity. We find the current robustness fine-tuning methods improve over standard fine-tuning and in-context learning, but the best-to-worst prompt gap remains as high as 40-57% of performance. Moreover, the recent robustness-enhancing methods we test - CoIN for contrastive alignment and PPCL for consistency regularization - often fail to outperform the simplest data construction strategy: training on one template per batch. Our diagnostics explain these results. The auxiliary objectives move the quantity they penalize, but do not generalize beyond it. Additionally, data construction strategies differ due to the conflicting signs of per-template gradients on 57-64% of parameters. Thus, batches that mix formulations force the optimizer to reconcile competing updates instead of finding a shared, prompt-agnostic one.

1 Introduction

The paper systematically compares train-time strategies for reducing prompt sensitivity and finds that robustness methods still leave substantial prompt-dependent performance variation.

  • Motivation: LLMs remain brittle because semantically equivalent prompts can produce drastically different outputs, while manual prompt engineering does not scale.This motivates building robustness into models during training rather than addressing sensitivity only at inference time.
  • Research gap: Prior multi-prompt tuning outperforms standard single-prompt instruction fine-tuning, but the relevant data-construction design space remains poorly understood.The paper asks whether more sophisticated robustness objectives add meaningful gains beyond data construction.
  • Approach: The study compares three train-time method families: multi-template data construction, consistency regularization, and contrastive methods.These categories are evaluated as strategies improving on instruction fine-tuning.
  • Diagnostics: 57–64% of parameters show conflicting gradient signs across prompt templates, indicating that mixed formulations can create template interference.Such interference helps explain why batch construction matters for prompt robustness.
  • Findings: COIN and PPCL fail to reliably improve upon data-construction strategies because their objective-specific gains do not generalize broadly.The reported comparison challenges the assumption that more elaborate robustness losses necessarily provide additional benefits.
  • Findings: 40–57% of performance remains between the best- and worst-performing prompts across every tested method.The result scopes the limits of current train-time robustness methods.

2 Related Work

Related work establishes prompt sensitivity as a persistent problem and motivates a controlled comparison of train-time robustness methods across major prompt perturbations.

  • Prompt sensitivity: Prompt sensitivity has been documented across model sizes and tasks, with paraphrases and minor structural changes causing substantial performance swings.This establishes that prompt brittleness is not confined to a particular scale or task.
  • Existing responses: Instruction fine-tuning raises mean performance but leaves prompt sensitivity largely intact.Inference-time approaches address brittleness through optimization, calibration, or self-correction, but add algorithmic overhead and require rerunning at test time.
  • Train-time robustness: Train-time methods build invariance directly into models through multi-prompt fine-tuning, consistency regularization, or contrastive alignment.These approaches use prompt variation during training rather than relying solely on test-time intervention.
  • Research gap: Earlier studies did not systematically compare robustness-training families under controlled conditions and major prompt perturbations.The paper positions its comparison as broader than studies focused mainly on local perturbations or test-time methods.

3 Experimental setup

The experiments compare batch-level template selection and auxiliary robustness losses across multiple model families, scales, datasets, and prompt-based evaluation metrics.

  • Evaluation: The evaluation trains each method on training tasks and tests it on disjoint unseen tasks using best-, worst-, and average-template performance.This directly measures both overall performance and sensitivity to prompt formulation.
  • Data construction: Four data-construction strategies vary prompt diversity within batches: SINGLE, ALL SHUFFLED, ALL-IN-ONE-BATCH, and ONE-AT-A-TIME.ONE-AT-A-TIME uses only one template per batch, while ALL-IN-ONE-BATCH groups all templates for an example together.
  • Consistency regularization: PPCL adds a Jensen–Shannon divergence penalty between output distributions for semantically equivalent prompt formulations.Its λ terms control the trade-off between instruction following and prompt robustness.
  • Contrastive alignment: COIN adds a contrastive loss over internal representations to align equivalent prompts and separate syntactically similar but semantically different prompts.The method is selected as a representative contrastive-alignment approach.
  • Models: The study evaluates base variants of four Llama and Qwen models spanning two families and two scales, with robustness differences attributable to training method.The models include 0.6B, 1B, and 8B parameter scales.
  • Datasets: Training uses 48 datasets and testing uses 11 datasets from unseen tasks, with PromptSource templates covering paraphrasing and structural changes.Datasets with fewer than three majority templates are excluded from majority-template training required by COIN and PPCL.
  • Metrics: Rouge-L is the main metric, and each method is assessed relative to instruction fine-tuning on average, worst-case, and best-case template performance.Rank classification accuracy is additionally reported in an appendix.

4 Results

Robustness training generally improves over IFT and ICL, but prompt sensitivity remains substantial. Simple template-homogeneous batching often matches or outperforms PPCL and COIN because mixed-template updates interfere.

  • Impact of robustness training: ICL improves over base models but usually remains below IFT; on Qwen3-8B it reaches 0.660 versus IFT’s 0.634 average Rouge-L yet remains less robust.Robustness training has a one-time cost, whereas ICL adds recurring inference cost.
  • Model choice: Smaller models are disproportionately prompt-sensitive: Llama3.2-1B shows a 0.419 Rouge-L worst-to-best spread, while Llama3.1-8B can lose worst-case performance under multi-prompt IFT.For Llama3.1-8B, strong IFT performance limits the ceiling for robustness gains and later training improvements are small and inconsistent.
  • Data construction: ONE-AT-A-TIME delivers the most consistent small-model improvements, raising worst-template Rouge-L from 0.210 to 0.363 on Llama3.2-1B and from 0.308 to 0.449 on Qwen3-0.6B.Its gains are statistically best or equivalent to best across all metrics for smaller models.
  • Data construction: 57–64% of parameters show conflicting signs between template gradients, causing ALL-IN-ONE-BATCH to reconcile interfering updates rather than learn a shared prompt-agnostic update.The mean pairwise cosine similarity of template updates is 0.54, and interference is strongest in early blocks.
  • Auxiliary objectives: PPCL and COIN provide only modest gains and are generally matched or exceeded by simpler data construction, with COIN the sole exception on Qwen3-8B average performance.PPCL attains the best average and worst-template performance only in the reported exception involving Llama3.1-8B.
  • Diagnostics: COIN shifts hidden-state separation at the label readout to 0.74–0.79, but its advantage shrinks to at most 0.03 at the prompt readout on Qwen models.On Llama models, COIN separates prompts worse than the strongest data-construction approaches.
  • Diagnostics: PPCL achieves the lowest validation cross-template JS divergence for both Llama models, but its advantage disappears on held-out tests and does not generalize.Multi-template schedules already reduce validation divergence roughly two- to fivefold over single-template training.

5 Conclusion

Simple train-time choices, especially ONE-AT-A-TIME batching, provide the most reliable robustness gains among the tested strategies. However, substantial prompt variance remains, and auxiliary objectives are difficult to justify when their benefits fail to generalize.

  • Conclusion: ONE-AT-A-TIME provides the most reliable worst-case gains by keeping same-template samples together within updates.ALL-IN-ONE-BATCH underperforms because conflicting template gradients cancel or interfere.
  • Conclusion: PPCL and COIN are mostly matched or outperformed by simple data construction, making their added complexity difficult to justify.Their auxiliary loss objectives improve the training setting but fail to generalize beyond it.
  • Conclusion: Even the best train-time method leaves ample room for improvement because prompt robustness remains limited after training.The paper therefore advises prioritizing simple ONE-AT-A-TIME or ALL-SHUFFLED schedules before auxiliary objectives.

Limitations

The study is limited by its model, benchmark, method, and hyperparameter coverage, constraining how broadly its robustness conclusions can be generalized.

  • Model scale and family coverage: The experiments cover models up to 8B parameters from only the Llama and Qwen families.Whether the observed scheduling and auxiliary-objective patterns hold at larger scales remains open.
  • Benchmark scope and evaluation protocol: The benchmark consists predominantly of classification and short-answer tasks with closed label sets.Long-form generation and reasoning tasks may exhibit prompt sensitivity differently.
  • Method selection within each category: The consistency-regularization and contrastive categories are represented by only PPCL and COIN.Conclusions therefore apply to these specific instantiations rather than the broader methodological families.
  • Hyperparameter optimization: Auxiliary-loss weights were not fully tuned per model, and COIN’s hyperparameters were not tuned beyond advised values.The authors cannot exclude that fuller tuning would narrow the reported gaps, although diagnostics argue against under-tuning as the sole explanation.

A Dataset statistics

The dataset setup adapts the T0-style PromptSource collection by replacing incompatible datasets, filtering unsuitable templates, and retaining only datasets with multiple usable templates.

  • Dataset selection: The study replaces tydiqa with Zaid/coqa_expanded and gigaword with scitldr because the original versions were incompatible with most PromptSource templates.These substitutions affect the training-data setup.
  • Template filtering: Templates are manually filtered when inapplicable or when they change the task type and could cause train/test leakage.HellaSwag topic-classification reformulations are given as an example.
  • Template filtering: social_i_qa, riddle_sense, and wiki_bio are removed because each has no more than one template remaining after filtering.Dataset-selection changes affect only the training split.
  • Template construction: PromptSource templates preserve task functionality while changing wording, framing, and answer-option ordering.Each template maps an input example to an instruction x and expected answer verbalization y.
  • Template examples: Eight filtered SuperGLUE COPA templates render the same example with substantial rephrasing and prompt-structure variation.Four additional templates were removed because they failed to render with the current Hugging Face dataset version.

C Hyperparameters

The experiments use shared training settings, tune learning rates under a limited sweep, and apply distinct handling for PPCL, COIN, and inference-time baselines.

  • Learning-rate selection: Learning rates are selected per model and method by grid search over 10^-5, 3×10^-5, 5×10^-5, 7×10^-5, 9×10^-5, and 10^-4.The sweep covers only the first 2,000 training steps because of compute constraints.
  • Auxiliary-loss settings: PPCL’s λ3 is tuned over 0.1, 1, 10, and 100, with λ3 = 1 stable among the best settings and collapse occurring only at 100 on Qwen3-0.6B.The same protocol is applied on Llama3.2-1B and Qwen3-0.6B.
  • Auxiliary-loss settings: COIN’s τ and λ use the advised values from Yan et al. without further tuning.This choice reflects computational constraints.
  • Compute: All experiments run on H200 GPUs for roughly 2,800 GPU hours.
  • Inference-time baselines: The base-model and ICL baselines load or update no weights, while ICL prepends four demonstrations consistently rendered with each query template.The same sampled demonstrations are reused across templates to isolate template effects.

E Validation Performance over Training

Validation performance improves rapidly during early training, while later checkpoints add little benefit and some methods show non-monotonic behavior.

  • Validation trajectory: Most performance improvement occurs within the first 1,000 training steps.No run improves by more than four points afterward, except ONE-AT-A-TIME, which oscillates by up to ten points.
  • Validation trajectory: No run improves monotonically across checkpoints.The base model initially fails the required output format, whereas checkpoints after 1,000 steps follow it.
  • Validation trajectory: LoRA tuning contributes little additional benefit after the early improvement phase.
  • Experimental settings: Table 3 lists shared hyperparameters for all experiments, with LoRA applied only to the 8B models.
  • Experimental settings: Table 4 reports learning rates selected per model and method, while template variants sharing the standard loss use one learning-rate tuning result.
  • Prompt variation: Figure 3 compares filtered COPA prompt renderings on one example, illustrating variation in question wording and prompt structure.
  • PPCL sensitivity: The PPCL sweep on Qwen3-0.6B reports Rouge-L on the test split and marks statistically best values in bold.

F Rank Classification Accuracy

The section explains why generative Rouge-L is preferred over rank classification accuracy for evaluating prompt robustness, because constrained candidate ranking can hide reformulation effects and separates methods poorly.

  • Metric choice: Generative Rouge-L is chosen as the primary metric because rank classification accuracy does not evaluate generative behavior across reformulations.Rank classification only asks models to order a provided candidate set and may show robustness even when generation shifts substantially.
  • Metric choice: Rank classification accuracy compresses method differences: on Qwen3-8B, all methods fall within 0.3 accuracy points of one another.
  • Metric scope: Rank classification accuracy is available only on the test split because training and validation include examples without concrete answer sets.
  • Metric scope: The evaluation reports average, best-template, and worst-template performance across templates.

H Gradient Interference

The gradient-interference analysis shows that mixed-template training combines substantially conflicting updates, while diagnostics indicate that robustness objectives can improve their targeted quantities without reliably transferring to held-out prompts.

  • Training schedules: ALL-IN-ONE-BATCH training mixes conflicting template gradients, whereas ONE-AT-A-TIME keeps those gradients from being co-located.
  • Objective diagnostics: PPCL obtains lower validation cross-template Jensen–Shannon divergence for Llama models, but this advantage disappears on held-out test data.The diagnostic therefore separates in-distribution objective improvement from held-out prompt consistency.
  • Objective diagnostics: COIN dominates at the label readout where its loss is applied, but not at the prompt readout where generation begins.
  • Interference results: 57% and 64% of trainable parameters receive opposing signs from templates in the two Llama models.These conflicts remain stable across training and accompany about 18–20% cancellation of gradient magnitude under averaging.
  • Interference results: Per-template gradients span about three effective directions rather than one, despite a mean pairwise cosine similarity of about 0.54.The measurements indicate coarse agreement alongside substantial coordinate-level disagreement.
Loading 2609.01217v1…