PaperlayerView exact revision on arXiv

Source-linked AI summary

A Single Character can Make or Break Your LLM Evals

Jingtong Su, Jianyu Zhang, Karen Ullrich, Léon Bottou, Mark Ibrahim

arXiv:2510.05152v1cs.CLcs.AI

TL;DR

The paper asks whether the separator between few-shot examples—a largely understudied prompt choice—affects LLM evaluations. It varies delimiters and finds substantial performance and ranking changes, while reporting that naming the delimiter in the prompt improves robustness.

  • Problem

    The choice of delimiter between demonstration examples is understudied, despite its use in evaluations and real-world prompts.

    Source: §1 · p. 1

  • Method

    The study varies example delimiters in a realistic evaluation pipeline and probes attention-head scores on a dictionary-search task.

    Source: §1 · p. 2, §3 · p. 5

  • Results

    18.3%−29.4% MMLU performance differences occur across leading models as the delimiter changes, and the choice can put any model in the lead.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Specifying the selected delimiter in the prompt boosts robustness, and the authors offer practical delimiter recommendations.

    Source: §1 · p. 2, §7 · p. 9

    The study considers single non-alphanumeric ASCII delimiters and fixes other choices, including example selection, answer delimiters, and system prompts.

    Source: §7 · p. 9

Abstract

from arXiv · show

Common Large Language model (LLM) evaluations rely on demonstration examples to steer models' responses to the desired style. While the number of examples used has been studied and standardized, the choice of how to format examples is less investigated. In evaluation protocols and real world usage, users face the choice how to separate in-context examples: use a comma? new line? semi-colon? hashtag? etc.? Surprisingly, we find this seemingly minor choice can dramatically alter model response quality. Across leading model families (Llama, Qwen, Gemma), performance on MMLU for example can vary by $\pm 23\%$ depending on the choice of delimiter. In fact, one can manipulate model rankings to put any model in the lead by only modifying the single character separating examples. We find LLMs' brittleness pervades topics, model families, and doesn't improve with scale. By probing attention head scores, we find that good-performing delimiters steer attention towards key tokens in the input. Finally, we explore methods to improve LLMs' robustness to the choice of delimiter. We find specifying the selected delimiter in the prompt boosts robustness and offer practical recommendations for the best-performing delimiters to select.

1 Introduction

The paper studies whether the single character separating demonstration examples affects LLM evaluations, a choice that common protocols have largely left understudied. It finds large performance shifts, ranking reversals, attention changes, and a prompt-based robustness mitigation.

1 Introduction

  • The study targets an understudied choice in few-shot evaluations: which character separates examples intended to guide response style or format.The authors motivate this gap by noting that fixed evaluation templates do not capture prompt sensitivity in real-world use.Source: §1 · p. 1
  • 18.3%−29.4% MMLU performance differences occur across leading models when the single example delimiter changes.The study also reports large fluctuations on other benchmarks and tasks.Source: §1 · p. 2
  • Changing only the delimiter can manipulate model rankings to put any model in the lead.The evaluation protocol fixes other choices and varies the character separating demonstration examples.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • Specifying the selected delimiter in the prompt boosts robustness; Qwen2.5-7B-instruct gains +14.2% on MMLU with this modification.The paper also identifies “\n” and “!” as strong delimiter choices when the delimiter is unspecified.Source: §1 · p. 2
  • On a dictionary search task, good delimiters steer attention toward relevant demonstration-example content and significantly increase attention scores for key tokens.The probed tokens are those needed to solve the task.Source: §1 · p. 2

2 Related work

Related work describes prompt sensitivity across formatting, example order, wording, and prompting strategies. It also discusses brittleness in benchmark-based comparisons and evaluation design.

2.1 Prompt sensitivity

  • Small prompt-format changes can cause substantial performance variation; prior work reports shifts up to 30% and 76 points on different task sets.Example-order effects can also vary across models, while long-context models with hundreds of demonstrations were reported as less sensitive to order.Source: §2.1 · pp. 2–3
  • Prompt strategies range from chain-of-thought and self-consistency to learned prompts, persona prompting, and instruction or tool-use formats.Prior work also studies deliberation, confidence-aware prompting, and prompting guides.Source: §2.1 · p. 3

2.2 Brittleness of benchmarks

  • Benchmark comparisons can be brittle because rankings respond to prompt-format perturbations, limited test-set sizes, and benchmark-design choices.The cited work also discusses leaderboard instability and test-set memorization risks.Source: §2.2 · p. 4

3 Method: A Common evaluation protocol reflecting real world usage of LLMs

The method varies example separators while holding prompt content fixed, using a standardized evaluation pipeline intended to reflect real-world prompting. It tests instruction-tuned models on representative benchmarks and additional in-context tasks.

3 Method: A Common evaluation protocol reflecting real world usage of LLMs

  • Demonstrations can provide task knowledge or establish expected response style, motivating their use in evaluation and real-world prompts.Users must choose how to separate these examples.Source: §3 · p. 4
  • The study compares instruction-tuned Llama, Gemma, and Qwen models, including approximately 8B-scale models and Llama-3.1-70B.GPT-4o results appear in the appendix, while the study focuses on open-source models.Source: §3 · p. 4
  • The authors evaluate MMLU, ARC-Challenge, and CommonsenseQA, with classification and dictionary-lookup tasks used to explore delimiter effects further.The tasks assess model performance under different demonstration separators.Source: §3 · pp. 4–5
  • The evaluation appends task-specific demonstrations and a question to the user-role message using each model’s chat template, then evaluates the response.The pipeline aims for consistent comparison and real-world alignment, not the best possible score for each model.Source: §3 · p. 5

4 Experiments: Changing a single delimiter character can dramatically change performance on leading benchmarks

Changing only the delimiter produces substantial performance variation across benchmarks and MMLU topics, and can reorder model rankings. This sensitivity also persists across model scale and demonstration counts.

4 Experiments: Changing a single delimiter character can dramatically change performance on leading benchmarks

  • MMLU performance drops 18.3% for Llama-3.1-8B-instruct, 23.5% for Qwen2.5-7B-instruct, and 29.4% for Gemma-2-9B-instruct as delimiters vary.Similar fluctuations also appear on arc-challenge and commonsense-qa; the evaluation tests 30 non-alphanumeric delimiter characters.Source: Figure 2 · p. 5, §4 · p. 5
  • Changing only the delimiter can manipulate model rankings to place any model in the lead.Source: §4 · p. 5
  • Delimiter-driven performance fluctuations occur across MMLU topics, from history and philosophy to science and math.Figure 3 shows accuracy by topic across three model families.Source: §4 · p. 5, Figure 3 · p. 6

4.1 LLMs’ brittleness to the choice of delimiter is pervasive

  • Scaling Llama-3.1-instruct from 8B to 70B improves performance across three benchmarks but does not improve delimiter robustness.On commonsense-qa, the 70B model fluctuates ±40%, versus ±29.1% for the 8B model.Source: Figure 4 · p. 6, §4.1 · p. 6
  • Delimiter sensitivity persists as demonstrations increase from two to ten in in-context classification tasks.On Banking77, Llama-3.1-8B-instruct ranges from 20% to 80% depending on whether the delimiter is “[space]” or “\n”.Source: §4.1 · pp. 6–7, Figure 5 · p. 7

5 Improving LLMs’ robustness to the choice of delimiter

The paper tests delimiter-randomized supervised fine-tuning and prompts that explicitly name the delimiter as robustness strategies. Naming the delimiter improves performance across delimiter choices, and newline and exclamation mark perform well across model families and benchmarks.

5 Improving LLMs’ robustness to the choice of delimiter

  • Naive supervised fine-tuning with randomly varying delimiters does not improve sensitivity to delimiter choice.The authors suspect a distributional mismatch because the SFT training data does not contain in-context examples.Source: §5 · p. 7
  • Specifying the delimiter in the prompt improves performance across delimiter choices on all three benchmarks, with gains ranging from 1.5% to 27.9%.The prompt adds a line stating which character separates the demonstration examples; Table 2 reports average performance across delimiter choices.Source: §5 · p. 7, Table 2 · p. 8
  • The newline and exclamation-mark delimiters provide average performance boosts of 5.3% and 12.2%, respectively, over the average across delimiters.The gains average accuracy across MMLU, arc-challenge, and commonsense-qa.Source: Table 3 · p. 8, §5 · p. 8

6 Understanding how delimiters steer attention to key tokens

In dictionary lookup, delimiter choice shifts model performance substantially, while newline directs more attention to target keys.

6 Understanding how delimiters steer attention to key tokens

  • The analysis compares newline with space using feature ablation to measure attention to target keys in a task that requires locating the corresponding value.The task isolates whether attention focuses on relevant in-context samples.Source: §6 · p. 8
  • 25% higher attention scores for the target key with newline were statistically significant for Llama-3.1-8B-instruct.The paired t-test had a t-statistic of 15.59.Source: §6 · p. 8, Figure 6 · p. 9
  • Llama-3.1-8B-instruct accuracy ranges from 0% to 95% across delimiter choices.Figure 6’s right panel shows performance by delimiter.Source: §6 · p. 8, Figure 6 · p. 9

7 Conclusions

The paper finds that example delimiters can substantially affect model performance and rankings, and reports practical mitigations alongside a bounded study scope.

7 Conclusions

  • Reasonable delimiter choices can alter performance across leading benchmarks and model families enough to put any model in the lead.The authors frame delimiter choice as one of the options users face when interacting with chat models.Source: §7 · p. 9
  • Specifying a delimiter in the prompt consistently boosts performance across delimiters, and the paper recommends delimiters with strong average performance.Source: §7 · p. 9
  • The study tests only single non-alphanumeric characters and fixes other choices, including examples, answer delimiters, and system prompts.The authors call for more research on inference-time interactions and how this behavior emerges during training.Source: §7 · p. 9

Reproducibility

The supplementary material documents evaluation resources and delimiter coverage, reports brittleness across benchmarks, and finds no consistent improvement from SFT with varying delimiters.

Reproducibility

  • Evaluations use the Open-LLM-leaderboard harness for Llama, Gemma, and Qwen checkpoints, plus GPT-4o via API; interpretability uses Captum feature ablation.Source: Reproducibility · p. 9

A mmlu Benchmark SoTA Evolution across Years

B The set of delimiters

C Additional Evaluations

D SFT use varying delimiters failed to boost performance

  • Tuning Llama-3.2-3B-Instruct with varying ASCII delimiters produces no consistent improvement, unlike specifying the delimiter in the prompt.The reported SFT uses the Tulu dataset and LoRA; similar results are noted for ARC-Challenge and CommonsenseQA.Source: D SFT use varying delimiters failed to… · pp. 13–14

D.1 Measuring the relative frequency of delimiters in SFT data

In the Tulu SFT dataset, newline has the highest relative frequency among the delimiters measured.

D.1 Measuring the relative frequency of delimiters in SFT data

  • Newline appears with the highest relative frequency among delimiters in Tulu SFT.Tulu SFT is described as an open-source dataset used for instruction tuning Olmo-2 models.Source: §D.1 · p. 14

E Delimiters are not consistently best per model across benchmarks

Delimiter choices can vary by MMLU topic, especially for the larger Llama model, and GPT-4o results are consistent with the open-source observations. In a dictionary lookup task, target-key attention scores also differ significantly across delimiters.

E Delimiters are not consistently best per model across benchmarks

  • Best delimiters are relatively stable across MMLU topics for 7–9B models but less stable for 70B Llama, suggesting topic interactions can matter.Table 14 presents the best delimiter by model across MMLU topics.Source: E Delimiters are not consistently best… · p. 15

F Evaluation on closed-source model

  • GPT-4o generation-based MMLU results are consistent with the observations on open-source models, using a customized answer filter.The original Gao et al. (2024) filter failed to parse the strings correctly in a preliminary experiment.Source: F Evaluation on closed-source model · p. 15

G Measuring attention scores for the dictionary lookup task

  • Target-key attention scores differed significantly across delimiter conditions, with a t-statistic of 15.59 (p-value of < 0.001).The analysis compared target lookup-key attention against other lookup keys, using newline and space delimiters.Source: G Measuring attention scores for the… · p. 16

H Reproducibility on lm-eval-harness (Gao et al., 2024)

The section gives instructions for reproducing delimiter evaluations with lm-eval-harness across MMLU, ARC-Challenge, and CommonsenseQA. It describes template changes for replacing the example delimiter and configuring prompts to name it.

  • Reproduce the evaluations by configuring example delimiters in lm-eval-harness for MMLU, ARC-Challenge, and CommonsenseQA, then running each benchmark normally.The instructions begin with cloning the repository and locating the task configuration folder.Source: H Reproducibility on lm-eval-harness… · pp. 16–17
  • For ARC-Challenge and CommonsenseQA, append YAML settings that use the selected example delimiter as the few-shot delimiter.The instructions specify the relevant benchmark configuration files.Source: H Reproducibility on lm-eval-harness… · pp. 16–17
  • To test delimiter specification, add benchmark descriptions that explicitly state which delimiter separates the demonstration examples.The instructions give separate descriptions for ARC-Challenge and CommonsenseQA.Source: H Reproducibility on lm-eval-harness… · p. 17
  • For MMLU, replace each subject YAML description with one that identifies the subject and says the question-answer examples are separated by the selected delimiter.The specified description also includes a blank line after the instruction.Source: H Reproducibility on lm-eval-harness… · p. 17
Loading 2510.05152v1…