Source-linked AI summary
Lexical Perturbations Disrupt LLM Reasoning: An Empirical Study of Attention Diversion
Jiaqian Zhu, Yang Zhang, Junhua Ding, Xiaowei Yu
TL;DR
The paper addresses limited evidence about how realistic, meaning-preserving lexical corruption affects LLM reasoning. It evaluates multiple models and benchmarks with controlled perturbations and internal interventions, finding that tokenizer fragmentation diverts attention and couples corrupted content with attention allocation. This coupling helps explain why inference-time repair strategies do not consistently recover performance.
Problem
Evidence on LLM robustness to realistic lexical corruption is limited, especially for reasoning tasks and internal processing changes.
Method
The study evaluates multiple instruction-tuned and frontier models across four reasoning benchmarks using lexical perturbations, controls, diagnostics, and factorial interventions.
Results
Character-level perturbations consistently degrade accuracy, especially on multi-step reasoning, while filler insertion has little effect; tokenizer fragmentation and attention diversion account for this asymmetry.
Takeaways & Limitations
Because corrupted token content and diverted attention are coupled, restoring either channel alone is insufficient and inference-time strategies do not consistently recover performance.
Takeaways & Limitations
The main analyses focus on synthetic perturbations in English QWERTY, standard-length single-turn tasks, and a factorial intervention on GSM8K at one severity.
Abstract
from arXiv · showhide
Large Language Models (LLMs) achieve strong reasoning performance, but their robustness to realistic lexical corruption remains poorly understood. We evaluate four open-weight instruction-tuned models and frontier models across four reasoning benchmarks under keyboard noise, character swaps, and filler insertion. Character-level perturbations substantially degrade accuracy, especially on multi-step reasoning tasks, while filler insertion has little effect. We trace this asymmetry to Attention Diversion: lexical corruption fragments subword tokenization, and the resulting fragments attract disproportionate attention mass, concentrated in middle and final transformer layers. Length-matched controls confirm that fragmentation, not prompt length, drives the loss. A factorial intervention then shows why the damage is hard to undo: fragmentation corrupts token content and attention allocation together, and the two are coupled. Restoring clean attention while the content remains corrupted is actively harmful, restoring content alone is insufficient, and only restoring both recovers a substantial share of the gap. This coupling explains why inference-time strategies, including chain-of-thought prompting, spell-checking, self-repair, and stronger repair models, fail to consistently recover performance: each addresses one channel at a time. Code and data are available at https://github.com/Jiaqian-Janelle/Attention-Diversion
1 Introduction
LLMs can reason well yet remain brittle to meaning-preserving lexical corruption. The study evaluates this asymmetry and attributes it to Attention Diversion caused by tokenizer fragmentation, with coupled effects on token content and attention allocation.
- Meaning-preserving character corruptions can markedly reduce LLM reasoning accuracy, whereas filler insertion leaves performance nearly unchanged despite increasing prompt length.
- Prior work leaves reasoning-intensive tasks, prompt length controls, and links between lexical noise and internal processing underexplored.
- Across four reasoning benchmarks and multiple model settings, character-level perturbations consistently hurt performance most on multi-step mathematical reasoning, while filler has little effect.
- Attention Diversion occurs when corrupted token fragments attract disproportionate attention and redirect it from task-relevant evidence and answer cues.The diversion concentrates in middle and final transformer layers, where contextual representations are composed and mapped toward predictions.
- Length-matched controls and interventions support tokenizer fragmentation as the primary driver, while factorial results show corrupted content and attention allocation are coupled.Inference-time strategies that address only one channel therefore do not consistently recover performance.
2 Lexical Perturbations in LLM Reasoning
The paper defines lexical perturbations as meaning-preserving changes to how prompts are written and compares character-level corruption with filler-based length inflation. Its design separates tokenization disruption from added prompt length.
- Lexical perturbations alter a prompt’s surface form while preserving intended meaning, modeling typing errors, speech-to-text transcription, and noisy prompt construction.
- Keyboard noise replaces characters with adjacent QWERTY keys, while typoswap transposes adjacent characters within words.
- Filler insertion adds conversational disfluencies such as “um” or “you know” without corrupting existing lexical items.
- Perturbations are applied at rates r ∈{0.05, 0.10, 0.20, 0.30}, with r denoting the fraction of words perturbed.
- Character corruption can break learned subword merges into rare fragments, whereas filler mainly adds familiar tokens while preserving the original tokenization.
3 Experimental Setup
The study evaluates lexical robustness across diverse reasoning tasks and model families using repeated, controlled perturbation experiments. It combines accuracy with tokenization and attention diagnostics.
- The benchmarks cover BoolQ, PIQA, HellaSwag, and GSM8K, spanning reading comprehension, commonsense reasoning, contextual inference, and multi-step mathematics.
- The models include four 7B–9B open-weight instruction-tuned families plus GPT-4o and GPT-5.4 frontier configurations.
- Each dataset contributes 1,000 evaluation examples, with three random seeds used to capture subset and perturbation variance.
- Greedy-decoding accuracy is averaged across seeds, with severity r = 0.1 emphasized in the main text and tokenization and attention diagnostics recorded alongside predictions.
4 How Robust Are LLMs to Lexical Perturbations?
Character-level lexical corruption reliably degrades reasoning, increasingly so as severity rises, while filler insertion remains near clean performance. The strongest vulnerability appears in multi-step mathematical reasoning and persists across model and language settings.
- Character-level perturbations consistently reduce accuracy across models and tasks, while filler insertion has minimal effect.For Qwen3.5-9B on GSM8K, keyboard noise lowers accuracy from 0.845 to 0.706 and typoswap to 0.569, while filler yields 0.840.
- Average accuracy decreases monotonically from r = 0.05 to r = 0.30 for keyboard noise and typoswap, reaching below 0.40 versus 0.74 on clean prompts at r = 0.30.Filler insertion remains near the clean baseline across severities.
- GSM8K shows the steepest degradation, falling from 0.619 to 0.191 under keyboard noise and from 0.549 to 0.112 under typoswap as severity rises from r = 0.05 to r = 0.30.The pattern indicates greater sensitivity for precise numerical interpretation and multi-step reasoning.
- At 70B scale, GSM8K still drops 13 points under typoswap while BoolQ remains stable; Chinese CMath shows a 30-point keyboard-perturbation drop while filler remains harmless.Frontier models show the same pattern, and reasoning does not absorb it.
- GPT-5.4 Thinking has the largest keyboard degradation evaluated, −0.147, despite near-identical clean accuracy to Standard.
- The results establish a robust pattern: character-level perturbations degrade reasoning, whereas filler insertion largely does not.
5 What Causes Lexical Fragility in LLM Reasoning?
Lexical corruption fragments subword tokenization and diverts attention toward corrupted fragments, with the resulting damage concentrated in reasoning-relevant content and coupled attention-content failures. Controlled comparisons and interventions show that fragmentation, rather than prompt length alone, drives errors, while restoring both token content and attention recovers substantially more than either alone.
- Fragmentation and diversion: Character-level perturbations increase subword-token fragmentation and reduce token overlap, whereas filler insertion largely preserves the clean token sequence.Keyboard noise and typoswap sharply reduce overlap; filler insertion remains close to the clean baseline.
- Fragmentation and diversion: Corrupted fragments attract disproportionate attention mass, with diversion concentrated in middle and final transformer layers.Llama-3.1-8B shows a bimodal pattern, while Qwen3.5-9B shows stronger late-layer diversion across four benchmarks.
- Fragmentation, Not Length, Drives Failure: At matched token counts, filler insertion preserves near-clean accuracy, while keyboard noise produces 8× more fragmentation than neutral padding and lower accuracy.On GSM8K, filler insertion adds 63 tokens versus 62 for keyboard noise but maintains 0.686 versus 0.694 accuracy; fragmentation is 0.080 versus 0.010 for matched padding.
- Fragmentation, Not Length, Drives Failure: Fragmentation is the strongest univariate predictor of correct-to-wrong flips, with β= + 0.66, OR=1.93, p<10−81.Controlling for token-count change strengthens the association to β= + 1.04, OR=2.84, while token-count change loses independent positive predictive power.
- Attention and Token Content Are Coupled: Restoring clean attention over corrupted content can worsen accuracy, while restoring both attention and token content recovers 45– 71% of the gap in three models.Embedding replacement alone recovers 26% of the gap, whereas the factorial intervention finds a significantly positive interaction in every model.
- What Determines Task Resilience: Attention Diversion harms reasoning in proportion to the irreplaceability of displaced evidence, explaining why GSM8K loses more accuracy than BoolQ despite less diversion.Numeric tokens are especially consequential in GSM8K; a case study expands a prompt from 48 to 63 tokens, redirects 67% of attention to corrupted fragments, and changes answer 14 to 4.
6 Can Inference-Time Strategies Mitigate Lexical Fragility?
Inference-time strategies do not consistently recover lexical-fragility losses because they address either corrupted content or diverted attention, while the two channels remain coupled. Oracle restoration recovers the clean baseline, but deployable repair detects corruption more reliably than it repairs it.
- Recovery is defined as the percentage of each comparison’s clean–corrupted accuracy gap, with every mitigation paired to its own baseline and clean anchor.Accuracies are not comparable across mitigation blocks.
- Spell-checking recovers 30% of the gap on BoolQ under keyboard noise but turns negative under typoswap on both tasks.Noise-aware prompting is negative in all four conditions.
- Self-repair recovers half the GSM8K keyboard-noise gap but drops BoolQ from 0.73 to 0.13 by rewriting passage content.A 70B rewriter fully recovers the BoolQ keyboard condition while leaving the other three unchanged.
- Chain-of-thought raises clean GPT-4o GSM8K accuracy from 0.42 to 0.90, yet typoswap still reduces it to 0.70.GPT-5.4’s built-in reasoning mode amplifies degradation, while attention calibration and prompt restoration show no consistent improvement.
- Full recovery occurs on all four models when an oracle replaces corrupted fragments with the clean words’ exact token IDs.This is a diagnostic ceiling requiring access to the clean token sequence, not a deployable method.
- Tokenizer-level detection reaches F1 0.70–0.75 from fragmentation alone, but repair recovers original words only 67–70% of the time.Recovery is positive in 0 of 8 model–task cells and negative in 1, although clean inputs are not damaged.
- Numeric corruptions dominate GSM8K failures yet remain valid, nonfragmented numerals with no dictionary neighbor, leaving no recoverable trace.Thus, clean-token restoration recovers performance, whereas corrupted-input correction cannot reliably reconstruct the original quantity.
- No strategy recovers consistently because each leaves either token content or attention allocation corrupted.The mitigation pattern matches the factorial intervention’s coupled-channel account.
7 Related Work
Related work places remedies at different stages of the lexical-robustness pipeline. This paper’s factorial intervention adds that diverted attention cannot be manipulated independently of corrupted token content.
- Prior typo-attack research locates the remedy at the input by restoring surface form before the model reads it.The cited line of work includes character-level perturbation studies in neural NLP systems.
- This paper extends that perspective by showing that attention diversion is functionally bound to the corrupted content it addresses.The intervention organizes the remedy around the coupled disruption of token identity and attention allocation.
8 Conclusion
The paper finds that realistic character-level noise harms reasoning by fragmenting tokenization and redirecting attention, whereas length-matched filler remains benign. The coupled disruption explains why inference-time strategies fail to recover consistently, and the damage is irreversible without clean text.
- Character-level noise alters token identities and redirects attention onto fragments together, while length-matched filler insertion remains benign.The study covers four benchmarks, four open-weight model families, and frontier systems.
- Controlled experiments identify tokenizer fragmentation rather than prompt length as the driver of reasoning loss.Per-example regression and representation-level interventions support this account.
- The factorial intervention shows that corrupted content and diverted attention are coupled rather than sequential.This coupling explains why no inference-time strategy recovers consistently and why harm tracks what corruption affects rather than diversion magnitude.
- The damage is localized at the input representation yet irreversible without access to clean text.
Limitations
The study’s evidence is bounded by its language, perturbation, task, intervention, and model settings. In particular, the factorial coupling result is evaluated on GSM8K at one severity.
- Main analyses focus on English with a QWERTY layout, while broader language and input-method coverage remains open.The protocol is extended to Chinese CMath and OCR/ASR controls.
- The perturbations are synthetic approximations rather than samples from user data.
- All benchmarks are standard-length single-turn tasks, so long-context and agentic settings require separate control.
- The factorial intervention is run on GSM8K at a single severity because BoolQ’s narrow gap makes recovery ratios ill-conditioned.Therefore, the coupling result rests on one task.
- The interventions are diagnostic and do not identify the internal circuits responsible for failure.
- The study covers open-weight models in the 7B–70B range and frontier configurations, so other recipes or scales may differ.
Ethical Considerations
The work involved no human subjects, sensitive personal data, or proprietary datasets, and used publicly available datasets.
- The study used no human subjects, sensitive personal data, or proprietary datasets.
- All datasets were publicly available and commonly used in prior research.
- The authors report no safety, privacy, or fairness concerns from their methods and results.
AI Assistance Disclosure
AI assistance was limited to language polishing, while the study evaluated frontier models through prediction-level experiments and compared direct with chain-of-thought prompting.
- AI assistance was used only for language polishing, not to generate experimental results or analyses.
- Frontier models were evaluated only at the prediction level because internal tokenization states and attention weights were unavailable.
- The frontier-model protocol used 100 examples per dataset with keyboard noise and typoswap at r = 0.1.
- CoT raised clean GPT-4o GSM8K accuracy from 0.42 to 0.90 but left typoswap accuracy at 0.70.
- On the other three benchmarks, CoT provided no consistent advantage over direct prompting.
A.2 GPT-4o Results
Frontier and larger models retain lexical fragility across tasks and perturbation types. Stronger or extended reasoning does not consistently mitigate corruption, while tokenization disruption remains associated with failures.
- GPT-4o Results: GPT-4o’s keyboard-noise accuracy drops range from 4 to 14 points, while typoswap causes an 18-point GSM8K drop but minimal PIQA impact.
- GPT-5.4 Results: GPT-5.4 Standard has 0.908 average clean accuracy and a −0.093 average keyboard degradation, while Thinking degrades by −0.147.
- GPT-5.4 Results: GPT-5.4 Thinking shows its largest keyboard-noise drops on HellaSwag (−0.23) and BoolQ (−0.19), despite near-identical clean performance to Standard.
- Extended Settings: At 70B scale, GSM8K drops 13 points under typoswap while BoolQ remains stable; keyboard noise has near-zero effect at −0.01 on both tasks.
- Mechanistic Evidence: Layer-wise diversion is bimodal, peaking at middle layers 11–15 and final layers 30–32 across all four benchmarks.
- Extended Settings: On MATH algebra, typoswap produces the largest drop (−0.08), followed by keyboard noise (−0.06), while filler drops accuracy by −0.04.
- Mechanistic Evidence: Failed cases show higher diversion, lower token overlap, and greater token-count inflation than cases remaining correct.
H Per-Example Regression Analysis
Per-example analyses identify fragmentation as the strongest predictor of reasoning failures and test interventions targeting token content or attention. These interventions show that corrupted-token representations and attention allocation are functionally coupled.
- Analysis Setup: The regression aggregates four open-weight models across four benchmarks, four perturbation rates, and keyboard and typoswap conditions.
- Analysis Setup: The analysis uses clean-correct examples, defining failure as a correct-to-wrong flip after perturbation.
- Embedding Intervention: Replacing about 2.95 corrupted-prompt embeddings raises accuracy from 0.42 to 0.52, recovering 26% of the 0.38-point clean-accuracy gap.
- Attention Intervention: Suppressing attention to outlier tokens consistently worsens accuracy, with GSM8K dropping up to −0.31.
- Perturbation Comparison: Keyboard perturbation causes 2–3× more fragmentation than filler at matched severity, explaining their asymmetric accuracy effects.