Systematic Outliers in Large Language Models

• Existing outlier research mainly mitigates their effects algorithmically, leaving their causes, functional roles, and interrelationships insufficiently examined.Source: §1 · p. 1
• The paper categorizes LLM outliers into activation, weight, and attention outliers and studies their systematic relationships.Source: §1 · pp. 1–2
• The analysis links outlier formation to self-attention’s softmax operation and interprets them as implicit, context-aware scaling factors.Source: §1 · p. 2
• Structurally eliminating outliers accelerates convergence and enhances model compression.Source: §1 · p. 2

• Prior studies identify outliers in weights, activations, and attention scores, including activation outliers, LayerNorm weight outliers, and attention sinks.Source: §2 · p. 2
• Existing solutions often target task-specific mitigation such as quantization or pruning rather than a unified account of outlier relationships.Source: §2 · p. 2
• Outliers can degrade performance and compression because they amplify quantization errors, complicate pruning, and help preserve context in KV-cache compression.Source: §2 · p. 2
• The paper studies outlier formation within self-attention, identifies implicit context-aware scaling, and proposes structural solutions for convergence and compression efficiency.Source: §2 · p. 2

3 SYSTEMATIC OUTLIERS: DEFINITION, EXISTENCE, AND LOCALIZATION

• The framework defines outliers using thresholded deviations from mean absolute values and sets τ = 1000 empirically to isolate extreme deviations.Source: §3 · pp. 2–3
• Systematic outliers are observed across pretrained and fine-tuned LLMs, including abnormal activation, weight, and attention values at recurring critical positions.Source: §3 · p. 3

3.1 WHERE ARE ACTIVATION OUTLIERS LOCATED?

• Activation outliers in layer outputs persist from shallow through middle layers, while down-projection-input outliers occur mainly in shallow and deep layers.Source: §3.1 · p. 4
• Both activation-outlier types concentrate in fixed feature dimensions and associate primarily with start tokens and weak semantic tokens.Source: §3.1 · p. 4

3.2 WHERE ARE WEIGHT OUTLIERS LOCATED?

• Weight outliers are concentrated in MLP down-projection matrices, especially the second layer and the final two layers of LLaMA2-7B.Source: §3.2 · pp. 4–5

3.3 WHERE ARE ATTENTION OUTLIERS LOCATED?

• Attention outliers persist across layers, associate strongly with start and weak semantic tokens, and vary substantially across heads and layers.Source: §3.3 · p. 5

4.1 HOW ARE THESE OUTLIERS RELATED?

• Weight outliers align with activation outliers in feature dimensions, while over 95% of activation outliers overlap with attention outliers in sequence dimensions.The overlaps are concentrated primarily in start and weak semantic tokens.Source: Table 1 · p. 5, §4.1 · p. 6

4.2 THE LIFECYCLE OF SYSTEMATIC OUTLIERS

• Weight outliers drive extreme activation responses that can reach up to a thousand times the average magnitude.In LLaMA2-7B’s second MLP layer, SiLU and GLU amplify responses, while down-projection outliers dominate selected residual dimensions.Source: §4.2 · p. 6, Figure 7 · p. 6
• Activation outliers align query and key dimensions, increasing dot products and assigning disproportionately high attention weights to outlier tokens.This propagates their influence through attention and across layers.Source: Figure 8 · p. 6, §4.2 · p. 7
• Outlier tokens attract substantial attention but have comparatively small value magnitudes, suggesting an anchor-like role that affects attention outputs indirectly.The query-key alignment and reduced value magnitude pattern is consistent across most heads and layers.Source: §4.2 · p. 7
• Outliers gradually vanish in final layers through cancellation by opposite-sign values, reducing activation and attention outliers progressively.Some final-layer heads show no outliers at all.Source: §4.2 · p. 7, Figure 9 · p. 7

5.1 HYPOTHESES ON THE ROLE OF SYSTEMATIC OUTLIERS

• The tested hypotheses distinguish fixed biases, context-aware biases, and context-aware scaling factors as separate attention modifications.The five formulations include default attention, fixed bias, context-aware bias, attention bias, and explicit context-aware scaling; Sc(x) is input-dependent.Source: §5.1 · pp. 7–8, §5.2 · p. 8

5.2 EMPIRICAL VALIDATION OF SYSTEMATIC OUTLIERS HYPOTHESES

• Context-aware scaling factors, rather than fixed or context-aware bias alone, are identified as the primary factor preventing systematic outliers.Attention bias and explicit context-aware scaling prevent outlier formation, while fixed and context-aware bias alone do not.Source: §5.2 · pp. 8–9, Figure 10 · p. 9

5.3 FURTHER ANALYSIS OF SYSTEMATIC OUTLIERS

• Softmax attention drives systematic outliers by forcing attention scores to sum to one, requiring large dynamic ranges even when tokens need minimal contextual updates.Layer Normalization compresses this range, prompting preceding MLPs to generate high-magnitude activations.Source: §5.3 · p. 9
• Context-aware scaling reduces compression sensitivity on WikiText2, lowering quantization PPL from 93.44 to 29.22 and pruning PPL from 7235.68 to 39.47.The parameter count rises from 123.59M to 123.70M, under a 0.1% increase.Source: §5.3 · p. 10, Table 3 · p. 10
• Context-aware scaling accelerates early training convergence, while final validation losses remain comparable to default attention.The paper attributes this to reduced reliance on extreme outliers and a smoother optimization landscape.Source: §5.3 · p. 10, Figure 12 · p. 10

6 CONCLUSION

• The paper systematically categorizes activation, weight, and attention outliers and analyzes their distribution, formation, roles, and interconnections.Source: §6 · p. 10, A ADDITIONAL RESULTS ON SYSTEMATIC… · p. 13
• Softmax-driven outliers act as implicit context-aware scaling factors that dynamically adjust attention distributions across diverse contextual demands.Source: §6 · p. 10
• Explicit context-aware scaling improves model convergence and compression efficiency while reducing unnecessary attention allocation.Source: §6 · p. 10

A.1 PRETRAINED LLMS

• Systematic outliers consistently appear across all evaluated pretrained models, including Phi-2, Mistral-7B, LLaMA2, LLaMA3, OPT-6.7B, MPT-7B, and Falcon-7B.Source: §A.1 · p. 13
• MLP architecture and activation function affect outlier distributions, with OPT-6.7B showing more dispersed patterns than LLaMA2-13B and Mistral-7B.Source: §A.1 · p. 13
• In deeper layers of LLaMA3 and Falcon-7B, down-projection activation outliers are more pronounced.The result suggests that architectural changes can amplify outlier formation in specific layers.Source: §A.1 · p. 13
• The additional results establish systematic outliers as pervasive across diverse model families and architectures.Source: §A.1 · p. 13

A.2 FINE-TUNED LLMS

• Outlier patterns in Mistral-7B and LLaMA2-7B remain consistent after fine-tuning into their instruction-tuned counterparts.The reported comparisons show minimal changes in outlier locations and distributions.Source: §A.2 · p. 13, Figure 13 · p. 14, Figure 14 · p. 14
• Instruction fine-tuning does not significantly alter the structural concentration of systematic outliers across spatial or feature dimensions.Some layers show minor magnitude or attention-score variations, including reduced attention-outlier intensity in Mistral-7B Instruct.Source: §A.2 · p. 13
• Their persistence across pretrained and fine-tuned models indicates a structural origin rooted in the Transformer architecture rather than task-specific fine-tuning artifacts.The authors characterize this pattern as intrinsic and robust across training paradigms.Source: §A.2 · p. 13

B DETAILED EXPERIMENTAL SETTINGS

• The experimental-settings section describes methodologies and configurations for systematic-outlier analyses in LLMs, with examples covering OPT-6.7B, MPT-7B, Falcon-7B, and LLaMA2-7B-Chat.It states that the detailed setups support reproducibility and clarity.Source: B DETAILED EXPERIMENTAL SETTINGS · p. 14, Figure 17 · p. 15, Figure 18 · p. 15

B.1 POSITION ANALYSIS SETTINGS

• Position analysis examines activation, weight, and attention outliers across layers, sequences, feature dimensions, modules, and attention heads.Activation analyses use 100 RedPajama sequences of length 2,048, while weight and attention statistics track concentration and persistence across layers.Source: §B.1 · pp. 15–16

B.2 CONSISTENCY ANALYSIS SETTINGS

• Consistency analysis quantifies activation–attention alignment across 100 RedPajama samples of length 2,048, analyzing attention separately for each head.Overlap is defined over sequence indices and averaged across samples and heads, with results summarized in Table 1.Source: §B.2 · pp. 16–17

B.3 GPT-2 ATTENTION VARIANT TRAINING SETTINGS

• GPT-2 attention variants use NanoGPT defaults, five model variants, 50,000 training iterations, approximately 2 billion tokens, and specified attention-bias initialization.The attention-bias variant initializes k′ and v′ from N(0, 0.02I).Source: §B.3 · p. 17

B.4 MODEL COMPRESSION EXPERIMENT SETTINGS

• Compression experiments apply 8-bit AbsMax quantization and 50% unstructured magnitude pruning, evaluating GPT-2 variants on WikiText2 using perplexity.The comparison tests whether context-aware scaling factors preserve performance under aggressive compression.Source: §B.4 · p. 17

C HOW SOFTMAX CAUSES SYSTEMATIC OUTLIERS IN TRANSFORMER MODELS

• Softmax-induced concentration satisfies near-zero updates for weakly semantic tokens but expands attention-score ranges and produces extreme values.Dominant attention weights grow exponentially while other weights become negligible, amplifying gradients and potentially destabilizing training.Source: C HOW SOFTMAX CAUSES SYSTEMATIC… · p. 17, §C.1 · p. 18, §C.2 · pp. 18–19
• Systematic outliers predominantly emerge in deeper layers, where semantic differentiation sharpens attention, while early layers maintain more balanced distributions.Outliers also concentrate at particular tokens and feature dimensions, limiting their spatial extent but intensifying local anomalies.Source: C HOW SOFTMAX CAUSES SYSTEMATIC… · p. 18

C.1 NECESSITY OF ZERO-UPDATE IN MULTI-HEAD ATTENTION (MHA)

• Near-zero updates require softmax to concentrate probability on a few keys, forcing large query–key score disparities and selective gradient updates.This mechanism is motivated by tokens such as initial or weakly semantic tokens that require minimal contextual changes.Source: §C.1 · p. 18

C.2 SOFTMAX-INDUCED DYNAMIC RANGE EXPANSION

• Softmax assigns nearly all attention to a dominant key as its score grows, while the remaining weights approach zero, creating highly imbalanced attention distributions.The resulting gradient amplification affects query and key vectors and contributes to subsequent outlier formation.Source: §C.2 · p. 19

C.3 PROPAGATION OF SYSTEMATIC OUTLIERS

• Shared MHA weights and nonlinear MLP transformations propagate softmax-induced anomalies into concentrated activation and weight outliers.Activation outliers localize in sequence positions and feature dimensions, while large gradients concentrate weights in WK, WV, and Wdown.Source: §C.3 · pp. 19–20

C.4 LOCALIZATION OF SYSTEMATIC OUTLIERS

• Outliers concentrate at initial or weak-semantic tokens and in a small subset of feature channels, preserving stability across most representations.Initial tokens aggregate global information, while punctuation-like tokens receive high attention; fixed rows or columns localize extreme feature values.Source: §C.4 · p. 20

C.5 CONCLUSION

• Softmax attention drives systematic outliers, which propagate through layers and are amplified by shared projections and nonlinear MLP activations.The resulting activation and weight outliers reflect the mechanism’s effort to satisfy near-zero updates for certain tokens.Source: §C.5 · p. 21

D.1 ABSENCE OF OUTLIERS IN SIGMOID ATTENTION

• Sigmoid attention eliminates systematic outliers in GPT-2, supporting the hypothesis that softmax normalization is a primary cause of their formation.Sigmoid attention can produce near-zero attention values without softmax’s extreme dynamic range, although some inputs may experience vanishing gradients.Source: §D.1 · p. 21, Figure 23 · p. 21

D.2 EXPLICIT CONTEXT-AWARE SCALING FACTOR IN TINY-LLAMA

• Explicit context-aware scaling completely eliminates systematic outliers in TinyLLaMA-120M while achieving the zero-update objective without generating large gradients.The same pattern observed in GPT-2 across a different Transformer architecture supports the approach’s robustness.Source: §D.2 · pp. 21–22, Figure 24 · p. 22

D.3 IMPACT OF SEQUENCE LENGTH ON OUTLIERS

• Outliers persist across sequence lengths, while their specific positions can shift as the sequence changes.The analysis reports no evidence that particular sequence lengths are more prone to generating outliers.Source: §D.3 · p. 22
• Extending sequences can add or relocate punctuation-token outliers, while the initial token remains a consistent outlier location.In the examples, “Summer” remains an outlier and punctuation positions change with sequence content.Source: §D.3 · p. 22