PaperlayerView exact revision on arXiv

Source-linked AI summary

A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models

Zeru Shi, Zhenting Wang, Fan Yang, Qifan Wang, Ruixiang Tang

arXiv:2605.08504v2cs.CL

TL;DR

Massive activations arise and propagate in LLMs through mechanisms that prior work has not clearly explained end-to-end. This paper identifies the ME Layer and introduces hidden-state masking that consistently improves performance while partially attenuating attention sinks.

  • Problem

    Existing work lacks a clear end-to-end account of how massive activations emerge and connect to downstream functional effects in LLMs.

  • Method

    The paper identifies the ME Layer and masks dimensions associated with large RMSNorm weights to relax massive-activation token rigidity without changing architecture or training objectives.

  • Results

    The method consistently improves performance across multiple tasks and training settings while partially attenuating attention sinks.

  • Takeaways & Limitations

    Massive activations and attention sinks are linked to directionally rigid, low-dimensional hidden representations originating at the ME Layer.

  • Takeaways & Limitations

    The study does not investigate integrating the method into pre-training, leaving its effects on representation learning during large-scale pre-training unresolved.

Abstract

from arXiv · show

We investigate the origins of massive activations in large language models (LLMs) and identify a specific layer named the \textbf{Massive Emergence Layer (ME Layer)}, that is consistently observed across model families, where massive activations first emerge and subsequently propagate to deeper layers through residual connections. We show that, within the ME Layer both the RMSNorm and the FFN parameters jointly contribute to the emergence of massive activations. Once formed, the massive activation token representation remains largely invariant across layers, reducing the diversity of hidden representations passed to the attention module. Motivated by this limitation, we propose a simple and effective method to reduce the rigidity of the massive activation token. Our approach consistently improves LLM performance across multiple tasks, including instruction following and math reasoning, in both training free and fine tuning settings. Moreover, we show that our method mitigates attention sinks by selectively weakening their influence, elucidating their origin at the hidden state level and shedding new light on principled mitigation strategies.

1. Introduction

The paper identifies a single Massive Emergence Layer (ME Layer) where massive activations first arise and then propagate through residual connections. It attributes their emergence to RMSNorm and FFN properties, links their cross-layer invariance to reduced representation diversity and attention sinks, and proposes selective masking to mitigate these effects.

  • Massive activation emergence: Massive activations first emerge at a single ME Layer, where their values increase by several hundred times, then propagate through residual connections.The ME Layer is the specific layer at which massive activations are generated before appearing in subsequent layers.
  • Massive activation emergence: Within the ME Layer, massive activation emergence is jointly driven by the pre-FFN RMSNorm and FFN layer.The analysis identifies both components as contributors to the phenomenon.
  • Representation invariance: Massive activations remain highly stable and consistent across inputs and layers, reducing hidden-state representation diversity and making attention patterns more similar.Their shared direction biases how tokens interact when propagated into self-attention.
  • Mitigation method: The proposed method selectively masks attention-input dimensions associated with large RMSNorm weights to relax massive-activation-induced directional rigidity.These weights tend to amplify dominant directions in the hidden state.
  • Attention sinks: Attention sinks emerge immediately after the ME Layer, exhibit low-rank attention weights, and are partially attenuated by the proposed method.The paper connects attention-sink behavior to the low-rank properties of massive activations produced in the ME Layer.

2. Related Work

Prior work established large outlier features and massive activations in language-model hidden states, examining their quantification, function, causes, training emergence, and behavioral effects. Related research also identified attention sinks in LLMs and VLMs and developed attention-level mitigation strategies.

  • Outlier features and massive activations: Timkey & Van Schijndel (2021) identified extremely large activations in certain GPT-2 feature dimensions, prompting studies of hidden-state outlier features.Later work examined these features through quantification analysis and functional analysis.
  • Outlier features and massive activations: Recent studies reported unusually large-magnitude hidden states called massive activations and investigated their FFN-weight drivers, training emergence, and effects on model performance and behavior.Oh et al. (2025) linked massive activations to large FFN weights; Gallego-Feliciano et al. (2025) studied their emergence during training; He et al. (2024) examined their effects.
  • Attention sinks: Attention sinks are tokens that consistently receive disproportionately large self-attention weights, and prior work has observed them in both LLMs and VLMs.Gu et al. (2024) characterized sinks as noninformative key biases arising from softmax-induced coupling.
  • Attention sinks: This characterization motivated studies that mitigate attention sinks by modifying the attention mechanism.The supplied passage cites Ramapuram et al. (2024), Zuhri et al. (2025), and Bondarenko et al. as examples of this research direction.

3. Emergence of Massive Activations in a Single Transformer Layer

Massive activations emerge abruptly in a single ME Layer, where RMSNorm and FFN computations selectively amplify one token before the representation propagates through residual connections. Afterward, that token’s hidden state remains directionally stable across inputs and layers, reducing representational diversity for downstream attention.

  • 3.1 Origin of Massive Activations: Massive activations first emerge abruptly at the ME Layer rather than accumulating gradually, then persist through residual connections.In Qwen3-4B, activations remain comparable across tokens before the ME Layer, while the first token suddenly increases in magnitude there.
  • 3.1 Origin of Massive Activations: At the ME Layer, RMSNorm disproportionately amplifies the first token through concentration on dimensions with large scaling factors.Before layer 7, amplification is similar across tokens; at layer 7, the first token receives much larger output magnitude and shows concentrated, scaling-aligned output.
  • 3.1 Origin of Massive Activations: Only at the ME Layer do all three FFN modules simultaneously produce the strongest selective activation difference for the first token.This indicates substantially stronger concentration of the first token’s representation under FFN transformations than in other layers.
  • 3.1 Origin of Massive Activations: Unusually large and directionally aligned RMSNorm and FFN parameters selectively amplify the massive-activation token at the ME Layer.The FFN primarily generates and sustains massive activations, while the preceding RMSNorm regulates their scale.
  • 3.2 Stability of Massive Activations: Once formed, the massive activation’s hidden state remains input-invariant in direction and highly consistent across tasks, instances, and subsequent layers.This directional stability reduces representational diversity and constrains downstream self-attention.

4. Weight Guided Dimension Masking

WeMask selectively masks dominant hidden-state dimensions identified by large RMSNorm weights to restore directional diversity in attention without changing the transformer structure or adding computational cost. The mask is applied before attention from the ME Layer onward to reduce massive-activation rigidity.

  • Motivation: Massive activations remain nearly identical across inputs, restricting attention’s ability to adapt conditionally and reducing input-dependent flexibility.Their fixed hidden-state direction can act as a stable global reference but limits representation diversity.
  • Motivation: RMSNorm dimension-wise scaling primarily amplifies already dominant components when massive-activation states concentrate in a small subset of dimensions.This concentration helps explain why directional similarity persists after normalization.
  • Motivation: Directionally invariant massive-activation keys create stable attention similarity patterns that function as fixed reference points and reduce input sensitivity.Stable directions produce approximately fixed key positions in attention similarity space, yielding rigid representations.
  • WeMask method: WeMask selects top-k dimensions with large RMSNorm weights and suppresses the corresponding hidden-state dimensions before attention.The selection count k is determined by the mask rate multiplied by the hidden dimension.
  • WeMask method: The masking module is inserted before attention in each subsequent layer, beginning at the ME Layer, to reduce directional rigidity without structural changes or extra computation.The method is designed to restore directional diversity for effective attention computation.

5. Experiments

Experiments on Qwen3-4B evaluate WeMask in training-free, supervised fine-tuning, and reinforcement-learning settings across instruction, math-reasoning, and safety-alignment tasks. The method consistently improves performance, generalizes beyond instruction fine-tuning, and outperforms alternative masking strategies.

  • Experimental Setup: WeMask is evaluated on Qwen3-4B across instruction fine-tuning, math reasoning, and safety alignment in both training-free and training-time settings.Additional experiments on Llama-3.1-8B-Instruct and Qwen3-8B examine scaling across model families and parameter sizes.
  • Math Reasoning and Safety Alignment: Across math reasoning and safety alignment, incorporating WeMask consistently improves performance beyond instruction fine-tuning.The reported gains span different optimization objectives, training paradigms, and data distributions, including reasoning-oriented and safety-critical tasks.
  • Math Reasoning and Safety Alignment: On XSTest, WeMask addresses the overly conservative refusal behavior that standard SFT can induce, improving overall performance.The passage identifies standard SFT’s conservative refusals as a source of noticeable degradation on this benchmark.
  • Ablation Study: Alternative strategies that randomly mask fixed proportions or mask the largest activations substantially degrade performance, whereas WeMask consistently improves it.This ablation supports the effectiveness of the method’s masking strategy relative to both comparison approaches.
  • Weight-guided Masking in RL Training: WeMask also improves the performance of reinforcement-learning models trained with DPO for safety alignment and GRPO for math reasoning.The RL experiments train Qwen3-4B on HH-RLHF and GSM8K, respectively, and evaluate safety alignment on XSTest and AdvBench.

6. Discussion: Rethinking Attention Sink from a Representation Perspective

Attention sinks emerge after massive activations begin, originating in the ME Layer’s rigid first-token representation rather than the same layer’s FFN output. This representation collapse produces similar first-token keys and queries, linking massive activations to downstream attention-sink behavior.

  • Representation Perspective: The first token’s input-invariant direction and larger magnitude suggest a low-rank effect in hidden representations rather than attention weights.Prior work similarly described attention weights concentrated on a single token across multiple heads, reducing information-aggregation richness.
  • Attention Sink Mechanism: Attention sinks consistently appear in layers following the onset of massive activation.The ME Layer sink is not caused by that layer’s FFN output because multi-head attention precedes the FFN in the forward pass.
  • Attention Sink Mechanism: The ME Layer amplifies the first token’s magnitude and makes its representation highly consistent across inputs, collapsing hidden states into a low-dimensional subspace before attention.This hidden-representation collapse provides a representation-level perspective on attention sinks.
  • Representation Perspective: The collapsed first-token representation leads to highly similar keys and queries, suggesting that attention sinks are a downstream consequence of massive activation emergence.This connects the ME Layer’s hidden-state dynamics to the subsequent attention pattern.

7. Conclusion

The paper identifies the ME Layer as the point where massive activations emerge in LLMs. It shows that these activations produce consistent hidden-state patterns and reduced representational diversity, motivating a hidden-state intervention that relaxes this rigidity without changing architecture or training objectives.

  • The ME Layer is identified as the point of emergence for massive activations in large language models.
  • Massive activation tokens exhibit highly consistent hidden-state patterns across layers and diverse inputs.
  • This consistency reduces representational diversity and increases directional rigidity.
  • The proposed method relaxes excessive consistency by directly intervening on hidden-state representations without modifying model architecture or training objectives.

Impact Statement

The paper advances understanding of LLM internal mechanisms and improves performance through principled representation-level interventions, while identifying no work-specific ethical concerns or societal risks beyond general machine-learning progress.

  • Impact Statement: The work targets better understanding of LLM internal mechanisms and improved performance through principled representation-level interventions.The authors note that enhanced capabilities may influence downstream applications.
  • Impact Statement: The authors identify no ethical concerns or societal risks specific to this work beyond those generally associated with progress in machine learning research.

A. Limitation and Future Works … D. Stability of ME Layer

The paper identifies limitations in its focus on intermediate-layer activations and post-training evaluation, then details component ablations and task-dependent WeMask settings. It also examines ME Layer stability across diverse input categories using Qwen3-4B.

  • A. Limitation and Future Works: Final layers produce massive activations in the first token during the last two layers, indicating qualitatively distinct behavior from intermediate layers.The study does not yet establish the functional roles of these final layers.
  • A. Limitation and Future Works: The evaluation primarily studies post-training applications and does not investigate integrating the proposed method into pre-training.The authors leave open whether pre-training suppression would provide similar or stronger benefits without harming representations.
  • B. Compare the Role of RMSNorm and FFN: Removing the FFN still allows the massive-activation token to emerge in intermediate layers, supporting a joint role for RMSNorm and FFN.The ablations separately remove the RMSNorm preceding the FFN or the FFN itself to analyze formation and propagation across layers.
  • C. More Experiment Settings: During training, WeMask is applied to every layer after massive activation begins.This setting addresses propagation and repeated reinforcement of a directionally rigid representation through the residual stream.
  • C. More Experiment Settings: During evaluation, WeMask covers all subsequent layers for knowledge-generalization tasks but only the first massive-activation layer for mathematical reasoning and safety alignment.The distinction reflects differing task sensitivities and the functional differences between training and inference.
  • D. Stability of ME Layer: Qwen3-4B tests ME Layer emergence across commonsense question answering, mathematical problem solving, logical reasoning, and open-ended text continuation.The section frames ME Layer emergence as systematic and input-agnostic rather than tied to specific examples.
  • D. Stability of ME Layer: Figure 10 tracks the first token’s L2 norm across layers for different input instances.Each curve represents a distinct example.

E. Performance of Different Mask Methods

Training-free masking interventions were evaluated using benchmark-specific best mask ratios. Among the strategies, indiscriminate masking substantially degraded performance, whereas the proposed method consistently improved it across benchmarks.

  • Evaluation setup: The study evaluates masking strategies as training-free inference interventions, selecting the best-performing mask ratio for each benchmark.This setup examines how masking affects model performance without additional training.
  • Masking strategies: Random Mask randomly removes a fixed proportion of dimensions from the massive-activation token’s hidden state.
  • Masking strategies: Magnitude Mask removes the top-k dimensions with the largest activation magnitudes in the massive-activation token.The comparative results are reported in Table 5.
  • Results: Except for the proposed method, alternative masking strategies substantially degraded performance and often severely harmed reasoning ability, while the proposed method consistently improved benchmarks.These findings indicate that indiscriminately masking dimensions is harmful.

F. Performance of Other Models

WeMask was evaluated for generality by fine-tuning Llama-3.1-8B-Instruct and Qwen3-8B, then testing them across five benchmarks. Compared with the training-free variant, SFT-based WeMask showed more stable performance.

  • Generality across models: WeMask was fine-tuned on Llama-3.1-8B-Instruct and Qwen3-8B to evaluate its generality across different base models.The selected models served as base models for supervised fine-tuning.
  • Evaluation benchmarks: The resulting models were evaluated on MMLU, PIQA, ARC-C, OpenBookQA, and MathQA.These benchmarks cover general knowledge, reasoning, question answering, and mathematics.
  • Performance comparison: Compared with the training-free variant, SFT-based WeMask exhibited more stable performance.The comparison was reported in Table 6.

G. Compared with Other Methods Which Eliminating Attention Sinks · H. The Universality of ME Layer

Section G compares the proposed method with existing attention-sink removal approaches using gated attention and supervised fine-tuning, while Section H shows that the ME Layer occurs consistently across evaluated architectures and is positioned similarly within model families.

  • G. Compared with Other Methods Which Eliminating Attention Sinks: The proposed method is directly compared with existing attention-sink removal approaches using gated attention and supervised fine-tuning.The evaluation covers MMLU, PIQA, ARC-C, OpenBookQA, and an additional truncated benchmark entry.
  • G. Compared with Other Methods Which Eliminating Attention Sinks: Table 7 compares the proposed method with other attention-sink removal methods at a mask rate of 0.1.The supplied passage identifies the comparison setting but does not provide the table’s numerical results.
  • H. The Universality of ME Layer: The ME Layer is a ubiquitous phenomenon across architectures, with its position largely consistent within the same model family.Table 8 reports ME Layer indices and magnification compared with the previous layer.
  • H. The Universality of ME Layer: Both Qwen3-8B and Qwen3-4B-Instruct locate the ME Layer at layer 7.This example illustrates the reported within-family positional consistency.
  • H. The Universality of ME Layer: Within the Qwen3 family, Qwen3-8B and Qwen3-4B show ME Layer emergence at the same layer.The passage presents this as further evidence of consistency within a model family.
Loading 2605.08504v2…