PaperlayerView exact revision on arXiv

Source-linked AI summary

Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding

Mingyu Jin, Kai Mei, Wujiang Xu, Mingjie Sun, Ruixiang Tang, Mengnan Du, Zirui Liu, Yongfeng Zhang

arXiv:2502.01563v4cs.CL

TL;DR

The paper addresses limited understanding of how attention representations support contextual knowledge understanding. It systematically analyzes massive values in Q, K, and V, their relation to contextual and parametric knowledge, quantization, and RoPE. It finds that concentrated Q and K massive values are critical for contextual tasks, are less influential for parametric retrieval, and are induced by RoPE from early layers.

  • Problem

    Existing understanding of LLM representations and their behavioral roles is limited, including why massive values appear in Q and K but not V and how they affect contextual knowledge understanding.

  • Method

    The paper systematically analyzes massive values across attention components, evaluates their functional role across datasets and quantization methods, and traces their origin through RoPE and inference representations.

  • Results

    Concentrated massive values occur in Q and K rather than V, support contextual knowledge understanding more than parametric retrieval, and are caused by RoPE from the initial layers.

  • Takeaways & Limitations

    Preserving massive values is important for contextual tasks and informs quantization, model optimization, and interpretation of RoPE-induced Q and K patterns.

  • Takeaways & Limitations

    The inference-workflow analysis omits the attention output layer and other parts of the inference workflow for ease of illustration.

Abstract

from arXiv · show

Large language models (LLMs) have achieved remarkable success in contextual knowledge understanding. In this paper, we show that these concentrated massive values consistently emerge in specific regions of attention queries (Q) and keys (K) while not having such patterns in values (V) in various modern transformer-based LLMs (Q, K, and V mean the representations output by the query, key, and value layers respectively). Through extensive experiments, we further demonstrate that these massive values play a critical role in interpreting contextual knowledge (knowledge obtained from the current context window) rather than in retrieving parametric knowledge stored within the model's parameters. Our further investigation of quantization strategies reveals that ignoring these massive values leads to a pronounced drop in performance on tasks requiring rich contextual understanding, aligning with our analysis. Finally, we trace the emergence of concentrated massive values and find that such concentration is caused by Rotary Positional Encoding (RoPE), which has appeared since the first layers. These findings shed new light on how Q and K operate in LLMs and offer practical insights for model design and optimization. The Code is Available at https://github.com/MingyuJ666/Rope_with_LLM.

1. Introduction

The paper investigates concentrated massive values in transformer attention and finds that they occur in specific Q and K regions, support contextual knowledge understanding, and arise through RoPE.

  • Motivation: The study motivates analyzing LLM internal mechanisms because limited understanding of them challenges efforts to improve reliability and performance.The paper frames massive values as representations with magnitudes significantly larger than typical values.
  • Findings: Massive values concentrate in specific regions of attention queries and keys, but this pattern is absent in value computations.The concentration is observed across heads at similar dimensional indices and is absent in models without RoPE.
  • Functional role: Experiments across ten datasets show that Q and K massive values are crucial for contextual knowledge processing but have subtle impact on parametric knowledge retrieval.Contextual tasks include passkey retrieval, IMDB review analysis, and mathematical reasoning, whereas World-Cities represents parametric knowledge retrieval.
  • Quantization: Quantization methods that explicitly address massive values better preserve contextual understanding capabilities.The comparison evaluates three representative quantization methods.
  • Origin: RoPE causes the concentrated massive values, which emerge as early as the initial layers.The paper uses causal and temporal analysis to connect the concentration to RoPE mechanisms.

2. Preliminary

This section introduces positional encoding and the LLM inference workflow, focusing on RoPE's application to queries and keys and the processing of cached key-value tensors during generation.

  • Position Encoding: RoPE incorporates positional information into query and key vectors so their inner product inherently contains relative positional information.The method applies vector transformations to the query and key representations.
  • Position Encoding: The RoPE formulation uses projection functions for queries and keys, with a rotation parameter defined by θ_d = b^-2d/|D| and b = 10,000.The projection matrices W_q and W_k map the embedding space to the head dimension.
  • Position Encoding: The resulting query-key dot product depends on the relative token distance m − n.The paper describes this dependence after retaining the real part of the complex inner product.
  • LLM Inference Workflow: LLM inference has prefilling and decoding phases, with prefilling establishing the context representation and decoding generating tokens autoregressively.The decode phase uses the current token embedding together with cached key and value tensors.
  • LLM Inference Workflow: During prefilling, key and value tensors are computed from the input tensor and cached for efficient decoding.The key and value layer weights produce the tensors stored in memory.
  • LLM Inference Workflow: During decoding, the KV cache is updated from the current token's key and value outputs before calculating the attention output.The query-layer weight matrix is used in the attention computation; the output layer and other workflow components are omitted for illustration.

3. The Effect of Massive Values in Knowledge Understanding

The study separates contextual knowledge understanding from parametric knowledge retrieval and finds that concentrated massive values strongly support the former while only modestly affecting the latter.

  • Massive Value Measurement: Massive values are measured from Q and K by reducing sequence-wise representations to head-by-dimension L2 norms.An element is designated massive using a threshold λ, empirically set to 5.
  • Experimental Setup: The experiments disrupt massive or non-massive values during prefilling, replacing selected values with averages or other substitutions.Targeting prefilling isolates context processing from the model’s fundamental generation capabilities.
  • Parametric Knowledge Retrieval: Massive-value disruption preserves relatively strong parametric retrieval, with Cities scores remaining between 76% and 88%, a 15–20% degradation.Sports, Arts, Technology, and Celebrity retrieval show varying but generally moderate impact.
  • Contextual Knowledge Understanding: Massive-value disruption severely harms contextual understanding, while non-massive manipulation has little effect across GSM8K, AQUA, and IMDB.Replacing non-massive values with averages across n=1 to 20 maintains relatively stable benchmark performance.
  • Disruption Robustness: In GSM8K, massive-value disruption reduces accuracy to single digits—4.00% with mean, 1.60% with zero, and 2.73% with minimum substitution—versus 76–77% after non-massive manipulation.The consistent effect across substitution methods indicates robustness to the disruption strategy.
  • Quantization: Quantization methods that preserve massive values maintain contextual performance, whereas GPTQ causes significant degradation on contextual knowledge understanding.AWQ protects important weights, while SmoothQuant redistributes activation magnitudes through an equivalent transformation.

4. Causal Mechanisms and Temporal Analysis of Concentrated Massive Values

The paper links concentrated massive values to RoPE’s selective, frequency-based rotations in Q and K, while layerwise evidence suggests training progressively forms the observed concentration.

  • RoPE Mechanism: RoPE divides dimensions into paired rotations whose frequencies decay exponentially across dimensional indices.The resulting spectrum spans high-frequency and low-frequency rotations.
  • Functional Implication: Disrupting massive values sharply impairs contextual understanding while preserving basic parametric retrieval, supporting a semantic-comprehension role beyond positional encoding.The reported IMDB accuracy drops from 94% to single digits under disruption.
  • RoPE Mechanism: RoPE applies positional transformations to Q and K so their inner product captures relative positional information, but it does not transform V.This selective design directly predicts concentration in Q and K and its absence in V.
  • Cross-Architecture Evidence: Experiments find concentrated massive values in RoPE-equipped models but not in OPT, GPT-2, or Jamba, which lack RoPE.OPT’s embedding maps appear disorganized compared with the consistent concentration observed in models using RoPE.
  • Localization: The concentration appears in low-frequency regions of Q and K and is absent in V, consistent with RoPE’s selective positional application.The paper frames this as a consequence of how RoPE is implemented.
  • Localization: RoPE’s paired rotations produce mirrored massive-value clusters across the two halves of the embedding dimensions.The observed two-cluster structure matches the hypothesized dimensional pairing.
  • Temporal Analysis: Massive values appear from the first layer in Llama and Gemma, but their layerwise patterns change gradually during processing.The authors infer that training progressively forms the concentration rather than RoPE alone causing it, because RoPE is not trainable.

5. Related Work

Related work studies outlier features, quantization, and positional encoding, while this paper examines massive values across attention depth and multiple heads.

  • Outlier Features: Prior studies describe massive or outlier activation features in LLM representations and identify characteristic attention patterns involving special or starting tokens.These studies motivate analysis of unusually large activation values.
  • Quantization: LLM quantization research includes W8A8 activation-and-weight quantization and low-bit weight-only methods such as W4A16.Existing approaches also suppress or protect outlier features to preserve model behavior.
  • Positioning: Unlike prior quantization analyses centered on individual attention heads, this work examines massive values across the depth dimension of multiple attention heads.The authors present this scope as a direction for future quantization approaches.
  • Positional Encoding: RoPE is widely used in modern LLMs, and increasing its θ parameter has been studied for reducing attention decay and improving long-context learning.The paper situates its RoPE analysis within broader work on positional encoding and context length.

6. Conclusion

The study finds that massive values in Q and K are critical for contextual knowledge understanding but have limited influence on parametric knowledge retrieval. It attributes these concentrated patterns to RoPE and highlights their implications for model optimization.

  • Massive values are critical for contextual knowledge understanding tasks but have limited influence on parametric knowledge retrieval.
  • RoPE induces massive value stripes exclusively in Q and K, particularly in low-frequency channel dimensions.
  • Preserving massive values is important for maintaining performance in reasoning and context-dependent tasks.
  • These findings inform model design and optimization, including quantization and parameter-efficient finetuning.

Impact Statement

The study advances understanding of massive values in attention mechanisms and their relationship with contextual knowledge understanding. Its findings may inform future model design, optimization, quantization, and parameter-efficient finetuning.

  • The study systematically investigates massive values in attention mechanisms and their relationship with contextual knowledge understanding.
  • The findings provide insights for future model design and optimization, particularly model quantization and parameter-efficient finetuning.
  • The relationship between massive values and contextual knowledge understanding could inform more efficient model compression.

A.1. Synthesizing Parametric Knowledge Retrieval Dataset

This appendix describes datasets and tasks used to evaluate parametric knowledge retrieval, passkey retrieval, and logical contextual knowledge understanding. It also documents dataset verification and task difficulty settings.

  • Parametric Knowledge Retrieval Dataset: The parametric knowledge retrieval dataset covers Sports, Arts, Technology, and Celebrity factual questions generated and verified through automated and manual stages.
  • Passkey Retrieval Task: Passkey retrieval tests locating and using a hidden passkey at varying document depths and context lengths without requiring text-content understanding.
  • Passkey Retrieval Task: 128-, 256-, and 1024-token passkey tasks form a hierarchy from simple to difficult, and removing all massive values can prevent responses at the 128-token level.
  • Inequality Relation Problems: The inequality dataset assesses logical contextual knowledge understanding through two premises and a query about the relation between variables.

B. More Visualization Results for LLMs without RoPE.

Visualization results for models without RoPE show more uniform attention-related values and no concentrated massive values. The appendix compares OPT and Jamba architectures, both described as not using RoPE.

  • OPT: OPT models without RoPE do not exhibit concentrated massive values in the visualized Q, K, and V representations.
  • OPT: OPT-350M shows a uniform distribution of values across tokens, layers, and selected heads.
  • OPT: The absence of concentrated massive values in OPT is associated with more balanced handling of input tokens and smoother attention scores.
  • Jamba: Jamba uses a hybrid Transformer-Mamba mixture-of-experts architecture and does not use RoPE in its positional encoding.

C. More Visualization Results for LLMs with RoPE

Across RoPE-equipped language and multimodal models, visualizations show concentrated massive values primarily in Q and K, often as bright high-frequency regions, while patterns vary across heads and layers.

  • Visualization scope: The visualizations cover K, Q, and V embeddings, three-dimensional head/layer maps, prompts, and inference outputs across Llama, LLaVA, Qwen, Mistral, and related models.
  • Llama models: Llama3-8B visualizations place massive values in high-frequency Q and K regions, appearing as bright lines across dimensions.These dimensions are described as heavily weighted during attention computation.
  • LLaVA: LLaVA shows a similar consistent bright line in high-frequency Q and K embedding regions, indicating concentration along specific dimensions across heads.
  • Qwen2-VL: Qwen2-VL uses M-RoPE, decomposing positional information into time, height, and width components; for text, this is functionally equivalent to 1D-RoPE.
  • Qwen and Mistral: Qwen2.5-7B and Mistral visualizations likewise show localized peaks or bright regions in Q and K, with intensities varying across heads and token ranges.

D.1. Perplexity Result

Perplexity remains relatively stable when non-massive values are disrupted but rises sharply when massive values are disrupted, indicating that massive values are important for language modeling performance.

  • Perplexity evaluation: Perplexity changes are reported across AQUA, IMDB, and GSM8K for Llama3-9B, Gemma2-9B, and Qwen2.5-7B under multiple disruption conditions.
  • Metric definition: Perplexity is defined as an exponential transformation of the average negative log-probability assigned to the sequence tokens.
  • Non-massive values: Non-massive-value disruption causes only slight perplexity increases across models and datasets.For Llama3-8B, PPL rises from 3.21 to 3.30 on AQUA and from 8.38 to 8.94 on IMDB; Qwen2.5-7B rises from 2.95 to 3.12 on AQUA.
  • Massive values: Massive-value disruption causes a dramatic perplexity increase and severely degrades language-modeling ability.For Llama3-8B, AQUA PPL rises from 3.21 to 10.34 after massive values are disrupted.

E. Experiment Details

The experiments use RoPE-equipped and additional transformer models, multiple contextual and parametric-knowledge datasets, and targeted prefilling-stage disruption of massive and non-massive values.

  • Models: The main experiments use Llama-3-8B-Instruct, Gemma2-9B-it, and Qwen2.5-7B-Instruct, with additional models evaluated in the appendix.
  • Datasets: Parametric-knowledge evaluation uses Cities and synthetic Sports, Arts, Technology, and Celebrity datasets.
  • Datasets: Contextual-knowledge evaluation includes GSM8K, AQUA, IMDB, and synthetic passkey-retrieval datasets.
  • Experimental materials: The study includes prompts and examples for GSM8K, AQUA, and IMDB, alongside figures documenting inference prompts and results.

F. More Experiments to Support the Conclusion.

Additional experiments show that preserving concentrated massive values is important for contextual reasoning in RoPE-based models, while non-RoPE models exhibit different disruption behavior. The experiments also examine when these patterns emerge and why readability requires metrics beyond perplexity.

  • RoPE contributes to Concentrated Massive Values: Qwen2.5-7B accuracy fell from 0.77% to 0.03% when massive values were destroyed in the Greater-Less Experiment.Mistral-7B similarly dropped from 0.59% to 0.12%.
  • Disrupting Massive Values in LLMs without RoPE: OPT-2.7B and Jamba-1.5-mini both degraded on in-context and parametric retrieval after massive-value disruption without RoPE.OPT-2.7B decreased 60% on GSM8k and 21% on Cities; Jamba-1.5-mini decreased 31% and 22%, respectively.
  • Experiment for Other Metrics: Perplexity alone was considered insufficient because a score as low as 2.99 could accompany endlessly repeated, incomprehensible text.The authors therefore combined human judgment and GPT-4 evaluations to introduce Readability Rate.
  • Explore when Concentrated Massive Values Appear: Concentrated massive-value patterns appeared as early as the first layer in Qwen2.5-7B, Llama3-8B, Llama2-7B, and Mistral-7B.The embedding-vector observations are reported in Figure 42.
  • Disruption Choices Exert Subtle Influences in Gemma: Gemma-9B performance remained remarkably stable across Mean, Zero, Minimum, and Maximum handling methods, especially for non-massive values.Passkey Retrieval remained perfect for non-massive values across configurations.
  • Controlled Experiment in Non Massive Value Region Disrupted in Gemma: Replacing 1–20 minimum values in Qwen2.5-7B left GSM8K, IMDB, and AQUA performance notably stable.The reported ranges were 75–85% for GSM8K, above 90% for IMDB, and 51–57% for AQUA.
Loading 2502.01563v4…