PaperlayerView exact revision on arXiv

Source-linked AI summary

Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Tasks

Yuval Ran-Milo

arXiv:2603.11487v5cs.LG

TL;DR

The paper asks whether attention sinks are optimization artifacts or functionally necessary in softmax Transformers. It formalizes the question with a trigger-conditional averaging task, proves necessity results, and shows that ReLU attention avoids sinks while experiments support the theory. The results suggest that changing how default off-states are represented is the relevant lever when sinks are undesirable.

  • Problem

    The paper investigates whether the widespread attention-sink behavior is merely an optimization artifact or can be functionally necessary in softmax attention.

  • Method

    The paper introduces a trigger-conditional task that outputs the mean of preceding tokens at a trigger and zero elsewhere, then analyzes softmax and ReLU attention theoretically and experimentally.

  • Results

    Softmax self-attention necessarily forms sinks for this task, while ReLU attention solves it without sinks; experiments support these findings in single-layer and deeper models.

  • Takeaways & Limitations

    When sinks are undesirable, non-normalized attention, explicit gating, or related mechanisms can represent an off-state without allocating probability mass.

  • Takeaways & Limitations

    The multi-layer theorem guarantees a sink in at least one layer at some non-trigger position but does not identify which layer, and experiments show sinks do not occur in all heads or layers.

Abstract

from arXiv · show

Transformers often display an attention sink: probability mass concentrates on a fixed, content-agnostic position. Are sinks a byproduct of the optimization/training regime? Or are they sometimes functionally necessary in softmax Transformers? We prove that, in some settings, it is the latter: computing a simple trigger-conditional behavior necessarily induces a sink in softmax self-attention models. Our results formalize a familiar intuition: normalization over a probability simplex must force attention to collapse onto a stable anchor to realize a default state (e.g., when the model needs to ignore the input). We instantiate this with a concrete task: when a designated trigger token appears, the model must return the average of all preceding token representations, and otherwise output zero, a task which mirrors the functionality of attention heads in the wild (Barbero et al., 2025; Guo et al., 2024). We also prove that non-normalized ReLU attention can solve the same task without any sink, confirming that the normalization constraint is the fundamental driver of sink behavior. Experiments validate our predictions and demonstrate they extend beyond the theoretically analyzed setting: softmax models develop strong sinks while ReLU attention eliminates them in both single-head and multi-head variants.

1 Introduction

The paper asks whether attention sinks are merely optimization artifacts or can be functionally necessary, and studies this through a trigger-conditional task in softmax attention. It proves sink necessity for this task, identifies normalization as the driver, and supports the results experimentally.

  • 1 Introduction: Attention sinks appear broadly across model scales, positional schemes, and modalities, suggesting a pervasive pattern rather than a peculiarity of one training regime.They concentrate attention on an early, largely content-insensitive position.
  • 1 Introduction: The paper introduces a task requiring the mean of preceding tokens at a trigger and a zero output elsewhere, modeling observed attention-head behavior.The formulation captures heads that aggregate context when triggered and remain dormant otherwise.
  • 1 Introduction: The paper limits its necessity claim to softmax attention rather than all architectures, noting that gated attention and Mamba-based models need not exhibit sinks.Empirically, sinks also do not form in all positions and layers.
  • 1 Introduction: Single-layer softmax models with vanishing error must place nearly all attention on BOS at every non-trigger position.This establishes a fixed sink as a necessity rather than a training preference.
  • 1 Introduction: Multi-layer softmax models must exhibit sink behavior in at least one layer at some non-trigger position.The theorem is existential and does not require sinks at every layer or position.
  • 1 Introduction: ReLU attention can solve the same task without sink formation, showing that softmax normalization is the primary driver of the behavior.Experiments likewise find sinks in softmax models and their elimination after replacing softmax with ReLU attention.

2 Sinks Empirically Enable No-Op Behaviors in Real Models

Prior empirical work shows attention heads switching between active, content-dependent computation and dormant sink behavior. These observations motivate the paper’s theoretical account of sinks as a mechanism for controlled information mixing and default no-op states.

  • 2 Sinks Empirically Enable No-Op Behaviors in Real Models: Barbero et al. report an apostrophe head in Gemma 7B that fires on apostrophe triggers and otherwise attends to BOS.The two modes correspond to triggered computation and a default no-operation.
  • 2 Sinks Empirically Enable No-Op Behaviors in Real Models: Guo et al. document an active–dormant head in Llama 2–7B, with diverse patterns on code-like inputs and a position-0 sink on text-like inputs.The contrast illustrates content-dependent activation versus dormant behavior.
  • 2 Sinks Empirically Enable No-Op Behaviors in Real Models: Replacing softmax with ReLU attention eliminates the reported sink behavior, consistent with the paper’s theorem that non-softmax attention can avoid sinks.This empirical observation links the prior work to the paper’s normalization-based explanation.
  • 2 Sinks Empirically Enable No-Op Behaviors in Real Models: The prior studies emphasize different explanations: controlled information mixing and training dynamics, whereas this paper analyzes functional necessity through expressiveness.The paper’s perspective complements rather than replaces those empirical accounts.

3 Theory and Results

The paper studies a trigger-conditional task that averages preceding content only at a trigger and outputs zero elsewhere, then proves contrasting sink requirements for softmax and ReLU attention. Softmax models require BOS sinks under low loss, whereas ReLU attention can solve the task without sinks.

  • Task Definition: The synthetic task outputs the mean of preceding non-BOS tokens at the trigger and the zero vector at all other positions.Inputs include BOS, trigger, non-trigger, and continuous content indicators.
  • Scope: The analysis likely extends to related trigger-conditional retrieval tasks, but formally characterizing the full class of sink-requiring tasks remains future work.The paper specifically mentions key-query retrieval as a related example.
  • Main Results: Theorem 1 proves that low-loss single-layer softmax models must place attention at least 1−ε on BOS at every non-trigger position.The result holds with probability at least 1−δ under the stated sequence, dimension, and content-distribution conditions.
  • Main Results: Theorem 2 proves that multi-layer softmax models must exhibit BOS sink behavior in at least one layer at some non-trigger position.This is an existential guarantee rather than a claim that every layer or head forms a sink.
  • Main Results: Theorem 3 constructs a one-layer ReLU attention model with zero loss and zero BOS attention at every non-trigger position.ReLU attention replaces softmax normalization and uses position-dependent scaling to represent averages.

4 Experiments

Experiments support the theoretical predictions: softmax models form sinks on the trigger-conditional task, whereas ReLU attention avoids them while retaining accuracy. In multi-layer models, sinks appear across some but not necessarily all heads.

  • Single-layer models were trained to validate theorems 1 and 3, while multi-layer multi-head models with residual connections tested theorem 2.
  • Single-Layer Models: Softmax attention placed near-unit mass on position 1 at every non-trigger position, with negligible variance across 1,000 test examples.The experiment used trigger position j = 8.
  • Single-Layer Models: ReLU attention achieved comparable task accuracy while keeping attention on position 1 near zero throughout the sequence.All parameters other than the attention normalization were kept identical.
  • Multi-Layer Multi-Head Models: In a 2-layer 2-head softmax model, all heads exhibited strong sink behavior across non-trigger positions.
  • Multi-Layer Multi-Head Models: In deeper softmax models, some heads lacked sinks while others developed clear sinks, consistent with theorem 2's existential guarantee.A 4-layer 4-head example had one head with near-zero BOS attention and other heads with sinks.
  • Multi-Layer Multi-Head Models: Replacing softmax with ReLU eliminated sink formation in the tested multi-layer models.This held for both 2-layer 2-head and 4-layer 4-head models.

5 Conclusions and Practical Implications

The paper concludes that sinks are structurally necessary for the studied trigger-conditional behavior under softmax normalization, not merely optimization artifacts. It therefore motivates changing how inactive states are represented when sinks are undesirable.

  • For trigger-conditional behaviors, softmax normalization forces sink formation when models must produce a stable no-op by default and compute content-dependent outputs after a trigger.
  • Within softmax, penalizing BOS attention, spreading attention mass, or post-hoc reweighting may weaken the no-op guarantee or shift the anchor elsewhere.
  • Non-normalized attention, explicit gating, and related mechanisms can represent an off state by outputting zero without allocating probability mass.The paper presents these as promising directions when sinks waste capacity, confound analyses, or create quantization-unfriendly outliers.
  • The authors aim for these results to guide future sink-free attention mechanisms that directly support no-op operations.

6 Limitations

The limitations concern the scope of the synthetic task and the incomplete localization of sinks in multi-layer models. Several broader extensions and related sink phenomena remain open.

  • The synthetic trigger-conditional task represents one computational pattern within a broader class of trigger-conditional problems.
  • The analysis may extend to key-query retrieval, but formally characterizing the full class of tasks that necessitate sinks is left for future work.
  • In multi-layer models, theorem 2 guarantees a sink somewhere, but does not identify which specific layer must exhibit it.
  • Whether other stable special tokens or secondary attention sinks show similar behavior remains an open question.

A Training Details

The paper motivates its study with reported effects of attention sinks on model performance and deployment, including context use, compression, long-context inference, vision, multimodality, and interpretability.

  • The empirical motivation section surveys evidence that attention sinks affect model performance and deployment.
  • Accuracy and context utilization: Concentrated probability mass can divert attention from other tokens and affect downstream accuracy.
  • Compression and quantization: Sink-associated outlier activations complicate model compression and quantization.
  • Streaming and long-context inference: Evicting sink tokens from rolling KV caches can cause catastrophic performance degradation during long-context generation.
  • Vision and multimodal models: Analogous sink effects occur in vision and multimodal models, where they can waste representational capacity or be exploited to induce hallucinations.
  • Interpretability: Sinks complicate attention-based interpretation by concentrating probability on tokens without content-relevant information.

C Additional Experimental Results

Larger 4-layer, 4-head experiments reproduce the predicted contrast: softmax develops attention sinks, whereas ReLU attention keeps BOS attention near zero.

  • These results extend the necessity of softmax attention sinks to deeper, wider architectures.
  • Softmax models exhibit strong sink behavior in at least one head per layer in the no-trigger regime.
  • ReLU models maintain near-zero attention on BOS throughout.

D Proof of theorem 1

The proof establishes that low-loss single-layer softmax models must place nearly all attention on BOS at both pre-trigger and post-trigger positions, yielding sink necessity at every non-trigger position.

  • Together, the pre-trigger and post-trigger necessity results establish sink necessity at all non-trigger positions.
  • Theorem 4 shows that, with probability at least 1 −δ, every pre-trigger position has αi,1 ≥1 −ε.
  • The contradiction arguments show that non-BOS attention would force value projections to be small for constructed tokens, conflicting with the task outputs.
  • Theorem 5 shows that, with probability at least 1 −δ, every post-trigger position has αi,1 ≥1 −ε.
  • The post-trigger proof shifts triggers to the sequence end, preserves probability density, and contradicts pre-trigger necessity.

E Proof of theorem 2

The multi-layer proof extends sink necessity by representing the network output as a nonnegative combination of transformed token values and deriving a contradiction from the absence of sinks.

  • The proof assumes a sequence of low-loss multi-layer softmax models lacking the required sink behavior.
  • On the no-sink event, recursive attention representations imply small value projections at pre-trigger positions across layers.
  • Constructed token pairs differ in one content coordinate, enabling a projection argument that eliminates the common value component.
  • The target requires a nonzero projected component, while the uniform loss bound makes the corresponding model difference too small, producing a contradiction.

F Proof of theorem 3

The ReLU construction solves the trigger-conditional task exactly without a sink: non-trigger queries receive zero attention, while the trigger aggregates preceding representations.

  • The construction sets WK, WV, and WO to identity and uses coordinate indicators for BOS, trigger, and non-trigger tokens.
  • At every non-trigger position, all ReLU attention scores and weights are zero, including BOS attention αi,1 = 0.
  • At the trigger position, the construction normalizes positive ReLU scores over preceding tokens to produce the required average.
  • The resulting model has zero loss while maintaining αi,1 = 0 for all positions.
  • Softmax normalization is monotone under added keys, a property used in the lemmas to bound attention weights in the necessity proofs.

H Related Work

Prior work examines why attention sinks arise, how softmax normalization constrains attention, and when sinks may be useful or mitigated. This paper contributes a concrete task where softmax normalization provably makes a sink necessary, while ReLU attention avoids it.

  • Theory and analyses of attention sinks: Recent studies analyze attention sinks as phenomena shaped by circuits, positional encodings, residual activations, normalization, compression, and functional redundancy.These accounts include concrete sink-forming circuits, reference-frame anchors, residual sinks, and BOS-focused heads used for pruning or redundancy analysis.
  • Softmax normalization implications: Softmax converts attention scores into nonnegative weights summing to one, and prior work studies how this simplex constraint restricts attention behavior.Alternative mechanisms relax or replace softmax normalization to expand the available attention behaviors.
  • Softmax normalization implications: This paper proves that trigger-conditional aggregation with zero output otherwise requires a sink under softmax, whereas ReLU attention solves the same task without one.The result identifies normalization as the cause of sink formation for this task rather than treating the sink as merely an optimization artifact.
  • Mitigating sinks: Mitigation work proposes modified normalizations, long-context training procedures, and inference-time methods addressing sink-related behavior or its effects.These approaches include sink-avoiding normalizations, sliding-window attention, and analyses of KV-cache quantization disruptions.
  • Usefulness of sinks: Other studies treat sinks as useful computational primitives that can support trigger-conditional computation, specialization, dormant-head identification, and sink-aware pruning.This perspective aligns sinks with concrete computations such as averaging and retrieval rather than viewing them only as artifacts to eliminate.
  • Usefulness of sinks: In multi-head experiments, softmax attention shows strong sinks across layers, while ReLU attention shows no sink behavior across layers.Figures 7 and 8 provide representative patterns for the 4-layer, 4-head softmax and ReLU models, respectively.
Loading 2603.11487v5…