PaperlayerView exact revision on arXiv

Source-linked AI summary

Disentangling Representation Evolution in Transformers through Directional Decomposition

Shwai He, Haichao Zhang, Shen Yan

arXiv:2609.15975v1cs.CLcs.LG

TL;DR

Transformers combine direction-preserving and direction-changing updates, but the functional importance of these components depends on where they are measured. This paper decomposes and intervenes on them across residual and attention value spaces, finding robust value-space parallel edits, more diagnostic perpendicular compression error, and improved pretraining outcomes from parallel suppression.

  • Problem

    The paper asks whether substantial parallel Transformer updates are behaviorally necessary or functionally redundant despite the residual identity path already supporting scalar rescaling.

  • Method

    It decomposes updates into parallel and perpendicular components in residual and attention value spaces, then applies targeted scaling interventions and component-resolved compression diagnostics.

  • Results

    Parallel edits are broadly more robust than perpendicular edits, especially when excluding the self message in value space; perpendicular compression error separates methods more clearly, and parallel suppression improves pretraining outcomes.

  • Takeaways & Limitations

    Directional geometry provides a framework for analyzing editing robustness, diagnosing compression, and biasing pretraining toward orthogonal contextual steering.

  • Takeaways & Limitations

    Experiments cover decoder-only models and selected architectures and training regimes, so generalization to other settings remains to be established.

Abstract

from arXiv · show

Transformer representations evolve through learned additive transformations that either preserve their current direction or redirect it. We study this evolution as a functional geometry, decomposing learned updates into parallel and perpendicular components. Across pretrained models, we find substantial parallel components beyond the residual identity path. We then apply the decomposition in two spaces: to attention and MLP updates relative to the hidden state, and to attention value aggregation relative to the current token's value. Targeted edits reveal a strongly space-dependent asymmetry: exclude-self value-space parallel manipulation is markedly more robust than residual-space and perpendicular counterparts, preserving the direct self message while scaling only the non-self aggregate. The same decomposition gives a component-resolved description of compression-induced update error: perpendicular error separates compression methods more clearly than parallel error. Extensive experiments further demonstrate that full-aggregate parallel suppression during from-scratch pretraining lowers validation-loss trajectories and improves downstream averages, with the value-space variant strongest. Together, these results connect representation geometry to editing robustness, compression diagnosis, and training-time intervention. Code is available in the \href{https://github.com/Shwai-He/Transformer-Geometry}{project repository}.

1 Introduction

The paper treats Transformer representation evolution as directional geometry, separating magnitude-preserving updates from direction-changing updates. Interventions show that parallel components are broadly robust, especially in attention value space, while perpendicular components are more behaviorally important; the same geometry diagnoses compression and improves pretraining.

  • 1 Introduction: The work frames Transformer updates as parallel magnitude modulation and perpendicular directional steering, then tests their behavioral sensitivity through geometric interventions.The decomposition is applied to residual updates and attention value aggregation.
  • 1 Introduction: Across pretrained models, substantial parallel update components persist despite the residual path already supporting scalar rescaling, raising questions about their functional necessity.The paper identifies this as a central motivation for testing whether parallel updates are redundant or essential.
  • 1 Introduction: Perpendicular scaling is consistently disruptive, whereas parallel scaling is comparatively benign across a broad interval, particularly for attention value-space cross-token aggregation.Residual-space parallel scaling is less stable than value-space scaling.
  • 1 Introduction: Directional error decomposition separates compression quality more clearly through perpendicular error, while parallel error is less discriminative.The paper reports this pattern across quantized and pruned models.
  • 1 Introduction: Across model scales from 296M to 2.7B parameters, parallel attention suppression lowers validation-loss trajectories and improves downstream performance, with the value-space variant strongest.Both pretraining variants improve downstream averages, and the value-space variant produces the largest gains.

2 Related Work

Prior work establishes residual pathways and attention-mediated token mixing as important to Transformer stability and information flow. This paper studies the directional geometry of learned updates as a complementary property of those mechanisms.

  • 2 Related Work: Residual connections preserve representations across layers while branch-scaling and normalization methods adjust update magnitude to stabilize optimization and reduce representational degeneracy.The paper positions directional update geometry as complementary to these residual-path studies.
  • 2 Related Work: Attention aggregates information across tokens, but attention mass can concentrate on special positions, motivating mechanisms that modulate outputs or reduce sink behavior.The related work connects these phenomena to self-directed value flow.

3 Background

The paper decomposes each Transformer update relative to a reference state into parallel and perpendicular components. The parallel part rescales the incoming representation, while the perpendicular part supplies direction-changing information whose substantial presence persists across depth.

  • 3 Background: Any update vector admits a unique decomposition into a component parallel to a reference state and an orthogonal component.The parallel component lies along the reference direction; the perpendicular component lies in its orthogonal subspace.
  • 3 Background: The framework uses residual space for sub-layer updates relative to the incoming hidden state and attention value space for aggregates relative to the token’s own value vector.These spaces provide complementary reference directions for analyzing representation evolution.
  • 3 Background: A parallel residual component rescales the incoming representation by a factor of 1 + α, whereas the perpendicular component introduces orthogonal direction-changing information.The residual update can describe individual attention or MLP outputs, or their accumulated block effect.
  • 3 Background: Figure 1 tracks the ratio of parallel to perpendicular residual-update magnitudes across layers and generation steps, with ratios above one indicating parallel dominance.The figure covers accumulated attention-plus-MLP updates at sampled layers in two Qwen3 models.

4 Directional Interventions

Directional interventions decompose and independently scale parallel and perpendicular components in residual and attention value spaces. Value-space editing further separates the direct self message from non-self aggregation so contextual scaling can preserve token identity.

  • 4 Directional Interventions: Attention value vectors are formed by mapping each source token through the value projection and applying a query-dependent linear mixing operator before output projection.The notation accommodates standard multi-head attention through block-diagonal head structure.
  • 4 Directional Interventions: The intervention framework applies component scaling in residual space and attention value space, with parallel-only scaling also expressible as an attention-diagonal change.Figure 2 summarizes both intervention sites and reports raw perplexity changes relative to the scale-one no-op.
  • 4 Directional Interventions: In value space, the natural reference is the current token’s own value vector, representing the direction obtained when attention routes no information from other positions.Residual-space geometry does not isolate this self-value-aligned structure because output projection mixes the aggregate into the residual stream.
  • 4 Directional Interventions: Exclude-self scaling decomposes and scales only the non-self aggregate, then restores the unchanged self message without masking the self edge or renormalizing the attention row.This avoids extinguishing the token’s identity carrier during parallel removal.
  • 4 Directional Interventions: For MLPs, the main analysis decomposes sub-layer outputs in residual space, while value-space decomposition is reserved for attention aggregates.Appendix analyses additionally apply the projection geometry to post-gating and grouped down-projected contributions.

5 Experimental Setup

Inference-time edits scale decomposed components during the forward pass without changing model weights, across residual-stream and attention-value sites. Experiments span multiple model families, language-modeling and downstream benchmarks, long-context tasks, and compression settings.

  • Intervention implementation: Forward-pass interventions scale parallel and perpendicular components across decoder layers while leaving model weights untouched.Value-space edits include both full-aggregate and exclude-self scaling, which preserves the query token’s direct self message.
  • Models and benchmarks: Experiments cover dense and mixture-of-experts Qwen3, Llama-3, and Gemma-3 models across perplexity, zero-shot, long-context, and compression benchmarks.
  • Pretraining setup: From-scratch GPT-style pretraining compares baseline optimization with residual-space and value-space parallel removal across models from 296M to 2.7B parameters.Training uses OpenWebText for the reported training-time investigations.

6 Inference-Time Component Editing

Inference-time component edits reveal that perpendicular steering is fragile, while parallel manipulation is comparatively robust, especially when applied to exclude-self value-space aggregation. Preserving the direct self message is central to this robustness across standard and long-context evaluations.

  • Directional asymmetry: Perpendicular scaling is hyper-fragile across value-space, residual-attention, and residual-MLP sites, with even minor deviations causing steep perplexity surges and removal causing catastrophic breakdown.
  • Site asymmetry: Parallel robustness depends on site: exclude-self value-space scaling stays within one perplexity point from baseline over [0,1], whereas residual MLP editing is most fragile.Residual attention is intermediate, and value-space shifts by at most 1.3 points up to scale 3.
  • Cross-space task robustness: V-Excl.-self leaves seven-benchmark performance virtually intact, trailing baseline by 1.5 points on 1.7B and matching the 30B-A3B MoE within 0.1 points.Naive full-aggregate value removal instead drops average performance by 8–10 points.
  • Context-length scalability: At 12k context, exclude-self editing remains substantially more resilient than full-aggregate scaling, whose complete parallel removal collapses accuracy by 26–40 points.Retaining half the parallel component stays within 2.4 points of baseline at 12k, while exclude-self retains over 75% accuracy at 4k.
  • Attention-map interpretation: Exclude-self value editing preserves self-token routing with bounded non-positive diagonal shifts, unlike volatile residual-space diagonal adjustments spanning −1.00 to 0.76.Despite a larger output perturbation norm than residual removal, value-space editing preserves capabilities by retaining self-identity routing and residual propagation trajectories.
  • Compression diagnostics: Perpendicular compression error separates AWQ and Wanda regimes by downstream fidelity, whereas parallel error interleaves and provides no consistent ranking.The fidelity order is 4-bit AWQ, unstructured Wanda, 4:8 Wanda, then 2:4 Wanda; perpendicular distortion dominates total error with r > 0.97.

7 Training-Time Allocation

The training experiments suppress parallel attention updates during both optimization and evaluation to test whether capacity shifts toward direction-changing components. Across model scales, suppression lowers validation-loss trajectories and improves downstream averages, with value-space removal strongest.

  • Training objective: Training enforces parallel removal during optimization and evaluation, avoiding a train–evaluation mismatch while testing allocation toward orthogonal contextual steering.
  • Training trajectories: Parallel attention suppression shifts OpenWebText validation-loss trajectories downward from early training through the final checkpoint across 296M, 436M, and 528M models.Figure 5 compares baseline, Attn Para-Rem., and V-Para Rem., retaining the perpendicular component in both interventions.
  • Downstream generalization: +0.7 points on 1.4B and +1.5 points on 2.7B: value-space parallel removal delivers the largest downstream gains over baseline across six benchmarks.Both parallel-removal interventions improve downstream performance at both model scales.
  • Additional controls: Fixed removal maintains the lowest training loss in the larger-model and gated-scaling control analyses.

8 Discussion

The paper interprets Transformer updates as magnitude modulation and semantic steering, showing that parallel changes are often tolerant while perpendicular changes are behaviorally critical. This geometry explains compression differences and motivates parallel suppression during pretraining.

  • Geometric Duality of Representation Evolution: Parallel updates act as adaptive gain controllers, while perpendicular updates steer representations into finely calibrated semantic subspaces and are therefore acutely fragile.This geometric duality distinguishes magnitude modulation from directional meaning in Transformer representation evolution.
  • Beyond Isotropic Compression Error: Perpendicular deviations directly distort semantic trajectories, making perpendicular error a stronger predictor of compression degradation than parallel error.Parallel shifts fall within a benign tolerance basin, whereas orthogonal deviations drive downstream differences even when global L2 errors are comparable.
  • Architectural Implications and Pretraining Dynamics: Suppressing parallel attention updates lowers validation-loss trajectories and improves downstream generalization by directing capacity toward orthogonal contextual steering.The intervention supports architectures that separate contextual routing from magnitude modulation.

9 Conclusion

The paper formalizes Transformer representation evolution through orthogonal decomposition into parallel magnitude modulation and perpendicular semantic steering. Across evaluated models and settings, perpendicular perturbations are fragile, value-space aggregation is especially robust to parallel edits, and directional error improves diagnosis and intervention.

  • 9 Conclusion: Orthogonal decomposition separates Transformer updates into parallel magnitude modulation and perpendicular semantic steering, revealing a universal asymmetry in intervention sensitivity.Representations tolerate substantial colinear scaling but are acutely fragile under perpendicular perturbations.
  • 9 Conclusion: Value-space aggregation is markedly more robust than residual-space editing because it preserves direct token routing.The conclusion identifies this site dependence as a central property of directional stability.
  • 9 Conclusion: Directional error decomposition clarifies compression behavior and supports parallel suppression during pretraining, which lowers validation loss and improves downstream performance across evaluated scales.The framework is presented as an alternative to isotropic Euclidean heuristics for model diagnosis, compression, and architecture design.

Limitations

The supplied material documents evaluation settings, editing constructions, intervention strength, computational overhead, and head-level sensitivity. These details bound how the reported editing and diagnostic results should be interpreted.

  • Limitations: The editing and diagnostic experiments primarily use Qwen3 base checkpoints, with Llama-3.2-3B used for long-context evaluation.Figure 1 is the stated exception, using Qwen3-4B-Instruct-2507.
  • Limitations: The experiments use standardized Language Model Evaluation Harness and RULER configurations, with task-specific few-shot settings and metrics.The supplied table caption identifies accuracy and length normalization as evaluation conventions.
  • Limitations: Exclude-self value-space editing preserves the direct self contribution while suppressing parallel components in the non-self aggregate.The construction isolates contextual mixing from the diagonal identity carrier.
  • Limitations: Value-space and residual-space Full No-Para edits differ substantially in applied strength: 49.10% versus 25.15% norm change and 25.89% versus 9.32% energy removal.Despite the larger output perturbation, value-space editing preserves the primary diagonal identity channel, whereas residual removal disrupts residual propagation.
  • Limitations: Directional decomposition is implemented with forward hooks and incurs less than 2.5% prefill, less than 1.0% decoding, and 3.39% pretraining overhead.The reported measurements use an RTX 6000 Ada and fused FlashAttention-2.
  • Limitations: Joint all-head parallel removal remains within the isolated single-head sensitivity envelope, except for a sharp layer-0 spike.The result indicates opposing head shifts mutually cancel in aggregate, while early heads collectively anchor initial parallel coordinates.

C Additional Editing Results

Additional experiments test whether directional asymmetry generalizes across models, internal MLP sites, compression settings, and training scales. They consistently find robust parallel suppression effects, fragile perpendicular perturbations, and downstream benefits from value-space removal.

  • C Additional Editing Results: Retained-scale value-space editing keeps RULER-4k scores within 0.5 points of matched unedited baselines across three model families and long-context settings.The evaluation uses s∥ = 0.5 for the non-self aggregate while preserving the direct self contribution.
  • C Additional Editing Results: Joint all-head parallel removal does not compound super-linearly and stays within the single-head sensitivity range, except for a sharp layer-0 spike.Opposing head shifts mutually cancel in aggregate, while early heads anchor initial parallel coordinates.
  • C Additional Editing Results: Across two MLP-internal editing sites, parallel removal changes fixed C4 loss by at most 0.0301, whereas perpendicular removal increases it by more than 13 points.The result supports directional asymmetry as an intrinsic property of internal activations rather than only final residual outputs.
  • C Additional Editing Results: Value-space parallel removal consistently improves downstream averages by +0.7 at 1.4B and +1.5 at 2.7B relative to same-scale baselines.The gains span ARC-E, BoolQ, HSwag, OBQA, PIQA, and WinoGr.
  • C Additional Editing Results: Fixed parallel removal outperforms learned tokenwise gating throughout 1.4B pretraining, with the lowest validation loss.The comparison supports parallel suppression as an invariant inductive bias rather than a learned modulation requirement.
  • C Additional Editing Results: Across MLP and combined-block compression analyses, perpendicular error tracks degradation while parallel error is non-monotonic.Perpendicular error aligns with total error at r ≥ 0.97 and ρ ≥ 0.90, accounting for 87.5%–98.0% of layer distortion.
  • C Additional Editing Results: Cosine diagnostics align with the decomposition because small updates make angular behavior primarily reflect perpendicular steering.Layers are safely pruned when perpendicular steering vanishes.
  • C Additional Editing Results: Baseline layer updates are measured relative to the incoming residual stream for combined block, attention, and MLP outputs across Qwen3-4B layers.Figure 10 reports means and one-standard-deviation bands over 12 prompts, with values below one indicating smaller updates than the residual stream.
Loading 2609.15975v1…