PaperlayerView exact revision on arXiv

Source-linked AI summary

When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models

Ding Zhang, Runtao Zhou, Wenqing Zheng, Rizal Fathony, Bayan Bruss, Chirag Agarwal

arXiv:2606.03712v1cs.LG

TL;DR

GLMs represent graph structure as tokens, but it is unclear whether LLMs interpret those tokens as meaningful carriers of topology. The paper audits graph-token behavior in representative GLMs and finds that activation saliency, attention, and graph-semantic utility can diverge. These results indicate that mapping graph topology into LLM token space alone does not guarantee a fully usable topology-aware internal representation.

  • Problem

    It remains unclear whether graph tokens in GLMs are meaningfully interpreted by LLMs and function as carriers of graph structure.

  • Method

    The paper mechanistically audits graph sink behavior in two representative GLMs, LLaGA and TEA-GLM, using activation analysis and token interventions.

  • Results

    Graph sink tokens consistently appear as activation-level outliers, but they do not reliably dominate attention or carry the main semantic and structural information for downstream prediction.

  • Takeaways & Limitations

    Mapping graph topology into LLM token space does not by itself guarantee a fully usable topology-aware internal representation.

  • Takeaways & Limitations

    The study examines two representative GLM architectures, and other architectures may exhibit different graph sink behavior.

Abstract

from arXiv · show

Graph Language Models (GLMs) have become a promising direction for adapting Large Language Models (LLMs) to graph learning tasks. By transforming graph topology and node information into graph tokens, GLMs allow LLMs to jointly process structured graph inputs and textual instructions. Yet, it remains unclear how LLMs internally interpret these graph tokens and whether graph tokens act as meaningful carriers of graph structure. In this work, we analyze how LLMs process graph information through graph-token behavior in representative GLM architectures. Findings. We find that the internal saliency of graph tokens in GLMs is not equivalent to graph information utilization. Graph sink tokens consistently emerge as activation-level outliers: they can be identified by massive activation values along a small set of hidden-state dimensions and are biased toward early graph-token positions. However, this activation-level saliency does not imply that these tokens are the main carriers of graph information. Unlike classical attention sinks in language and vision-language models, graph sink tokens do not necessarily attract the largest attention weights from query tokens. Through pruning, repositioning, and swapping interventions, we show that graph sink tokens are not the most important semantic or structural tokens for downstream prediction. Implications. Together, these results suggest that after current GLMs map graph structure into the LLM token space, the resulting graph-token representations do not naturally form a fully usable topology-aware internal representation; instead, they exhibit a decoupling between activation-level saliency and graph-semantic utility. This decoupling points to limitations in existing graph-token construction, placement, and alignment mechanisms.

1 Introduction

GLMs convert graph structure into tokens for joint processing with text, but it remains unclear whether LLMs faithfully use those tokens for topology-aware reasoning. This uncertainty reflects possible architectural sink behavior, where internally salient graph tokens may not carry graph semantics.

  • GLMs transform graph topology and node features into graph tokens that LLMs process alongside textual instructions.
  • The central representational assumption—that sequential token streams can faithfully encode non-Euclidean graph topology—remains largely unverified.
  • Transformers can develop attention sinks: tokens with massive internal saliency and attention scores that stabilize computation rather than convey semantic meaning.
  • Graph tokens may therefore become internally salient without supporting topological reasoning or meaningful graph-information use.
  • In LLaGA and TEA-GLM, graph sink tokens appear as sparse activation-level outliers with large spikes on a small set of hidden dimensions.

2 Graph Sink Tokens Emerge as Activation-Level Outliers

Graph sink tokens are defined by unusually large activations on a small set of hidden dimensions and consistently appear near the beginning of graph-token sequences. Across representative GLMs and graph tasks, these activation outliers show stable dimensions and positional biases, including frequent [PAD] tokens in LLaGA.

  • Graph Sink Definition: Graph sink tokens are graph tokens with large hidden-state values on identified sink dimensions.The definition uses activation magnitude as the sink characteristic and applies a predefined threshold.
  • Interpretation: The identified sink dimensions may reflect graph-token processing after graph information is mapped into the LLM token space.The study also considers whether graph tokens inherit sink dimensions from their LLaMA-family backbones or induce new ones.
  • Experimental Setup: The analysis compares node-aligned LLaGA with encoder-generated TEA-GLM across node classification and link prediction.Both models use fixed-length graph-token sequences, enabling positional comparisons across samples.
  • Activation Magnitudes: A sparse activation pattern recurs across datasets and architectures, with only a few hidden dimensions producing large spikes.Dimension 1512 repeatedly appears as a dominant sink dimension in both GLM designs; LLaGA also shows dimension 2533 for node classification.
  • Sink Token Positions: Graph sink tokens are biased toward early graph-token positions across tasks, datasets, and models.For TEA-GLM, positions 0 and 1 dominate; LLaGA shows the same bias across longer sequences.
  • Sink Token Positions: In LLaGA, the two graph sink tokens with the largest activation scores are always [PAD] tokens across datasets and tasks.Under the Neighborhood Detail template, index 0 is the center node, while later positions represent neighbors or [PAD] tokens.

3 Activation-Level Saliency Does Not Imply Attention Dominance

Graph sink tokens are activation-salient, but this saliency does not reliably translate into attention dominance. Across both GLMs, query attention can favor non-sink or later graph-token positions.

  • Graph sink tokens emerge as activation-level outliers biased toward early graph-token positions, prompting tests of whether they attract most query attention.
  • In LLaGA, sink tokens are not consistently the highest-attended graph-token positions across query offsets.
  • In TEA-GLM, attention often shifts to later graph-token indices rather than the main sink-token positions 0 and 1.
  • Across layers, TEA-GLM maintains stronger attention to later positions, while LLaGA distributes stable attention bands across sink and non-sink regions.
  • Thus, graph sink tokens are activation-salient but do not necessarily become dominant attention targets of query tokens.

4 Graph Sink Tokens Are Not the Main Semantic or Structural Carriers

Direct interventions show that graph sink tokens are not the main semantic or structural carriers in GLMs. Their behavior instead depends on architecture, graph-token sparsity, and padding patterns.

  • The study tests whether removing, swapping, or repositioning graph sink tokens causes meaningful prediction degradation.
  • Top-2 sink pruning usually causes little performance drop, whereas random non-sink pruning can be more harmful, especially for LLaGA.
  • For LLaGA, attention to top-2 sink tokens decreases as the proportion of non-padded graph tokens increases across Arxiv, Cora, and PubMed.
  • These results indicate that LLaGA sink attention is strongly shaped by graph-token construction and padding patterns rather than reliable graph-semantic content.
  • In LLaGA, sink behavior reappears after pruning but redistributes across many graph-token positions.
  • In TEA-GLM, pruning original sinks reduces activation magnitudes, and remaining graph tokens rarely satisfy the sink criterion.
  • Across pruning, swapping, repositioning, and sparsity analyses, graph sink tokens do not behave as the main semantic or structural carriers.

5 Mechanistic Evidence for Limited Graph-Token Semantics

Logit lens analysis finds that graph-token hidden states expose limited topology-aware or task-specific information. Sink positions instead tend to decode generic citation-domain terms with low confidence, supporting a gap between activation saliency and graph-semantic utility.

  • Logit lens setup: The logit lens projects graph-token hidden states through the language-model head and records the most frequent top-1 decoded vocabulary token and its average probability.The analysis evaluates whether hidden states expose labels, node semantics, or topology-related concepts.
  • Decoded graph-token states: Decoded graph-token states are weakly topology-aware, often producing fragmented subwords, punctuation, or generic terms rather than class labels or structural concepts.Decoded probabilities are generally low, indicating weak alignment with stable vocabulary predictions.
  • Sink-token decoding: From approximately layer 20 onward, sink positions g0 and g1 frequently decode to paper across three datasets.The repeated term is consistent with a broad citation-domain signal inherited from Arxiv pretraining, rather than dataset-specific labels or topology information.
  • Interpretation: Graph sink tokens mainly expose generic domain-level terms rather than task-specific or topology-aware graph information.This pattern is consistent with the intervention results and indicates that activation-level saliency is not sufficient evidence of graph-semantic utility.

6 Related Work

Related work situates GLMs within graph representation learning and connects graph-token analysis to established attention-sink and massive-activation phenomena in Transformer models.

  • Graph Language Models: Graph representation learning models relational data across scientific and real-world domains, while recent GLMs adapt LLMs to graph-structured data including text-attributed and citation graphs.Existing approaches differ in the role assigned to the LLM, including prediction over verbalized graph prompts.
  • Attention Sinks: Attention sinks are tokens that receive disproportionate attention despite limited semantic relevance, and related massive-activation work links sink behavior to extreme values on a few feature dimensions.These phenomena have been observed beyond text-only LLMs, including vision-language models.

7 Conclusion

Across LLaGA and TEA-GLM, graph sink tokens are activation outliers without corresponding functional importance or reliable graph-semantic content. The study concludes that tokenizing graph topology alone does not ensure a usable topology-aware representation, while broader architectural coverage remains necessary.

  • Conclusion: Across LLaGA and TEA-GLM, graph sink tokens are activation-level outliers whose saliency does not translate into dominant query attention or downstream functional importance.Intervention and logit lens analyses further indicate that they are not the main carriers of graph-semantic or structural information.
  • Conclusion: Mapping graph topology into the LLM token space does not by itself guarantee a fully usable topology-aware internal representation.The result identifies a limitation of relying on token-space mapping alone for graph understanding.
  • Limitations and Future Work: The study examines two representative GLM architectures, so other GLMs may exhibit different graph sink token behavior.Future work concerns improving graph-token construction, placement, and graph-text alignment to better preserve topology-aware information.

A.1 More Intervention Results

Additional interventions evaluate whether graph sink tokens are prediction-critical by pruning, swapping, and repositioning them. These operations also disturb the graph-token order encoded by LLaGA’s fixed template, while repositioning is not applied to TEA-GLM because its sinks are already near the front.

  • Intervention design: Both swapping and repositioning alter graph-token order and disturb the topology pattern encoded by LLaGA’s fixed graph-token template.Repositioning is the stronger positional intervention because it moves detected sinks to the front of the sequence.
  • Intervention design: Repositioning is not applied to TEA-GLM because its graph sink tokens already appear at positions 0 and 1 in most cases.Moving them to the front would therefore not create a meaningful additional intervention.
  • Intervention results: Table 2 compares baseline link prediction with pruning top-2 sink tokens, swapping sink and non-sink positions, and randomly pruning two non-sink tokens.The table marks cases where non-sink pruning causes a larger performance drop than top-2 sink pruning and baseline results.
  • Repositioning: LLaGA repositioning moves detected graph sink tokens to the front of the graph-token sequence and evaluates downstream performance on node classification and link prediction.Table 3 reports this performance comparison.

A.2 Attention Patterns of Graph Sink Tokens on Link Prediction

Figures 10 and 11 present query-to-graph attention maps for link prediction, averaged across heads and test samples. Figure 11 additionally shows how these maps vary across transformer layers.

  • Figure 10 visualizes query-to-graph attention weights for link prediction after averaging over heads and test samples.
  • Both figures summarize attention patterns using averages over heads and test samples.
  • Figure 11 provides a layer-wise view of query-to-graph attention weights for link prediction.The y-axis denotes transformer layers.

A.3 More Logit Lens Results

Additional analyses examine graph-token decoding and activation patterns across datasets and GLM architectures. They report generic or fragmented decoded tokens, sparse activation outliers, and a bias toward early graph-token positions.

  • Logit Lens Analysis: Sink-token positions g0 and g1 in TEA-GLM often decode to generic citation-domain terms such as paper in later layers on Arxiv.The analysis uses the most frequent top-1 decoded vocabulary token at each graph-token position and layer.
  • Logit Lens Analysis: On Cora, TEA-GLM graph-token states mostly decode to fragmented subwords or generic terms, including later-layer citation-domain tokens such as paper.
  • Activation Patterns: InstructGLM graph sink tokens show sparse activation-level outliers, with dominant spikes concentrated in a small number of hidden dimensions.Figure 14 averages results over 300 test samples for Cora, Arxiv, and PubMed.
  • Position Distributions: Most detected graph sink tokens occur near the beginning of the graph-token sequence across Cora, Arxiv, and PubMed.These appendix results support the same early-token bias observed in the main results.
  • Implementation Details: The reported LLaGA analyses use deterministic greedy decoding with sampling and beam search disabled.The fixed random seed is 42, and the experiments use two NVIDIA A100 GPUs.
Loading 2606.03712v1…