PaperlayerView exact revision on arXiv

Source-linked AI summary

Position: Don't be Afraid of Over-Smoothing And Over-Squashing

Niklas Kormann, Benjamin Doerr, Johannes F. Lutzeyer

arXiv:2601.07419v1stat.MLcs.LG

TL;DR

The paper questions whether over-smoothing and over-squashing are practically relevant across established GNN learning tasks. It examines these phenomena through literature review and experiments, finding no consistent performance gains from their mitigation and advocating task-level measures of information localisation and factorisation.

  • Problem

    The practical relevance of over-smoothing and over-squashing remains questioned despite sustained theoretical attention to these phenomena in established learning tasks.

    Source: §1 · p. 2, §2 · p. 2, §3 · p. 5

  • Method

    The paper reviews proposed accounts and interventions, then evaluates over-smoothing and over-squashing mitigation empirically on established graph-learning tasks.

    Source: §2.3 · p. 5, §3.3 · p. 9

  • Results

    Across the examined datasets, accuracy and over-smoothing do not consistently track, and mitigation yields no consistent performance gains for either phenomenon.

    Source: §2.3 · p. 5, §3 · p. 5

  • Takeaways & Limitations

    Analyze learning tasks with statistics of feature, structure, and label distributions to measure information localisation and factorisation and guide theoretical research.

    Source: §1 · p. 2, §4 · pp. 9–10

    The argument is scoped to reasonably constructed practical graphs and assumes relevant interaction information is usually contained within a small k-hop neighbourhood.

    Source: §3.2 · p. 7

Abstract

from arXiv · show

Over-smoothing and over-squashing have been extensively studied in the literature on Graph Neural Networks (GNNs) over the past years. We challenge this prevailing focus in GNN research, arguing that these phenomena are less critical for practical applications than assumed. We suggest that performance decreases often stem from uninformative receptive fields rather than over-smoothing. We support this position with extensive experiments on several standard benchmark datasets, demonstrating that accuracy and over-smoothing are mostly uncorrelated and that optimal model depths remain small even with mitigation techniques, thus highlighting the negligible role of over-smoothing. Similarly, we challenge that over-squashing is always detrimental in practical applications. Instead, we posit that the distribution of relevant information over the graph frequently factorises and is often localised within a small k-hop neighbourhood, questioning the necessity of jointly observing entire receptive fields or engaging in an extensive search for long-range interactions. The results of our experiments show that architectural interventions designed to mitigate over-squashing fail to yield significant performance gains. This position paper advocates for a paradigm shift in theoretical research, urging a diligent analysis of learning tasks and datasets using statistics that measure the underlying distribution of label-relevant information to better understand their localisation and factorisation.

1. Introduction

GNNs provide a foundation for many graph-learning applications, while research has emphasized challenges including over-smoothing and over-squashing. This paper questions their practical relevance in established tasks and calls for statistics of relevant-information distributions to guide theoretical research.

1. Introduction

  • GNNs support applications including molecular-property prediction, recommendation, and traffic forecasting, while their theoretical challenges include over-smoothing, over-squashing, robustness, and expressivity.The paper notes that MPNNs underpin most modern GNNs, with GCNs and GATs among their variants.Source: §1 · p. 1
  • The paper places its proposal alongside calls for impactful graph-learning tasks and methods to measure dataset structure and feature relevance.It advocates additionally quantifying the relevance of theoretical phenomena as learning problems.Source: §1 · pp. 1–2
  • The paper questions the practical relevance of over-smoothing and over-squashing in currently established learning tasks.It calls for diligent task analysis and statistics of relevant-information distributions to assess theoretical problems' practical impact and guide future research.Source: §1 · p. 2
  • The paper frames graph data using nodes, edges, adjacency, neighborhoods, and node features, and describes most GNNs as iterative message-passing and update networks.It introduces the GCN equation as an example of this model class.Source: §1 · p. 2
  • An MPNN's receptive field for node v comprises nodes whose initial features influence its final embedding.The supplied definition identifies these as nodes at distance at most L from v.Source: §1 · p. 2

2. Over-Smoothing

The section argues that over-smoothing has limited practical relevance in the examined learning tasks: deep-model performance losses may reflect uninformative receptive fields, while mitigation does not usually make large depths worthwhile. It calls for task- and dataset-specific measures of label-relevant information to guide theoretical and architectural research.

2.1. Related Work

  • Keriven (2022) showed that a limited number of message-passing layers can improve learning before complete collapse, while surveyed mitigation methods still show performance stagnation or decline as depth increases.Source: §2.1 · p. 3

2.3. Empirical Findings

  • In most examined cases, optimal performance was achieved with shallow models; mitigation prevented over-smoothing but did not improve performance, and large-depth gains were marginal relative to added computational cost.Experiments compared a standard GCN, PairNorm, GCNII, and G2-GCN across 2–64 message-passing layers on common node-classification datasets.Source: Table 1 · p. 4, §2.3 · p. 5

2.2. Position

  • For node-level tasks, performance losses may stem from uninformative receptive fields rather than over-smoothing; when the problem radius is no greater than the optimal smoothing depth, performance need not suffer.Source: §2.2 · p. 4
  • For graph-level tasks, over-smoothing may be beneficial when the common node representation aligns with the label distribution.Source: §2.2 · p. 4
  • Because message passing is costly on large graphs, the authors question arbitrarily many message-passing operations and point to Graph Transformers or rewiring for problems with very large radii.Source: §2.2 · p. 4
  • The authors propose measuring label relevance across receptive-field sizes and comparing problem radius with optimal smoothing depth to guide architecture research.Source: §2.2 · pp. 4–5

3. Over-Squashing

The paper questions whether over-squashing limits performance on practical tasks, arguing that relevant information is often localised or factorises. Experiments on common benchmarks find no consistent accuracy gains from mitigation and no significant evidence that structural bottlenecks limit predictive performance.

3.1. Related Work

  • Prior work frames over-squashing as growing receptive fields overwhelming fixed-size node embeddings, especially when tasks require long-range information.Other accounts examine bottlenecks through Jacobian bounds and graph topology.Source: §3.1 · pp. 5–6
  • The paper reviews rewiring approaches that add edges around low-curvature bottlenecks or use expander edges to reduce effective path lengths.It also discusses virtual nodes as a way to reduce commute time and over-squashing.Source: §3.1 · p. 7

3.2. Position

  • The paper posits that relevant interaction information is usually within a small k-hop neighbourhood and that receptive-field distributions often factorise.Under this view, subsets can be processed independently, making fixed-size representations sufficient for most real-world tasks.Source: §3.2 · p. 7
  • The authors assume label distributions may correlate with structural communities, making information exchange across bottleneck edges irrelevant for many tasks.Their findings suggest that negative curvature does not limit predictive performance.Source: §3.2 · p. 7
  • Future work should measure label-information localisation and factorisation to distinguish relevant long-range interactions from large receptive fields.The paper also recommends measuring model and graph-structure effects separately and examining interaction effects within k-hop receptive fields.Source: §3.2 · pp. 7–8

3.3. Empirical Findings

  • Neither individual mitigation methods nor graph rewiring consistently improve accuracy, and curvature slopes are not significantly greater than zero on any tested dataset.The authors conclude that local over-squashing through structural bottlenecks does not significantly limit predictive performance on common benchmark datasets.Source: Table 2 · p. 8, §3.3 · p. 9, Table 3 · p. 9

4. Conclusion

The conclusion urges closer analysis of learning tasks and datasets before judging the practical relevance of over-smoothing and over-squashing. It calls for statistics that measure how feature, structure, and label information is localised and factorised.

  • The practical relevance of over-smoothing and over-squashing should be assessed through diligent analysis of learning tasks and datasets.The paper notes that new applications and datasets may emerge and should be analysed carefully.Source: §4 · pp. 9–10
  • The paper calls for understanding real-world learning problems to guide theoretical research in graph representation learning.Source: §4 · p. 9
  • Statistics should measure localisation and factorisation in feature, structure, and label distributions.Source: §4 · p. 10

A. Over-Smoothing Measures

The experiments measure over-smoothing with multiple metrics on final hidden embeddings, selecting measures appropriate to each message-passing model. These measures capture representation convergence, neighborhood smoothness, or similarity to the global mean.

A. Over-Smoothing Measures

  • The study computes multiple proposed over-smoothing measures, including neighborhood MAD based on cosine similarity within a target mask.For the reported neighborhood smoothness, the target is a given k-hop neighborhood.Source: A. Over-Smoothing Measures · p. 13
  • Dirichlet energy quantifies convergence of node representations and is normalized by feature dimension to compare models with different hidden widths.The multidimensional definition sums energy over feature dimensions; the adopted definition differs from Rusch et al. (2023a) by rescaling, which does not affect expected decay toward zero.Source: A. Over-Smoothing Measures · p. 13
  • GCNs require degree-normalized Dirichlet energy because their over-smoothed representations are expected to scale with node degree.The degree-normalized measure converges to zero for over-smoothed GCN representations.Source: A. Over-Smoothing Measures · p. 13
  • Wu et al. (2023) define node similarity from deviations from the global mean feature vector; the metric converges to zero under GAT-based over-smoothing.Source: A. Over-Smoothing Measures · p. 13
  • Measures are computed on final hidden embeddings for every test initialization and all graph nodes, with metrics selected by message-passing model.GCN models use degree-normalized Dirichlet energy and 3-hop MAD; GATv2 and GraphSAGE use Dirichlet energy, node similarity, and 3-hop MAD.Source: A. Over-Smoothing Measures · p. 14

B. Experimental Details

The experiments span node-level and graph-level datasets, with models tuned across depths and evaluated using dataset-specific split and testing protocols. The appendix figures and tables report the measures, hyperparameter spaces, and results used in these experiments.

B. Experimental Details

  • Seven node-level datasets span citation, large-scale citation, heterophilic, and co-purchase graphs; graph-level tasks were excluded because convergence-related performance degradation is not expected after node embeddings are aggregated.Source: B. Experimental Details · pp. 14–15
  • Figures 3 and 4 report Dirichlet energy, 3-hop MAD, and node similarity across datasets and depths for GATv2 and GraphSAGE models, with 25th- and 75th-percentile bands.Source: Figure 3 · p. 15, Figure 4 · p. 16
  • Four graph-level datasets—Mutag, Enzymes, Proteins, and Peptides-func—extend the over-squashing experiments across typical application domains and a larger benchmark.Source: B. Experimental Details · p. 15
  • Each model was tuned per dataset at depths of 2, 4, 8, 16, 32, and 64 layers, with shared and method-specific search spaces.Tables 5 and 6 summarize standard and over-squashing preprocessing hyperparameters.Source: B. Experimental Details · p. 15, Table 5 · p. 17, Table 6 · p. 17
  • Citation networks used the Planetoid split with 10-fold cross-validation, OGBN-arXiv used 8-fold cross-validation, and Roman-Empire used 10 random splits.Source: B. Experimental Details · p. 16
  • Amazon datasets used 20/30/rest splits, TU graph datasets used 80%/10%/10% splits, and Peptides-func used 70%/15%/15% splits.For datasets outside the Errica protocol, testing used the configuration with best mean validation loss and averaged over 10 random initializations.Source: B. Experimental Details · p. 16

C. Extended Report of Experimental Results

Tables 7 to 9 provide the full experimental results, including GATv2 and GraphSAGE message-passing methods.

C. Extended Report of Experimental Results

D. Curvature Results

The curvature analysis relates predictive performance to curvature values for selected models and rewiring configurations. The reported analysis finds no apparent curvature–accuracy correlation and tests this relationship with linear regression.

D. Curvature Results

  • The analysis evaluates Ollivier–Ricci and Balanced Forman curvature, averaging the highest-curvature bottleneck edges up to the maximum number edited by either rewiring method.Curvature is evaluated for the original graph, the EGP expander graph, and BORF and SDRF rewiring variations.Source: D. Curvature Results · p. 17
  • For each dataset, the analysis pairs the best-performing model across depths with curvature for its method and hyperparameter configuration.Roman-Empire, Mutag, Enzymes, and Proteins contribute ten times as many points per method under the Errica protocol, where hyperparameters vary across initializations.Source: D. Curvature Results · pp. 17–18
  • No apparent correlation between curvature and accuracy is observed; linear regressions test the relationship for each dataset and rewiring method.A t-test assesses whether each regression slope differs significantly from zero; Table 4 reports the regressions.Source: D. Curvature Results · p. 18, Figure 5 · p. 18
Loading 2601.07419v1…