PaperlayerView exact revision on arXiv

Source-linked AI summary

The Underappreciated Power of Vision Models for Graph Structural Understanding

Xinjian Zhao, Wei Pang, Zhongkai Xue, Xiangru Jian, Lei Zhang, Yaoyao Xu, Xiaozhuang Song, Shu Wu, Tianshu Yu

arXiv:2510.24788v1cs.CVcs.AIcs.LG

TL;DR

The paper examines whether vision models can capture graph structure in ways that differ from bottom-up message-passing GNNs, especially where existing benchmarks mix topology with domain features. It introduces GraphAbstract to test global structural properties across graph scales and finds vision models stronger than GNNs on holistic structural tasks. The findings support visual processing as a complementary pathway for graph learning.

  • Problem

    Existing graph benchmarks can couple domain-specific features with topology, limiting assessment of models’ ability to recognize abstract structural patterns across scales and contexts.

    Source: §1 · p. 1, §4.1 · p. 6

  • Method

    The authors introduce GraphAbstract, a benchmark of four global graph-property tasks, and apply vision encoders to graph images without graph-specific architectural modifications.

    Source: §1 · p. 2

  • Results

    Vision models significantly outperform GNNs on global-property abstraction tasks and generalize better across graph scales; on topology classification, vision models drop only 5-6% accuracy from ID to Far-OOD settings.

    Source: §1 · p. 2, §4.4 · p. 9

  • Takeaways & Limitations

    Visual processing offers a complementary pathway to GNNs’ explicit structural priors and domain-specific inductive biases for capturing global graph patterns.

    Source: §5 · p. 10

    The tasks are designed to mirror human cognitive capabilities, but the study does not directly compare model performance with human behavior.

    Source: §A.1 · p. 18

Abstract

from arXiv · show

Graph Neural Networks operate through bottom-up message-passing, fundamentally differing from human visual perception, which intuitively captures global structures first. We investigate the underappreciated potential of vision models for graph understanding, finding they achieve performance comparable to GNNs on established benchmarks while exhibiting distinctly different learning patterns. These divergent behaviors, combined with limitations of existing benchmarks that conflate domain features with topological understanding, motivate our introduction of GraphAbstract. This benchmark evaluates models' ability to perceive global graph properties as humans do: recognizing organizational archetypes, detecting symmetry, sensing connectivity strength, and identifying critical elements. Our results reveal that vision models significantly outperform GNNs on tasks requiring holistic structural understanding and maintain generalizability across varying graph scales, while GNNs struggle with global pattern abstraction and degrade with increasing graph size. This work demonstrates that vision models possess remarkable yet underutilized capabilities for graph structural understanding, particularly for problems requiring global topological awareness and scale-invariant reasoning. These findings open new avenues to leverage this underappreciated potential for developing more effective graph foundation models for tasks dominated by holistic pattern recognition.

1 Introduction

The introduction contrasts human-like holistic visual perception with bottom-up graph message passing and explores vision models as an alternative for graph understanding. It introduces GraphAbstract to evaluate global structural reasoning and reports advantages for vision models on such tasks and across graph scales.

1 Introduction

  • Humans perceive global structure intuitively, whereas GNNs use bottom-up message passing and can struggle with long-range dependencies and basic topological patterns.The introduction notes that prior work has addressed message-passing limitations, while vision-based graph learning remains comparatively underexplored.Source: §1 · pp. 1–2
  • The proposed approach renders graphs as images and applies vision encoders to graph-level tasks without graph-specific architectural modifications.The authors describe this as closer to human visual perception than explicit message passing.Source: §1 · pp. 2–3
  • Vision encoders perform comparably to specialized GNNs on established graph benchmarks, although those benchmarks can conflate domain features with topology.The introduction notes that fixed-structure expander graphs can match or exceed molecular-benchmark performance, suggesting node features may dominate structural information.Source: §1 · p. 2
  • GraphAbstract tests four global properties—organizational archetypes, symmetry, connectivity strength, and critical elements—and evaluates generalization to larger graphs.Its controlled graph families and increasing test sizes probe whether models recognize structural patterns across scales.Source: §1 · p. 2
  • On tasks requiring global-property abstraction, vision models outperform GNNs and generalize better across graph scales.The paper also reports that positional encodings improve GNNs more than message-passing architectural innovations, framing global topology as central to graph understanding.Source: §1 · p. 2

2 Related Work

Related work spans graph visualization and learning, alongside practical and synthetic graph benchmarks. The paper positions its approach against work that integrates layouts into existing architectures and benchmarks that do not directly assess human-like structural understanding.

Graph visualization and graph learning

  • Recent work combines visual graph layouts with vision-language models, adds probabilistic layout sampling to GNNs, or uses visual representations for molecular graphs.The paper distinguishes these approaches from its focus on differences between vision models and GNNs in graph understanding.Source: §2 · p. 3
  • The paper frames its distinction from prior layout-based work as examining model families’ cognitive differences rather than treating layouts and architectures as black boxes.Its stated benchmark gap is evaluating human-like recognition of connectivity patterns across scales and contexts.Source: §2 · p. 3

Graph learning benchmarks

  • Graph benchmarks include real-world datasets and synthetic expressivity tests, while newer LLM-oriented benchmarks probe tasks such as cycle detection and path finding.The paper notes that these tests often use a limited range of random graph generators and do not assess intuitive structural reasoning across scales and contexts.Source: §2 · p. 3

3 Visual vs. Message-Passing: Distinct Cognitive Patterns

Across benchmark datasets, vision models and GNNs achieve competitive performance but exhibit distinct prediction patterns and learning behaviors. Case studies and training curves further contrast vision models’ flexible attention and strong memorization with GNNs’ more locally constrained processing and the vision models’ validation gap.

Prediction Overlap Analysis

  • Vision models achieve performance competitive with GNNs across five graph-classification datasets, while their prediction patterns differ from GNN variants.GNN variants show high internal consistency, and the model families often succeed on different samples.Source: §3 · p. 3, Table 1 · p. 4, Figure 1 · p. 4

Case Studies

  • Vision models shift attention across local, regional, bridge, and global patterns in PROTEINS case studies, unlike GNNs’ relatively uniform, locally constrained patterns.The cases include hierarchical structures, critical bridges, and chain-like configurations.Source: §3 · p. 3, Figure 2 · p. 5

Training dynamics & confidence

  • Across architectures, vision models achieve strong training performance but have substantial validation gaps, while traditional GNNs plateau at lower training and validation levels.CNNs learn rapidly; transformers progress more slowly, and GPS reaches higher training accuracy but similarly limited validation performance.Source: §3 · p. 4, Figure 3 · p. 5

Why an intuitive topology benchmark is essential

  • Existing evaluations mix structural understanding with domain-specific features, motivating a benchmark that isolates intuitive topological perception.The section points to cases where models perform well even with random graph topologies.Source: §3 · p. 5

4 A New Benchmark: GraphAbstract

GraphAbstract targets a gap in evaluating topology understanding separately from domain-specific features, using tasks that probe global graph properties across scales. Its results favor vision models on global structural reasoning and scale generalization, while layouts, positional encodings, and computational cost shape the comparison.

4.1 Motivation and Design Principles

  • GraphAbstract separates structural comprehension from domain-specific attributes through tasks for archetypes, symmetry, connectivity strength, and critical elements.The benchmark varies graph scale between training and testing to assess scale-invariant structural understanding.Source: §4.1 · pp. 5–6

4.2.1 High-level Topology Classification

  • Topology classification asks models to identify a graph’s dominant pattern across six designed topological types, beyond local connectivity.The patterns include cyclic, random geometric, hierarchical, community, bottleneck, and multicore-periphery structures.Source: §4.2.1 · p. 6

4.2.2 Symmetry Classification

  • Symmetry classification labels graphs by whether their automorphism group contains a non-identity mapping, using several constructions of symmetric and asymmetric graphs.Symmetric examples include Cayley graphs with automorphism groups containing at least n elements; asymmetric examples include verified edge-swap perturbations that preserve degree distribution.Source: §4.2.2 · pp. 7–8

4.2.3 Spectral Gap Regression

  • Spectral gap regression tests whether models infer a graph’s global connectivity from topology, using graph families with varied spectral properties.The spectral gap is associated with mixing speed and connectivity; generated examples include stochastic block, geometric, and configuration models.Source: §4.2.3 · p. 8

4.2.4 Bridge Counting

  • Bridge counting predicts the total number of edges whose removal increases the graph’s connected-component count, requiring local edge assessment and global connectivity reasoning.The challenge varies with graph structure, and the benchmark frames critical connectivity awareness as relevant to molecular stability analysis and retrosynthetic planning.Source: §4.2.4 · p. 8

4.3 Evaluation Protocol

  • Evaluation tests scale transfer on ID graphs with 20-50 nodes, Near-OOD graphs with 40-100 nodes, and Far-OOD graphs with 60-150 nodes.The baselines include GCN, GIN, GAT, and GPS with positional encodings, plus four vision backbones and three graph layouts.Source: §4.3 · pp. 8–9

4.4 Main Results

  • Vision models lose only 5-6% accuracy from ID to Far-OOD on topology classification, while basic GNNs with one-hot degree features decline by over 45%.On symmetry detection, vision models with spectral layouts achieve 20% higher accuracy than even the best GNN variants.Source: §4.4 · p. 9
  • Vision models maintain performance across scale shifts, while GNNs degrade; spectral layouts outperform force-directed approaches by 8-10% accuracy on symmetry detection.Spectral layouts can cause node overlap in symmetric or repetitive graphs, whereas force-directed methods emphasize node separation and edge routing; layout choice should match the task.Source: §4.4 · pp. 9–10

5 Conclusion

The conclusion presents visual processing as a complementary pathway to graph learning, particularly for global structural properties and scale generalization. It points toward combining visual perception with structural reasoning in future graph foundation models.

5 Conclusion

  • Vision models offer distinctive strengths in perceiving global graph patterns, complementing GNNs’ explicit structural priors and domain-specific inductive biases.The paper proposes integrating visual perception with structural reasoning for more robust and generalizable graph understanding.Source: §5 · p. 10

A.1 Limitations

The paper identifies open theoretical and empirical questions, alongside directions for building vision-centric graph learning infrastructure. It also situates positional encodings as a route for injecting global structural information into GNNs.

A.1 Limitations

  • The work lacks a complete theoretical account of why particular layouts benefit particular reasoning tasks, leaving layout geometry and neural-network learnability open.The authors call for rigorous frameworks connecting layout properties to learning guarantees.Source: §A.1 · p. 18
  • GraphAbstract’s tasks are designed to mirror human cognition, but the paper does not directly compare model performance with human behavior.Suggested future studies include eye-tracking or timed reasoning tasks.Source: §A.1 · p. 18

A.2 Future Work

  • Vision-centric graph foundation models require an ecosystem including diverse graph-visualization pretraining data, topology-preserving augmentations, attribute encodings, and specialized architectures.The paper also identifies interactive graph visualization and visual graph analytics as possible application scenarios.Source: §A.2 · p. 18

B Extended Related Work

  • Positional encodings add pre-computed global information to graph models, including spectral methods, SPE, and SignNet.The cited methods encode global connectivity patterns or produce sign-invariant representations.Source: B Extended Related Work · p. 18

C Benchmark Statistics

The benchmark covers four graph-understanding tasks with task-specific sample sizes and node-count ranges. It evaluates each task on test sets with increasing distribution shift.

C Benchmark Statistics

  • The benchmark overview reports task-specific sample sizes and node-count ranges, and Figures 4 and 5 visualize bridge-count and spectral-gap distributions across graph types and splits.The supplied table caption describes the statistics layout but does not provide the individual counts or ranges.Source: C Benchmark Statistics · p. 18, Table 4 · p. 19
  • Table 4 distinguishes two classification tasks—six-way topology and two-way symmetry—from spectral-gap and bridge-count regression tasks.Its cells report graph counts and node-count ranges; the supplied passages do not give the cell values.Source: Table 4 · p. 19

D.1 Dataset Generation

  • Each of the four tasks has ID, Near-OOD, and Far-OOD test sets, and the datasets are implemented in PyTorch Geometric.ID matches the training distribution; Near-OOD and Far-OOD represent moderate and significant shifts, respectively.Source: §D.1 · p. 19

D.1.1 Topology Classification Dataset Generation

Topology classification uses six graph generators to produce visually and structurally distinct patterns. The generators cover cyclic, geometric, community, hierarchical, bottleneck, and multi-core structures.

D.1.1 Topology Classification Dataset Generation

  • Annular random geometric graphs provide cyclic structures, with sampled inner and connection radii and an outer radius offset from the inner radius.The inner radius is sampled from [0.7, 1.2], the outer-radius offset from [0.05, 0.3], and the connection radius from [0.5, 0.8].Source: §D.1.1 · p. 19
  • Random geometric graphs place nodes uniformly in a unit square and connect nodes within a sampled radius, adding edges between components when needed for connectivity.The connection radius is sampled from [0.15, 0.25].Source: §D.1.1 · p. 19
  • Community graphs use 3–5 groups with dense within-group and sparse between-group connections.Within-group edge probabilities are [0.6, 0.8], while between-group probabilities are [0.01, 0.05].Source: §D.1.1 · p. 20
  • Hierarchical graphs have 2–4 levels, decreasing level sizes, and nodes connected to 1–3 nodes in the level above.The level-size ratio decreases by approximately 0.4 at higher levels, while intra-level connectivity decreases for lower levels.Source: §D.1.1 · p. 20
  • Bottleneck graphs comprise 2–4 communities linked by narrow pathways 1–3 nodes wide.Within-community edge probabilities are [0.4, 0.6].Source: §D.1.1 · p. 20
  • Multi-core graphs combine 2–3 dense cores, bridging nodes between core pairs, and peripheral nodes attached to a selected core.Cores occupy 50–60% of nodes, and periphery nodes connect to 1–2 nodes in a randomly selected core.Source: §D.1.1 · p. 20

D.1.2 Symmetry Classification Dataset Generation

Symmetry classification uses generated symmetric and asymmetric graphs, including constructions from real-world base graphs. Spectral-gap regression varies graph connectivity and structure, then samples across connectivity levels and computes the normalized-Laplacian spectral gap.

D.1.2 Symmetry Classification Dataset Generation

  • Real-world base graphs from MUTAG and sampled Cora subgraphs supply connected, diverse inputs for graph covers and Cartesian-product constructions.Cora subgraphs are obtained by BFS, random walks with restart, or GraphSAGE-style neighborhood sampling; approximately 30 base graphs are prepared per target size range.Source: §D.1.2 · p. 20
  • Cartesian products involving MUTAG and Cora graphs are filtered by automorphism checks to retain verified symmetric or asymmetric variants.The passage notes that real-world graphs typically lack perfect symmetry, so products are not assumed to preserve it.Source: §D.1.2 · p. 21
  • Symmetric graphs are generated with cyclic covers, Cayley cyclic graphs, bipartite double covers, and Cartesian products of symmetric components.Cyclic covers use 2–5 layers; the other constructions use cyclic groups, varied base graphs, or cycle, path, and star components.Source: §D.1.2 · pp. 20–21
  • Asymmetric graphs are produced by degree-preserving edge swaps or Cartesian products with real graphs, with connectivity and symmetry checked during generation.Perturbed graphs are verified to have |Aut(G)| = 1; real-graph products are also checked for asymmetry.Source: §D.1.2 · p. 21

D.1.3 Spectral Gap Regression

  • Spectral-gap graphs vary structure through stochastic block models, geometric graphs, and configuration-model rewiring.Block-model mixing controls between-community connectivity, geometric mixing adds connections, and rewiring preserves degree distribution while disrupting the original structure.Source: §D.1.3 · pp. 21–22
  • The spectral-gap samples allocate 40% to low, 30% to medium, and 30% to higher connectivity; λ2 is the second-smallest eigenvalue of the normalized Laplacian.The normalized Laplacian is L = I − D^-1/2 A D^-1/2.Source: §D.1.3 · p. 22

D.1.4 Bridge Counting Dataset Generation

Bridge-count data use five topology generators and exact bridge labels. The implementation also specifies GNN features and positional encodings, model configurations, training and evaluation protocols, and graph-image rendering.

D.1.4 Bridge Counting Dataset Generation

  • Bridge-count graphs use five topology generators, and NetworkX computes the exact number of edges whose removal increases the number of connected components.The generators cover geometric, community, hierarchical, bottleneck, and multi-core structures.Source: §D.1.4 · p. 22

D.2 Node Features and Positional Encodings

  • GNNs use one-hot degree node features capped at degree 100, with 16-dimensional positional encodings.Source: §D.2 · p. 22

D.3 Model Architecture and Hyperparameters

  • GNNs use 2–4 layers and hidden dimension 128; vision models use four standard backbones and receive 224×224 graph images.Source: §D.3 · p. 22

D.4 Training Protocol

  • Training uses batches of 128 for up to 200 epochs, early stopping, Adam, task-specific losses and metrics, and five fixed random seeds.The passage also specifies different learning rates for vision-backbone parameters and GNN models and classifier heads.Source: §D.4 · p. 22

D.5 Graph Image Generation for Vision Models

  • Graph images use sky-blue circular nodes with white borders and white edges, rendered consistently for each dataset split.Nodes have size 50; edges use width 1.5 and alpha 0.8.Source: §D.5 · p. 22

E Proof of Symmetry in Graph Coverings

This section defines graph covers and proves that their construction creates non-identity automorphisms, guaranteeing symmetry. It also gives algorithms that implement both cover constructions.

E.1 Definitions

  • The section defines automorphisms and symmetry, then specifies bipartite double covers and k-fold cyclic covers by their vertices and edges.Symmetry means having more than one automorphism.Source: §E.1 · p. 23

E.2 Symmetry of Bipartite Double Cover

  • Every bipartite double cover is symmetric because swapping its two layers is a non-identity automorphism.The proof verifies that the layer swap is bijective and preserves edges and non-edges.Source: §E.2 · pp. 23–24
  • A bipartite double cover has at least 2m distinct automorphisms when its base graph has m automorphisms.The construction lifts each base automorphism in two ways, preserving or swapping the layers.Source: §E.2 · p. 24

E.3 Symmetry of k-fold Cyclic Cover

  • For k ≥2, shifting each vertex's layer index by one modulo k is an automorphism of the k-fold cyclic cover.The shift is bijective and preserves edges and non-edges.Source: §E.3 · p. 24
  • The k-fold cyclic cover contains a cyclic automorphism subgroup isomorphic to Zk, so its automorphism group has size at least k.The successive layer shifts are distinct and the kth shift is the identity.Source: §E.3 · pp. 24–25

E.4 Algorithmic Implementation

  • The implementation constructs each cover by adding the prescribed copies of base vertices and connecting copies according to the cover definition.The bipartite cover makes two copies per vertex; the cyclic cover makes k copies and links successive layers modulo k.Source: §E.4 · p. 25

F Extended Experimental Results

Extended experiments report computational costs, scaling behavior, comparisons with an advanced GNN, and resolution robustness. The discussion contrasts global computations in graph layouts with local message passing, while the visual examples show benchmark graph patterns.

F Extended Experimental Results

  • Vision models require 8.2× to 12.2× more time per epoch than GNN+SPE models across the four benchmark tasks.The authors attribute the overhead mainly to larger model capacity and more complex feature extraction.Source: Table 5 · p. 26, F Extended Experimental Results · p. 26
  • 84.80% to 82.53%: GPS+SPE topology-classification accuracy falls on Near-OOD tasks as model size scales to 212.2M parameters.The authors report degradation with increased capacity across all tasks, despite models exceeding the 25.6M-parameter ResNet-50.Source: F Extended Experimental Results · p. 26, Table 6 · p. 27
  • I²-GNN has competitive in-distribution performance but significantly worse out-of-distribution generalization, particularly on Far-OOD settings.The comparison concerns scale generalization against vision-based approaches.Source: F Extended Experimental Results · p. 26, Table 7 · p. 27
  • Vision-model performance remains relatively stable across 64×64 to 448×448 resolutions, and some tasks perform better at lower resolutions.The resolution study uses ResNet on symmetry classification.Source: F Extended Experimental Results · p. 26, Table 8 · p. 27

G Discussion: Visual vs. Message-Passing Paradigms

  • Layouts perform global computations before image recognition, making structural features such as symmetry, clusters, and bridges visible as geometric patterns.The discussion contrasts this pathway with GNNs' iterative local message passing.Source: G Discussion: Visual vs.… · p. 28
  • Visual representations can reveal distinctions that message-passing methods miss, including cycle-connectivity differences and bridge structures.The cited discussion notes that standard GNNs fail to distinguish some NECKLACES examples and cannot identify bridges.Source: G Discussion: Visual vs.… · p. 28

H Visualization Examples from GraphAbstract

I Analysis of Graph Layout Algorithms

The section describes four graph layouts and examines how they affect visual assessment of asymmetry and task performance. It also provides implementation details for preliminary experiments.

I.1 Layout Algorithm Details

  • Kamada-Kawai and ForceAtlas2 use force-directed placement, spectral layouts use Laplacian eigenvectors, and circular layouts place nodes evenly around a circle.The methods have different heuristics and can produce different visual outputs.Source: §I.1 · p. 31

I.2 Visualizing Asymmetric Graphs

  • For a non-symmetric Cartesian-product graph, spectral and circular layouts make the lack of global symmetry more visually evident than force-directed layouts.In the spectral layout, irregular geometry and line thickness or brightness provide cues to asymmetry.Source: Figure 9 · p. 31, §I.2 · p. 31

I.3 Characteristics and Performance of Circular Layout

  • Circular layouts remain effective for symmetry detection despite disrupting topology cues, but performance drops sharply on Far-OOD topology classification.Balanced edge density signals symmetry; uneven density or symmetry-breaking edges signal asymmetry.Source: Table 9 · p. 32, §I.3 · p. 32

J.1 Implementation Details of Preliminary Experiments

  • Preliminary experiments used four GNNs and four vision models, with ImageNet-1K initialization for vision models and five random seeds for evaluation.Vision inputs were resized to 224×224; models used Adam, batch size 64, and early stopping.Source: §J.1 · p. 33

J.2 Training dynamics and confidence

The supplied material covers reported confidence and prediction-pattern observations, including attention case studies, alongside code and reproducibility information and study-practice disclosures.

J.2 Training dynamics and confidence

  • On PROTEINS, vision models concentrate predictions in the 0.8–1.0 confidence range across training and test splits, unlike typically lower-confidence GNNs; GPS is the GNN exception.Training dynamics are plotted over 100 epochs for PROTEINS, IMDB-BINARY, IMDB-MULTI, and ENZYMES, with loss, training accuracy, validation accuracy, and across-run standard deviations.Source: §J.2 · p. 33, Figure 10 · p. 33, Figure 11 · p. 34

J.3 Case Studies

  • On ENZYMES, Grad-CAM highlights known discriminative patterns more sharply than GNNExplainer, including multiple patterns near cut-vertices and cut-edges; the authors suggest vision models use these patterns as shortcuts.The discriminative pattern was previously characterized in [12] as a square with two diagonal connections, appearing in >90% of one class’s graphs and <10% of others.Source: Figure 16 · p. 37, §J.3 · p. 37

J.4 Heatmap

  • GNNs show high prediction overlap across layer depths, while their predictions remain distinct from those of vision models across five benchmark datasets.Figures 20–24 compare correct-prediction and error-overlap patterns for GNNs with 1–6 layers and vision models.Source: §J.4 · p. 40, Figure 20 · p. 40, Figure 21 · p. 41

NeurIPS Paper Checklist

  • The authors report that implementation details are in Appendix D, code for dataset generation and experiments will accompany supplementary material, and experiments use multiple random seeds with means and standard deviations reported.The checklist states that experimental setup details appear in Section 4.4 and Appendix D.Source: §2 · pp. 46–47
  • The authors state that they will release the code.The checklist marks the new-asset documentation question Yes.Source: §2 · p. 50
  • The paper reports no crowdsourcing or research with human subjects.The checklist also marks the related participant-risk and IRB question not applicable.Source: §2 · p. 50
  • LLMs were used only for writing, not as a core research component.The checklist marks declaration of LLM usage not applicable.Source: §2 · p. 51
Loading 2510.24788v1…