PaperlayerView exact revision on arXiv

Source-linked AI summary

NodeGround: A Node Classification Benchmark in the Graph Foundation Model Era

Jinmo Lee, Dooho Lee, Minho Jeong, Jaemin Yoo

arXiv:2609.39673v1cs.LG

TL;DR

NodeGround asks whether pretrained graph models can replace dataset-specific training and tuning while remaining competitive in prediction quality and workflow cost. It evaluates GFMs and supervised methods across 51 node-classification datasets, finding tuned GNNs strongest overall and current GFM advantages limited and condition-dependent.

  • Problem

    Graph foundation models aim to reduce the repeated training and tuning needed on new graphs, but their consistency and cost-effectiveness versus separately trained supervised methods remain unclear.

    Source: §1 · p. 1, §2 · p. 3

  • Method

    NodeGround benchmarks six GFMs and 15 supervised methods across 51 datasets using shared splits, controlled searches, validation-only selection, predictive metrics, and workflow-cost measurements.

    Source: §1 · p. 2

  • Results

    Tuned GNNs remain strongest overall; GraphPFN rises from sixth under 10/10/80 to third under 50/25/25.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Current GFMs do not yet provide a broadly reliable route to stronger predictions or cheaper workflows, and their strengths depend on task, graph conditions, and label availability.

    Source: §6 · p. 9

    The evaluation focuses on node classification with fixed label budgets and shared splits, and does not cover non-IID shifts or edge- and graph-level tasks.

    Source: §6 · p. 10

Abstract

from arXiv · show

Can a pretrained graph model replace training and tuning a separate predictor for each dataset? Answering this requires evaluating prediction quality alongside computational cost. We present NodeGround, a node classification benchmark that puts graph foundation models (GFMs) and dataset-specific supervised learning under a common evaluation framework. The benchmark spans 51 datasets and evaluates six GFMs alongside 15 supervised methods under two label-availability regimes. Shared data partitions, validation-only model selection, controlled hyperparameter searches, and multiple predictive metrics make comparisons systematic, while workflow measurements account for adaptation, training, tuning, and inference. The results favor carefully tuned graph neural networks overall. GraphPFN reaches third place by Elo when more labels are available, yet its relative strengths vary substantially with dataset properties. Efficiency comparisons further qualify the benefits of pretrained reuse: GVT and GraphPFN appear on the Pareto frontiers when supervised methods are represented by their default and fully tuned configurations. Adding intermediate tuning budgets removes this advantage for GVT and leaves GraphPFN extending the estimated frontier in the label-rich setting alone. Thus, reusing pretrained parameters does not yet provide a broadly reliable route to either stronger predictions or cheaper workflows. We release the evaluation pipeline, run-level records, and an open leaderboard at https://github.com/nums-ai/nodeground.

1 INTRODUCTION

Graph foundation models aim to reduce the repeated training and tuning required for node classification, but their consistency and cost-effectiveness relative to supervised methods remain unclear. NodeGround introduces a controlled benchmark across diverse datasets and finds tuned supervised GNNs lead overall, with GraphPFN the strongest GFM.

Motivation

  • The benchmark addresses fragmented prior evaluations that differed in dataset suites, splits, implementations, and hyperparameter tuning.The paper identifies the need to assess both predictive consistency and adaptation cost against separately trained and tuned supervised methods.Source: §1 · p. 2

Benchmark design

  • NodeGround evaluates 15 supervised methods and six GFMs across 51 datasets using two label regimes, fixed splits, controlled searches, and uniform model selection.The label regimes are 10/10/80 and 50/25/25, each with five fixed splits.Source: §1 · p. 2

Resources

  • The paper releases its evaluation pipeline, exact splits, raw results, and selected hyperparameters, and provides a leaderboard and submission system.These resources support reproducibility and evaluation of new methods under the same protocol.Source: §1 · p. 2

Main findings

  • GCNII and GPRGNN lead mean-rank comparisons under both label splits, while GraphPFN rises from sixth to third with more labels.GVT and GraphPFN appear on both performance–cost Pareto frontiers, but with intermediate tuning budgets only GraphPFN remains on the frontier.Source: Figure 1 · p. 1, §1 · p. 2

2 BACKGROUND

Graph foundation models seek to reuse pretrained components across graph datasets, in contrast with supervised GNNs that typically require dataset-specific training and tuning. Existing GFM evaluations leave predictive and computational comparisons uncertain; NodeGround brings broad coverage, controlled evaluation, and cost measurement together.

Graph foundation models

  • Supervised GNNs typically require training and extensive hyperparameter tuning for each graph dataset, motivating GFMs’ reusable pretrained knowledge.GFMs vary in what they learn from each new dataset: some fit predictors per graph, some train lightweight predictors on pretrained representations, and PFN-based approaches use labeled examples as context.Source: §2 · pp. 2–3

Node-classification benchmarks

  • GFMs aim to move per-dataset learning into reusable pretrained components, while node classification benchmarks must test broad transfer, strong baselines, and computational savings.The background frames this as a distinct evaluation question from assessing supervised GNN performance alone.Source: §2 · pp. 2–3

Limitations of current evaluations

  • Prior GFM evaluations report matching or outperforming supervised GNNs, but their evidence is limited in dataset coverage or baseline strength.NodePFN uses graphs below 50,000 nodes; G2T-FM and GraphPFN consider fewer than 500 input features, while some evaluations compare against only two GNNs.Source: §2 · p. 3
  • Whether pretrained reuse offers consistent predictive gains at competitive computational cost remains unclear because GFM and supervised workflow costs have not been systematically compared.Some GFMs require target-specific preprocessing or search, and dense interactions among labeled nodes can make PFN-based inference costly on large graphs.Source: §2 · p. 3

Node-classification benchmarks

  • No benchmark in the comparison covers all five evaluation aspects; NodeGround combines 51 datasets, two label regimes, controlled evaluation, cost measurement, and released run-level results.The five aspects include dataset coverage, comparison control, robustness, computational cost, and transparency.Source: §2 · p. 3, Table 1 · p. 3

3 NODEGROUND DATASET COLLECTION

NODEGROUND curates 51 in-scope node-classification datasets and characterizes their domains, graph structure, features, and targets. The collection spans varied individual properties and combinations that can create distinct learning conditions.

3 NODEGROUND DATASET COLLECTION

  • NODEGROUND focuses on transductive, single-label classification in small- and medium-scale homogeneous attributed graphs, drawing candidate datasets from four sources.Multilabel classification, heterogeneous graphs, and web-scale training are outside its current scope.Source: §3 · p. 4, §3.1 · p. 4

3.1 DATASET SELECTION PROTOCOL

  • Web, social, and scholarly datasets each contribute 13 of the 51 retained datasets, while finance contributes one.The collection spans six application domains, with no domain accounting for more than approximately one quarter of the datasets.Source: Figure 2 · p. 4, §3.1 · p. 4

3.2 COVERAGE OF DATASET PROPERTIES

  • Graph structure spans several orders of magnitude, adjusted homophily ranges from strongly heterophilous to strongly homophilous, and features and targets vary widely.The tasks include compact dense and high-dimensional sparse features, as well as binary to high-cardinality classification with balanced to highly imbalanced classes.Source: §3.2 · pp. 4–5, Figure 3 · p. 4
  • Datasets similar on one property can differ on another, including node count versus average degree, median degree versus hub concentration, and feature dimension versus sparsity.In eight datasets, features distinguish labels despite negative adjusted homophily.Source: Figure 4 · p. 5, §3.2 · p. 5

4 STANDARDIZED EVALUATION PROTOCOL

NODEGROUND standardizes how datasets, model selection, predictive performance, and computational costs are evaluated. It compares supervised models trained from scratch with GFMs using shared partitions, validation-based tuning, and common reporting measures.

4 STANDARDIZED EVALUATION PROTOCOL

  • The benchmark evaluates 15 supervised models trained from scratch alongside six recent GFMs, using a standardized data representation and evaluation protocol.Datasets are represented as PyG Data objects with repeated edges coalesced, graphs made undirected, and one self-loop added per node.Source: Table 2 · p. 5, §4 · p. 5, §4.1 · p. 5

4.1 MODEL OPTIMIZATION AND DATA PROTOCOL.

  • Two shared, class-stratified split regimes represent limited label availability (10/10/80) and a label-rich setting (50/25/25), each with five fixed experimental indices.Each index identifies a shared partition and deterministically sets random seeds for stochastic evaluation components.Source: §4.1 · p. 5
  • Supervised methods are evaluated with default and tuned configurations, while GFMs retain released pretrained components and optimize only choices exposed by their interfaces.Tuned supervised configurations are selected from the default plus up to 200 sampled configurations using validation cross-entropy.Source: §4.1 · p. 6

4.2 EVALUATION AND REPORTING

  • Accuracy for multiclass tasks and AUROC for binary tasks are the primary metrics, with overall comparisons reported as Elo ratings derived from pairwise outcomes.Additional metrics include macro-F1 and cross-entropy for multiclass tasks, and accuracy, positive-class precision, and binary cross-entropy for binary tasks.Source: §4.2 · p. 6, Figure 5 · p. 6
  • The cost measures include training-and-tuning time, prediction time, total workflow time, and peak allocated GPU memory during fitting and prediction.Tuned timing includes executed search trials, while GFM measurements exclude upstream pretraining and previously cached downloads.Source: §4.2 · p. 6

5 EXPERIMENTAL RESULTS

Across 51 datasets, tuned GNNs lead overall, while GraphPFN’s standing varies with label availability, task type, and graph conditions. GFMs reach workflow-time Pareto frontiers in endpoint comparisons, but intermediate tuning budgets leave only GraphPFN extending the frontier in the label-rich regime.

5.1 HOW DO GFMS COMPARE WITH SUPERVISED METHODS (RQ1)?

  • GCNII and GPRGNN lead both label-split Elo comparisons after tuning; GraphPFN is the strongest GFM, ranking third under 50/25/25.GVT and GraphPFN score above default supervised configurations, but their gains do not necessarily persist against tuned methods.Source: §5.1 · p. 7, Figure 6 · p. 7
  • GraphPFN leads mean AUROC rank on binary tasks under both splits, but GPRGNN and GCNII lead multiclass tasks.Its label-rich overall standing combines binary strength with improved multiclass performance as more labels become available.Source: §5.1 · p. 7, Figure 6 · p. 7

5.2 HOW BROADLY DO GFMS TRANSFER ACROSS DATASET CONDITIONS (RQ2)?

  • GraphPFN’s transfer is uneven: it leads both GNNs in some settings, but high-dimensional features, imbalance, strong homophily, and tasks above ten classes remain unfavorable.With 50/25/25, weaknesses on high-dimensional features, imbalanced targets, and tasks above ten classes persist; those class counts exceed its backbone’s output space and may require output coding.Source: §5.2 · pp. 7–8, Figure 7 · p. 8

5.3 DO GFMS IMPROVE THE PERFORMANCE–COST FRONTIER (RQ3)?

  • GVT and GraphPFN lie on workflow-time Pareto frontiers under both splits, but GVT is cheaper and GraphPFN achieves stronger predictions at greater cost.Under 50/25/25, GraphPFN has the lowest Mean Improvability, with approximately 6.4 mean workflow-time overhead, or roughly 84× the reference cost.Source: §5.3 · pp. 8–9, Figure 8 · p. 8
  • With intermediate tuning budgets, no GFM expands the frontier under 10/10/80; under 50/25/25, GraphPFN is the sole exception.Partially tuned supervised methods overtake both GVT and GraphPFN in the lower-label regime.Source: §5.3 · p. 9, Figure 9 · p. 9
  • Avoiding target-specific training does not guarantee lower workflow cost, because adaptation and inference can absorb computation.PFN-based models use self-attention with cost that grows quadratically with context length; the paper’s target is competitive predictions within the workflow-time budget of supervised training and tuning.Source: §5.3 · p. 9

6 CONCLUSION

NodeGround finds tuned GNNs strongest overall, while GraphPFN’s relative performance depends on task, graph conditions, and available labels; avoiding parameter updates does not ensure an inexpensive workflow. The evaluation is limited to node classification under fixed label budgets and shared splits, and the appendix examines robustness of the findings.

6 CONCLUSION

  • Across 21 methods and 51 node-classification datasets, tuned GNNs remain strongest overall, while GraphPFN’s results depend on task, graph conditions, and available labels.Adaptation and inference can absorb computation even when model parameters are not updated.Source: §6 · p. 9
  • Future GFMs should support competitive predictions across varied graphs within the time required to train and tune a supervised method.The paper identifies transfer across feature and label spaces and the cost of using pretrained models as priorities.Source: §6 · p. 9
  • The evaluation covers node classification under fixed label budgets and shared splits, not non-IID shifts or edge- and graph-level tasks.The paper notes that labeled and query nodes may differ systematically by community or time period in non-IID settings.Source: §6 · p. 10

A ADDITIONAL ANALYSES AND ROBUSTNESS

  • The appendix checks whether conclusions change with dataset and method subsets, missing-result handling, predictive metrics, aggregation rules, cost measures, and tuning budgets.It also compares pairwise win rates and statistical significance.Source: A ADDITIONAL ANALYSES AND ROBUSTNESS · p. 15

A.1 ANALYSIS COHORTS AND SENSITIVITY TO EVALUATION CHOICES

The appendix tests the main comparisons under alternative cohorts, metrics, aggregation rules, and cost measures. The strongest methods remain broadly similar, while GraphPFN’s position and efficiency trade-offs depend on the evaluation setting.

A.1 ANALYSIS COHORTS AND SENSITIVITY TO EVALUATION CHOICES

  • Across cohorts, tuned GNNs remain the strongest competition, and GraphPFN remains the sole GFM among leading methods, though its exact Elo position changes.Under 50/25/25, GraphPFN rises from third in V1 to first in V2; V2 uses 28 complete-case datasets, while V3 restricts methods to 15 near-complete entries.Source: §A.1 · p. 15, Figure 10 · p. 16
  • Across cohorts, GraphPFN leads on binary accuracy, AUROC, and cross-entropy, while multiclass results favor GNNs; positive-class precision yields a different ordering.The multiclass contrast is not created by imputation or a single predictive metric.Source: §A.1 · p. 15, Figure 11 · p. 17, Figure 12 · p. 18
  • The workflow-time trade-off persists across cohorts: GraphPFN is costly despite its label-rich predictive strength, while GVT is cheaper but leaves a larger predictive gap.GraphAny is a low-overhead exception in the V2 frontier under 10/10/80; apart from that exception, remaining GFMs are dominated.Source: §A.1 · p. 21, Figure 15 · p. 21
  • Changing the aggregation rule preserves the winner in each split but shifts individual methods by up to four or six places and can change the top four.GCNII remains first under 10/10/80 and GraphPFN under 50/25/25 across the three rules; overall rankings have Spearman correlations from 0.955 to 0.993.Source: §A.1 · p. 22, Table 5 · p. 22

A.2 PERFORMANCE–MEMORY TRADEOFFS

  • Under the memory comparison, GraphAny anchors the low-memory frontier but leaves a large predictive gap; GraphPFN reaches the label-rich frontier at roughly 26× reference memory.GraphPFN is dominated under 10/10/80, so the two frontier GFMs trade different resources for their positions.Source: §A.2 · p. 22, Figure 16 · p. 23

A.3 PAIRWISE WIN RATES AND STATISTICAL SIGNIFICANCE

  • Across all 51 datasets, GPRGNN wins more often than every competitor under 10/10/80, while GraphPFN does so under 50/25/25 despite ranking third by Elo.On 28 complete-case datasets, GraphPFN has seven significant wins and no losses under 50/25/25; no method significantly outperforms every competitor.Source: §A.3 · p. 23

A.4 TUNING REQUIREMENTS AND MODEL-SELECTION ROBUSTNESS

This appendix examines how tuning budgets affect test performance, workflow cost, and validation–test gaps, alongside describing the dataset sources used to build the benchmark.

A.4 TUNING REQUIREMENTS AND MODEL-SELECTION ROBUSTNESS

  • Under 10/10/80, Polynormer gains 9.4 percentage points from tuning, while under 50/25/25, GAT and GATv2 each improve by about ten points.GCN, GPRGNN, and Polynormer need 50 configurations to come within one point of full search under 10/10/80; MLP reaches that threshold after two under 50/25/25.Source: §A.4 · p. 25, Table 7 · p. 26
  • Intermediate workflow costs are estimated by linear interpolation between default and full-search measurements, not directly measured as cumulative trial time.The 50-configuration budgets cost approximately 28× the default under 10/10/80.Source: §A.4 · p. 25
  • From 50 to 201 configurations, every method gains less than one percentage point on average; most test gains appear within 25–50 configurations.Partially tuned supervised methods can approach full-search performance at substantially lower cost.Source: Figure 18 · p. 26, §A.4 · pp. 26–27
  • Under 10/10/80, validation–test improvement gaps reach approximately three points, versus below 1.3 points and sometimes negative under 50/25/25.The gap generally grows with search budget, so validation gains can overstate gains on unseen test nodes when labels are fewer.Source: §A.4 · p. 27

B.1 CANDIDATE SOURCES

  • The dataset-catalog appendix records four candidate sources and reports how records remaining after eligibility checks become canonical retained tasks.The sources are PyTorch Geometric, HF TAG, the Heterophilous Graphs collection, and OGB; 117 loader records were assembled.Source: §B.1 · p. 27, Table 8 · p. 27, B DATASET CURATION AND CATALOG · p. 27

B.2 ORDERED SCREENING, CANONICALIZATION, AND TASK IDENTITY

The curation procedure applies ordered eligibility checks, consolidates loader aliases only when they represent the same task, and catalogs the retained tasks and their provenance.

B.2 ORDERED SCREENING, CANONICALIZATION, AND TASK IDENTITY

  • Five ordered checks screen for task scope, verified categorical targets, usable label-independent features, scale limits, and duplicate tasks.The funnel reduces 117 records to 51 tasks, assigning each record to its first failed check.Source: §B.2 · p. 27, Table 9 · p. 28
  • After the first four checks retain 67 records, 15 alias groups consolidate 16 redundant records when graph structure, node attributes, and target semantics match.Cora, PubMed, and Wiki-CS remain separate from their TAG variants because their node features differ, yielding 51 tasks over 48 graph topologies.Source: §B.2 · pp. 27–28, Table 10 · p. 28

B.3 RETAINED DATASET CATALOG

  • The retained-task catalog records each task’s source, domain, target, citation, and upstream license information.Its links trace entries to their original sources; the benchmark does not redistribute or relicense the datasets.Source: §B.3 · p. 28
  • For 49 tasks, NODEGROUND retrieves and preprocesses data from upstream sources; deezer and lastfm asia use pinned Hugging Face copies with provenance, license information, and checksums.Their original graphmining.ai download endpoints were unavailable when the release was prepared.Source: §B.3 · p. 28

B.4 DATASET PROPERTIES

Dataset properties define overlapping subgroups for comparing model performance across graph scale, topology, features, and targets, while additional statistics describe coverage without defining those subgroups.

B.4 DATASET PROPERTIES

  • Seven properties define 22 overlapping subgroups across graph scale, topology, features, and targets, with each dataset assigned to one subgroup per property.Subgroups overlap across different properties but are mutually exclusive within a property; boundaries are fixed across both label regimes.Source: §B.4 · pp. 28–29, Table 12 · p. 30
  • Structural statistics use a simple undirected graph without self-loops, while adjusted homophily uses edges whose endpoints are both labeled.Adjusted homophily compares same-class edge agreement with agreement expected from endpoint class proportions; feature nonzero fraction uses a seeded sample of up to 4,096 rows.Source: §B.4 · p. 29
  • Density, degree quantiles, hub concentration, feature-similarity AUROC, and label entropy describe dataset coverage but do not define Figure 7’s subgroups.Source: §B.4 · p. 29

B.5 PER-DATASET STATISTICS

The appendix reports dataset-level statistics, documents criteria for comparing benchmarks, and specifies the evaluated method scope, including its fixed-backbone definition of graph foundation models.

B.5 PER-DATASET STATISTICS

  • Tables 14–16 report per-dataset grouping values and additional coverage statistics, while Table 13 defines statistics that do not determine RQ2 subgroups.Feature and target assignments include feature dimension, nonzero fraction, class count, and class imbalance; structural assignments include node count, average degree, and adjusted homophily.Source: §B.5 · p. 30, Table 13 · p. 31, Table 14 · p. 32

B.6 BENCHMARK COMPARISON CRITERIA AND EVIDENCE

  • NODEGROUND compares 51 node-classification tasks using controlled tuning, validation-only checkpoint and configuration selection, five shared partitions per label regime, multiple metrics, measured costs, released run records, and a public leaderboard.The comparison criteria define support for tuning control, validation selection, splits, multi-metric evaluation, compute cost, run-level results, and leaderboard access.Source: §B.6 · pp. 35, 37

C.1 METHOD SCOPE AND GFM DEFINITION

  • The benchmark includes 21 methods: one feature-only MLP, ten GNNs, four graph Transformers, and six GFMs.Methods must support homogeneous single-graph node classification with released features and a runnable, sufficiently specified implementation or pretrained artifact.Source: §C.1 · p. 37
  • Evaluated GFMs keep pretrained parameters fixed during target-dataset adaptation and can be applied to unseen graphs with different sizes, feature dimensions, and label spaces.Methods that update the pretrained backbone on each target dataset are excluded.Source: §C.1 · p. 37

C.2 IMPLEMENTATIONS AND TARGET-DATASET ADAPTATION

NodeGround standardizes how supervised methods and GFMs are implemented, adapted, selected, and evaluated. Its protocols use shared splits and validation-only model selection while preserving each method’s target-dataset adaptation approach.

C.2 IMPLEMENTATIONS AND TARGET-DATASET ADAPTATION

  • The benchmark runs supervised methods through benchmark-maintained implementations and distinguishes official GFM implementations from adapters loading official pretrained artifacts.Seven supervised implementations wrap PyG operators; eight are written directly in NodeGround.Source: §C.2 · p. 37, Table 17 · p. 38

C.3 PRETRAINED ARTIFACTS, PREPROCESSING, AND OVERLAP

  • Pretraining overlap is audited by canonical graph identity; GraphAny and GVT results on ogbn arxiv are omitted and replaced in common-universe analyses with matched GCN-Default results.The other four GFMs use synthetic pretraining, with no exact real-target overlap among the 51 datasets.Source: §C.3 · p. 38

C.4 OPTIMIZATION PROTOCOL, SEARCH SPACES, AND OUTCOMES

  • Supervised methods start with a default trial and sample up to 200 additional unique configurations; failed or non-finite additional trials are skipped rather than replaced.Trials use method-specific search spaces and validation-based selection.Source: §C.4 · pp. 38–39
  • GFM searches vary only exposed target-evaluation settings and keep pretrained backbones fixed; GVT and Node4All train prediction heads, while other GFMs use inference or analytical adaptation.GraphPFN uses training nodes as labeled context; GVT varies encoder depth, and NodePFN searches preprocessing and inference configurations.Source: §C.4 · pp. 39–40

D.1 SPLITS AND RANDOMNESS

  • Each dataset uses shared stratified partitions under 10/10/80 and 50/25/25 train/validation/test ratios, with five fixed partitions per ratio.Only valid-label nodes are eligible, and classes are remapped after exclusions.Source: §D.1 · p. 43

D.2 PREDICTIVE METRICS AND MODEL SELECTION

  • Validation loss selects checkpoints and configurations without test metrics; accuracy and AUROC are the primary test metrics for multiclass and binary tasks, respectively.Additional test metrics describe selected predictions but do not trigger separate model selection.Source: §D.2 · p. 43

D.3 AGGREGATION, MISSING RESULTS, AND UNCERTAINTY

The evaluation aggregates performance across datasets using ranks and pairwise Elo outcomes, and reports uncertainty and missing-result handling separately. It measures workflow costs across fitting, search or adaptation, and prediction, then compares predictive gap with relative cost using estimated intermediate-budget trajectories.

D.3 AGGREGATION, MISSING RESULTS, AND UNCERTAINTY

  • Mean ranks weight datasets equally after averaging each method’s scores across five partitions, while Elo pools partition-level pairwise wins on AUROC or accuracy to avoid averaging unlike task metrics.Ties count as half-wins in Elo, and each partition comparison has weight 1/5 so each dataset contributes equally.Source: §D.3 · pp. 43–44
  • Missing built-in results are imputed only for designated comparisons using matched GCN-Default results, and affected results are marked as imputed; user submissions are never imputed.Elo uncertainty uses 200 dataset-bootstrap resamples; rank-figure interquartile ranges describe rank variation across datasets, not bootstrap intervals.Source: §D.3 · p. 44

D.4 COMPUTATIONAL MEASUREMENT

  • Workflow cost includes supervised preprocessing and training, the full executed tuning search, or GFM target-specific preprocessing and adaptation; prediction is timed separately.Peak workflow memory is the larger of fitting/search/adaptation and prediction peaks because these stages run sequentially.Source: §D.4 · pp. 44–45

D.5 MEAN IMPROVABILITY, MEAN OVERHEAD, AND PARETO FRONTIERS

  • Mean Improvability measures error relative to the best evaluated configuration, while Mean Overhead compares cost with the least expensive eligible reference on the same dataset and averages log-ratios.Lower values are better on both Pareto-frontier axes; a Mean Overhead of one corresponds to a geometric-mean cost multiplier of 2.Source: §D.5 · p. 45
  • Intermediate-budget trajectories select among stored trials using validation loss, but estimate costs by interpolating measured default and full-search endpoints rather than measuring cumulative trial time.The trajectories compare aggregate performance with estimated relative workflow cost, not equal wall-clock budgets on each dataset.Source: §D.5 · p. 46

E ADDITIONAL DATASET-LEVEL AND CONFIGURATION RESULTS

  • The appendix reports dataset-level outcomes, configuration rankings, and predictive coverage across the evaluated methods.The dataset-level leaders use tuned supervised configurations or canonical GFM configurations; coverage identifies methods requiring GCN-Default replacements.Source: E ADDITIONAL DATASET-LEVEL AND… · p. 46

E.1 DATASET-LEVEL PREDICTIVE OUTCOMES

Graph foundation models lead some individual datasets, but supervised methods lead most of the benchmark and tuned supervised configurations rank ahead overall. The ranking shifts substantially with optimization, while coverage records where comparisons rely on imputation.

E.1 DATASET-LEVEL PREDICTIVE OUTCOMES

  • GFMs lead 15 datasets under 10/10/80 and 13 under 50/25/25, with GraphPFN accounting for 14 and 13 wins, respectively, while supervised methods collectively lead most datasets.GraphAny contributes one GFM win under 10/10/80; the reported leaders use each method’s selected tuned or canonical configuration.Source: §E.1 · p. 46, Table 22 · p. 47

E.2 CONFIGURATION-LEVEL RANKINGS

  • Tuned supervised methods occupy the first eight ranks under 10/10/80, with GraphPFN ninth; under 50/25/25, tuned GNNs take the first four positions and GraphPFN ranks fifth.Optimization substantially changes ordering: GPRGNN moves from default ranks 15 and 16 to rank 2 under both ratios, while GCNII moves from ranks 19 and 27 to rank 1.Source: §E.2 · pp. 47–48, Table 23 · p. 48

E.3 PREDICTIVE COVERAGE

  • Thirteen of the 21 methods have native results on all 51 datasets under both label ratios; the remaining eight require at least one GCN-Default replacement.A dataset counts as imputed if any of its five partitions is replaced.Source: §E.3 · p. 48, Table 24 · p. 48

Related papers

Loading 2609.39673v1…