PaperlayerView exact revision on arXiv

Source-linked AI summary

Bridging Academia and Industry: A Comprehensive Benchmark for Attributed Graph Clustering

Yunhui Liu, Pengyu Qiu, Yu Xing, Yongchao Liu, Peng Du, Chuntao Hong, Jiajun Zheng, Tao Zheng, Tieke He

arXiv:2602.08519v1cs.LG

TL;DR

AGC benchmarks remain disconnected from industrial conditions because they emphasize small, homophilous citation graphs, non-scalable full-batch training, and supervised metrics. PyAGC introduces a production-ready benchmark and library that unifies AGC methods, supports memory-efficient mini-batching, and evaluates diverse datasets with structural and efficiency measures. Its 12-dataset benchmark spans 2.7K to 111M nodes and is positioned as a reproducible platform for realistic deployment-oriented research.

  • Problem

    AGC evaluation is limited by small, high-homophily citation datasets, full-batch training, and supervised metrics that do not cover label-scarce industrial settings.

  • Method

    PyAGC unifies AGC methods under Encode-Cluster-Optimize and provides modular, memory-efficient mini-batch implementations with diverse benchmark data and holistic evaluation.

  • Results

    12 datasets spanning 2.7 × 10^3 to 1.1 × 10^8 nodes establish a benchmark spanning academic and industrial scales and structural conditions.

  • Takeaways & Limitations

    PyAGC provides a production-oriented, reproducible platform for evaluating AGC across supervised alignment, structural quality, and efficiency.

  • Takeaways & Limitations

    On Papers100M, deep methods are trained for a single epoch because of prohibitive computational cost.

Abstract

from arXiv · show

Attributed Graph Clustering (AGC) is a fundamental unsupervised task that integrates structural topology and node attributes to uncover latent patterns in graph-structured data. Despite its significance in industrial applications such as fraud detection and user segmentation, a significant chasm persists between academic research and real-world deployment. Current evaluation protocols suffer from the small-scale, high-homophily citation datasets, non-scalable full-batch training paradigms, and a reliance on supervised metrics that fail to reflect performance in label-scarce environments. To bridge these gaps, we present PyAGC, a comprehensive, production-ready benchmark and library designed to stress-test AGC methods across diverse scales and structural properties. We unify existing methodologies into a modular Encode-Cluster-Optimize framework and, for the first time, provide memory-efficient, mini-batch implementations for a wide array of state-of-the-art AGC algorithms. Our benchmark curates 12 diverse datasets, ranging from 2.7K to 111M nodes, specifically incorporating industrial graphs with complex tabular features and low homophily. Furthermore, we advocate for a holistic evaluation protocol that mandates unsupervised structural metrics and efficiency profiling alongside traditional supervised metrics. Battle-tested in high-stakes industrial workflows at Ant Group, this benchmark offers the community a robust, reproducible, and scalable platform to advance AGC research towards realistic deployment. The code and resources are publicly available via GitHub (https://github.com/Cloudy1225/PyAGC), PyPI (https://pypi.org/project/pyagc), and Documentation (https://pyagc.readthedocs.io).

1 Introduction

AGC research remains poorly aligned with industrial deployment because prevailing benchmarks emphasize small, homophilous citation graphs, full-batch methods, and supervised evaluation. PyAGC addresses these gaps with a scalable benchmark, unified methodology, diverse datasets, and holistic assessment.

  • AGC combines graph topology and node attributes to partition unlabeled nodes into latent groups.
  • Existing AGC evaluations rely heavily on small, textual, high-homophily citation datasets that differ from large, noisy, heterophilous industrial graphs with tabular features.
  • Full-batch matrix operations create a scalability bottleneck for methods intended for massive graphs.
  • Industrial deployment requires mini-batch scalability, robustness to heterogeneous features and structures, and unsupervised structural validation.
  • PyAGC stress-tests AGC across domains, scales, and structural properties using a unified Encode-Cluster-Optimize framework and industrially validated workflows.
  • 12 datasets span 2.7 × 10^3 to 1.1 × 10^8 nodes, while evaluation combines supervised metrics, unsupervised structural metrics, and efficiency profiling.

2 Preliminaries

The paper formalizes AGC as clustering nodes using both structural connectivity and attributes, then organizes methods into modular encoding, projection, and optimization components. The framework also captures the transition from full-batch objectives to mini-batch approximations for very large graphs.

  • 2.1 Problem Formalization: AGC learns a mapping from adjacency and attributes to K cluster assignments without ground-truth labels.
  • 2.2 The Encode-Cluster-Optimize Framework: The Encode-Cluster-Optimize framework decomposes AGC into representation encoding, cluster projection, and optimization strategy.
  • Representation Encoding: Encoders may be parametric GNNs that fuse topology and attributes or non-parametric filters that smooth features over graph structure.
  • Cluster Projection: Cluster projection maps latent embeddings to soft or hard assignments through differentiable or discrete post-hoc mechanisms.
  • Optimization Strategy: Optimization combines representation and clustering objectives, with training either decoupled into stages or performed jointly end to end.
  • Scalable Training: Mini-batch training approximates full-graph objectives on sampled subgraphs, trading sampling error against throughput and memory efficiency for graphs with hundreds of millions of nodes.

3 The PyAGC Library

PyAGC implements AGC as interchangeable encoders, cluster heads, and model-level optimization components, with unified evaluation and targeted speed and memory optimizations. These design choices support both small-graph full-batch processing and massive-graph mini-batch training.

  • 3.1 Modular Architecture: PyAGC decouples the clustering lifecycle into interchangeable modules so researchers can swap components without rewriting training loops.
  • Encoders: Encoder implementations support full-batch processing for small graphs and neighbor-sampling mini-batches for massive graphs.
  • Clusters: Standalone cluster heads provide consistent interfaces for differentiable pooling, prototypical assignment, and discrete partitioning.
  • Models: High-level model classes orchestrate encoder–cluster interactions and compute joint losses such as reconstruction plus clustering loss.
  • Unified Evaluation: Configuration-driven data loading, augmentation, metric computation, and experiment management support reproducible evaluation.
  • Efficiency Optimizations: GPU-accelerated clustering and subgraph-based mini-batch objectives reduce runtime and memory barriers for large-scale AGC.

4 Benchmark Setup

The benchmark evaluates AGC across diverse datasets, representative methods, scalable implementations, and supervised, unsupervised, and efficiency criteria. Its design emphasizes industrial scale, heterogeneous tabular attributes, varied graph structure, and reproducible resource-aware evaluation.

  • Dataset Curation: 12 datasets span five orders of magnitude, from canonical small graphs to Papers100M with over 111 million nodes and 1.6 billion edges.The atlas includes citation, social, e-commerce, and web graphs, with large-scale datasets designed to stress mini-batch strategies.
  • Dataset Curation: Industrial datasets HM, Pokec, and WebTopic introduce heterogeneous categorical and numerical attributes with skewed, multimodal, and noisy distributions.These tabular graphs move beyond clean textual features and reflect metadata such as user demographics and transaction counts.
  • Dataset Curation: The atlas covers a broad homophily-heterophily spectrum, including Physics with edge homophily H_e=0.93 and HM and WebTopic with H_e<0.25.This structural diversity tests robustness beyond highly homophilous academic graphs.
  • Evaluated Representative Algorithms: The evaluation covers 17 representative AGC methods spanning traditional, non-parametric, deep decoupled, and deep joint approaches.Methods are organized across the Encode-Cluster-Optimize framework, whose modules represent encoding, cluster projection, and optimization strategy.
  • Scalable Adaptation: All deep learning algorithms are reimplemented with neighbor sampling and mini-batch training, enabling methods such as DAEGC to run on 100M+ node graphs.Standard implementations otherwise produce out-of-memory errors on medium, large, and massive datasets.
  • Holistic Evaluation Protocol: The protocol combines supervised alignment metrics, unsupervised structural metrics, and efficiency profiling of training time, clustering time, and peak GPU memory.Structural evaluation uses modularity and conductance, while supervised reporting retains ACC, F1, NMI, and ARI.

5 Benchmark Results

The benchmark exposes a sharp generalization gap between small, homophilous academic graphs and heterogeneous industrial graphs, while evaluating structural quality and deployment efficiency alongside label alignment. Mini-batch PyAGC implementations enable memory-efficient training on massive graphs, but method choice involves speed–scalability trade-offs.

  • Generalization Gap: Most methods perform strongly on small, high-homophily datasets, where Node2Vec and NS4GC both exceed 65% NMI on Photo.These datasets make clustering relatively easy because structure and attributes are highly correlated.
  • Generalization Gap: 38.33% NMI is achieved by the best method, SAGSC, on Pokec, while most deep methods fail to surpass 5% NMI.The decline is associated with heterophily, noisy tabular features, and over-smoothing or insufficiently expressive representations.
  • Generalization Gap: Deep decoupled methods such as NS4GC, MAGI, and S3GC show the most consistent robustness by separating representation learning from clustering.Stable self-supervised pre-training helps avoid the trivial solutions associated with joint-training clustering collapse.
  • Metric Paradox: 84.56% Modularity and 12.62% Conductance make SAGSC structurally strongest on Products, despite NS4GC achieving higher NMI at 54.63%.The result demonstrates that label alignment and topological community quality can diverge.
  • Metric Paradox: 12.54% Modularity makes DMoN the strongest structural method on HM despite mediocre supervised scores, showing why label metrics alone can miss useful partitions.Standard GNN methods produce low or negative Modularity on this highly heterophilous dataset.
  • Scalability and Efficiency: ≈14GB of GPU memory is sufficient for NS4GC on Papers100M, whereas the full-batch equivalent would require terabytes of RAM.PyAGC also trains GAE and NS4GC on 111M-node Papers100M in 0.67 and 1.24 hours per epoch, respectively.
  • Scalability and Efficiency: 0.16 minutes is achieved by MS2CAG on Reddit, while deep mini-batch methods trade sampling overhead for linear scalability to massive graphs.This supports non-parametric solvers for rapid medium-scale prototyping and mini-batch deep models for massive deployment.

6 Conclusion and Future Work

The paper presents PyAGC as a production-ready benchmark library that organizes AGC under Encode-Cluster-Optimize and evaluates methods across realistic scales and conditions. It identifies robust heterophily handling and unsupervised model selection as priorities for future work.

  • Conclusion: PyAGC systematizes AGC under the Encode-Cluster-Optimize framework and benchmarks 17 algorithms across 12 datasets.The benchmark is intended to connect academic evaluation with industrial application.
  • Future Work: Future work should develop encoders robust to heterophily and tabular noise without over-smoothing.The paper frames this as a need to move beyond the “Coraverse.”
  • Future Work: Reliable unsupervised model-selection criteria remain an urgent open problem for label-scarce industrial applications.The conclusion specifically highlights this need for realistic deployment.

A Dataset Descriptions

The benchmark’s 12 datasets span four domains and two feature modalities, with scale-based categories designed to evaluate AGC across diverse data conditions.

  • Dataset Scope: The dataset atlas covers Citation, Co-purchase, Social, and Web domains.This domain diversity broadens evaluation beyond citation networks.
  • Dataset Scope: Feature modalities include Textual and Tabular data, complementing the domain categories.The datasets are categorized by scale according to the benchmark’s taxonomy.

A.1 Tiny Scale Datasets (𝑁< 104)

Tiny-scale datasets include standard academic and co-purchase graphs whose nodes and edges represent papers, products, and their relationships, with attributes supporting topic or item clustering.

  • Cora: Cora represents papers as nodes, citations as edges, and sparse bag-of-words attributes, with seven research-topic clusters.It is a standard citation network in graph clustering literature.
  • Photo: Photo is an Amazon co-purchase graph whose nodes are products and edges connect frequently co-purchased goods.The passage introduces Photo as another tiny-scale benchmark dataset.

A.2 Small Scale Datasets (104 ≤𝑁< 105)

The small-scale dataset descriptions include a co-authorship graph with textual node features and an e-commerce co-purchasing graph.

  • A.2 Small Scale Datasets (104 ≤𝑁< 105): Physics is a co-authorship graph whose nodes are authors and whose edges connect authors who co-authored papers.Its node features are bag-of-words representations of paper keywords, and authors are clustered into five research fields.
  • A.2 Small Scale Datasets (104 ≤𝑁< 105): Physics represents node attributes with bag-of-words features derived from paper keywords.
  • A.2 Small Scale Datasets (104 ≤𝑁< 105): HM is derived from the H&M Kaggle competition and represents a co-purchasing network of products.The passage states that undirected edges connect products frequently bought together.

A.3 Medium Scale Datasets (105 ≤𝑁< 106)

The medium-scale datasets cover citation, social, and tabular-feature settings, including subject-area and region-oriented clustering tasks.

  • A.3 Medium Scale Datasets (105 ≤𝑁< 106): ArXiv is a citation network of Computer Science papers with 128-dimensional title-and-abstract features and 40 subject-area clusters.
  • A.3 Medium Scale Datasets (105 ≤𝑁< 106): Reddit contains posts made in September 2014, with ground-truth labels representing post communities.
  • A.3 Medium Scale Datasets (105 ≤𝑁< 106): Pokec is a social-network graph with tabular user-profile features and a region-clustering task involving 183 classes.Its features include numerical values and categorical indicators derived from profile information.

A.5 Massive Scale Datasets (𝑁> 108)

The massive-scale benchmark combines very large citation data with a broader dataset atlas and a structured taxonomy of AGC methods and computational complexity.

  • A.5 Massive Scale Datasets (𝑁> 108): Papers100M is a citation network of approximately 111 million papers, with 128-dimensional title-and-abstract features and 172 subject-area clusters.Approximately 1.5 million nodes have labels from arXiv subject areas.
  • A.5 Massive Scale Datasets (𝑁> 108): The benchmark spans five orders of magnitude, from Tiny to Massive, across diverse domains and textual or tabular feature types.Table 4 identifies edge homophily and node homophily as benchmark statistics.
  • A.5 Massive Scale Datasets (𝑁> 108): The ECO taxonomy classifies AGC methods by encoder type, clusterer type, optimization strategy, and complexity.It distinguishes parametric from non-parametric encoders, differentiable from discrete clusterers, and joint from decoupled optimization.
  • A.5 Massive Scale Datasets (𝑁> 108): The benchmark method descriptions are organized according to the Encode-Cluster-Optimize framework.
  • A.5 Massive Scale Datasets (𝑁> 108): Non-parametric methods use fixed or simplified encoders and separate clustering from representation learning.
  • A.5 Massive Scale Datasets (𝑁> 108): Traditional baselines include KMeans on raw node features and Node2Vec on structural embeddings.These baselines assess attribute-only and topology-only representations, respectively.

B.3 Deep Joint Methods

Deep joint methods optimize encoders and cluster assignments end-to-end, and their evaluation compares clustering quality with memory and runtime efficiency.

  • B.3 Deep Joint Methods: Deep joint methods optimize the encoder and cluster assignments simultaneously in an end-to-end fashion.
  • B.3 Deep Joint Methods: Table 6 compares clustering performance using ARI and F1 percentages reported as mean ± SD.Best and second-best results are highlighted, while an em dash denotes out-of-memory errors from full-graph processing.
  • B.3 Deep Joint Methods: DinkNet reports ARI values across twelve dataset columns, ranging from 0.29±0.02 to 53.54±0.23.The row also reports F1 values ranging from 1.65±0.01 to 70.12±0.59.
  • B.3 Deep Joint Methods: MinCut reports ARI values ranging from 0.61±0.04 to 50.99±2.33 across the listed datasets.Its corresponding F1 values range from 2.30±0.05 to 66.99±2.97.
  • B.3 Deep Joint Methods: Neuromap reports ARI values ranging from 1.45±0.07 to 53.01±3.67 across the listed datasets.Its corresponding F1 values range from 2.62±0.14 to 69.13±3.76.
  • B.3 Deep Joint Methods: Efficiency profiling records peak GPU memory and total training-plus-clustering time for smaller datasets.
Loading 2602.08519v1…