PaperlayerView exact revision on arXiv

Source-linked AI summary

Learn Beneficial Noise as Graph Augmentation

Siqi Huang, Yanchen Xu, Hongyuan Zhang, Xuelong Li

arXiv:2505.19024v1cs.LGstat.ML

TL;DR

Graph contrastive learning lacks reliably stable augmentations because heuristic perturbations can disrupt graph structure. PiNGDA learns beneficial noise for topology and attributes through a trainable generator, using a Gaussian auxiliary variable to connect GCL loss with task entropy. Across graph tasks, the method improves performance and stability, while its analysis and experiments remain focused on stated node-level and graph-level settings and a diagonal Gaussian attribute-noise assumption.

  • Problem

    Graph contrastive learning needs effective, stable augmentations because heuristic perturbations may disrupt crucial graph structure and predefined augmentations may not reduce task entropy.

  • Method

    PiNGDA uses a trainable pi-noise generator for topology and attributes, with a Gaussian auxiliary variable linking GCL training loss to information entropy.

  • Results

    PiNGDA improves performance and stability across graph tasks, with high accuracy, lower variance, and top or near-top graph-classification performance across datasets.

  • Takeaways & Limitations

    Learning beneficial noise provides a more adaptive graph-augmentation approach than simple predefined estimation, covering both topology and node attributes.

  • Takeaways & Limitations

    The analysis focuses on node-level GCL while extending to graph-level and hybrid models, and attribute noise assumes an uncorrelated diagonal covariance matrix.

Abstract

from arXiv · show

Although graph contrastive learning (GCL) has been widely investigated, it is still a challenge to generate effective and stable graph augmentations. Existing methods often apply heuristic augmentation like random edge dropping, which may disrupt important graph structures and result in unstable GCL performance. In this paper, we propose Positive-incentive Noise driven Graph Data Augmentation (PiNGDA), where positive-incentive noise (pi-noise) scientifically analyzes the beneficial effect of noise under the information theory. To bridge the standard GCL and pi-noise framework, we design a Gaussian auxiliary variable to convert the loss function to information entropy. We prove that the standard GCL with pre-defined augmentations is equivalent to estimate the beneficial noise via the point estimation. Following our analysis, PiNGDA is derived from learning the beneficial noise on both topology and attributes through a trainable noise generator for graph augmentations, instead of the simple estimation. Since the generator learns how to produce beneficial perturbations on graph topology and node attributes, PiNGDA is more reliable compared with the existing methods. Extensive experimental results validate the effectiveness and stability of PiNGDA.

1. Introduction

GCL needs graph augmentations that preserve crucial topology, but heuristic and predefined perturbations can be unreliable. PiNGDA addresses this by theoretically learning beneficial noise for topology and attributes, with experiments showing improved performance and stability.

  • Graph augmentation remains difficult because non-Euclidean topology complicates stable perturbations that retain crucial structural properties.
  • Random edge and node dropping increases diversity but may disrupt inherent graph structure, while learning-based methods often select predefined operations or focus on edge modifications.
  • Pi-noise is defined as beneficial noise that reduces task complexity, enabling learnable perturbations on both graph topology and node attributes.
  • A Gaussian auxiliary variable related to the GCL loss connects the pi-noise framework with GCL and interprets predefined augmentation as point estimation of pi-noise.
  • The analysis finds predefined augmentations may be unreliable estimates of pi-noise, motivating PiNGDA's pi-noise generator for topology and attributes.
  • PiNGDA uses an efficient differentiable noise-generation algorithm and improves GCL performance and stability over baselines in extensive experiments.

2. Related Work

Prior graph augmentation methods create diverse views but struggle with graph complexity, motivating adaptive approaches and a positive-impact noise perspective. PiNGDA frames augmentation through task entropy and learns noise intended to reduce task uncertainty.

  • Traditional node dropping, edge perturbation, and feature masking create diverse graph views but struggle with inherent graph-topology complexity.
  • Graph structure learning methods modify graph structure for specific goals but differ from heuristic edge dropping.
  • Noise can improve performance, motivating contrastive-learning approaches that introduce diverse transformations across domains.
  • PiNGDA combines a pi-noise generator with contrastive learning, jointly perturbing graph structures and node features.
  • The paper defines pi-noise as noise whose additional components reduce task uncertainty after noise is introduced.

3. π-Noise Driven Graph Data Augmentation

The paper reframes GCL augmentation through π-noise by converting training loss into task entropy and showing predefined augmentations are point estimates of beneficial noise. PiNGDA instead learns π-noise distributions for topology and attributes, aiming to provide more stable graph augmentations.

  • 3.2. A General Framework to Bridge π-Noise and Training Loss: A Gaussian auxiliary variable converts GCL loss into information entropy, making task complexity measurable through the contrastive loss.The construction uses a monotonically increasing mapping from loss to the variance of a Gaussian distribution; lower loss corresponds to lower entropy and an easier task.
  • 3.3. Rethink GCL under π-Noise Framework: Standard GCL with predefined augmentation is equivalent to optimizing with a point estimation of π-noise.Under this view, the fixed augmentation acts as a strongly definite noise hypothesis, represented by a Dirac delta distribution.
  • 3.2. A General Framework to Bridge π-Noise and Training Loss: The framework models π-noise as a distribution over augmentations and estimates conditional entropy with Monte Carlo sampling over graph nodes.The augmented positive and negative losses are defined using views generated by noise ε, with p(ε|u) representing the π-noise distribution to learn.
  • 3.3. Rethink GCL under π-Noise Framework: Heuristic graph augmentations may be poor point estimates of π-noise, causing instability in GCL with heuristic augmentations.The paper contrasts this instability with the lack of stable methods for generating augmented graphs that preserve reliable topology and attributes.
  • 3.4. Loss of π-Noise Driven Data Augmentation: PiNGDA learns π-noise by optimizing the π-noise objective rather than randomly editing edges or nodes.The method is compatible with existing GCL models and represents noise through both topological and attribute components.
  • 3.4. Loss of π-Noise Driven Data Augmentation: PiNGDA’s loss samples noise from a trainable distribution and uses the resulting augmented embedding as a positive sample against other nodes as negatives.The contrast model parameters are denoted by θ, and similarity is computed with cosine similarity in this paper.

4. Implementation Details of Topological Noise and Attribute Noise

PiNGDA learns differentiable noise generators for both graph topology and node attributes. Topological noise uses node-pair edge-drop probabilities with Gumbel-Softmax, while attribute noise uses a conditioned Gaussian with reparameterization.

  • Topological Noise: Topological noise models edge dropping with a factored distribution over edges incident to each node.The distribution p(εedge|u) determines edge-dropping probabilities for node u.
  • Topological Noise: A learnable module gψ(u, v) parameterizes each edge-deletion probability using a Bernoulli distribution over node pairs.The module is implemented as a two-layer MLP processing concatenated node features.
  • Topological Noise: Gumbel-Softmax provides a differentiable approximation to Bernoulli sampling for discrete edge-dropping decisions.This preserves gradient flow through the topology perturbation process.
  • Attribute Noise: Attribute noise is modeled as a Gaussian distribution conditioned on node attributes, with learnable mean and covariance parameters.An MLP takes node attributes as input to learn the distribution parameters.
  • Attribute Noise: The attribute covariance is assumed diagonal, reducing parameters from d2 to d, and reparameterization makes sampling differentiable.Standard normal noise is transformed using the learned distribution before being added to node attributes.

5. Experiments

Experiments evaluate PiNGDA across node, graph, and heterogeneous graph classification, augmentation strategies, generalization, efficiency, and learned-noise behavior. Results report high accuracy and stability, broad competitiveness, and advantages from learning noise over topology and attributes.

  • Experimental setup: Experiments evaluate PiNGDA against supervised, self-supervised, and learnable graph representation methods across multiple classification settings.Node classification uses linear evaluation with 20 random data splits; comparisons include GCN, GAT, GCL methods, and AD-GCL.
  • Node classification results: PiNGDA maintains high accuracy with lower variance across diverse node-classification graphs and generally outperforms AD-GCL by augmenting topology and node attributes.AD-GCL learns only topology, whereas PiNGDA uses both topology and attributes; PiNGDA is also nearly as accurate as GCN on ogbn-arxiv while avoiding the reported out-of-memory issues affecting several methods.
  • Graph classification results: PiNGDA achieves top or near-top performance across five graph-classification datasets, with stronger generalization on MUTAG and RDT-B while AD-GCL is slightly better on NCI1 and PROTEINS.Both learnable methods outperform the other baselines across most datasets, while PiNGDA shows balanced performance across tasks.
  • Heterogeneous graph results: PiNGDA consistently surpasses other methods on Texas and Cornell and remains competitive on Wisconsin, indicating generalization across heterogeneous graph structures.The reported results rank PiNGDA closely behind the top-performing approach on Wisconsin.
  • Augmentation ablations: Learnable augmentation outperforms random augmentation, which outperforms no augmentation across most datasets, although edge and feature effects vary by dataset.Edge augmentation helps more than feature augmentation on Cora and PubMed, while WikiCS illustrates that modifying edges or features can have dataset-dependent effects.
  • Generalization and analysis: PiNGDA improves GRACE and Sp2GCL across datasets while offering an accuracy–efficiency trade-off through lower memory use and runtime comparable to other methods.The learned topological noise tends to remove inter-class edges while preserving intra-class connections, and its learned weights represent connection strength.

6. Conclusion

PiNGDA learns adaptive positive-incentive noise for graph contrastive learning, while framing standard GCL through task entropy. The method is reported to improve performance across graph tasks and offers directions for alternative noise distributions.

  • 6. Conclusion: PiNGDA learns adaptive positive-incentive noise to address instability in traditional graph augmentations.The approach uses a noise generator to learn beneficial perturbations rather than relying on fixed augmentation choices.
  • 6. Conclusion: Task entropy provides an information-theoretic view in which standard GCL is an approximation under the positive-incentive-noise optimization framework.
  • 6. Conclusion: PiNGDA achieves performance improvements across various graph tasks.
  • 6. Conclusion: Future work may investigate alternative ways to incorporate positive-incentive noise with diverse noise distributions.

A. Derivation of Eq. (12)

The derivation expands a Gaussian density, substitutes it into the loss, and simplifies the result while separating out parameter-independent constants.

  • A. Derivation of Eq. (12): The derivation expands the density of N(0, κθ(u, ε)−1) before substituting it into L.
  • A. Derivation of Eq. (12): The intermediate derivation details are provided in Appendix A.
  • A. Derivation of Eq. (12): Substitution into L simplifies the loss expression.
  • A. Derivation of Eq. (12): The derivation treats C as a constant independent of learnable parameters and uses an additional stated identity in its final step.

B.1. Datasets

The experiments use seven benchmark datasets for semi-supervised node classification and TUDataset benchmarks for semi-supervised graph classification, with dataset statistics reported in Tables 6 and 7.

  • B.1. Datasets: Seven benchmark datasets are used for semi-supervised node classification: Cora, Citeseer, Pubmed, Wiki-CS, Amazon-Photo, Coauthor-CS, and ogbn-arxiv.Nodes are randomly split into 10% training, 10% validation, and 80% test sets.
  • B.1. Datasets: Semi-supervised graph classification is evaluated on the benchmark TUDataset.Its detailed dataset statistics are summarized in Table 7.
  • B.1. Datasets: Table 6 reports statistics for the datasets used in node classification experiments.

B.2. Implementation Details

Implementation uses GCN-based encoders and separate trainable generators for edge and feature noise, with experiments run on NVIDIA 4090 and A100 GPUs.

  • B.2. Implementation Details: Most datasets are run on an NVIDIA 4090 GPU, while OGB-arxiv uses an NVIDIA A100 GPU.The reported memories are 24 GB for the 4090 and 40 GB for the A100.
  • B.2. Implementation Details: The proposed method uses a two-layer GCN with PReLU activation, a 512-dimensional hidden layer, and a 256-dimensional final embedding.
  • B.2. Implementation Details: The edge noise generator processes node features with an MLP and uses Gumbel-Softmax sampling, while the feature generator estimates noise mean and variance with two MLPs and reparameterization.

B.3. Ablation Study on Graph Classification

The ablation study examines feature and edge augmentation across PROTEINS, DD, and NCI1, finding that augmentation effectiveness varies with dataset-specific node-attribute semantics. In some cases, neither random nor learnable perturbations alone captures the optimal pattern.

  • Table 8 reports classification accuracy for feature and edge augmentation methods, highlighting the highest value within each dataset row and column.
  • Augmentation effectiveness varies across PROTEINS, DD, and NCI1 because node attributes play different semantic roles in each dataset.
  • In some cases, neither random nor learnable perturbations alone fully captures the optimal augmentation pattern.

B.4. Hyperparameter Analysis

The hyperparameter analysis varies optimization and augmentation settings across datasets and evaluates temperature and hidden-dimension sensitivity. PiNGDA is not very sensitive to temperature or hidden dimension, while smaller graphs benefit from lower temperature.

  • Dataset-specific hyperparameters are summarized in Table 9, with separate learning rates for edge and feature noise generators.The learning rates are 0.0001 for edges and 0.001 for features, with weight decay of 0.0001 for both.
  • The grid search varies training epochs, random edge-dropping rates, and random feature-masking rates.Epochs range from 500 to 3000; edge-dropping rates range from 0.1 to 0.6; feature-masking rates are 0.1, 0.3, or 0.5.
  • For each dataset, hyperparameters are selected to obtain the best average accuracy.
  • PiNGDA is not very sensitive to temperature or hidden dimension.For small graphs, smaller temperature leads to better performance.
Loading 2505.19024v1…