PaperlayerView exact revision on arXiv

Source-linked AI summary

From Euclidean to Graph-Structured Data: A Survey of Collaborative Learning

Rémi Bourgerie, Šarūnas Girdzijauskas, Viktoria Fodor

arXiv:2609.02984v1cs.LGcs.MAcs.SI

TL;DR

Fragmented, privacy-constrained data and the limited study of collaborative learning for relational graphs motivate this survey. It maps established Euclidean collaborative-learning principles to graph settings, introducing distribution taxonomies and organizing challenges around effectiveness, efficiency, and privacy. The survey finds a nascent but rich graph-learning landscape in which collaborative inference is especially central because GNNs rely on message passing, while joint topology, communication, privacy, and heterogeneity create unique constraints.

  • Problem

    Data access is fragmented by privacy, ownership, integration, and coordination constraints, while collaborative learning on graph-structured data lacks a systematic intersection-focused analysis.

  • Method

    The survey maps collaborative learning from Euclidean to graph-structured data, introducing graph distribution taxonomies and analyzing effectiveness, efficiency, privacy, and heterogeneity.

  • Results

    Collaborative inference is more central for graph-structured data, where distributed relational data jointly introduce constraints involving topology, communication, privacy, and heterogeneities.

  • Takeaways & Limitations

    The survey provides a coherent framework for developing collaborative graph-learning methods across scientific and industrial domains.

  • Takeaways & Limitations

    GNN convergence and generalization remain poorly understood, and topologically distinct agents can experience fundamentally different optimization landscapes.

Abstract

from arXiv · show

The conventional approach to machine learning, that is, collecting data, training models, and performing inference in a single location, faces fundamental limitations, including scalability and privacy, that restrict its applicability. To address these challenges, recent research has explored collaborative learning approaches, including federated learning and decentralized learning, where individual agents perform training and inference locally, with limited collaboration. Most collaborative learning research focuses on Euclidean data with regular, grid-like structure (e.g., images, text). However, these approaches fail to capture the relational patterns in many real-world applications, best represented by graphs. Learning on graphs relies on message-passing mechanisms to propagate information between connected nodes, making it conceptually well-suited for collaborative environments where agents must exchange information. Yet, the opportunities and challenges of learning on graph-structured data in collaborative settings remain largely underexplored. This survey provides a comprehensive investigation of collaborative learning from Euclidean to graph-structured data, aiming to consolidate this emerging field. We begin by reviewing its foundational principles for Euclidean data, organizing them along three core dimensions: learning effectiveness, efficiency, and privacy preservation. We then extend the discussion to graph-structured data, introducing a taxonomy of graph distribution scenarios, characterizing associated statistical heterogeneities, and developing standardized problem formulations and algorithmic frameworks. Finally, we systematically identify open challenges and promising research directions.

1 Introduction

Collaborative learning addresses fragmented, privacy-constrained data by enabling agents to train shared models locally, but graph-structured settings introduce distinct distribution and communication challenges. The survey connects collaborative learning, graph representation learning, and network diffusion while organizing solutions around effectiveness, efficiency, and privacy.

  • Access to massive datasets remains difficult because privacy legislation, ownership, integration complexity, and multi-agent coordination fragment data ecosystems.
  • Collaborative learning exchanges model updates or aggregated information instead of entire datasets, keeping private data at its source.
  • Graph learning involves either independent graph instances treated as samples or partitions of a shared global graph requiring inter-agent information exchange.
  • Graph settings amplify distributed-learning challenges because MPNNs iteratively propagate messages across agent-held partitions of a shared topology.
  • The survey bridges Euclidean and graph-structured collaborative learning through three pillars: network diffusion, graph representation learning, and collaborative learning.
  • Its review organizes approaches around learning effectiveness, communication and computation efficiency, and privacy preservation, then introduces graph-data partition taxonomies and open questions.

2 Collaborative Learning on Euclidean Data

Collaborative learning keeps local data private while coordinating model training across agents, distinguishing it from distributed parallel computing with centralized data. For Euclidean data, the section formulates local empirical learning, explains FedAvg's round-based aggregation, and highlights privacy vulnerabilities.

  • Collaborative learning scope: Collaborative learning distributes training across agents while local data remains private, unlike parallel computing, which assumes centralized data.It includes federated and decentralized learning approaches.
  • Problem formulation: Because local datasets may be too small to estimate their underlying distributions accurately, agents collaborate under an assumption of shared distributional similarities.Aggregation allows agents to benefit from knowledge across the union of local datasets.
  • Problem formulation: The Euclidean collaborative-learning objective is commonly approximated with local empirical losses because population losses are uncomputable, assuming the generalization gap is negligible.Models are typically parameterized by neural-network weights θ.
  • Federated learning framework: FedAvg uses repeated communication rounds in which selected agents receive a global model, perform local SGD updates, and return models for server-side averaging.The averaged global model is redistributed for the next round, and the final model supports independent local inference.
  • Privacy vulnerabilities: Keeping data private does not eliminate leakage: model parameters, local updates, and the temporal evolution of aggregated models can reveal training information.Reported attacks include model inversion, gradient inversion, reconstruction, temporal inference, and active manipulation of contributions.

The Collaborative Learning Trilemma

Collaborative learning must balance effective learning, efficient learning, and privacy preservation. These competing objectives form the trilemma that guides subsequent design analysis.

  • Collaborative learning balances effective learning, efficient learning, and privacy preservation as three competing objectives.Effective learning concerns accurate predictions, efficient learning constrains communication and computation, and privacy preservation anonymizes agents’ data.
  • The trilemma concerns trade-offs among effectiveness, efficiency, and privacy in collaborative learning design.

2.3 Improving Learning Effectiveness

FedAvg offers convergence guarantees under ideal i.i.d. local data, but statistical heterogeneity and uneven participation can make it suboptimal. The survey organizes effectiveness improvements into data-based interventions that reduce client drift through regularization, augmentation, and client selection.

  • Motivation: FedAvg has strong convergence guarantees when local datasets are i.i.d., but real-world deployments often violate this assumption.The gap between the idealized setting and real deployments motivates alternative effectiveness strategies.
  • Data-based techniques: Client drift arises when heterogeneous data or unbalanced participation increases local-model variance, causing the averaged model to depart from the optimum.Data-based methods retain the shared global-model paradigm while modifying FedAvg.
  • Local regularization: Local regularization reduces discrepancies among local updates through gradient control, selective updates, objective modification, or prototype-based embedding alignment.These approaches target gradient variance, update noise, objective divergence, or representation alignment.
  • Data augmentation: Data augmentation addresses quantity, feature, and label shifts, as well as missing labels, by modifying local training datasets.Server-instructed methods redistribute samples, down-sample overrepresented classes, or generate synthetic samples for agents.
  • Client selection: Client selection addresses statistical imbalance by changing which agents participate, using strategies for total or partial availability.Methods include prioritizing underperforming or accuracy-contributing clients and modeling cyclical or Markovian participation.
  • Trade-offs: Data-based interventions trade effectiveness gains from variance reduction against resource and privacy costs that depend on the intervention.Regularization may require correction terms, augmentation may require generative models or redistribution, and sharing-based techniques require additional protection.

2.4 Improving Learning Efficiency

Collaborative learning efficiency depends on reducing communication and computation costs while preserving convergence. The survey organizes solutions by aggregation topology and synchronization strategy, covering centralized, decentralized, hierarchical, and asynchronous methods.

  • Efficient learning seeks fast convergence despite constrained communication, computation, network topology, and heterogeneous devices.
  • Centralized aggregation: Centralized aggregation collects local parameters after E local epochs, combines them at a server, and redistributes the resulting model.
  • Centralized aggregation: One-shot aggregation minimizes communication but assumes statistically similar agent datasets, whereas local SGD periodically aggregates after multiple local updates.
  • Decentralized aggregation: Decentralized aggregation removes the central coordinator, using distributed averaging, gossip, or dual-based techniques over agent communication networks.
  • Hierarchical aggregation: Hierarchical aggregation localizes operations through edge servers, conserving communication resources, reducing time, supporting personalization, and limiting data traffic to administrative or social groups.
  • Asynchronous aggregation: Asynchronous schemes address stragglers; bounded staleness preserves synchronous convergence rate and accuracy, although delays can reduce practical convergence speed.

2.5 Privacy-preserving Collaborative Learning

Privacy-preserving collaborative learning addresses leakage from exchanged computations and evolving global models through cryptographic blind computation and differential privacy. These protections introduce a trilemma involving learning effectiveness, system efficiency, and privacy.

  • Keeping data local does not provide formal privacy guarantees because intermediate computations and global model parameters may leak private information.
  • Differential privacy: Differential privacy provides formal statistical bounds by randomizing released aggregate statistics and limiting an adversary’s ability to distinguish sample inclusion.
  • Differential privacy: Common DP mechanisms include data subsampling and shuffling, per-sample gradient clipping or quantization, and calibrated noise scaled to (ε, δ).
  • Differential privacy: Collaborative learning naturally motivates agent-level DP because the global dataset is partitioned across agents and neighboring datasets differ by one agent’s entire local dataset.
  • Privacy mechanisms: Local DP protects data from an honest-but-curious aggregator but has high communication costs, while distributed DP combines local mechanisms with server-side amplification.
  • Blind computation uses TEE, HE, or SMPC to prevent adversaries from accessing intermediate values during collaborative computation.
  • Privacy protection trades off learning effectiveness and system efficiency through noise, clipping bias, cryptographic computation, communication overhead, and reduced robustness to asynchronicity.

3 Collaborative Learning on Graph-structured Data

The survey extends collaborative learning from Euclidean data to graph-structured data, where distinctive graph characteristics require adapted formulations and solutions. It first reviews graph learning before applying the Euclidean analytical structure to the graph domain.

  • The graph extension begins with an overview of state-of-the-art graph learning solutions, then formulates collaborative learning and analyzes effectiveness, efficiency, and privacy in the graph domain.

3.1 Machine Learning on Graphs and GNNs

Machine learning on graphs models relational data for node-, edge-, and graph-level tasks, commonly using GNNs and especially MPNNs. These models encode neighborhoods through iterative, permutation-compatible message passing and produce task-specific representations.

  • Graphs represent entities and relationships with irregular structure, and graph samples may include adjacency matrices, node features, and task-specific labels.
  • Graph representations must account for variable neighborhood sizes and the absence of canonical node ordering through permutation-equivariant or permutation-invariant mappings.
  • GNNs learn node encoders, predictors, and optional link or graph encoders for node-level, link-level, or graph-level prediction.
  • Message Passing Neural Networks: Modern spatial GNNs predominantly use MPNNs, which aggregate information from each node’s L-hop ego-graph through iterative message computation and state updates.
  • Message Passing Neural Networks: The MPNN framework is modular: choices of message, aggregation, and update functions define architectures such as GCN, GraphSAGE, and GAT.
  • Spectral and spatial GNNs: Spectral GNNs filter graph signals using Laplacian eigendecomposition, but practical polynomial approximations reduce computational cost and induce multi-hop information propagation.
  • Message Passing Neural Networks: MPNN inference operates on ego-graphs through hierarchical computational trees, and the operations can be parallelized across batches of nodes.

3.2 Collaborative Learning on Graph-structured Data Problem Definition

Collaborative graph learning distinguishes learning from multiple graph instances from learning over subgraphs of one global graph. In the subgraph setting, message passing, inference, and optimization require exchanges of intermediate information across agents.

  • Problem settings: Graph learning is divided into multiple graph instances and subgraphs of a shared global graph.Multiple graph instances support graph-level tasks such as molecular property prediction, whereas subgraph learning covers partial views of one global graph.
  • Multiple graph instances: For multiple graph instances, local objectives have the same form as Euclidean federated learning objectives, so FedAvg applies when distributions are i.i.d.Each agent holds graphs sampled from a local distribution and estimates its local objective.
  • Subgraphs: In the subgraph setting, each agent’s objective uses its local graph expanded by the target node’s L-hop neighborhood in the global graph.The expanded topology is denoted ˜Gk and contains information needed by the GNN for message passing.
  • Subgraphs: Agents lacking the expanded neighborhood cannot minimize their local loss, making collaboration among agents holding parts of that neighborhood necessary.This requirement motivates collaborative inference for nodes whose message-passing neighborhoods cross partition boundaries.
  • Algorithmic framework: A unified subgraph-GNN procedure broadcasts model parameters, exchanges boundary-node embeddings during collaborative inference, and aggregates node representations and local updates.Isolated inference uses only local subgraphs, while collaborative inference iteratively exchanges foreign boundary-node embeddings across the L MPNN layers.

3.3 Graph Data Partition, Heterogeneity and Privacy Challenges

Graph partitioning and heterogeneity determine what agents observe and must exchange. The survey categorizes partitions across graph instances and global-graph subgraphs, while highlighting structural shifts and privacy risks specific to collaborative graph learning.

  • Partition taxonomy: The survey organizes graph partitions around two cases: agents holding multiple graph instances or subgraphs of a single global graph.This taxonomy adapts Euclidean partition concepts to graph topology, node and relation types, features, and labels.
  • Multiple graph instances: Multiple graph instances may be horizontally partitioned across unrelated samples or vertically partitioned across agents sharing the same samples.Vertical topology partitions can represent different modalities for one graph instance, but this setting had not yet been implemented in distributed training.
  • Subgraph partitions: Subgraph partitions include horizontal topology partitions, edge cuts, and vertex cuts, with agents observing different node or edge subsets and possibly foreign boundary nodes.Topology partitioning can also distinguish ego-graph and larger subgraph partitions according to the observed subgraph size.
  • Statistical heterogeneity: Graph-specific heterogeneity includes structural shifts in topology distributions, including differences in graph size, edge density, and motifs across agents.Structural shifts may arise initially or when agents or nodes with distinct structures or labels arrive.
  • Privacy: Shared node embeddings can leak private node features, reconstruct edges, or reveal whether nodes and edges participated in training.These vulnerabilities remain even when raw node features and distributed message-passing features are not communicated.

3.4 Improving Inference and Learning Effectiveness

The survey reviews data- and model-based techniques for improving collaborative GNN effectiveness, including graph augmentation, regularization, and representation alignment. It also identifies topology-dependent optimization and foundation-model integration as open challenges.

  • Effectiveness goals: Effectiveness methods target tighter convergence to a global collaborative objective or to agents’ local optima despite graph partitioning and statistical heterogeneity.The survey frames effectiveness challenges through generalization gaps, data isolation, and heterogeneous graph data.
  • Data-based techniques: Data augmentation addresses generalization, statistical heterogeneity, and data isolation by modifying graph instances or adding new graph instances.Approaches include graph generation, contrastive perturbations, node dropping, edge perturbation, attribute masking, and subgraph sampling.
  • Graph manipulation: Graph-specific instance manipulations include L-hop recovery, node rewiring, and virtual-node addition, but L-hop recovery can require cryptographic primitives and become expensive.The expansion may therefore be restricted to a 1-hop neighborhood.
  • Feature and label augmentation: Feature and label augmentation reduces graph-specific heterogeneity through neighbor-feature averaging, synthetic features, virtual edges, and label smoothing.These methods target homophily bias, feature imbalance, and enhancement of local subgraphs in vertex-cut partitions.
  • Model-based techniques: Regularization aligns local models or vertex-cut embeddings using graph perturbations, structural properties, contrastive losses, or structurally relevant prototypes.Prototype methods can stratify representations by node centrality or label influence within subgraphs.
  • Open challenges: Topology-dependent optimization and graph foundation models remain open directions because structural differences and billion-parameter models create convergence, resource, communication, and privacy challenges.The survey specifically calls for optimization strategies accounting for topology-dependent dynamics and methods addressing foundation-model adaptation in collaborative settings.

3.5 Improving Inference and Learning Efficiency

Graph collaborative learning efficiency depends on coordinating computation and communication topologies while reducing embedding-exchange costs during training and inference.

  • Aggregation patterns: Aggregation can be centralized, decentralized, or hierarchical for both local-model training and node-embedding message passing.The same or distinct topologies may be used for model and embedding aggregation.
  • Topology coupling: Communication topology may follow the computational graph induced by partitioning or remain decoupled as an independent agent network.In subgraph partitions, links can derive from graph cuts, agent locations, or an independently defined communication network.
  • Embedding aggregation: Embedding aggregation improves efficiency by changing aggregation timing, topology, or exchanged content.These techniques target both embedding transmission and aggregation computation during collaborative inference.
  • Timing: Pre-communication can replace repeated collaborative inference with isolated inference after expanded subgraphs are shared before training.Its upfront communication cost may be worthwhile when collaborative inference would require many training rounds.
  • Timing: Stale embeddings and truncated message passing reduce communication by requesting embeddings periodically or transmitting only later-layer embeddings.Periodic requests can be optimized against a communication budget and training convergence speed.
  • Topology and computation: Pre-aggregating an l-hop neighborhood reduces GCN communication to an (l −1)-hop neighborhood and SGC communication to the node itself.For SGC, non-learnable operations can also be pre-computed locally.
  • Model aggregation: Centralized model aggregation dominates moderate-constraint cross-silo settings, while graph-specific convergence depends on structural shift and homophily.The cited analysis covers i.i.d. and label-shift scenarios in subgraph edge-cut configurations.

3.6 Privacy Preservation Techniques for Graph Data

Privacy preservation for collaborative graph learning must protect model parameters, exchanged embeddings, and topology information against leakage during training and inference.

  • Protection scope: Graph privacy techniques target model-parameter leakage, embedding leakage, and topology leakage, especially in cross-silo applications.The surveyed protections address honest-but-curious agents and central servers handling sensitive information.
  • Model parameters: Model parameters or gradients can be anonymized using federated-learning techniques, including adaptive noise scaling for the stability–privacy trade-off.These methods apply directly to exchanged GNN training parameters.
  • Embeddings: Exchanged embeddings create attack surfaces for feature extraction and membership inference, so intermediate representations require protection even when local computations are isolated.The cited discussion states that shared and aggregated embeddings must be protected through differential privacy.
  • Embeddings: Blind computation securely aggregates embeddings or representations without revealing their values, but its high computational cost often limits use to selected layers.Secure multiparty computation can implement GNN aggregation and matrix multiplication.
  • Embeddings and topology: Local differential privacy adds noise, clipping, subsampling, and budget adjustment to communicated vectors, but topological noise can degrade GNN performance.Excessive noise may densify graphs and cause oversmoothing.
  • Topology: Topology protection uses blind computation or local differential privacy for edges, adjacency matrices, degree statistics, and private subgraph expansion.Other approaches blend ego-graphs or infer pseudo-edges from node-representation similarity.
  • Open challenges: Privacy mechanisms remain insufficient against Byzantine agents and may reduce learning effectiveness through information compression.The survey highlights privacy amplification along collaborative computation chains as an unexplored graph-learning direction.

4 Conclusion

The survey maps collaborative learning from Euclidean to graph-structured data and finds graph learning to be a nascent landscape centered on isolated and collaborative inference. Graph settings introduce unique constraints requiring joint consideration of topology, communication, privacy, and heterogeneity.

  • Conclusion: Collaborative graph learning extends access to distributed data across domains including sensor networks and molecular biology.The survey frames this extension as an opportunity beyond centralized graph machine learning.
  • Conclusion: The survey systematically maps design choices across Euclidean and graph-structured collaborative learning, including heterogeneity and the effectiveness–efficiency–privacy trilemma.For graph data, it also identifies emerging research challenges.
  • Conclusion: Graph collaborative learning comprises isolated inference and collaborative inference, with collaborative inference more central because GNNs use message passing.The analysis describes the overall problem-and-solution landscape as nascent yet rich.
  • Conclusion: Distributed relational data introduces unique constraints requiring novel solutions and joint consideration of topologies, communication, privacy, and heterogeneities.These constraints extend beyond challenges already present in Euclidean collaborative learning.
  • Conclusion: The survey aims to support development of collaborative graph-learning methods with practical impact across scientific and industrial domains.Named areas include drug discovery, healthcare, sustainable industries, and smart cities.

Organization of the solutions

Figure 21 organizes surveyed collaborative-learning solutions for Euclidean and graph data along effectiveness, efficiency, and privacy preservation.

  • Organization of the solutions: The taxonomy covers solutions for effective, efficient, and privacy-preserving collaborative learning across Euclidean and graph data.

Glossary

The glossary defines collaborative systems, agent roles, distribution patterns, and core learning concepts used throughout the paper. It also distinguishes Euclidean from relational non-Euclidean data and characterizes system topology.

  • Entities: Agents are autonomous entities with local data, computation, and communication capabilities; servers centrally supervise agents and communicate with them.
  • Systems: A collaborative system is a distributed group of federated or decentralized agents pursuing shared objectives under local constraints or incentives.These constraints may involve objectives, computation, communication, data availability, or privacy.
  • Systems: Federated systems operate under central supervision, whereas decentralized systems operate without a central server.
  • Scope: Local properties concern one agent, while global properties are shared across or computed from many agents without requiring every agent to have a full view.A global graph is an example of a property that need not be fully visible to each agent.
  • Learning: Training optimizes model parameters to reduce prediction error, and inference uses the learned model to predict a sample’s label from its features.Learning encompasses methodologies, algorithms, and theoretical foundations for effective training and inference.
  • Data: Euclidean data represent each sample as a feature vector in R^n with the standard inner product, whereas non-Euclidean data contain relational structure such as graphs.

Notations

Table 3 presents the paper’s notation system, organized into comprehensive categories and arranged alphabetically.

  • Notation table: Table 3 organizes the paper’s notations by comprehensive categories and alphabetical order.
Loading 2609.02984v1…