Source-linked AI summary
Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling
Chuizheng Meng, Sirisha Rambhatla, Yan Liu
TL;DR
Decentralized sensor and IoT data require models that capture spatio-temporal dependencies without sharing node-level data, a gap left by existing federated and centralized approaches. CNFGNN separates temporal modeling on devices from spatial GNN modeling on the server and trains them with alternating optimization. On traffic-flow forecasting, it achieves the best performance in transductive and inductive settings with no extra edge computation and modest communication cost.
Problem
Existing federated methods do not adequately model inherent spatio-temporal dependencies, while centralized forecasting models assume access to data from all devices.
Method
CNFGNN disentangles local temporal modeling from server-side spatial GNN modeling and uses Split Learning, FedAvg, and alternating optimization to train the modules.
Results
CNFGNN achieves the best forecasting performance in transductive and inductive traffic-flow prediction with no extra edge computation and modest communication cost.
Takeaways & Limitations
The approach enables GNN-based spatio-temporal modeling while keeping node data decentralized across the federated network.
Takeaways & Limitations
The formulation requires each node’s features, labels, and model outputs to remain visible only to that node.
Abstract
from arXiv · showhide
Vast amount of data generated from networks of sensors, wearables, and the Internet of Things (IoT) devices underscores the need for advanced modeling techniques that leverage the spatio-temporal structure of decentralized data due to the need for edge computation and licensing (data access) issues. While federated learning (FL) has emerged as a framework for model training without requiring direct data sharing and exchange, effectively modeling the complex spatio-temporal dependencies to improve forecasting capabilities still remains an open problem. On the other hand, state-of-the-art spatio-temporal forecasting models assume unfettered access to the data, neglecting constraints on data sharing. To bridge this gap, we propose a federated spatio-temporal model -- Cross-Node Federated Graph Neural Network (CNFGNN) -- which explicitly encodes the underlying graph structure using graph neural network (GNN)-based architecture under the constraint of cross-node federated learning, which requires that data in a network of nodes is generated locally on each node and remains decentralized. CNFGNN operates by disentangling the temporal dynamics modeling on devices and spatial dynamics on the server, utilizing alternating optimization to reduce the communication cost, facilitating computations on the edge devices. Experiments on the traffic flow forecasting task show that CNFGNN achieves the best forecasting performance in both transductive and inductive learning settings with no extra computation cost on edge devices, while incurring modest communication cost.
1 INTRODUCTION
CNFGNN addresses decentralized spatio-temporal modeling by separating local temporal learning from server-side spatial modeling under cross-node federated constraints. It uses explicit graph structure and alternating training to achieve strong forecasting performance without extra edge computation and with modest communication cost.
- Motivation: Centralized spatio-temporal modeling is impractical for the large data volumes generated by edge devices, motivating decentralized computation.Applications include traffic prediction, forecasting, and user activity detection.
- Motivation: Existing federated methods often omit spatio-temporal dependencies or encode graph structure only implicitly, limiting inductive learning.Regularization-based approaches assume graphs represent node similarity and cannot handle training with only a fraction of devices observed.
- Approach: CNFGNN separates temporal feature extraction on each device from spatial dependency modeling with a server-side GNN.The architecture uses an encoder-decoder on devices and a GNN on the server while keeping local data decentralized.
- Approach: CNFGNN uses explicit graph structure, Split Learning, alternating optimization, and FedAvg to train spatial and temporal modules under federated constraints.Alternating optimization reduces communication overhead, while FedAvg trains a shared temporal feature extractor.
- Results: CNFGNN achieves the best traffic-flow forecasting performance in transductive and inductive settings with no extra edge computation and modest communication cost.The reported result compares CNFGNN with related techniques on traffic flow prediction.
2 RELATED WORKS
Related work spans graph neural networks, federated learning, alternating optimization, and privacy-preserving graph learning. CNFGNN combines these lines to model spatial and temporal relationships while preserving decentralized data.
- Graph Neural Networks: GNNs perform well on graph-structured tasks, but most spatio-temporal GNN work requires centralized training data.Prior applications include graph embedding, node classification, spatio-temporal modeling, and trajectory prediction.
- Federated Learning: Federated learning keeps each client’s raw data local but suffers from heterogeneous client distributions and limited information exchange.Optimization methods address non-IID and unbalanced data, but decentralized data can reduce information utilization.
- Federated Learning: Prior federated approaches mitigate missing-neighbor information but remain less effective than GNNs because feature exchange and aggregation are absent.A cited multi-task framework captures relationships among data without providing full GNN-style feature exchange and aggregation.
- Alternating Optimization: Alternating optimization has been used in federated learning to reduce communicated parameters and transfer knowledge from server models to edge models.CNFGNN applies it to jointly train on-device temporal modules and a server spatial module.
- Privacy-Preserving Graph Learning: Privacy-preserving graph learning uses graph statistics, differential privacy, secure computation, or homomorphic encryption to limit node-information leakage.These approaches include MPC and HE for GNN learning with vertically split data.
3 CROSS-NODE FEDERATED GRAPH NEURAL NETWORK
CNFGNN models node-level temporal dynamics locally and spatial dependencies on a server-side graph network while preserving decentralized node data. Its alternating training procedure reduces Split Learning communication overhead and combines local temporal feature extraction with server-generated graph embeddings.
- Problem formulation: Cross-node federated learning keeps each node’s features, labels, and outputs visible only to that node.The formulation targets settings where sensor records cannot be directly exchanged because of edge-computation or data-access constraints.
- Node-level temporal dynamics: Each node uses an encoder-decoder to extract temporal features and make predictions from locally stored data.The encoder summarizes the input sequence into a hidden state, while the decoder predicts autoregressively from the final input frame.
- Spatial dynamics: The server-side Graph Network propagates node hidden states to produce embeddings containing relational information across nodes.The GN uses node hidden states as inputs, adjacency-derived edge features, and outputs a corresponding embedding for each node.
- Communication challenge: 4|V|S data units are transmitted per GN training round under straightforward end-to-end Split Learning when hidden states and embeddings each have size S.The cost includes forward and backward exchanges between the server and all nodes.
- Alternating optimization: Alternating optimization fixes node embeddings while training node models, then fixes node models while optimizing the server GN.Because node hidden states remain constant during GN training, the server fetches them before optimization, reducing communication to the procedure’s server-round exchange.
- Federated temporal learning: FedAvg trains node encoder-decoder models so devices share a common temporal-feature space, avoiding potential overfitting and empirically improving convergence and prediction performance.The shared feature extractor is trained across nodes rather than independently on each node.
4 EXPERIMENTS
Experiments evaluate CNFGNN for traffic-flow forecasting under decentralized data constraints, comparing forecasting, computation, communication, inductive generalization, and training-strategy trade-offs.
- Dataset and task: Traffic forecasting uses PEMS-BAY and METR-LA sensor datasets, with 5-minute windows and 12-step prediction from 12 observed steps.PEMS-BAY contains 325 sensors over six months; METR-LA contains 207 detectors over four months.
- Models and baselines: CNFGNN combines a 64K-parameter GRU encoder-decoder on each node with a 1M-parameter two-layer Graph Network.The model uses local temporal processing and graph-based spatial modeling under the federated setting.
- Forecasting performance: Spatially informed methods outperform baselines without relation information, indicating that modeling spatial dependencies is critical for forecasting.Both GRU+FMTL and CNFGNN incorporate spatial relations among nodes.
- Forecasting performance: CNFGNN achieves the lowest forecasting error on both datasets while keeping edge computation almost unchanged and communication modest relative to larger on-device baselines.Baselines that increase on-device complexity provide slight or no improvement at higher computation and communication cost.
- Inductive learning: CNFGNN outperforms GRU+FedAvg under most inductive settings, except when 25% of METR-LA nodes are observed during training, where performance is similar.The result supports stronger generalization by CNFGNN in the evaluated inductive settings.
- Training-strategy ablation: AT+FedAvg consistently outperforms alternative training strategies while achieving the lowest communication cost on METR-LA and second-lowest on PEMS-BAY.On PEMS-BAY, SL+FedAvg has lower communication cost but higher prediction error than AT+FedAvg, 4.383 versus 3.822.
5 CONCLUSION
CNFGNN enables GNN-based spatio-temporal modeling under decentralized federated learning by separating local temporal learning from server-side spatial learning. Experiments on two real-world traffic datasets show superior performance over competing techniques, while future work targets scalability, decentralization, and privacy.
- CNFGNN enables GNNs to model complex spatio-temporal data within federated learning despite decentralized data processing.
- The method decouples local temporal models from the server-side spatial model using alternating optimization with Split Learning and Federated Averaging.
- Experiments on traffic-flow prediction across two real-world datasets show superior performance compared with competing techniques.
- Future work includes scalable sampled GNNs, fully decentralized training, and privacy-preserving graph learning.
A.1 The Histograms of Data on Different Nodes
The appendix visualizes traffic-speed distributions across nodes in PEMS-BAY and METR-LA. These distributions vary by node, indicating that node data are not identically distributed.
- Traffic-speed histograms are shown for PEMS-BAY and METR-LA in Figures A1 and A2.Each figure displays the first 100 nodes ranked by ID.
- The histograms display traffic-speed distributions for the first 100 nodes ranked by ID in each dataset.
- Traffic-speed distributions vary across nodes, so data on different nodes are not independent and identically distributed.
A.2 Table of Notations
The appendix provides a notation reference for the paper. Table A1 summarizes the symbols and their definitions.
- Table A1 summarizes the notations used throughout the paper.
- The notation table pairs symbols with their definitions.
- The appendix labels this reference as the table of notations.