PaperlayerView exact revision on arXiv

Source-linked AI summary

Privacy-preserving Traffic Flow Prediction: A Federated Learning Approach

Yi Liu, James J. Q. Yu, Jiawen Kang, Dusit Niyato, Shuyu Zhang

arXiv:2003.08725v1cs.LGcs.CRstat.ML

TL;DR

Traffic flow forecasting needs large datasets but must address privacy risks and the trade-off between prediction accuracy and privacy. The paper proposes FedGRU with secure parameter aggregation, Federated Averaging, a Joint-Announcement protocol, and ensemble clustering. On a real-world PeMS dataset, the method remains comparable to competing models while preserving privacy, and the protocol reduces communication overhead by 64.10%.

  • Problem

    Traffic flow prediction relies on data that may contain private information, creating a need for accurate forecasting under privacy-protection constraints.

  • Method

    FedGRU combines federated learning with a GRU network, using secure parameter aggregation, improved Federated Averaging, Joint-Announcement, and ensemble clustering.

  • Results

    64.10% communication-overhead reduction was achieved by the joint-announcement protocol compared with centralized models, while FedGRU performed comparably to competing methods with minuscule accuracy degradation.

  • Takeaways & Limitations

    FedGRU can provide accurate and timely traffic flow prediction while preserving the privacy of distributed organizational data.

  • Takeaways & Limitations

    Statistical heterogeneity remains a challenge because organizations’ local data are not necessarily i.i.d., increasing modeling, analysis, and evaluation complexity.

Abstract

from arXiv · show

Existing traffic flow forecasting approaches by deep learning models achieve excellent success based on a large volume of datasets gathered by governments and organizations. However, these datasets may contain lots of user's private data, which is challenging the current prediction approaches as user privacy is calling for the public concern in recent years. Therefore, how to develop accurate traffic prediction while preserving privacy is a significant problem to be solved, and there is a trade-off between these two objectives. To address this challenge, we introduce a privacy-preserving machine learning technique named federated learning and propose a Federated Learning-based Gated Recurrent Unit neural network algorithm (FedGRU) for traffic flow prediction. FedGRU differs from current centralized learning methods and updates universal learning models through a secure parameter aggregation mechanism rather than directly sharing raw data among organizations. In the secure parameter aggregation mechanism, we adopt a Federated Averaging algorithm to reduce the communication overhead during the model parameter transmission process. Furthermore, we design a Joint Announcement Protocol to improve the scalability of FedGRU. We also propose an ensemble clustering-based scheme for traffic flow prediction by grouping the organizations into clusters before applying FedGRU algorithm. Through extensive case studies on a real-world dataset, it is shown that FedGRU's prediction accuracy is 90.96% higher than the advanced deep learning models, which confirm that FedGRU can achieve accurate and timely traffic prediction without compromising the privacy and security of raw data.

I. INTRODUCTION

Traffic flow prediction supports transportation operations but commonly relies on shared data that may expose private information. The paper proposes federated GRU-based prediction, secure aggregation, and clustering to preserve privacy while maintaining accuracy and scalability.

  • Traffic flow prediction provides information for congestion mitigation, traffic-light control, transportation efficiency, and travel planning.
  • Centralized prediction methods typically require agencies and companies to share sensor data, which may contain sensitive information such as vehicle license plates.
  • The paper proposes FedGRU, combining federated learning with a GRU neural network so organizations train locally without exchanging raw data.
  • A Joint-Announcement protocol enhances Federated Averaging by using random sub-sampling to reduce communication overhead and improve scalability.
  • An ensemble clustering-based FedGRU scheme groups organizations to integrate global models and capture spatio-temporal traffic correlations for improved prediction accuracy.
  • Extensive experiments on a real-world dataset compare the proposed schemes with non-federated learning methods.

1) Parametric models:

Traffic prediction research includes parametric and non-parametric models, while privacy constraints limit direct data exchange among organizations. The paper formulates federated traffic prediction around local datasets and secure aggregation of model parameters.

  • 1) Parametric models:: Parametric traffic prediction models, including ARIMA variants, improve forecasting by modeling statistical correlations in traffic data.
  • Non-parametric models such as k-NN, stacked autoencoders, SVM, and feed-forward neural networks have achieved success in traffic flow prediction.
  • Privacy awareness and legal requirements can prevent direct data exchange among users and organizations in intelligent transportation systems.
  • Federated learning trains models on datasets distributed across devices while preventing data leakage, and prior work applies it to several privacy-sensitive domains.
  • In the paper’s formulation, organizations are treated as federated-learning clients, each holding a local database used to predict vehicle counts without sharing raw data.
  • Secure parameter aggregation lets detector stations upload locally updated parameters so the cloud constructs a global model without exchanging traffic-flow data.

IV. METHODOLOGY

The methodology formulates traffic flow prediction as federated learning with GRU local models, keeping data distributed while aggregating model updates securely. It combines local optimization, global aggregation, and GRU gating for time-series prediction.

  • Federated learning formulation: Federated learning trains a globally predicted model from datasets stored separately across many organizations.The organizations retain local datasets while contributing to global model learning through secure parameter aggregation.
  • Federated learning formulation: The local objective learns model parameters from each organization’s samples under local storage and processing constraints.The formulation uses local input-output pairs, a loss function, and a regularizer for each device dataset.
  • GRU model: GRU serves as the local model for traffic flow prediction and handles time-series data using reset and update gates.The update gate uses the current input and previous hidden state, while the reset gate is computed similarly.
  • GRU model: GRU’s candidate activation and final memory update combine gated hidden-state information across time steps.The tanh activation maps values to (-1,1), which the passage associates with reduced calculations and prevention of gradient explosions.
  • Aggregation architecture: The architecture uses FedAVG for small-scale organizations and the Joint-Announcement Protocol for large-scale organizations through subsampling.The figure caption identifies these as alternative update calculations based on organization scale.

B. Privacy-preserving Traffic Flow Prediction Algorithm

FedGRU addresses centralized learning’s privacy and communication problems through secure aggregation of locally trained updates. FedAVG forms the aggregation core, while an improved protocol targets larger participant populations.

  • Privacy-preserving algorithm: FedGRU replaces centralized raw-data merging with secure aggregation of model updates from participating organizations.The algorithm is designed to address both data privacy concerns and expensive communication overhead associated with centralized learning.
  • Scalable aggregation: The Joint-Announcement Protocol improves FedAVG by randomly subsampling organizations in each round for large-scale federated learning.This design specifically targets communication overhead and convergence difficulties as the participant population grows.
  • FedAVG aggregation: FedAVG collects local gradient information and performs weighted averaging to reduce communication overhead during parameter transmission.Clients optimize locally, then the cloud aggregates uploaded model updates.
  • FedAVG aggregation: FedAVG training selects organizations, broadcasts the global model, performs local SGD updates, and securely aggregates the resulting parameters.The described process includes volunteer selection, local training for E epochs with mini-batch size B, and cloud aggregation.

2) Federated Learning-based Gated Recurrent Unit neural network algorithm:

FedGRU combines a pre-trained global model, local GRU training, encrypted parameter sharing, and secure aggregation. The repeated exchange of model updates produces a new global model without sharing private data.

  • Federated training workflow: FedGRU initializes the cloud model through pre-training on domain-specific public datasets without privacy concerns.This initialization precedes distribution of the global model to participating organizations.
  • Federated training workflow: The cloud distributes the global model, organizations train copies locally, and each organization uploads model updates rather than raw data.The workflow keeps training data at the organizations while transmitting model-related information.
  • Federated training workflow: Secure parameter aggregation combines uploaded updates to build a new global model, which the cloud redistributes to organizations.The process iterates through global-model construction and redistribution.
  • Local GRU training: FedGRU uses GRU processing for traffic-flow sequences and updates gates, parameters, and hidden states during local training.The described algorithm includes mini-batch input and true traffic-flow steps, gate updates, and parameter collection.
  • Local GRU training: The algorithm defines organization inputs, outputs, and cell hidden states as time-series quantities for the FedGRU objective.These quantities support the stated objective function and pseudocode formulation.

3) Joint-Announcement Protocol:

The Joint-Announcement Protocol scales federated traffic prediction by selecting only a fixed proportion of organizations in each training round. Its preparation, training, and aggregation phases coordinate participation and model updates.

  • Joint-Announcement Protocol: Large-scale FedGRU uses the Joint-Announcement Protocol because FedAVG becomes difficult to converge with many participants.The protocol randomly selects a certain proportion of organizations in each training round.
  • Protocol phases: The protocol has preparation, training, and aggregation phases involving participating organizations and the cloud.These phases define the round-level coordination process.
  • Protocol phases: During preparation, organizations voluntarily check in with the cloud and may reject participation if unwilling or unable to participate.The passage describes rejection as representing unwillingness or other failures.
  • Protocol phases: During training, the cloud loads a pre-trained model, sends checkpoints, randomly selects a fixed proportion of organizations, and receives local parameters.Example sampling proportions include 10% and 20%.
  • Protocol phases: During aggregation, the cloud securely aggregates uploaded parameters with FedAVG, stores updated checkpoints, and sends update parameters to each organization.FedAVG is used to reduce uplink communication costs.

C. Ensemble Clustering Federated Learning-Based Traffic Flow Prediction Algorithm

The ensemble clustering-based FedGRU algorithm groups organizations by location before federated training, then combines cluster models to improve prediction accuracy and scalability. Its clustering and ensemble stages produce an optimal global model subset for distribution to organizations.

  • Clustering-based FedGRU: Organizations are grouped into K clusters using latitude and longitude information before implementing FedGRU.The clustering uses a constrained K-Means algorithm.
  • Clustering-based FedGRU: The constrained K-Means procedure updates cluster centers by minimizing the sum of squared Euclidean distances subject to cluster-assignment constraints.The assignment variables satisfy nonnegativity, membership, and minimum cluster-size constraints.
  • Ensemble learning: The cloud executes FedGRU for each cluster and uses ensemble learning to select an optimal subset of cluster global models.The selected models are integrated to obtain the best accuracy.
  • Ensemble learning: The cloud sends the new global model to each organization after the ensemble-learning stage.This is the final step of the ensemble clustering federated learning algorithm.

V. EXPERIMENTS

The experiments evaluate FedGRU and its clustering-based variant on real-world PeMS traffic data under a distributed, privacy-constrained setting. Prediction accuracy is assessed with four standard error metrics while encrypted parameter aggregation supports the experimental privacy assumption.

  • Dataset: The study uses three months of 2013 PeMS traffic data, training on the first two months and testing on the third.The database contains real-time data from over 39,000 detectors across California freeway systems.
  • Dataset: The time-series input uses previous intervals xt−1, xt−2, ···, xt−r to predict traffic flow at interval t.Here, r denotes the history data-window length.
  • Evaluation metrics: Prediction accuracy is measured with MAE, MSE, RMSE, and MAPE.The observed traffic flow is denoted by yi and the predicted traffic flow by ŷp.
  • Experimental setting: The experiment assumes independently distributed detector stations whose data cannot be exchanged arbitrarily, with encrypted parameters handled through PySyft.The PeMS dataset is split equally among 20 organizations for simulation.

A. FedGRU Model Architecture

The FedGRU architecture is tuned through a hyperparameter search over history-window length, hidden-layer count, and hidden-unit configurations. The selected architecture uses a history window of 12 and two hidden layers with 50 units each.

  • Hyperparameter selection: The optimal history-data window is r = 12.The search varies hidden-layer counts from 1 to 3 and hidden-unit configurations including 50 and 100 units.
  • Selected architecture: The best FedGRU architecture has two hidden layers, each containing 50 hidden units.The results identify two hidden layers as optimal in this experiment.
  • Evaluation: FedGRU performance is evaluated on a 5-min traffic flow prediction task using MAE, MSE, RMSE, and MAPE.The architecture is selected after a grid search.

B. Traffic Flow Prediction Accuracy

FedGRU is compared with GRU, SAE, LSTM, and SVM on the same PeMS-based 5-min prediction task. Its predictions and loss are close to GRU while retaining local training for privacy protection, and its MAE improves over the weakest centralized baseline.

  • Compared methods: FedGRU is compared with GRU, SAE, LSTM, and SVM under an identical simulation configuration.FedGRU is federated, whereas the other four competing methods are centralized.
  • Prediction accuracy: 9.04% lower MAE is reported for FedGRU than the worst case, SVM, in the 5-min prediction experiment.FedGRU's MAE is lower than SAE, LSTM, and SVM but higher than GRU.
  • Prediction accuracy: FedGRU prediction results are very close to GRU results for the 5-min traffic flow task.The paper attributes this comparability to FedGRU's GRU structure.
  • Convergence: FedGRU loss is not significantly different from GRU loss, indicating comparable convergence in the reported experiment.The comparison is shown in Fig. 6(b).

C. Performance Comparison of FedGRU Model Under Different Client Numbers

FedGRU performance is evaluated as client participation scales, motivating joint announcement and clustering mechanisms for large-scale traffic prediction. Joint announcement reduces communication overhead while preserving stable prediction behavior, and clustering further improves accuracy.

  • Client-number effects: FedAVG performs well with fewer than 8 clients but declines when client participation reaches or exceeds 8.The reported explanation is increased client-failure probability at larger client counts.
  • Joint-announcement protocol: The joint-announcement protocol randomly subsamples organizations before applying FedAVG to reduce participants per training round.This combines lower communication demand with averaged gradient information for model training.
  • Joint-announcement protocol: 64.10%: communication overhead reduction for FedGRU with joint announcement at C = 10 and β = 50% versus FedGRU with FedAVG.The protocol reduces participants through subsampling before gradient aggregation.
  • Participation-ratio effects: 29.08%: MAE increase for large-scale FedGRU versus FedGRU when C = 10 and β = 50%.The passage attributes this difference to FedAVG degradation at C ≥8 and describes joint announcement as controlling participation.
  • Participation-ratio effects: Larger β increases early-training loss, but β does not affect model convergence.The reported conclusion is that joint-announcement FedGRU maintains stability, robustness, and efficiency across tested participation ratios.
  • Ensemble clustering: The ensemble clustering-based FedGRU achieves the best prediction accuracy among tested clustering settings and can improve the original FedGRU model.For K = 8 and 10, it can outperform centralized GRU while retaining the federated setup described in the passage.

F. Discussion

The discussion identifies communication overhead, unsynchronized updates, and non-i.i.d. local data as important constraints, while reporting privacy-preserving performance comparable to centralized GRU. It also positions ensemble clustering and future GCN integration as ways to improve the approach.

  • Limitations and remedies: Communication overhead is a bottleneck for FedGRU, and joint announcement mitigates it by reducing participants in each communication round.This discussion links the bottleneck directly to the protocol’s participant-reduction strategy.
  • Limitations and remedies: Client failures can leave local models unsynchronized with the current global model, affecting subsequent global updates.Random subsampling is used to reduce the probability of client failure and alleviate this out-of-sync issue.
  • Limitations and remedies: Large-scale FedGRU faces statistical heterogeneity because organizations collect local data in different ways rather than under an i.i.d. assumption.The passage states that this can increase modeling, analysis, and evaluation complexity.
  • Privacy and performance: FedGRU aggregates encrypted parameters instead of accessing original data, supporting privacy-preserving traffic prediction.The privacy discussion frames this as the model’s data-access protection mechanism.
  • Privacy and performance: FedGRU performs comparably to centralized GRU while preserving privacy under the reported traffic-flow prediction setting.The comparison is explicitly described as involving a trade-off between prediction and privacy.
  • Empirical findings: On the PeMS dataset, FedGRU shows comparable performance to GRU, LSTM, SAE, and SVM with minuscule accuracy degradation and privacy preserved.The study also reports ensemble clustering for further performance improvement and a 64.10% communication-overhead reduction from joint announcement.
  • Future direction: Future work proposes federated GCNs to better capture spatial-temporal dependencies and further improve prediction accuracy.This is presented as a planned extension rather than a demonstrated result.
Loading 2003.08725v1…