PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challenges

Vishnu Pandi Chellapandi, Liangqi Yuan, Christopher G. Brinton, Stanislaw H Zak, Ziran Wang

arXiv:2308.10407v2cs.LGcs.CRcs.DCcs.NI

TL;DR

CAV machine learning requires extensive vehicular data, creating privacy, security, and communication challenges. This survey reviews FL frameworks, data and security techniques, applications, challenges, and future directions, concluding that properly designed FL can support privacy preservation, network efficiency, and collaborative intelligence.

  • Problem

    Training CAV machine-learning models requires extensive vehicular data, raising concerns about privacy, security, data misuse, and communication overhead.

  • Method

    The survey reviews FL algorithms and frameworks, data modalities, model-security techniques, CAV applications, datasets, challenges, and future research directions.

  • Results

    Across reviewed applications, FL achieved comparable or improved performance for steering prediction, object recognition, and traffic-flow prediction while reducing communication costs or preserving local data.

  • Takeaways & Limitations

    With proper design and implementation, FL offers CAVs privacy preservation, network efficiency, and collaborative intelligence, with further applications anticipated in driver modeling, anomaly detection, and predictive maintenance.

  • Takeaways & Limitations

    FL4CAV still faces challenges involving data integrity, communication latency, heterogeneous data sources, model synchronization, resource utilization, and incentives for data sharing.

Abstract

from arXiv · show

Machine learning (ML) is widely used for key tasks in Connected and Automated Vehicles (CAV), including perception, planning, and control. However, its reliance on vehicular data for model training presents significant challenges related to in-vehicle user privacy and communication overhead generated by massive data volumes. Federated learning (FL) is a decentralized ML approach that enables multiple vehicles to collaboratively develop models, broadening learning from various driving environments, enhancing overall performance, and simultaneously securing local vehicle data privacy and security. This survey paper presents a review of the advancements made in the application of FL for CAV (FL4CAV). First, centralized and decentralized frameworks of FL are analyzed, highlighting their key characteristics and methodologies. Second, diverse data sources, models, and data security techniques relevant to FL in CAVs are reviewed, emphasizing their significance in ensuring privacy and confidentiality. Third, specific applications of FL are explored, providing insight into the base models and datasets employed for each application. Finally, existing challenges for FL4CAV are listed and potential directions for future investigation to further enhance the effectiveness and efficiency of FL in the context of CAV are discussed.

I. INTRODUCTION

The introduction motivates FL4CAV as a response to CAVs’ massive data volumes, privacy concerns, and communication burdens. It outlines the survey’s coverage of FL frameworks, data and security, applications, challenges, and research directions.

  • I. INTRODUCTION: CAVs generate 20–40 TB of raw data per vehicle daily from components, sensors, and V2X communications.These data support monitoring, prognostics, diagnostics, connectivity, and ML applications.
  • I. INTRODUCTION: Privacy risks arise from data collection, misuse, breaches, and regulatory requirements governing sensor-derived personal data.The introduction cites GDPR as an example of stringent privacy protection requirements.
  • I. INTRODUCTION: FL keeps local datasets at edge devices while clients transmit gradients or learnable parameters for secure aggregation and iterative global-model updates.This approach reduces network strain while preserving local-data privacy.
  • I. INTRODUCTION: FL4CAV research includes centralized and decentralized frameworks, diverse data modalities, ML models, and security techniques.The survey specifically reviews how ML models are integrated into FL deployments for CAV applications.
  • I. INTRODUCTION: Applications reviewed include driver monitoring, steering-angle and trajectory prediction, object detection, motion control, traffic-flow prediction, and V2X communication.The survey also summarizes relevant modalities, algorithms, and datasets.
  • I. INTRODUCTION: The survey highlights performance, safety, fairness, applicability, and scalability as challenges and identifies future research directions for FL4CAV.It compares its scope with related surveys of federated learning for CAVs.

II. FEDERATED LEARNING METHODS

The survey distinguishes centralized and decentralized FL frameworks and reviews their training, aggregation, and optimization methods for CAVs. Centralized FL commonly uses local SGD with server aggregation, while heterogeneous non-IID vehicle data challenge global optimization.

  • Frameworks: The survey categorizes FL4CAV into centralized and decentralized frameworks and reviews the ML techniques used as local base models.Fig. 2 illustrates the two framework categories.
  • Centralized Federated Learning: Centralized FL distributes a global model to K vehicles, which train locally, apply privacy-preserving techniques, upload parameters, and receive aggregated updates.The process repeats through communication rounds, with the server securely aggregating vehicle updates and testing model performance.
  • Averaging: FedAvg uses local SGD and weighted server averaging, with multiple local updates reducing the number of communication rounds.Vehicle data are dynamically updated at each communication round.
  • Averaging: The central-server objective aggregates local vehicle objectives, where each local objective is an expected loss over samples drawn from that vehicle’s data.The optimization seeks a minimizer of the global objective.
  • Averaging: Non-IID CAV data can make a single global model perform poorly for individual vehicles and cause local updates to deviate from the global objective.FedProx, Dynamic Federated Proximal, and FedAdam are cited as variants addressing heterogeneity, drift, communication, and optimization challenges.

2) Knowledge Distillation:

Knowledge distillation is integrated with FL to transfer model knowledge while reducing communication size and addressing non-IID samples. The reviewed approaches use server-side teacher models or shared outputs to support smaller client models.

  • Knowledge Distillation: Federated Distillation transfers knowledge in a decentralized manner, significantly reducing communication size compared with traditional FL and handling non-IID samples.FDCAV has vehicles share outputs such as bounding boxes with a central server, which returns averaged global-model outputs.
  • Knowledge Distillation: A teacher model can reside on the server while student models on clients mimic its behavior, reducing client computational complexity and improving efficiency.This design transfers knowledge from a larger, more complex server model to smaller client models.
  • Knowledge Distillation: The described CFL workflow initializes vehicle models, performs parallel local SGD, sends vehicle models to the server, aggregates them, and redistributes the result across communication rounds.The algorithm outputs the aggregated global model after T rounds.
  • Knowledge Distillation: FedGKT places ResNet-55 or ResNet-109 on the server and ResNet-8 on clients.The example illustrates asymmetric model sizes between server and vehicles.

B. Decentralized Federated Learning

Decentralized FL replaces central-server aggregation with consensus among vehicles that exchange model data over V2X links. It improves scalability and robustness but faces convergence, latency, synchronization, and topology challenges.

  • Decentralized Federated Learning: DFL enables vehicles to update global parameters through consensus without a central server, whereas CFL transmits parameters or gradients to a central server such as an RSU.CFL can suffer server computational limits, single points of failure, and communication congestion.
  • Decentralized Federated Learning: DFL accommodates many vehicle clients and can continue collaborative training when an individual vehicle is unavailable by exchanging model data with neighboring vehicles.Vehicle-to-vehicle model exchange uses V2X communication.
  • Consensus mechanism: The DFL algorithm aggregates neighboring models using a mixing matrix and outputs a personalized model for each vehicle.The algorithm performs local SGD, exchanges models, aggregates client-side updates, and deploys each vehicle’s personalized result.
  • Consensus mechanism: Neighbor communication establishes consensus through a gossip matrix whose non-negative weights encode vehicle connectivity, with zero weight indicating no communication.Self-loops have positive weights, and the graph determines which vehicles exchange updates.
  • Limitations: DFL faces hindered convergence from heterogeneous data, network latency, parameter synchronization and arbitration requirements, and adaptation to dynamic network topologies.A four-vehicle example contrasts ring and fully connected topologies.

III. OVERVIEW OF DATA MODALITIES, BASE MACHINE LEARNING MODELS, AND SECURITIES

FL4CAV relies on diverse onboard sensing modalities and supports perception, prediction, planning, control, and human-monitoring tasks. The survey organizes these modalities alongside local ML models and security considerations.

  • Data modalities: CAV clients perform sensing, signal processing, storage, communication, perception, and decision-making using GNSS, cameras, radar, LiDAR, and IMUs.These sensors capture vehicle, driver, passenger, and external information.
  • CAV applications: CAV applications include target-speed tracking, behavior prediction, motion planning, motion control, object detection, and in-vehicle human monitoring.After local training, clients send models to a server and receive a generalized model for perception, prediction, and decision-making.
  • Overview: The overview connects CAV data modalities and base ML models with data-security techniques for privacy and confidentiality.The survey presents these three elements together before reviewing applications.
  • Data modalities: Sensor modalities affect FL-based sensor fusion dynamically, while sensor range, precision, placement, and operating conditions determine the collected data.Snow, heavy rain, and fog can reduce sensor visibility and deteriorate data quality.
  • Data modalities: RGB images support driver monitoring, steering-wheel-angle prediction, object detection, traffic-sign recognition, and semantic segmentation, but their intrusive nature raises privacy concerns.Camera images are identified as an important CAV data modality.

2) LiDAR:

LiDAR and radar provide complementary sensing for CAV perception, while vehicle-status data expose privacy-sensitive information. FL4CAV combines multimodal learning with ML architectures suited to vehicle tasks and resource constraints.

  • LiDAR: LiDAR produces accurate 3D point clouds for object detection and cooperative perception, including under adverse weather, but their density makes transmission demanding.FL for LiDAR can improve learning efficiency and save communication resources.
  • Radar: Radar supports object detection and collision avoidance by measuring surrounding objects’ positions and speeds across varying weather conditions.Its long operating range and measurement accuracy support applications including emergency braking, traffic alerts, and adaptive cruise control.
  • Vehicle-status data: Vehicle-status data include velocity, acceleration, commands, GNSS position, and other parameters that can reveal drivers’ locations, habits, and behaviors.These data primarily describe the vehicle rather than its external environment, making privacy protection important.
  • ML base models: ML in FL4CAV is considered feasible in real time despite vehicles’ limited computing and communication resources.The reviewed base models include MLPs, CNNs, and RNNs for vehicle-related tasks.
  • RNN: RNNs capture temporal dependencies and are applied to vehicle trajectory and traffic-flow prediction using time-series data.LSTM and GRU are identified as popular RNN architectures.

4) Transformer:

Transformer-based and generative models extend FL4CAV beyond conventional CNN and RNN architectures, while privacy techniques address threats to collaborative training. These methods involve trade-offs among privacy, security, communication, and model performance.

  • Transformer: Transformers and Vision Transformers use self-attention to capture global dependencies and long-range relationships across input sequences or images.They have emerged as alternatives to traditional CNNs and RNNs across domains including computer vision.
  • Generative networks: Generative networks such as GANs and VAEs can produce realistic images and support FL4CAV applications including trajectory prediction and image super-resolution.Super-resolution can enhance low-resolution imagery for license-plate recognition and surveillance.
  • Security threats: FL4CAV training remains vulnerable to compromised participants sending false parameters and to attacks on central servers.Such attacks can hinder global-model performance or collapse the learning process.
  • Privacy preservation: Homomorphic encryption, differential privacy, and blockchain-based techniques are reviewed as privacy-preserving approaches for FL4CAV.They seek to balance data security and privacy with effective model performance.
  • Homomorphic encryption: Homomorphic encryption lets servers compute on encrypted vehicle data and obtain decrypted results without directly decrypting the data during training.This provides a mechanism for protecting data privacy and security.
  • Differential privacy: Differential privacy adds random noise before transmission to limit sensitive-information extraction, but privacy settings can reduce model accuracy.The approach also supports data ownership and regulatory compliance.

4) Physical Security:

FL4CAV security mechanisms span trusted hardware and blockchain, while surveyed applications address human monitoring, prediction, and control. These applications combine privacy protection with collaborative or efficient learning, but dataset scarcity and user heterogeneity remain boundaries.

  • 4) Physical Security: Trusted Execution Environments provide FL participants with isolated, secure, and confidential environments for processing and storing data and refining local models.TEE is presented as a hardware-level approach to model security enhancement.
  • 5) Blockchain: Blockchain improves data integrity, transparency, and security through decentralized, tamper-resistant storage for collaborative FL in CAV networks.It is described as a secure and credible framework for collaborative model training while safeguarding data privacy.
  • 5) Blockchain: Blockchain can store vehicle model parameters as verifiable transactions and use smart contracts to reward high-quality updates or computational contributions.These incentives are intended to encourage participation and collaboration.
  • Applications: The survey organizes FL4CAV applications around privacy, learning efficiency, generalization ability, and communication-overhead reduction.Its application review covers configurations, modalities, models, algorithms, and datasets.
  • A. In-Vehicle Human Monitoring: Driver monitoring uses FL for privacy-sensitive safety functions, but personalized behavior creates user heterogeneity that challenges FL systems.Personalized FL is identified as the dominant solution for human-related applications.
  • A. In-Vehicle Human Monitoring: A gossip-based framework achieved personalized models without additional processing and accelerated training through knowledge dissemination.The cited result connects decentralized information sharing with both personalization and training speed.
  • A. In-Vehicle Human Monitoring: Passenger monitoring remains limited by a lack of available datasets and the difficulty of monitoring multiple users simultaneously.The application includes detecting boarding and leaving intentions and warning about dangerous behavior in public transportation.
  • B. Steering Wheel Angle Prediction: For steering-angle prediction, FL supports collaborative learning with low communication overhead and data security for robust prediction models.Studies using images and optical flow reported improved edge-model quality, while another comparison found FL and centralized learning comparable under different noise levels.

C. Vehicle Trajectory Prediction

Vehicle trajectory prediction supports motion planning and proactive responses but requires substantial sensitive data. FL addresses this setting through collaborative learning across diverse vehicles, while related CAV applications show benefits for perception, recognition, and control.

  • C. Vehicle Trajectory Prediction: Accurate trajectory prediction helps CAVs plan motion and anticipate dangerous behaviors such as sudden lane changes, skidding, and hard braking.Training trajectory models requires substantial amounts of sensitive vehicle data.
  • C. Vehicle Trajectory Prediction: FL keeps local trajectory data private while learning from diverse vehicles and scenarios, supporting generalization to rare events and continuous model updates.The cited examples include traffic accidents, adverse weather, and risky behaviors.
  • C. Vehicle Trajectory Prediction: Trajectory models commonly use time-series position, velocity, and acceleration data with RNNs or Transformers to capture spatio-temporal and behavioral patterns.FL has been shown effective for learning spatio-temporal features with Transformer models.
  • D. Object Recognition: Object-recognition data from LiDAR and high-resolution imagery are large and privacy-sensitive, limiting traditional centralized training; FL mitigates these concerns.Adverse weather can also reduce object-detection accuracy.
  • D. Object Recognition: FL improved object-detection accuracy and outperformed centralized and gossip-decentralized models in reported studies.Resource allocation and vehicle selection further improved performance relative to centralized learning and baseline FL approaches.
  • D. Object Recognition: FL supports license-plate recognition by protecting privacy, enabling collaboration, and reducing network-bandwidth requirements.These properties address the difficulty of real-time edge-device training on data collected from all vehicles.
  • E. Motion Control: FL dynamically updates motion-controller parameters, improving target-speed achievement, driver comfort, and safety, while supporting control optimization across vehicles.Reported applications include intersection collision avoidance and brake optimization under different driving conditions.
  • E. Motion Control: Reinforcement learning is widely used for vehicle motion control because it learns control policies in complex, dynamic environments.Open FL-related problems include platooning, lane changes, merging, and signalized or unsignalized intersections.

F. Traffic Flow Prediction

FL is applied to traffic-flow prediction using data from vehicles, roadside units, and traffic sensors, supporting privacy-preserving collaborative learning. Reported studies use recurrent and spatial-temporal models, with results ranging from centralized-comparable predictions to improved accuracy over several baselines.

  • Traffic-flow prediction: FL-based traffic-flow prediction uses data from CAVs, RSUs, and traffic sensors for real-time or future traffic-pattern forecasting.The approach supports collaborative learning while addressing privacy concerns.
  • Traffic-flow prediction: A FL-trained GRU achieved predictions comparable to traditional centralized approaches on a real-world dataset.
  • Traffic-flow prediction: FedSTN reportedly achieved higher prediction accuracy than ARIMA, XGBoost, FedGRU, and ST-ResNet.FedSTN combines recurrent long-term capture, attentive federated, and semantic capture networks to learn spatial-temporal and semantic information.
  • Traffic-flow prediction: An FL-LSTM framework was trained on real VANET data based on V2V and V2R communication for network-traffic prediction.The reviewed developments are presented as beneficial for complex traffic-flow prediction tasks.
  • Vehicular cyber-physical systems: FL is also used in vehicular cyber-physical systems to preserve privacy, address resource constraints, and improve resilience against adversarial attacks.Reported security combinations include differential privacy and blockchain-based approaches.
  • V2X communication: V2X communication faces increasing transmission, communication, energy, latency, and security demands as the number of vehicles grows.FL is described as a decentralized approach for reducing data transmission and communication overload while supporting secure learning and resource management.

A. Resource Limitations and Utilization

FL4CAV faces resource limitations from large perception datasets, unreliable networks, heterogeneous device capabilities, and privacy requirements. The survey discusses decentralized, split, and cooperative approaches, alongside incentives, to improve communication, computation, and resource efficiency.

  • Resource limitations and utilization: Vision and LiDAR perception tasks generate large datasets that increase communication costs and global-model computation demands as more CAVs participate.Decentralized and clustered FL are being explored to reduce communication overhead.
  • Resource limitations and utilization: High communication demands and low 5G reliability motivate 6G-V2X integration with V2X, MEC, and ML for more efficient edge processing.The proposed direction targets improved future-network performance and reliability.
  • Resource limitations and utilization: FL requires more real-world datasets, high-fidelity simulators, and strong pre-trained base models for varied weather and traffic scenarios.
  • Resource limitations and utilization: Split learning is proposed to reduce edge-client inference requirements while retaining generalization and personalization capabilities.The survey identifies a trade-off between personalized accuracy and high edge-device computational requirements.
  • Resource limitations and utilization: Resource optimization must account for idle powerful devices, underused networks, disconnected devices, and diverse privacy requirements.The survey calls for joint optimization of device, server, and network resources.
  • Resource limitations and utilization: Cooperative FL is reported as feasible and beneficial for improving ML performance and resource efficiency.A related architecture combines cooperative FL with reinforcement learning for resource allocation in CAV networks.
  • Resource limitations and utilization: Massive vehicular data creates privacy and security concerns that must be addressed without compromising model accuracy and redundancy.
  • Resource limitations and utilization: Rewarding policies and incentive mechanisms are needed to encourage CAVs to share quality data for efficient model training.

1) Lack of methods for efficient vehicle selection and resource allocation:

Efficient vehicle selection and resource allocation remain open FL4CAV challenges because useful data, connectivity, device speed, storage, and update quality vary across vehicles. The survey points to adaptive selection, trust-aware learning, diagnostics, and broader FL design directions as responses.

  • Lack of methods for efficient vehicle selection and resource allocation: No efficient method currently filters useful CAV data to reduce network loading, although client-selection and resource-allocation schemes are being developed.Reported approaches select clients using resource availability or trust-based deep reinforcement learning.
  • Lack of methods for efficient vehicle selection and resource allocation: Limited vehicle storage and continuously generated data can cause the global model to forget previous knowledge during iterative updates.The survey identifies catastrophic forgetting as an open FL4CAV problem.
  • Lack of methods for efficient vehicle selection and resource allocation: Poor connectivity and slower straggler devices lengthen training and increase communication rounds because traditional rounds wait for all selected-device updates.Adaptive strategies aim to reduce or eliminate straggler effects.
  • Lack of methods for efficient vehicle selection and resource allocation: Newcomer vehicles may lack information for informed decisions, motivating trust-aware deep-RL assistance for trajectory and motion planning.
  • Lack of methods for efficient vehicle selection and resource allocation: Faulty sensors and incorrect missing-data imputation can introduce noisy updates, requiring diagnostics to identify and eliminate them.
  • Lack of methods for efficient vehicle selection and resource allocation: The survey identifies data integrity, communication latency, heterogeneous data, and model synchronization as distinctive FL4CAV challenges.It reports potential benefits in privacy preservation, network efficiency, and collaborative intelligence with proper design and implementation.
  • Lack of methods for efficient vehicle selection and resource allocation: Future applications include privacy-preserving driver behavior modeling, anomaly detection, and predictive maintenance.The survey also connects future FL4CAV development with cloud infrastructure, 6G, V2X, and flying cars.
Loading 2308.10407v2…