PaperlayerView exact revision on arXiv

Source-linked AI summary

Decentralized Federated Learning: A Survey and Perspective

Liangqi Yuan, Ziran Wang, Lichao Sun, Philip S. Yu, Christopher G. Brinton

arXiv:2306.01603v2cs.LGcs.CYcs.DCcs.NI

TL;DR

DFL addresses the limited comprehensive coverage of decentralized federated learning, which removes central-server coordination while retaining model-based collaboration. The paper reviews CFL, systematically organizes DFL taxonomies and variants, and discusses challenges, solutions, and future directions. It concludes by synthesizing the DFL literature and proposing perspectives intended to support researchers and future collaborations.

  • Problem

    Existing FL research has focused largely on centralized FL, while DFL lacks a comprehensive survey covering its system-building logic and emerging variants.

  • Method

    The paper reviews CFL and systematically organizes DFL by iteration order, communication protocol, network topology, paradigm proposal, temporal variability, and topology-based variants.

  • Results

    The paper provides a comprehensive perspective synthesizing DFL definitions, communication protocols, network topologies, paradigms, extension variants, challenges, and potential solutions.

  • Takeaways & Limitations

    The synthesis establishes a theoretical foundation for understanding DFL and is intended to facilitate introductions, new ideas, and collaborations among researchers.

Abstract

from arXiv · show

Federated learning (FL) has been gaining attention for its ability to share knowledge while maintaining user data, protecting privacy, increasing learning efficiency, and reducing communication overhead. Decentralized FL (DFL) is a decentralized network architecture that eliminates the need for a central server in contrast to centralized FL (CFL). DFL enables direct communication between clients, resulting in significant savings in communication resources. In this paper, a comprehensive survey and profound perspective are provided for DFL. First, a review of the methodology, challenges, and variants of CFL is conducted, laying the background of DFL. Then, a systematic and detailed perspective on DFL is introduced, including iteration order, communication protocols, network topologies, paradigm proposals, and temporal variability. Next, based on the definition of DFL, several extended variants and categorizations are proposed with state-of-the-art (SOTA) technologies. Lastly, in addition to summarizing the current challenges in the DFL, some possible solutions and future research directions are also discussed.

I. INTRODUCTION

Federated learning shares model weights rather than user data, while decentralized FL removes the central server and enables direct client communication. This paper surveys DFL by organizing its architectures, variants, challenges, solutions, and future directions.

  • Federated learning background: Federated learning shares model weights instead of user data, supporting privacy-preserving collaboration across application domains.The paper describes FL applications spanning transportation, IoT, healthcare, manufacturing, agriculture, energy, and remote sensing.
  • Centralized and decentralized architectures: Centralized FL has clients train locally, upload model parameters to a server, and receive an aggregated global model.Clients contribute to one global model while withholding users’ raw data from the server.
  • Centralized and decentralized architectures: Decentralized FL removes server coordination, allowing clients to exchange model parameters directly through more diverse communication networks.The server-free design is presented as more customizable and potentially capable of saving communication and computational resources.
  • Research gap: Existing DFL surveys provide limited comprehensive coverage of system-building logic across iteration, communication protocol, network topology, paradigm, and related dimensions.The paper positions its survey as addressing this gap in the literature.
  • Paper contributions: The paper defines five DFL taxonomies, proposes topology-based variants, and summarizes five challenges with possible solutions and future research directions.Its organization covers iteration order, communication protocol, network topology, paradigm proposal, and temporal variability.

II. REVIEW OF CENTRALIZED FEDERATED LEARNING

Centralized federated learning uses a server to aggregate client models, but the server introduces a single point of failure and other challenges motivating alternative variants.

  • FedAvg has clients upload local models each round for server-side weighted averaging by client sample counts.
  • Server centralization creates a single point of failure, prompting variants and technologies intended to address server-related challenges.

A. Challenges in Centralized Federated Learning

Centralized federated learning faces challenges from scaling, heterogeneous clients, limited resources, trust concerns, and unreliable connections.

  • Growing client populations strain large service providers serving as centralized servers.
  • Client heterogeneity arises from individual, group, and systemic differences.
  • Communication remains constrained because many parallel clients can create delays that leave the server waiting.
  • Server computation and storage may reach petabyte scale when storing and aggregating models for billions of clients.
  • Fairness, security, and trust concerns affect system reliability, user confidence, and data integrity.
  • Unreliable communication, attacks, or server malfunctions can cause delays, packet loss, noise, or system-wide interruption through a single point of failure.

B. Variants of Centralized Federated Learning

Centralized federated learning variants adapt architecture, personalization, computation, data structure, synchronization, and deployment to different challenges and scenarios.

  • Hierarchical FL: Hierarchical FL adds edge aggregation between clients and the cloud to reduce communication overhead and distribute communication and computing pressure.
  • Personalized FL: Personalized FL uses global-model personalization or personalized model architectures to adapt models to users and non-IID data.
  • Split FL: Split FL divides model learning across client and server layers, sending hidden representations or gradients instead of user data.
  • Graph FL: Graph FL targets graph-structured applications and can also use client-topology graphs based on connectivity, availability, similarity, or data generality.
  • Asynchronous FL: Asynchronous FL lets heterogeneous clients train and update at different times without the server waiting for all clients to synchronize.
  • CFL variants may combine single, multiple, sub, or master servers for large-scale and heterogeneous environments.

C. Definition of Client and Server

The paper defines FL roles relationally and describes DFL through taxonomies covering iteration, protocols, topologies, and related network configurations. These choices matter because DFL lacks centralized coordination and must integrate models across diverse connections and versions.

  • Definition of Client and Server: A communication endpoint is a client when it generates and locally stores raw data, while client and server roles can coexist institutionally.
  • Definition of Client and Server: Institutions acting as both data holders and aggregators raise fairness concerns because local data may receive preferential treatment.
  • Table II organizes DFL using five taxonomies that may overlap, conflict, or complement one another.
  • Iteration Order: In DFL, client iteration order can affect convergence and performance, unlike parallel CFL reception order.
  • Communication Protocol: DFL shares model weights through pointing, gossip, or broadcast protocols to obtain models across clients.
  • Communication Protocol: Hybrid protocols combine gossip and broadcast structures for different scenarios and constraints.

C. Network Topology

DFL network topology determines how clients exchange and accumulate model knowledge, while the Continual and Aggregate paradigms trade communication, computation, convergence, personalization, and generalization differently.

  • Network Topology: DFL lacks centralized management, so diverse topologies and protocols create multiple model versions and make complete knowledge access more difficult.This motivates paradigms that are not limited to centralized-style aggregation.
  • Paradigm Proposal: Continual clients learn directly from one peer’s model, whereas Aggregate clients combine received models before local learning.Aggregate exchanges more model information; Continual forwards only the trained model to the next client.
  • Paradigm Proposal: Continual supports sequential learning with minimal retained model data and accommodates continuously generated data and concept drift.Clients can learn without waiting for all data and receive the latest model version.
  • Paradigm Proposal: Continual requires more communication rounds, while Aggregate incurs greater per-round communication overhead.The paradigms differ in preprocessing, model sharing, and aggregation requirements.
  • Paradigm Proposal: Continual offers stronger personalization and lower aggregation-related resource demands, whereas Aggregate is expected to provide more stable and faster convergence and greater generalization.Aggregate’s convergence advantage is accompanied by increased communication overhead.

E. Temporal Variability

DFL topology is shifting toward dynamic structures that adapt to changing environments, interference, resource constraints, and fairness requirements.

  • Temporal Variability: DFL topologies increasingly shift from static to dynamic configurations in response to time-varying external environments.The paper relates this design direction to adaptive group behaviors in fish schools and bee swarms.
  • Temporal Variability: External interference, single points of failure, and malicious attacks can trigger topology changes or node discards to avoid whole-network failure.Topology adjustment is presented as a response to communication barriers and other disruptions.
  • Temporal Variability: Clients can dynamically select nearby neighbors or elect a central client to optimize communication resources and efficiency.These choices adapt communication relationships during operation.
  • Temporal Variability: Fig. 5 uses arrow length in parameter space as an ℓ2-norm-related indicator of learning difficulty and parameter-change magnitude.Shorter arrows are associated with more accessible, stable, and accurate learning and convergence.
  • Temporal Variability: Random communication-target selection supports fairness by limiting bias toward particular clients.Dynamic topology design combines robustness, efficiency, and equal participation opportunities.

IV. VARIANTS OF DECENTRALIZED FEDERATED LEARNING

The paper extends its DFL framework into multiple topology variants and reviews their advantages and limitations for application-specific deployment choices.

  • Variants of Decentralized Federated Learning: The paper proposes viable DFL topology variants by combining CFL-variant insights with underlying network topologies.The variants are intended as alternative options for researchers deploying DFL.
  • Variants of Decentralized Federated Learning: Each topology variant is discussed with associated advantages and limitations to support selection for specific usage scenarios.The paper frames deployment choice as dependent on application requirements.
  • Variants of Decentralized Federated Learning: The survey also connects DFL variants to real-world applications and deployments across diverse domains.This situates topology choices within practical deployment contexts.

A. Real-World Applications

DFL is positioned for privacy-sensitive, distributed, mobile, and resource-constrained applications, with deployment patterns spanning vehicles, healthcare, IIoT, UAVs, satellites, social networks, and AGI.

  • Real-World Applications: DFL application domains include connected vehicles, healthcare institutions, industrial IoT, mobile services, UAVs, satellites, social networks, and AGI.These settings combine distributed data, heterogeneous devices, or privacy and communication constraints.
  • Real-World Applications: Vehicle-to-vehicle FL enables vehicles to exchange and share up-to-date knowledge, while vehicular DFL studies emphasize privacy protection and data-leakage risks.Vehicles provide batteries, sensors, computing, storage, and established V2V communication infrastructure.
  • Real-World Applications: Healthcare institutions favor DFL because they possess sensitive patient data alongside substantial computational and storage resources.Clinicians can observe, analyze, fine-tune, and match models without relying on a traditional server-centric approach.
  • Real-World Applications: DFL supports IIoT robustness against single points of failure and suits mobile IoT systems with flexible connectivity and diverse sensors.Its autonomy is presented as suitable for varied industrial physical and operational environments.
  • Real-World Applications: For UAVs and satellites, broadcast-gossip DFL can reduce bandwidth and communication-resource consumption while supporting responsiveness in dynamic environments.These platforms have sensitive data and limited resources, making bandwidth a key constraint.
  • Real-World Applications: DFL applications in AGI remain nascent because billion-scale models pose unique challenges, despite methods reducing communication to a few random seeds and scalar gradients.The paper notes that decentralized LLM design has also been addressed theoretically.

B. Variant: Line

The line variant is the simplest sequential DFL topology and a useful baseline, but it has substantial limitations in learning continuity, generalization, convergence, and fault tolerance.

  • B. Variant: Line: The line variant is a sequential pointing topology used as a simple, intuitive, and efficient DFL baseline.Its ease of implementation supports comparison and initial experimentation.
  • B. Variant: Line: It cannot accommodate continuous learning effectively and risks catastrophic forgetting or redundant learning, depending on the paradigm.The Continual paradigm risks catastrophic forgetting, while the Aggregate paradigm can cause redundant and excessive learning.
  • B. Variant: Line: The topology limits generalization for starting clients and remains vulnerable to a single point of failure.These limitations constrain reliability and performance at the front of the client queue.
  • B. Variant: Line: Without cyclic connections, each client is limited to one iteration, preventing full convergence and worsening performance for clients at the queue front.The line variant can therefore serve as a baseline or initial implementation rather than a complete topology.

C. Variant: Ring

DFL topology variants trade off simplicity, communication patterns, resilience, adaptability, resource efficiency, security, and personalization. Ring and mesh structures support cyclic or multidirectional exchange, while star and hybrid structures organize communication through central or hierarchical clients.

  • C. Variant: Ring: The ring variant adds cyclic communication to the line topology, helping clients acquire knowledge from other clients and enhance generalization.It retains the line variant’s simplicity while avoiding purely sequential exchange.
  • D. Variant: Mesh: Mesh topology sends each client’s local model to all other clients, increasing communication frequency while reducing the model-packet size per transmission.This creates a bandwidth and communication-frequency trade-off relative to the ring variant.
  • D. Variant: Mesh: Mesh topology significantly mitigates the impact of a single point of failure compared with ring topology.The multidirectional structure distributes communication across more client connections.
  • E. Variant: Star: Star topology assigns one client to coordinate, aggregate, and distribute models, either by serving as a computational leader or a geographic center.The surrounding clients transmit models to the central client, which forwards them to others in the geographic mode.
  • F. Variant: Hybrid: Hybrid topology combines multiple variants and is considered adaptable for practical scenarios, although configuring it can be complex.One configuration connects two ring variants through central clients to provide global connectivity.
  • F. Variant: Hybrid: Hybrid organization can reduce communication overhead by sharing aggregated global models between organizational leaders and limiting irrelevant information exchange.The cited configuration also supports organization-specific models and permits leaders to disconnect inter-organization communication.

V. CHALLENGE AND POTENTIAL SOLUTIONS IN DFL

DFL remains constrained by communication costs and client-side resource demands despite eliminating server-mediated communication. Proposed directions include topology optimization, model compression, leader-based communication, and transfer learning.

  • V. CHALLENGE AND POTENTIAL SOLUTIONS IN DFL: Further reductions in communication resources and complexity remain an important DFL challenge despite DFL’s recognized efficiency relative to CFL.The survey identifies topology construction and model compression as a direction for addressing this challenge.
  • V. CHALLENGE AND POTENTIAL SOLUTIONS IN DFL: Star and hybrid variants can improve communication efficiency by selecting geocentric or resource-rich clients as leaders.Other strategies focus on efficient communication lines and differences in client bandwidth.
  • V. CHALLENGE AND POTENTIAL SOLUTIONS IN DFL: Deployment in real-world applications is expected to benefit from continued efforts to reduce communication complexity and optimize communication resources.The paper presents these directions as ongoing areas for exploration.
  • V. CHALLENGE AND POTENTIAL SOLUTIONS IN DFL: The Aggregate paradigm places greater computational and storage demands on clients because they retain previous parameters and aggregate models without a dedicated server.These burdens challenge client hardware.
  • V. CHALLENGE AND POTENTIAL SOLUTIONS IN DFL: Fixing lower-layer weights through transfer learning can reduce gradient, aggregation, communication, and storage requirements.Lower layers act as task-specific feature extractors while higher-level representations remain task-specific.

C. Vulnerability in Cybersecurity

DFL’s server-free structure creates cybersecurity, participation, coordination, and trust challenges. The survey discusses privacy, attack defenses, reputation mechanisms, leader coordination, and broader topology-based solutions, while identifying incentives as unresolved.

  • C. Vulnerability in Cybersecurity: Direct client-to-client knowledge exchange makes network security especially prominent in DFL compared with the more regulated communication of CFL.The section frames attacks and data poisoning as important concerns in decentralized exchange.
  • C. Vulnerability in Cybersecurity: Blockchain, differential privacy, secure aggregation, and poisoning defenses are proposed as protections for privacy, trust, and model integrity.Examples include Multi-Krum, differential-privacy noise, and secure aggregation in Biscotti.
  • D. Lack of Incentive Mechanism: Although DFL can address fairness in aggregation without server management, lacking incentives may encourage free-riding and reduce clients’ willingness to contribute.Clients may benefit from models without contributing their own knowledge.
  • D. Lack of Incentive Mechanism: Effective incentive mechanisms remain an open problem because game-theoretic approaches may require leaders, participants, and rewards that are difficult to realize in DFL.Reputation scores, blockchain, and smart contracts are identified as possible directions.
  • E. Lack of Management: Without central management, heterogeneous client resources and communication capabilities complicate organized knowledge sharing and synchronization.Ring clients wait for the previous client, whereas mesh clients wait for models from all other clients.
  • E. Lack of Management: Pre-requesting client status and appointing leaders can regulate knowledge dissemination in decentralized networks.These approaches use model-version information or designated leaders to organize transfers.
  • VI. CONCLUSION: The survey organizes DFL’s variants, challenges, solutions, and future directions to provide a foundation for further research and collaboration.It also proposes unexplored combinations such as Continual learning with mesh topology.
Loading 2306.01603v2…