Source-linked AI summary
Federated Machine Learning: Concept and Applications
Qiang Yang, Yang Liu, Tianjian Chen, Yongxin Tong
TL;DR
Federated learning addresses isolated data and growing privacy requirements without requiring data owners to expose their local data. The paper surveys existing work and proposes a secure framework spanning horizontal, vertical, and transfer learning, concluding that federated mechanisms can support privacy-preserving knowledge sharing.
Problem
AI development is constrained by limited or isolated data in many fields and growing requirements for data privacy and security.
Method
The paper surveys federated learning and proposes a secure framework with definitions, categorizations, architectures, and applications across horizontal, vertical, and transfer learning.
Results
The paper presents federated learning as a possible solution that integrates knowledge while keeping data owners’ local data unexposed.
Takeaways & Limitations
Federated mechanisms could establish united models and data networks that enable organizations to share knowledge while treating data security as a premise.
Takeaways & Limitations
Federated learning can experience greatly reduced performance with non-IID local data.
Abstract
from arXiv · showhide
Today's AI still faces two major challenges. One is that in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated learning framework, which includes horizontal federated learning, vertical federated learning and federated transfer learning. We provide definitions, architectures and applications for the federated learning framework, and provide a comprehensive survey of existing works on this subject. In addition, we propose building data networks among organizations based on federated mechanisms as an effective solution to allow knowledge to be shared without compromising user privacy.
1 INTRODUCTION
The introduction identifies isolated data sources and growing privacy and security requirements as major obstacles to AI development. It presents federated learning as a possible solution and outlines a comprehensive secure framework, survey, and business applications.
- Data fragmentation: Most fields have limited or poor-quality data, while required AI data spans multiple sources that are difficult or impossible to combine across organizational barriers.The paper illustrates this issue with AI-driven product recommendation, which requires multiple types of organizational data.
- Privacy and security: Data privacy and security have become worldwide major issues, prompting public concern and stronger laws protecting data security and privacy.The introduction cites public data leaks, Facebook’s breach, and worldwide regulatory responses.
- Traditional data processing: Traditional AI data-processing models transfer, clean, fuse, and integrate data across multiple parties, creating challenges under new data regulations.These models typically involve one party collecting data, another cleaning and fusing it, and other parties using the resulting models.
- Federated learning approach: The article presents federated learning as a possible solution, surveying existing work and proposing definitions, categorizations, applications, and a comprehensive secure framework.It also discusses applying the framework successfully across various businesses.
2 AN OVERVIEW OF FEDERATED LEARNING
Federated learning trains models across distributed datasets while preventing data leakage. Research has focused on statistical challenges, security, personalization, and optimization issues in on-device settings.
- Core concept: Federated learning builds machine learning models from datasets distributed across multiple devices while preventing data leakage.The concept was recently proposed by Google.
- Research directions: Recent federated learning improvements address statistical challenges, security, and personalization.These efforts extend beyond the initial framework toward more secure and personalizable systems.
- On-device federated learning: On-device federated learning must optimize communication cost, unbalanced data distribution, and device reliability across massive distributions.These settings involve distributed mobile-user interactions, with data partitioned by user or device identifiers.
2.1 Definition of Federated Learning
Federated learning enables multiple data owners to collaboratively train a model without exposing their individual datasets. Its accuracy should remain close to that of centralized training, with the difference characterized as δ-accuracy loss.
- Definition: Federated learning lets N data owners collaboratively train a model while keeping each owner’s dataset private.Unlike centralized training, the process does not expose any data owner’s Di to others.
- Definition: The federated model’s accuracy is required to remain close to centralized training accuracy, with |V_FED − V_SUM| < δ defining δ-accuracy loss.The inequality compares federated and centralized model accuracy.
2.2 Privacy of Federated Learning
Privacy is essential in federated learning, requiring security models that provide meaningful guarantees while addressing indirect leakage. Existing approaches include secure multi-party computation, differential privacy, k-anonymity, homomorphic encryption, and blockchain-based protection mechanisms.
- Privacy requirements: Federated learning requires security models and analyses that provide meaningful privacy guarantees and address potential indirect leakage.The section reviews and compares privacy techniques while identifying challenges in preventing indirect leakage.
- Privacy techniques: Secure multi-party computation provides simulation-based security and complete zero knowledge, but its protocols may be computationally complicated and inefficient.Partial knowledge disclosure may be acceptable in certain scenarios.
- Privacy techniques: Differential privacy, k-anonymity, and diversification obscure sensitive attributes through noise addition or generalization to protect user privacy.These methods aim to prevent third parties from distinguishing individuals and restoring the protected data.
- Privacy techniques: Homomorphic encryption protects privacy by enabling parameter exchange under encryption, limiting leakage at the raw-data level without transmitting the data or model itself.The cited approach has also been adopted for centralized cloud-based training.
- Indirect leakage: Exposed federated-learning parameter updates, including SGD gradients, can lack security guarantees and leak important data information when combined with structures such as image pixels.Researchers have also considered malicious attacks by one federated-learning member against others.
- Blockchain-based federated learning: BlockFL uses blockchain to exchange and verify mobile devices’ local model updates while addressing block generation, network scalability, and robustness.The architecture treats blockchain as a platform for facilitating federated learning.
2.3 A Categorization of Federated Learning
Federated learning is categorized by how participating datasets overlap in samples, features, and labels. The framework distinguishes horizontal learning for shared features, vertical learning for shared sample IDs, and federated transfer learning when both samples and features differ.
- Categorization basis: The categorization analyzes each owner’s data matrix by sample rows, feature columns, optional labels, and the spaces X, Y, and I.These spaces represent features, labels, and sample IDs, respectively.
- Horizontal Federated Learning: Horizontal federated learning applies when datasets share a feature space but contain different samples, such as regional banks serving distinct user groups.Its defining relation is Xi = Xj and Yi = Yj, while sample-ID spaces differ.
- Security assumptions: Horizontal systems typically assume honest participants and protection against an honest-but-curious server, while vertical systems typically assume honest-but-curious, non-colluding parties.The vertical two-party definition limits an adversary to data from the corrupted client beyond what input and output reveal.
- Vertical Federated Learning: Vertical federated learning applies when datasets share the same sample-ID space but have different feature spaces, such as a bank and an e-commerce company.The parties aggregate different features and compute training loss and gradients collaboratively while preserving privacy.
- Federated Transfer Learning: Federated transfer learning applies when participating datasets differ in both samples and feature spaces, leaving only limited overlap across institutions.The paper illustrates this with a China-based bank and a United States-based e-commerce company whose user and feature intersections are both small.
2.4 Architecture for a federated learning system
The federated learning architecture supports privacy-preserving collaboration through distinct horizontal and vertical designs, with secure aggregation, encrypted computation, and iterative model updates. It also addresses broader deployment through transfer learning and incentive mechanisms for organizational participation.
- Horizontal federated learning: Horizontal federated learning lets participants with the same data structure jointly train a model through a parameter or cloud server without revealing participant information.Participants are typically honest while the server is honest-but-curious.
- Horizontal federated learning: Participants compute and mask gradients locally, the server securely aggregates them, and participants decrypt the aggregate to update their models until convergence.Masking can use encryption, differential privacy, or secret sharing; the architecture is independent of the specific machine learning algorithm.
- Security analysis: Secure aggregation with secure multiparty computation or homomorphic encryption protects against leakage to a semi-honest server, but malicious participants can still mount attacks such as GAN-based attacks.The security guarantee depends on the assumed threat model.
- Vertical federated learning: Vertical federated learning aligns common users through encryption and trains on shared entities using encrypted intermediate results, masked gradients, and a non-colluding collaborator.The parties keep their data locally and obtain model parameters associated only with their own features.
- Extensions and deployment: Transfer learning extends vertical federated learning beyond a small overlap of samples, while fair platforms and consensus-based incentives support commercialization among organizations.Federated learning is presented as a closed-loop mechanism that rewards data contributors.
3 RELATED WORKS
Federated learning is framed as privacy-preserving decentralized collaborative machine learning related to several established fields. The section connects it to privacy-preserving machine learning, distributed machine learning, and federated database systems while noting challenges from Non-IID data.
- Federated learning enables multiple parties to collaboratively construct a model while keeping their private training data private.
- Privacy-preserving machine learning: It is tightly related to multi-party privacy-preserving machine learning, including secure protocols for vertically partitioned decision trees, association mining, k-means, and Naive Bayes.
- Privacy-preserving machine learning: Prior work developed privacy-preserving linear regression for horizontally and vertically partitioned data, alongside SGD-based protocols for logistic regression and neural networks.
- Distributed machine learning: Horizontal federated learning resembles distributed machine learning, but distributed learning also covers storage, computing, model-result distribution, and Parameter Server coordination.
- Distributed machine learning: With non-iid local data, federated learning performance can be greatly reduced, prompting a transfer-learning-like method to address the issue.
- Federated database systems: Federated database systems similarly integrate independent, heterogeneous databases using distributed storage, creating similarities with federated learning in data type and storage.
4 APPLICATIONS
Federated learning enables organizations to train shared models without directly aggregating sensitive data, supporting applications across sales, finance, smart retail, and healthcare. Combined with transfer learning, it addresses privacy, security, and data heterogeneity while enabling services such as personalized recommendations and multiparty-borrowing detection.
- Cross-industry applications: Federated learning can train a unified model across parties without compromising data privacy and security where intellectual property, privacy, or security prevent direct aggregation.The paper identifies sales, financial, and other industries as promising application domains.
- Smart retail: In smart retail, federated learning combines purchasing power, personal preference, and product-characteristic data distributed across departments or enterprises without exporting enterprise data.The resulting model supports personalized and targeted customer services, while transfer learning addresses data heterogeneity.
- Finance: Federated learning supports multiparty database queries without exposing data, including detecting borrowers who use one bank’s loan to repay another bank.The paper presents multiparty borrowing as a major banking risk and a threat to financial stability.
- Smart healthcare: Smart healthcare can use federated learning to improve machine-learning models despite sensitive medical data, isolated hospitals and medical centers, insufficient sources, and limited labels.The paper cites disease symptoms, gene sequences, and medical reports as examples of sensitive medical data.
5 FEDERATED LEARNING AND DATA ALLIANCE OF ENTERPRISES
Federated learning is presented as both a technology standard and a business model arising from tensions between centralized data aggregation and growing concerns over privacy, security, and data-related profits.
- Federated learning is characterized as both a technology standard and a business model.The passage explicitly frames federated learning in both technological and commercial terms.
- Traditional big-data approaches aggregate data centrally, process models remotely, and return results for further use.Cloud computing is described as emerging from this demand for centralized computation.
- The growing importance of data privacy, data security, and companies’ financial relationships with their data challenges centralized aggregation.
6 CONCLUSIONS AND PROSPECTS
Federated learning is presented as a promising response to AI’s challenges of isolated data and strengthened privacy. The article introduces its concepts, architecture, techniques, and potential applications while envisioning secure collaboration among enterprises.
- Conclusions and Prospects: Federated learning could unite multiple enterprises around a shared model while protecting local data.This approach frames data security as a premise for mutually beneficial enterprise collaboration.
- Conclusions and Prospects: The article introduces the basic concept, architecture, and techniques of federated learning.
- Conclusions and Prospects: The paper discusses federated learning’s potential across various applications.