Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator

1 Introduction

• Data-sharing risks and legal restrictions make joint machine-learning training across organizations challenging.The introduction identifies risks of data abuse or secondary distribution, as well as data that cannot be circulated under laws and regulations.Source: §1 · p. 1
• Vertical federated learning applies when parties share sample IDs but hold different features; horizontal learning shares features but has different samples.Federated transfer learning covers parties differing in both samples and feature space.Source: §1 · p. 1
• The proposed architecture removes the third-party coordinator, allowing two parties to train a joint model without finding a trusted intermediary.The paper cites reduced system complexity and lower deployment cost as benefits of removing the coordinator.Source: §1 · p. 2
• The implementation uses parallel distributed training to handle very large volumes of data rather than relying on one machine at each party.The proposed system runs the training algorithm on a cluster of machines at each party.Source: §1 · p. 2
• The proposed architecture is also presented as extensible from two-party to multi-party model training.The paper describes multi-party support as an extension of its system architecture.Source: §1 · p. 2

2 Related Work

• Federated learning trains a joint model on decentralized data while preserving data privacy, with horizontal, vertical, and transfer variants organized by data distribution.The cited framework categorizes these variants according to how parties' samples and feature spaces differ.Source: §2 · p. 2
• Homomorphic encryption supports computation on encrypted data whose decrypted result matches computation on the plaintext.The paper notes that additively homomorphic encryption is commonly used because of its relatively superior computation efficiency.Source: §2 · p. 2
• Sending raw data to a central server encrypted with AHE can protect raw data, but adds memory and processing costs and leaves data off-site.The paper describes an alternative of exchanging encrypted intermediate results while keeping raw data locally and minimizing the amount encrypted.Source: §2 · pp. 2–3
• Compared with Hardy et al. [2017], the solution removes the coordinator, keeps the loss function unchanged instead of using a polynomial approximation, and targets large datasets with parallel scalable training.Source: §2 · p. 3

3.1 Overview of Logistic Regression

• In two-party vertical learning, logistic regression models a binary dependent variable using feature sets and parameters held across parties A and B.The section introduces the training objective and its gradients for the partitioned feature space.Source: §3.1 · p. 3

3.2 Protocol of Model Training

• The protocol keeps raw features, labels, and model parameters local while exchanging intermediate data, and repeats Steps 2–6 until an iteration limit or convergence condition.The protocol description identifies encrypted residuals and randomly masked gradients among the exchanged intermediates.Source: §3.2 · p. 3

3.3 System Architecture

• Each party uses a centralized parameter server with worker nodes, and the parties communicate only between their corresponding parameter servers.Worker nodes communicate only with their own party’s parameter server.Source: §3.3 · p. 4, Figure 3 · p. 4

3.4 Security Analysis

• The analysis says B can recover residuals from gradient combinations when m≥n, and residual signs then reveal labels; encrypted residuals are transmitted to protect them.The authors conclude the solution is secure in practice through careful boundary-condition checking.Source: §3.4 · p. 4
• For A’s inference of B’s feature data, the analysis counts transmitted equations against unknowns and warns that a malicious A can isolate feature values by setting residuals selectively.The authors’ practical-security conclusion depends on careful boundary-condition checking.Source: §3.4 · p. 4

4.1 Settings

• Experiments evaluated correctness, efficiency, and scalability using MNIST and Citeseer, a sparse, highly skewed binary-classification dataset.MNIST was transformed into an odd-versus-even digit binary-classification task.Source: §4.1 · p. 5, §4 · p. 5
• Experiments ran on Baidu Cloud with one parameter server and multiple workers per party; each machine had 16 CPU cores and 64 GB RAM, using eight threads.Source: §4.1 · p. 5

4.2 Results

• Parameter-server computation remained a performance bottleneck; offloading some computations to worker nodes could further reduce training time.Source: §4.2 · p. 5
• The learned models showed excellent AUC, rapid convergence to a steady state, and sub-linear speedup, with runtime comparable to traditional logistic regression.Table 5 reports AUC; Figures 4 and 5 show convergence and scalability results, respectively.Source: Table 5 · p. 5, Figure 4 · p. 5, Figure 5 · p. 5

• Removing the third-party coordinator lets any two parties run federated logistic regression directly, avoiding coordinator trust difficulties and reducing deployment complexity.Source: §5 · p. 6
• Evaluation on two datasets, one sparse and one dense, showed efficiency and scalability of the system.Source: §5 · p. 6
• The architecture can be extended to multiple parties; future work will combine logistic regression with deep neural networks for vertical federated learning.Source: §5 · p. 6