Multi-Participant Multi-Class Vertical Federated Learning

1 Introduction

• Federated learning suits settings where data are isolated across organizations and privacy is emphasized, unlike approaches that centralize data and learning.The introduction frames this as a motivation for distributed, privacy-preserving machine learning.Source: §1 · p. 1
• VFL assumes participants share sample IDs but have different feature spaces, with label information held by one participant.A central challenge is enabling privacy-preserving use of that participant’s labels for training.Source: §1 · p. 1
• Prior VFL approaches generally handle only two participants and binary classification, limiting their suitability for complex multi-participant tasks.The introduction identifies these restrictions as a barrier to complex VFL applications.Source: §1 · p. 1
• MMVFL extends multi-view learning to multi-class VFL with multiple participants, separate participant models, and privacy-preserving label sharing.Data and labels remain with their owners during training.Source: §1 · p. 1
• MMVFL’s feature-importance scheme quantifies each participant’s feature contributions and can reduce later incremental-training communication, computation, and storage costs by discarding redundant or harmful features.The feature discarding is described for initial training periods.Source: §1 · pp. 1–2
• MMVFL’s multi-class classification performance matches existing approaches while sharing label information among multiple VFL participants.The framework is presented as the first VFL framework for multi-class problems with multiple participants.Source: §1 · p. 2

2 Related Work

• Hardy et al. [Hardy et al., 2017] proposed federated logistic regression with additively homomorphic encryption and analyzed entity-resolution mistakes; Nock et al. [Nock et al., 2018] extended the error analysis across losses.Yang et al. [Yang et al., 2019a] reduced communication rounds, while Yang et al. [Yang et al., 2019d] removed the third-party coordinator.Source: §2 · p. 2
• Wang et al. [Wang et al., 2019] assessed feature-group importance with Shapley Values, whose computation has exponential complexity.The method dynamically removes feature groups to measure their impact on model performance.Source: §2 · p. 2
• Existing VFL approaches generally support only two participants and focus on binary classification, limiting complex multi-participant applications.MMVFL is designed for multi-class, multi-participant VFL settings.Source: §2 · p. 2

Multi-View Learning

• Multi-view learning models each view with a function and jointly optimizes the functions to improve generalization.Its shared sample IDs and heterogeneous features fit VFL, but existing methods require raw data interaction across views, conflicting with privacy preservation.Source: Multi-View Learning · p. 2

Feature Selection

• Feature selection chooses a useful subset of dataset features for a learning task and can save federated communication costs by compressing data according to feature importance.This common practice measures feature importance and discards less important features.Source: Feature Selection · p. 2

4 The Proposed MMVFL Framework

• Only locally predicted labels cross the privacy barrier; raw data, labels, and local models remain on their owners’ machines.Source: §4 · p. 2

4.1 Notations and Problem Definition

• The setting assumes K participants with local datasets for an Nc-class task, participant 1 holding the labels, and sample IDs already matched.Source: §4.1 · p. 2

4.2 Sparse Learning-based Unsupervised Feature Selection

• For participants without labels, MMVFL learns a transformation and pseudo-label embedding; the row ℓ2-norms of the transformation matrix provide feature-importance scores.The linear transformation scheme is adopted as a proof of concept rather than an exhaustive choice of feature-selection methods.Source: §4.2 · pp. 2–3

4.3 Privacy-Preserving Label Sharing

• The VFL objective aligns participants’ local pseudo-label matrices with a shared matrix and anchors participant 1’s matrix to the true one-hot labels, enabling indirect label sharing.Source: §4.3 · p. 3

4.4 Optimization

• Large penalty terms relax Zk=Z and Z1=Y, while alternating updates address the ℓ2,1-regularized objective until convergence or the iteration limit.Source: §4.4 · p. 3

5.1 Convergence

• MMVFL’s objective is consistently non-increasing during optimization, with each stated subproblem having a global minimum.The Z1, Zk, and Z subproblems are simple convex optimization problems when other parameters are fixed.Source: §5.1 · p. 4

5.2 Time Complexity

• Each federated iteration requires communication among all participants, and its time complexity depends on the participant with the largest feature dimension.The paper identifies the slowest participant in each round as the straggler.Source: §5.2 · p. 4

5.3 Privacy Preservation

• MMVFL transmits participants’ Zk values to the server while Xk and Y remain with their owners, supporting privacy-preserving label sharing.The paper states that the transformation matrices alone are insufficient to recover the original data, even if intercepted over multiple rounds.Source: §5.3 · p. 4

6.1 Real-world Data

• Handwritten and Caltech7 provide five and six views, respectively; class counts are balanced across training and validation sets, and a six-feature Handwritten view was removed.The removed view had morphological features; the paper says its six features made feature selection insignificant.Source: §6.1 · p. 4, §6.2 · p. 4, Table 1 · p. 4

6.2 Comparison Baselines

• supFL selects features independently assuming all participants know labels; supMVLFL extends it to multi-view linear transformations, and neither preserves data privacy during information sharing.Both are described as distributed approaches capable of sharing information across participants.Source: §6.2 · p. 5

6.3 Experiment Settings

• Five-fold cross-validation uses four folds for training, tunes algorithms at each selected-feature percentage, and reports mean classification accuracy for each view and dataset.The selected-feature percentages range from 2% to 100%.Source: §6.3 · p. 5

6.4 Results and Discussion

• In some cases, MMVFL matches or exceeds approaches using all features while using fewer important features, reducing resource needs especially in incremental-learning settings.The stated resources include communication bandwidth, computing devices, and memory space.Source: §6.4 · p. 5
• Across five Handwritten participants, MMVFL beats supFL by 1.42% and supMVLFL by 2.31%; across six Caltech7 participants, it trails them by 1.21% and 0.88%, respectively.The reported differences are averaged over participants; Table 2 gives differences across selected-feature choices.Source: Table 2 · p. 5, §6.4 · p. 5

• Feature-selection experiments show MMVFL achieves performance comparable to its supervised counterparts.Source: §7 · p. 5
• MMVFL shares owner-held labels with other participants without data leakage and supports more complex scenarios than existing two-participant techniques.The authors describe it as the first attempt to transfer multi-view learning into VFL.Source: §7 · p. 5
• Future work targets more sophisticated classifiers, communication efficiency, straggler handling, and the effect of relationships across participants' tasks on overall model performance.Source: §7 · p. 5
• Figures 3 and 4 plot classification performance against the percentage of selected features on Handwritten and Caltech7, respectively.Source: Figure 3 · p. 6, Figure 4 · p. 6