PaperlayerView exact revision on arXiv

Source-linked AI summary

Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer

Hongyan Chang, Virat Shejwalkar, Reza Shokri, Amir Houmansadr

arXiv:1912.11279v1stat.MLcs.CRcs.LG

TL;DR

Federated learning’s parameter sharing exposes private information, enables poisoning, and restricts collaboration to homogeneous models. Cronus transfers knowledge through black-box predictions, reducing update dimensionality and supporting heterogeneous models; across evaluated attacks, its accuracy drops by at most 2% while membership inference approaches random guessing.

  • Problem

    Federated learning is vulnerable to poisoning and inference attacks, leaks information through shared parameters, and cannot aggregate heterogeneous models.

  • Method

    Cronus extracts, aggregates, and transfers knowledge through black-box model predictions rather than shared parameters, using aggregated predictions to fine-tune local models.

  • Results

    Across strongest evaluated poisoning attacks, Cronus’s prediction accuracy drops by at most 2%.

  • Takeaways & Limitations

    Black-box prediction exchange supports heterogeneous collaboration, tight robustness guarantees, reduced information leakage, and privacy-preserving mechanisms for model outputs.

Abstract

from arXiv · show

Collaborative (federated) learning enables multiple parties to train a model without sharing their private data, but through repeated sharing of the parameters of their local models. Despite its advantages, this approach has many known privacy and security weaknesses and performance overhead, in addition to being limited only to models with homogeneous architectures. Shared parameters leak a significant amount of information about the local (and supposedly private) datasets. Besides, federated learning is severely vulnerable to poisoning attacks, where some participants can adversarially influence the aggregate parameters. Large models, with high dimensional parameter vectors, are in particular highly susceptible to privacy and security attacks: curse of dimensionality in federated learning. We argue that sharing parameters is the most naive way of information exchange in collaborative learning, as they open all the internal state of the model to inference attacks, and maximize the model's malleability by stealthy poisoning attacks. We propose Cronus, a robust collaborative machine learning framework. The simple yet effective idea behind designing Cronus is to control, unify, and significantly reduce the dimensions of the exchanged information between parties, through robust knowledge transfer between their black-box local models. We evaluate all existing federated learning algorithms against poisoning attacks, and we show that Cronus is the only secure method, due to its tight robustness guarantee. Treating local models as black-box, reduces the information leakage through models, and enables us using existing privacy-preserving algorithms that mitigate the risk of information leakage through the model's output (predictions). Cronus also has a significantly lower sample complexity, compared to federated learning, which does not bind its security to the number of participants.

I. INTRODUCTION

Federated learning exchanges model parameters to build a shared model, but this creates privacy, poisoning, dimensionality, and heterogeneity problems. Cronus instead transfers aggregated predictions from black-box models, enabling robust and privacy-aware collaboration while retaining accuracy.

  • Federated learning exchanges local model parameters through a server, assumes homogeneous architectures, and aims to produce a shared global model without pooling private data.
  • Existing federated learning is vulnerable to adversarial updates, backdoor attacks, sensitive-data leakage, and collaboration across heterogeneous models.
  • High-dimensional parameter updates raise robust aggregation error bounds and sample complexity, while overwriting local models increases susceptibility to poisoning and backdoors.
  • Cronus extracts, aggregates, and transfers model knowledge through black-box predictions, supporting heterogeneous architectures using an unlabeled public dataset.
  • Cronus fine-tunes local models with aggregated predictions and reduces exchanged update dimensionality to output size, enabling tight robustness guarantees even with few participants.
  • Under strongest evaluated poisoning attacks, Cronus loses at most 2% accuracy, with drops of 1.6% on Purchase, 1.3% on SVHN, 1.5% on MNIST, and 2.1% on CIFAR10.
  • Cronus reduces membership-inference risk, with attacks near random guessing and negligible accuracy loss, and supports heterogeneous models without compromising final accuracy.

II. THREAT MODELS

The threat model considers poisoning and inference attacks against federated learning, where parties share parameter updates with a central server. Its robustness depends on update dimensionality, and standard aggregation remains vulnerable to malicious participants.

  • The paper studies poisoning attacks against aggregation robustness and inference attacks against privacy of models’ sensitive training data.
  • The adversary controls an ǫ fraction of parties, has same-distribution data, and crafts malicious updates using knowledge of benign updates.
  • Membership inference evaluates whether a sample belongs to a party’s private training data using classifiers trained on victim-model features.
  • FedAvg repeatedly trains local models on private datasets, sends parameter updates to a server, aggregates them using data-size-based weights, and broadcasts the result.
  • Table I defines aggregation error in terms of party count, malicious fraction, update dimension, and per-dimension variance; Cronus’s error is independent of update dimension.
  • Weighted averaging cannot tolerate even one malicious party, while robust alternatives replace the mean with statistics such as coordinate-wise weighted median.

B. Krum

Krum selects one received update based on its distances to nearby updates, relying on benign updates being close together in parameter space.

  • Krum assumes benign party updates are normally distributed and close to one another in parameter space.
  • For each received update, Krum scores the distances to its closest (1−ǫ)n−2 neighbors and selects the update with the lowest score.

C. Bulyan

Bulyan combines vector-wise outlier filtering with coordinate-wise trimmed-mean aggregation to address weaknesses in Krum-like defenses. It retains updates selected by a Byzantine-resilient rule before applying TrimmedMean.

  • C. Bulyan: Krum selects the update closest to its neighbors but does not filter coordinate-wise outliers.Bulyan was proposed to address this limitation by combining vector-level and coordinate-level filtering.
  • C. Bulyan: Bulyan performs vector-wise and coordinate-wise filtering in two steps.It first uses a resilient aggregation rule such as Krum to select updates, then applies a TrimmedMean variant to the selected set.
  • C. Bulyan: TrimmedMean aggregates the values in each coordinate using the subset nearest to that coordinate’s median.The implementation uses the original Bulyan variant, where U_j contains the top-(1−2ϵ)n nearest values to median µ_j.
  • C. Bulyan: Multiplicative weight update reduces parties’ weights according to their distance from the aggregate under assumptions about malicious updates and party counts.The framework initializes weights equally, repeatedly updates them, and recomputes the aggregate until convergence.

1) MWU with mean aggregation:

MWU with mean aggregation updates party weights from their distances to the aggregate and then recomputes a weighted mean. The section also describes poisoning attacks designed to exploit these aggregation mechanisms.

  • 1) MWU with mean aggregation:: MWU begins with equal party weights and updates them according to distance from the weighted aggregate.The weighted aggregate is recomputed after each update until the convergence criterion is satisfied.
  • 1) MWU with mean aggregation:: An omniscient adversary estimates the benign-update distribution and crafts a malicious update arbitrarily far from its mean.The malicious vector can use arbitrarily large coordinate values and can jeopardize weighted averaging and weighted-median aggregation.
  • 1) MWU with mean aggregation:: The LIE attack uses small perturbations across many dimensions to affect convergence and circumvent robust aggregation defenses.It targets the sensitivity of neural-network parameters rather than relying on one arbitrarily large coordinate.
  • 1) MWU with mean aggregation:: The LIE attack is evaluated using an algorithmic input containing the number of workers, malicious fraction, and benign-update mean and variance vectors.The attack outputs a malicious update after perturbing coordinate values using the estimated distribution.
  • 1) MWU with mean aggregation:: The cited limitation states that one aggregation variant works against Krum but not Bulyan and is therefore omitted from evaluation.This scope boundary concerns the evaluation of that attack or defense variant.

D. Our poisoning attack (OFOM)

OFOM targets weighted aggregation schemes whose weights start equally and are updated using distances from an aggregate. It uses two malicious updates to manipulate the first weight-update epoch.

  • D. Our poisoning attack (OFOM): OFOM targets weighted aggregators that assign weights according to distances from an aggregate.The attack exploits the equal initialization of all parties’ weights.
  • D. Our poisoning attack (OFOM): OFOM crafts one malicious update arbitrarily far from the benign mean and another at the empirical benign mean.The two updates are formalized together in equation (9).
  • D. Our poisoning attack (OFOM): After the first MWU epoch, the adversary assigns a weight close to 1 to the malicious update located at the empirical benign mean.This exploits the distance-based weight update used by the targeted aggregators.
  • D. Our poisoning attack (OFOM): With MWUAvg and MWUOpt, benign parties receive negligible weights, substantially damaging aggregation accuracy.The attack requires only two malicious parties sharing the two crafted updates.

E. Membership inference attacks

Cronus addresses federated learning’s privacy, robustness, dimensionality, and heterogeneity limitations by exchanging black-box predictions rather than model parameters. This supports robust aggregation and avoids overwriting local models.

  • Motivation: Federated learning parameter sharing enables active and passive membership inference attacks against private training data.Attackers can observe or manipulate updated parameters to infer whether target samples belong to a party’s training set.
  • Cronus design: Knowledge transfer shares a model’s learned knowledge through predictions rather than raw model parameters.Predictions on data disjoint from the training set can also reduce information leakage and support privacy-preserving mechanisms.
  • Heterogeneity: Cronus supports heterogeneous architectures because participating models need only agree on their output vectors for the shared task.The approach can accommodate different model architectures or even different machine-learning algorithm families.
  • Cronus design: Cronus parties fine-tune local models using aggregated predictions instead of overwriting parameters with a global model.This limits the impact of harmful updates and allows local models to retain separate architectures.
  • Cronus design: Cronus reduces update dimensionality from model-parameter size to model-output size, enabling tighter robust aggregation guarantees.Its sample complexity is Θ((d/ε) log d), with d equal to the number of prediction classes rather than the parameter count.

B. Robust Mean Estimation in Cronus

Cronus uses robust mean estimation on low-dimensional prediction updates rather than high-dimensional parameter updates. The selected estimator provides dimension-independent error guarantees with near-optimal sample complexity.

  • Estimator: Cronus uses Diakonikolas et al.’s robust mean estimator, which provides dimension-independent error guarantees.The estimator operates on corrupted samples whose covariance is bounded by σ^2I.
  • Estimator: The estimator identifies directions with unusually high empirical variance and filters inputs farthest from the empirical mean along those directions.Filtering repeats until the stopping condition is satisfied.
  • Guarantee: Θ((d/ε) log d) samples suffice for the robust mean estimation guarantee under ε-corruption.The theorem states that the output mean has bounded Euclidean error with probability 9/10.
  • Implementation: The practical evaluation uses deterministic filtering of an ε/2 fraction per iteration for a constant number of iterations.This differs from the randomized filtering and repeated stopping procedure used in the theoretical analysis.

VI. EXPERIMENTAL SETUP

The evaluation compares Cronus with stand-alone, centralized, and FedAvg learning across benign accuracy, robustness, privacy, heterogeneity, and communication efficiency. Robustness is tested against strong poisoning attacks, with Cronus remaining largely unaffected.

  • Evaluation measures: Robustness is measured as accuracy under the strongest poisoning attack divided by benign-setting accuracy.The strongest attack is selected from the attacks evaluated in the paper.
  • Evaluation measures: Membership inference risk is measured by attack accuracy, where 50% corresponds to random guessing.The test uses equal numbers of member and non-member records.
  • Benign evaluation: The experiments compare Cronus with stand-alone, centralized, and FedAvg training in benign settings.Classification accuracy is used to compare the resulting models.
  • Robustness results: Existing federated-learning aggregation schemes are significantly vulnerable to at least one poisoning attack.FedAvg approaches random-guess accuracy across datasets, while Median, Bulyan, and Krum each have identified vulnerabilities.
  • Robustness results: Cronus robustness remains nearly 1.0, with maximum accuracy reductions of 0.4% for Purchase, 1.3% for SVHN, 1.5% for MNIST, and 4.8% for CIFAR10.The reported resilience is attributed to lower-dimensional updates, non-overwritten local models, and stronger agreement among benign predictions.

B. Privacy

Cronus substantially reduces membership-inference risk compared with FedAvg and remains resilient to active attacks, including when adversarial regularization is applied.

  • For Purchase, passive attack accuracy is 78.1% and 80.1% against FedAvg updates, versus 51.7% and 51.9% against Cronus updates.
  • Adversarial regularization reduces Purchase attack risk by 9.9% and 12.7% for FedAvg, but only 0.6% and 1.1% for Cronus.
  • Cronus accuracy in the adversarial setting is almost the same as in the benign setting.
  • Under active attacks on Purchase, FedAvg risk rises by 7.8% for individual updates and 8% for aggregated updates, while Cronus rises only 0.3% and 1.1%.
  • For active attacks on SVHN, FedAvg risk increases by 2.5% and 4.4%, whereas the increase is negligible for Cronus.

C. Differential privacy

The supplied evidence contrasts differential-privacy approaches with Cronus and describes heterogeneous collaboration experiments, including robustness and accuracy limitations under DP-SGD.

  • User-level DP [14]: User-level DP with few parties yields close to random-guess global-model accuracy even with a privacy budget of ǫ = 100.
  • User-level DP [14]: User-level DP requires many participants to reduce aggregation noise; one cited method uses 5000 parties per epoch.
  • Record-level DP: Record-level DP-SGD prevents federated or Cronus models from reaching stand-alone accuracy, making collaboration useless in the reported SVHN setting.
  • Cronus with heterogeneous model architectures: Heterogeneous collaboration between equivalent-capacity models does not reduce party-model accuracy compared with homogeneous counterparts.
  • Cronus with heterogeneous model architectures: Adding four low-accuracy models leaves the accuracy of twelve good models unaffected while benefiting the bad models.
  • Cronus with heterogeneous model architectures: Heterogeneous collaboration improves A1 accuracy by 8%, from 70.1% to 78.1%, with smaller gains for A2, A3, and A4 models.

E. Communication overhead of Cronus

Cronus reduces communication by exchanging prediction-based updates rather than model parameters, and random public-data subsampling further lowers communication without compromising final accuracy.

  • Prediction sharing reduces communication relative to sharing party-model parameters.
  • Using random subsets of public data in each epoch further reduces communication without compromising Cronus accuracy.
  • With public-data samples of 2k, 4k, 6k, 8k, and 10k, corresponding accuracies are 88.4, 89.4, 88.7, 89.3, and 89.6.
  • Knowledge transfer improves student-model convergence, so subsampling reduces both per-epoch and overall communication.

APPENDIX

The appendix documents evaluation settings and reports that Cronus has only slight benign-accuracy degradation relative to existing algorithms while outperforming stand-alone training.

  • Experimental details: SVHN experiments train 32 party models on 5,000 samples each and use 10,000 public samples.
  • Robustness evaluation: The appendix provides complete robustness evaluations across datasets, aggregation algorithms, and poisoning attacks.
  • Benign-setting comparison: Cronus incurs only slight accuracy degradation compared with existing algorithms in the benign setting and improves significantly over stand-alone training.
Loading 1912.11279v1…