PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Models for Effective ML on Private, Decentralized Datasets

Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, Blaise Aguera y Arcas

arXiv:1911.06679v2cs.LGstat.ML

TL;DR

Privacy-sensitive and federated datasets limit the direct data inspection needed for ML debugging and related workflows. The paper trains federated generative models with formal differential privacy to synthesize useful examples, demonstrating debugging of text and image problems with RNNs and GANs. The authors also identify tuning, low-signal bugs, and GAN mode collapse as remaining practical challenges.

  • Problem

    Direct inspection is problematic for privacy-sensitive data and impossible when federated learning exposes only aggregated outputs instead of raw examples.

  • Method

    The paper combines federated learning, user-level differential privacy, and generative models to synthesize representative examples without exposing raw user data.

  • Results

    The approach enables modelers to debug natural-language and image problems using differentially private federated RNNs and GANs, including a novel federated DP GAN algorithm.

  • Takeaways & Limitations

    Privacy-preserving federated generative models can substitute for some direct data-inspection needs in private, decentralized ML workflows.

  • Takeaways & Limitations

    Practical deployment still requires minimal tuning, useful synthesis for low-signal bugs, and improved handling of GAN mode collapse.

Abstract

from arXiv · show

To improve real-world applications of machine learning, experienced modelers develop intuition about their datasets, their models, and how the two interact. Manual inspection of raw data - of representative samples, of outliers, of misclassifications - is an essential tool in a) identifying and fixing problems in the data, b) generating new modeling hypotheses, and c) assigning or refining human-provided labels. However, manual data inspection is problematic for privacy sensitive datasets, such as those representing the behavior of real-world individuals. Furthermore, manual data inspection is impossible in the increasingly important setting of federated learning, where raw examples are stored at the edge and the modeler may only access aggregated outputs such as metrics or model parameters. This paper demonstrates that generative models - trained using federated methods and with formal differential privacy guarantees - can be used effectively to debug many commonly occurring data issues even when the data cannot be directly inspected. We explore these methods in applications to text with differentially private federated RNNs and to images using a novel algorithm for differentially private federated GANs.

1 INTRODUCTION

Privacy-sensitive and federated datasets prevent the direct inspection that modelers use to debug, form hypotheses, and refine labels. The paper proposes privacy-preserving federated generative models as synthetic substitutes for examining inaccessible data.

  • Privacy-sensitive data creates stewardship obligations while modelers still need to develop and improve machine-learned models.
  • Direct inspection can be audited or disallowed, while federated learning keeps raw examples on devices and exposes only model parameters and statistics.
  • The paper proposes auxiliary privacy-preserving generative models that synthesize representative examples for debugging data errors during inference or training.
  • The approach combines federated learning, user-level differential privacy, and deep generative models for text and image workflows.

2 CHALLENGES OF ML ON NON-INSPECTABLE DATA

Direct access to training data is assumed by many ML debugging and data-quality workflows, but privacy-sensitive and decentralized settings remove that option. The paper frames programmatic selection and generative modeling as alternatives, while noting that some applications require high-fidelity private models.

  • ML practice commonly assumes unfettered access to training or inference data, leaving private and decentralized data largely unaddressed.
  • Sanity checking and model debugging (T1–T4): Modelers inspect samples, misclassifications, out-of-vocabulary words, and performance slices to detect preprocessing, feature, labeling, and segmentation problems.
  • Data labeling (T5): Federated learning complicates human labeling because users’ data cannot be sent to the cloud, motivating private synthesis of realistic examples for raters.
  • Detecting bias in training data (T6): Bias can arise when public labeled data differs distributionally from larger private datasets observed at serving time.
  • The general method expresses inspection criteria as programmatic selection procedures for training generative models on selected devices or local data.
  • The paper experiments with three tasks, while leaving further details for some workflows and future work.
  • Generative models must not simply reproduce private user examples, so privacy guarantees are required.

3 DIFFERENTIALLY PRIVATE FEDERATED GENERATIVE MODELS

The paper combines generative modeling, federated learning, and differential privacy to synthesize useful information from decentralized data without releasing raw user examples. Federated training keeps data on devices, while user-level DP limits memorization risk.

  • Deep generative models synthesize examples, federated learning trains and evaluates against distributed data, and differential privacy provides user privacy protections.
  • Deep generative models learn joint distributions and support applications across text, audio, and imagery.
  • Federated training selects devices that compute local updates, which the server aggregates over repeated rounds until the global model converges.
  • Raw user data never leaves the edge device during federated generative-model training.
  • User-level differential privacy addresses generative-model memorization through per-user update clipping and post-aggregation Gaussian noise.

4 RELATED WORK

Prior work studied private GANs, decentralized generative modeling, and individual federated generative tasks, but did not provide the paper’s combination of federated training, user-level privacy, and broad workflow coverage.

  • Prior differentially private GAN work focused on centralized training rather than decentralized data with federated learning and user-level guarantees.
  • Decentralized GAN training had been studied without privacy protections, unlike this paper’s focus on user-level privacy.
  • FedGP addressed a task resembling labeling with an empirical average-case privacy measure, while federated character-level RNN work addressed OOV generation without differential privacy.
  • The cited task-specific studies were not connected to a general solution spanning the broader T1–T6 workflow class.

5 AN APPLICATION TO DEBUGGING DURING TRAINING WITH RNNS

DP federated RNNs generate text and words that let modelers diagnose preprocessing bugs without inspecting decentralized training data. Word- and character-level models provide complementary evidence: anomalous OOV patterns localize the bug, while generated OOV words reveal its tokenization mechanism.

  • DP Federated RNNs for Generating Natural Language Data: RNNs learn next-token probabilities and generate sequences by repeatedly sampling successor tokens.Word-LMs generate sentences, while char-LMs generate words; training under FL and DP uses DP-FedAvg.
  • Experiment: A tokenization bug concatenating the first two tokens makes affected tokens fall outside the word-LM vocabulary and appear as OOVs.The experiment concatenates the first two tokens in 10% of sentences derived from Stack Overflow data.
  • Experiment: Two DP federated language models are used complementarily: a primary word-LM and an auxiliary char-LM trained on OOV words.Comparing models trained before and after the bug supports debugging across training conditions.
  • Results: With the bug, word-LM generations show an unusually high OOV rate at the first sentence position.Figure 1 compares OOV percentages by sentence position for models trained with and without the bug.
  • Results: With the bug, most top OOV words generated by the char-LM contain spaces, indicating a tokenization bug that the modeler can fix.The OOV ranking uses joint character probability from Equation 1 and case-insensitive tokenization.
  • Privacy: The models are trained with user-level differential privacy; one reported configuration gives ϵ = 9.22 with δ = 2.92 × 10^-6.The dataset contains 342,477 users, and each round selects 5,000 users.

6 AN APPLICATION TO DEBUGGING DURING INFERENCE WITH GANS

The paper adapts GANs to federated, differentially private training so synthetic images can help diagnose preprocessing bugs without exposing decentralized user data. Contrasting models trained on high- and low-accuracy user subpopulations reveals pixel inversion and supports verifying its correction.

  • DP Federated GANs for Generating Image Data: GANs alternate a generator that creates images from latent vectors with a discriminator that distinguishes real from generated images.The generator and discriminator are trained against each other in the standard GAN framework.
  • DP Federated GANs for Generating Image Data: DP-FedAvg-GAN trains the discriminator with federated differential privacy and updates the generator at the server through the discriminator.Because the generator sees no real user data and depends only on the private discriminator, it inherits the discriminator’s differential privacy guarantees.
  • Application to Debugging During Inference: The experiment monitors an on-device handwriting CNN whose preprocessing bug inverts pixel intensities and causes most handwriting to be misclassified.A rising user correction rate signals that a problem exists, but does not reveal its nature because the inputs remain private and decentralized.
  • Application to Debugging During Inference: Two DP federated GANs are trained on processed images from high- and low-accuracy user subpopulations, then their outputs are contrasted to diagnose the bug.The selection procedure forms distinct user groups according to primary-model classification accuracy.
  • Application to Debugging During Inference: Comparing the GAN outputs makes the pixel inversion clear, enabling the modeler to locate and fix the issue without viewing the original images.Retraining after the fix produces outputs that verify the inversion bug is gone and may reveal additional data or model issues.
  • Privacy at Scale: For a realistic two-million-user population, the experiments achieve single-digit ϵ values between 1.48 and 2.38 for the reported selection scenarios.The simulations use 3,400 users and do not provide good privacy protections, whereas the scaled scenario indicates a reasonable amount of user privacy.

7 CONCLUSION

The paper presents differentially private federated generative models as substitutes for direct data inspection in private, decentralized ML workflows. It demonstrates this approach with RNNs and GANs while identifying practical barriers to broader deployment.

  • Differentially private federated generative models synthesize examples representative of private data when direct inspection is impossible.
  • The paper applies the methodology to DP federated RNNs and GANs for debugging natural-language and image problems.
  • Broader use requires less tuning, better performance when bug signal is weak, and further study of GAN mode collapse and higher-fidelity generation.
  • The approach uses user-level privacy through per-user update clipping, bounded-sensitivity aggregation, and Gaussian noise.
  • Fixed-size federated rounds sample qN users without replacement, introducing privacy-accounting differences from randomly sized DP-FedAvg rounds.

B.2 RNN MODEL ARCHITECTURES AND TRAINING

The appendix specifies DP word- and character-level recurrent language models trained with the federated procedure. The models use CIFG-LSTM architectures with projection layers and shared training settings.

  • The DP word-LM uses a 10K-word vocabulary, 670 hidden units, 96-dimensional embeddings, and 2,000 training rounds.
  • The DP char-LM uses three CIFG-LSTM layers, 256 hidden units, 128 projected dimensions, and a 258-symbol character vocabulary.
  • The char-LM and word-LM use the same remaining training and optimization parameters.
  • Table 5 provides the privacy hyperparameters used by DP-FedAvg to produce the reported RNN text results.

B.3 EXPANDED RESULTS

Expanded experiments show that DP language models reproduce signals from a token-concatenation bug in generated text. Word- and character-level outputs expose complementary evidence about the corrupted corpus.

  • The experiments compare 0%, 1%, 10%, and 100% affected-sentence settings, with models trained for 2,000 communication rounds.
  • As the fraction of bug-affected sentences increases, the word-LM increasingly reflects the bug in its generated output.
  • Overall OOV rate rises with the percentage of sentences affected, making it an early abnormality indicator for the corpus.
  • Generated word-LM phrases contain more UNK-marked OOVs as the concatenation bug affects more sentences.
  • In the 100% setting, all top 20 char-LM-generated OOV words are concatenated words.

C.2 FEDERATED GAN ARCHITECTURE AND TRAINING

The federated GAN experiments train generators and discriminators on user subpopulations selected by primary-model accuracy. Synthetic images can reveal whether an image-processing bug persists, but the selection criterion mixes bug-related and unrelated users.

  • The experiments use Improved Wasserstein GAN losses with a gradient penalty coefficient of 10.0 and a 1:1 discriminator-to-generator update ratio.
  • The GAN method trains separate models on subpopulations with the highest and lowest likelihood of being related to an unknown bug.
  • Users are partitioned by local CNN classification accuracy into poorly and strongly performing groups using accuracy thresholds.
  • When primary-model performance drops, the poorly performing group grows while the strongly performing group shrinks.
  • The poorly performing group includes users with unrelated low accuracy, so the method assumes the generator can model both modes or affected data dominates.
  • After the pixel inversion bug is fixed, synthetic images from both DP federated GANs are high enough quality to verify its removal.

C.4 ALTERNATIVE SELECTION CRITERIA: FILTERING BY EXAMPLE

Filtering by example trains separate differentially private federated GANs on correctly classified and misclassified examples, enabling comparison of their synthetic outputs to diagnose a primary-model bug. In simulation, the GANs distinguished reversed black-and-white data when the bug affected 50% of devices and after it was fixed.

  • Filtering by example: Filtering by example means training on some examples from all users, rather than all examples from only selected users.This selection produces smaller variation in the number of users contributing to each training subpopulation.
  • Filtering by example: The approach trains one DP federated GAN on correctly classified examples and another on misclassified examples, then infers the bug from contrasting synthetic content.Each GAN was trained for 1,000 communication rounds with 10 users per round.
  • Results: After the inversion bug was fixed, the corresponding GAN outputs were generated from data with the bug present on no phones.Figures 8a and 8b show the post-fix generated examples.
  • Privacy at scale: The privacy analysis used simulations with 3,400 users and projected realistic scenarios with 2,000,000 devices and 1,000 users per round.The projected realistic scenarios achieved single-digit epsilon bounds, while the small simulation scale did not guarantee meaningful privacy protection.

D OPEN PROBLEMS

The paper identifies open problems in privacy-preserving federated generative modeling, including privacy notions, sensitivity to bug prevalence, model design, alternative generative architectures, and cross-user training needs.

  • Alternative notions of privacy: The paper uses user-level (epsilon, delta)-differential privacy and treats single-digit epsilon bounds as acceptable, while noting that alternative privacy notions may be more practically useful.The paper highlights exposure as an empirical measure that can differ substantially from differential privacy’s risk measure.
  • Sensitivity: The sensitivity of federated generative models to the fraction of affected users or examples remains unresolved, especially given GAN mode collapse.The experiments demonstrated debugging with 50% of affected devices for GANs and 10% of affected examples for RNNs.
  • Federated GAN training: The paper’s DP-FedAvg-GAN uses a relatively simplistic GAN, leaving open which losses and architectures best tolerate privatization and federation.The authors specifically identify scaling and noising for differential privacy as design factors to study.
  • Other classes of generative models: Other generative models, including VAEs with personalized private encoders and a common decoder, present additional federation questions.The proposed VAE arrangement separates user-specific mapping from shared synthesis.
  • Federating contrastive or triplet losses: Differentially private federated generative models could provide synthetic negative examples for triplet-loss calculations that cannot directly access other users’ data.Federated learning keeps each user’s data in a distinct silo, preventing direct access to cross-user negative examples.
Loading 1911.06679v2…