PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Learning of AnDE Classifiers

Pablo Torrijos, Juan C. Alfaro, José A. Gámez, José M. Puerta

arXiv:2609.28695v1cs.LG

TL;DR

Federated Naive Bayes leaves open whether its strong independence assumption can be relaxed with AnDE classifiers. FedAnDE trains discriminative AnDE by sharing weights, and experiments report accuracy gains as dependency order rises.

  • Problem

    Whether AnDE classifiers can relax Naive Bayes’s strong independence assumption in federated learning remains unaddressed by prior work.

    Source: §1 · p. 2

  • Method

    FedAnDE trains discriminative AnDE models locally while keeping generative parameters private and sharing only discriminative weights for aggregation.

    Source: §3 · p. 3

  • Results

    Across federation scenarios, discriminative models consistently outperform generative ones, accuracy improves with dependency order, and A2DEw achieves the highest accuracy across federation levels.

    Source: §4.1 · p. 5

  • Takeaways & Limitations

    Higher-order discriminative AnDE models can be trained effectively in federated settings while offering stronger privacy guarantees than generative approaches.

    Source: A Complete Results of the Experimental… · p. 9

    Higher-order models need sufficient data to estimate probability tables reliably, and performance degrades with sparse data, especially under strong federation (C = 100).

    Source: A Complete Results of the Experimental… · p. 8

Abstract

from arXiv · show

This work presents a federated framework for training Averaged $n$-Dependence Estimators (AnDE) in distributed environments. The proposed method focuses on the discriminative setting, where model weights are learned locally and aggregated globally, supporting any dependency order $n$. This design allows federated training without transmitting semantically meaningful parameters, improving privacy. Additionally, generative AnDE models are federated to provide a comparative baseline, with optional differential privacy applied to the aggregation of probability tables. Experiments on 12 discrete datasets show that discriminative models with $n \geq 1$ consistently outperform federated Naive Bayes (NB, $n=0$), and that privacy-preserving aggregation is effective with limited accuracy loss. These results establish federated AnDE as a viable and privacy-preserving framework, showing that probabilistic models remain applicable in modern federated learning settings.

1 Introduction

FedAnDE extends discriminative federated Naive Bayes to AnDE models with higher dependency orders, sharing model weights rather than interpretable probability tables. The introduction reports that increasing n improves discriminative accuracy, while the generative baseline degrades more under privacy constraints and with more clients.

1 Introduction

  • AnDE averages models where each attribute depends on the class and up to n other attributes; it recovers NB at n = 0 and AODE at n = 1.FL enables collaborative training across distributed clients without sharing raw data.Source: §1 · p. 1
  • Generative FL for NB aggregates conditional probability tables, which may expose sensitive information even with differential privacy; discriminative NBw shares less semantically meaningful weights.NBw had previously been federated for n = 0.Source: §1 · pp. 1–2
  • FedAnDE extends federated NBw beyond n = 0, letting clients train more expressive AnDE models while sharing only weights.The framework is introduced as a response to the lack of prior work on federated AnDE training.Source: §1 · p. 2
  • Increasing n improves accuracy in federated discriminative models, whereas the generative AnDE baseline degrades more under privacy constraints and as client numbers rise.The generative AnDE implementation serves as a baseline for assessing discriminative training and differential privacy.Source: §1 · p. 2

2 Background

AnDE averages classifiers built from n-sized attribute parent sets, with increasing n capturing more dependencies at greater combinatorial complexity. Its weighted extension supports federated training by keeping generative counts local and sharing learned weights instead.

AnDE

  • AnDE averages SPnDE classifiers, each conditioning non-parent attributes on the class and a parent set P, to predict the class.Each base classifier is obtained by normalizing its joint distribution over the class variable.Source: §2 · p. 2
  • For n = 0 and n = 1, AnDE reduces to NB and AODE; larger n captures more dependencies but increases complexity combinatorially.Source: §2 · p. 2

Weighted Naive Bayes

  • NBw learns parameter weights by maximizing conditional log-likelihood, and AnDEw assigns a separate weight set wP to each parent set P.AnDEw averages the weighted predictions across parent sets.Source: §2 · pp. 2–3

Federated training

  • Federated AnDEw keeps class-conditional counts on clients and shares only learned weights wP, which lack semantic meaning, for server aggregation.Unprotected generative counts can be directly interpretable and may leak private information; the weighted strategy avoids extra protection steps.Source: §2 · p. 3

3 FedAnDE: Federated Weighted AnDE

FedAnDE federatively trains discriminative AnDE models by keeping generative parameters local and fixed while sharing structure-specific discriminative weights. This design improves privacy over federated generative AnDE, which shares class-conditional statistics.

3 FedAnDE: Federated Weighted AnDE

  • FedAnDE keeps each client’s generative parameters fixed and local, sharing only discriminative weights for each structure with the server.Clients train their weighted models using only their local datasets.Source: §3 · p. 3
  • Across T rounds, clients optimize the shared weights on local data using L steps of L-BFGS-B to maximize CLL, then return updates for server aggregation.The server sends the current global weights to clients before each optimization round.Source: §3 · p. 3
  • Training is performed independently for each structure, allowing parallelization.Source: §3 · p. 3
  • By communicating only weights, FedAnDE improves privacy over federated generative AnDE, which shares class-conditional statistics that may still be reconstructed or inferred.Source: §3 · p. 3

4 Experimental Evaluation

Across the evaluated settings, higher-order discriminative models performed best, while federated training improved on client-local models. The evaluation also found that sparse local data and differential privacy could weaken generative models as client counts increased.

4 Experimental Evaluation

  • Experiments evaluated FedAnDE on 12 categorical datasets across 5–100 clients, using five-fold cross-validation repeated with five random seeds and comparing federated and client-local models.The federated comparison included a differentially private generative variant; larger client counts reduced each client’s local data.Source: §4 · p. 4
  • FedAnDE trained for 10 rounds with five local L-BFGS-B steps, transmitting only weights; the generative baseline aggregated counts once with Laplace noise at ε = 1.0.The discriminative method applied no differential-privacy mechanism, while the generative baseline used ε-differential privacy.Source: §4 · p. 4

4.1 Results

  • A2DEw achieved the highest accuracy across federation levels; federated training beat client-local models, discriminative models beat generative counterparts, and accuracy improved with dependency order.The gains from federation and increasing dependency order were especially noticeable with more clients.Source: Figure 1 · p. 5, §4.1 · p. 5
  • On Kr-vs-K, A2DE outperformed A1DE, which outperformed NB, in both generative and discriminative panels.The dataset is described as having strong attribute dependencies, such as chess-piece positions in an endgame.Source: Figure 2 · p. 6, §4.1 · p. 6
  • As client counts increased, generative A1DE fell below NB and A2DE dropped sharply from 20 clients, while discriminative A2DEw remained more robust.The passage attributes the generative decline to reduced local data and differential-privacy noise affecting the larger set of statistics used by higher-order models.Source: §4.1 · p. 6

5 Conclusions and Future Work

FedAnDE enables federated training of discriminative AnDE classifiers through higher-order NBw models and weight-based aggregation. Experiments show performance improves with increasing n, and discriminative models outperform their generative counterparts.

  • FedAnDE extends NBw to higher-order discriminative AnDE classifiers and uses weight-based aggregation, improving accuracy and privacy; experiments show increasing n improves performance and discriminative models outperform generative counterparts.Source: §5 · p. 6
  • Although L = 5 local steps and T = 10 rounds provided stable results, convergence, communication cost, and runtime need further analysis, while n = 2's combinatorial cost may limit scalability for large d.Future work will explore sparse structures or dimensionality reduction to address scalability.Source: §5 · pp. 6–7
  • Future work includes comparing with logistic regression, combining generative and discriminative updates, supporting continuous attributes through Gaussian AnDE, and evaluating alternative privacy mechanisms.Source: §5 · pp. 6–7

A Complete Results of the Experimental Evaluation

Federated AnDE results show that aggregation benefits performance, while dependency order and privacy noise interact with data availability. Discriminative models match or surpass generative models, supporting effective federated training.

  • All federated variants outperform their client-local counterparts, and the performance gap widens as the number of clients increases.The passage attributes this widening gap to aggregation of local updates under data fragmentation.Source: A Complete Results of the Experimental… · p. 8
  • Across the accuracy figures, FedAnDE (n ∈{0, 1, 2}) consistently matches or surpasses generative models, supporting effective federated higher-order discriminative AnDE training.The advantage is more stable on larger datasets, while smaller datasets show greater variability.Source: A Complete Results of the Experimental… · pp. 8–9, Figure 3 · p. 10, Figure 4 · p. 11
  • A2DE > A1DE > NB generally holds on large datasets, but order differences are negligible on some small datasets and inconsistent for discriminative models on House Votes 84.Higher-order models need sufficient data to estimate probability tables reliably, and can degrade with sparse data, especially under strong federation (C = 100).Source: A Complete Results of the Experimental… · p. 8
  • Privacy noise degrades generative models, especially higher-order ones, while accuracy often peaks at intermediate client counts because noise averaging and per-client sample scarcity trade off.Averaging attenuates noise variance as 1/C, but larger C leaves each client with fewer samples; the non-convex pattern is especially pronounced on Connect-4.Source: A Complete Results of the Experimental… · pp. 8–9
Loading 2609.28695v1…