PaperlayerView exact revision on arXiv

Source-linked AI summary

Certified Uncertainty Propagation in One-Shot Federated Bayesian Models via Posterior Event Transport

Mahyar Mohammadi, Mohammad Hossein Badiei, Abolfazl Yaghmaei, Hamed Kebriaei

arXiv:2609.16373v1cs.LG

TL;DR

One-shot federated Bayesian learning deploys a model by aggregating draws from client-specific posteriors, so local certificates do not directly establish the deployed model’s safety probability. The paper transports local posterior events through aggregation and exactly characterizes their FedAvg images for nonnegative weights. Across reported configurations, predictive accuracy and certifiable posterior mass need not align, and neither FedAvg nor Product-of-Gaussians uniformly dominates across architectures.

  • Problem

    Local posterior certificates do not generally guarantee safety of the aggregated model because deployment depends on joint client draws and the aggregation rule.

    Source: §I · p. 2

  • Method

    The method transports disjoint local posterior events through the deployment aggregation rule and counts their mass only when the complete aggregate image is verified safe.

    Source: §I · p. 2

  • Results

    Across reported configurations, predictive accuracy and certifiable posterior mass need not follow the same ordering, and neither FedAvg nor PoG uniformly dominates across architectures.

    Source: B. Simulation results and discussion · pp. 8–9, §V · p. 9

  • Takeaways & Limitations

    For FedAvg with nonnegative weights, the transported event’s hyper-rectangle image is exact, while transported and direct certificates capture distinct regions or posterior constructions.

    Source: §I · p. 2, §V · p. 9

    The experiment does not fully separate width and depth effects.

    Source: B. Simulation results and discussion · p. 8

Abstract

from arXiv · show

Probabilistic certification of Bayesian neural networks lower-bounds the posterior probability that a model satisfies a verifier-defined safety property. In one-shot federated Bayesian learning, however, the deployed model is obtained by aggregating parameters drawn from client-specific posterior distributions, so local certificates do not directly guarantee safety of the aggregated model. This paper develops a deployment-consistent certification framework by propagating local posterior events through the deployment aggregation rule, with an exact geometric characterization for Federated Averaging (FedAvg). Each client constructs disjoint hyper-rectangular regions in parameter space and computes their probability masses. The server forms Cartesian products of these regions, maps them through the deployment rule, and retains a product event only when its aggregation image is verified to satisfy the safety property. Under independent client posteriors, each product-event probability factorizes into local masses, and summing verified disjoint events yields a lower bound on safety probability of the deployed model. For FedAvg with nonnegative aggregation coefficients, the image of a Cartesian product of axis-aligned hyper-rectangles is exactly a weighted hyper-rectangle, introducing no set over-approximation. We distinguish the proposed transported-event certificate from direct certification under posterior distributions induced by FedAvg and Product-of-Gaussians aggregation. Experiments on MNIST and Fashion-MNIST under label-Dirichlet heterogeneity show that the transported FedAvg certificate ranges from 22.51% to 46.89%, while direct global certificates range from 72.05% to 91.39%. Results show that predictive accuracy and certifiable safety do not necessarily follow the same trend, and that global posterior constructions can exhibit distinct certification behavior across architectures.

I. INTRODUCTION

In one-shot federated Bayesian learning, the deployed model depends on aggregating draws from distinct client posteriors, so local safety certificates do not directly guarantee the aggregated model’s safety. The paper transports disjoint local posterior events through deployment aggregation and distinguishes that certificate from direct certification under global posterior laws.

I. INTRODUCTION

  • Local certificates do not generally yield a valid global guarantee because deployment depends on joint client draws and aggregation may not preserve safety.Neural-network safety regions in weight space are generally nonconvex.Source: §I · pp. 1–2
  • The framework combines disjoint local posterior regions into joint events and counts their mass only when each complete aggregation image is verified safe.Under independent client posteriors, each joint-event probability factorizes into the product of local masses.Source: §I · p. 2
  • For FedAvg with nonnegative aggregation weights, Cartesian products of axis-aligned hyper-rectangles map exactly to weighted hyper-rectangles, with no added set over-approximation.The certificate’s conservatism then comes from finite posterior coverage and the verification procedure.Source: §I · p. 2

II. RELATED WORK

  • Direct certification under the FedAvg pushforward targets the same deployment safety probability, while Product-of-Gaussians defines a separate posterior fusion rule.The paper evaluates transported and direct certificates on MNIST and Fashion-MNIST as controlled one-shot benchmarks.Source: §I · p. 2, §II · pp. 2–3
  • Earlier one-shot Bayesian federated methods focus mainly on prediction, calibration, and uncertainty representation, rather than certifying safety under the deployment aggregation map.Probabilistic BNN verification methods generally assume a single posterior over the deployed model.Source: §II · p. 2

III. METHODOLOGY

The certification target is the safety probability of the random global model produced by the server’s aggregation rule. The methodology defines the evaluated robustness property and seeks a computable lower bound for that deployment probability.

III. METHODOLOGY

  • The system has n clients that independently train local Bayesian neural networks, followed by one server-side aggregation step.The derivation begins with one safety property and applies the procedure separately to each evaluated property.Source: §III · p. 3

A. Safety Certification Objective

  • For classification robustness, safety requires the true-class logit to exceed every competing-class logit by margin δ throughout the admissible input region X.X contains inputs within ℓ∞ distance ϵ of the reference input.Source: A. Safety Certification Objective · p. 3
  • The objective is a computable certificate lower-bounding the safety probability of the random global model produced by aggregation.That deployed model is formed from random client parameter draws.Source: A. Safety Certification Objective · p. 3

B. Local Bayesian Posterior

Clients represent parameter uncertainty with local Gaussian posteriors and construct finite candidate hyper-rectangles in their weight spaces. Disjointness permits their posterior masses to be accumulated without double-counting.

B. Local Bayesian Posterior

  • Each client represents parameter uncertainty with a Gaussian posterior around its local MAP estimate, while the certification principle is not tied to one posterior approximation method.The implementation uses a diagonal covariance.Source: B. Local Bayesian Posterior · p. 3
  • Independent deployment draws let joint-event probabilities factor into products of local posterior probabilities; the set-based argument also works when joint probabilities are directly available.Source: B. Local Bayesian Posterior · p. 3

C. Local Hyper-rectangle Construction

D. Safety Verification via IBP

Interval bound propagation (IBP) recursively bounds network outputs over bounded input and weight regions. A region is accepted only when the bounds certify the required logit margin, and conservative bounds can reduce the resulting certificate.

D. Safety Verification via IBP

  • IBP recursively propagates interval bounds through network layers to obtain certified lower and upper bounds on output logits.Source: D. Safety Verification via IBP · p. 4
  • A parameter region is certified safe when the reference-class logit’s lower bound exceeds each competitor’s upper bound by at least δ.This certifies the margin for every parameter vector in the region and every admissible input.Source: D. Safety Verification via IBP · p. 4
  • Local IBP screening cannot establish safety after aggregation, so each combined global region must be verified again.Source: D. Safety Verification via IBP · p. 4
  • A conservative verifier may reject genuinely safe regions, reducing the certified probability and tightening the lower bound less, but accepted regions remain guaranteed safe.Source: D. Safety Verification via IBP · p. 4

E. One-Shot FedAvg Deployment

FedAvg combines selected client regions using nonnegative aggregation coefficients, producing an exact weighted hyper-rectangle for each joint region.

E. One-Shot FedAvg Deployment

  • FedAvg coefficients may be uniform or proportional to local dataset sizes, and one selected region from each client forms a joint event.The tuple indexes the chosen client regions.Source: E. One-Shot FedAvg Deployment · p. 4
  • Nonnegative FedAvg weights make each joint hyper-rectangle’s aggregation image exact, with no additional interval relaxation.A tuple is accepted only when its complete aggregated image satisfies the safety verifier.Source: E. One-Shot FedAvg Deployment · p. 4
  • A joint tuple is retained only when its complete FedAvg image is verified safe.The verifier checks the entire aggregate region, rather than only its individual client regions.Source: E. One-Shot FedAvg Deployment · p. 4

F. Deployment-Consistent Safety Certificate

The certificate sums probability mass from disjoint joint client events whose complete FedAvg images are verified safe, yielding a lower bound on deployment safety.

F. Deployment-Consistent Safety Certificate

  • The global safe set is the verifier-defined set, and local certified sets are combined through deployment before their aggregate images are checked.It is not defined as the union of local certified sets.Source: F. Deployment-Consistent Safety… · p. 4
  • Under independent client posterior draws, disjoint retained regions, and the stated verifier assumption, the resulting quantity is a valid lower bound on FedAvg deployment safety.Each accepted tuple contributes the product of its selected clients’ region probabilities.Source: F. Deployment-Consistent Safety… · p. 4
  • Independent client posterior draws make each joint-event probability factorize, while pairwise-disjoint client regions make distinct tuples disjoint.Summing the probabilities of verified tuples therefore contributes only disjoint safe events.Source: F. Deployment-Consistent Safety… · p. 5
  • The procedure samples local posterior centers, constructs and retains disjoint hyper-rectangles, then verifies aggregate images of selected client-region tuples.The server accumulates the independent product of local region masses for accepted tuples.Source: F. Deployment-Consistent Safety… · p. 5

G. Direct Certification Under Global Posterior Laws

Direct certification constructs and verifies regions under a selected global posterior, whereas transported certification maps combinations of local posterior regions through aggregation.

G. Direct Certification Under Global Posterior Laws

  • Direct global certification constructs regions under the selected global posterior and retains only regions verified safe over their full extent.The paper considers global distributions induced by FedAvg and Product-of-Gaussians fusion.Source: G. Direct Certification Under Global… · pp. 5–6
  • Transported and direct FedAvg certificates lower-bound the same deployment safety probability, but neither is guaranteed to dominate because they cover different probability-space regions.The transported method verifies aggregate images of local posterior events; the direct method searches under the FedAvg-induced global distribution.Source: G. Direct Certification Under Global… · p. 6
  • The direct PoG certificate concerns safety under the PoG fusion distribution, not the safety probability of FedAvg deployment.FedAvg and PoG define different random global models.Source: G. Direct Certification Under Global… · p. 6

IV. EXPERIMENTS

The evaluation compares distinct certification quantities and examines how certification behaves across architectures and data partitioning.

IV. EXPERIMENTS

  • The evaluation compares direct global-posterior certification, the transported FedAvg certificate, and Monte Carlo verifier acceptance as distinct quantities.They have distinct probabilistic meanings.Source: §IV · p. 6
  • The experiments examine whether the transported FedAvg certificate is non-vacuous and how certification varies with architecture and data partitioning.Source: §IV · p. 6
  • The evaluation also tests whether FedAvg and Product-of-Gaussians fusion exhibit consistent differences in certification.Source: §IV · p. 6

A. Experimental setup

The experiments use controlled MNIST and Fashion-MNIST federations to compare certification across client heterogeneity, federation scales, and network architectures.

A. Experimental setup

  • MNIST and Fashion-MNIST training subsets contain 12,000 examples each and are partitioned among 2, 3, or 5 clients using label-Dirichlet sampling.Smaller Dirichlet concentration α produces more label-skewed client datasets.Source: A. Experimental setup · p. 6
  • Clients train fully connected Bayesian neural networks with 1×64, 1×128, or 2×64 hidden architectures, using ReLU activations and shared initialization.Each client trains a diagonal mean-field posterior for five epochs with Adam, a 10^-3 learning rate, batch size 128, and fixed post-training standard deviation 10^-5.Source: A. Experimental setup · p. 6
  • The study compares equal-weight FedAvg with PoG; it evaluates accuracy on posterior-mean networks and certification on 50 correctly classified inputs at radius ϵ = 10^-3 and margin δ = 0.Reported property-level values are macro-averaged; standard deviations reflect federation configurations, not independent training repetitions.Source: A. Experimental setup · pp. 6–7

B. Simulation results and discussion

Direct and transported certification behave differently across federation settings, architectures, and datasets; predictive accuracy does not consistently track certified posterior mass.

B. Simulation results and discussion

  • Direct certified mass ranges from 72.05% to 91.39% for PoG and 72.58% to 91.20% for FedAvg; MC–IBP exceeds direct certification by 5.47–13.25 percentage points.All evaluated direct-certification and MC–IBP points lie above the equality line.Source: Figure 2 · p. 7, B. Simulation results and discussion · p. 7, Table II · p. 8
  • 22.51%–46.89% for transported FedAvg certificates, versus 72.58%–91.20% for direct FedAvg certificates across the six dataset–architecture configurations.Transported certification is positive in every reported FedAvg configuration but is more conservative and more variable than direct FedAvg certification.Source: B. Simulation results and discussion · p. 7, Table II · p. 8
  • The 1×128 architecture has the highest mean accuracy, but 1×64 has the strongest direct certificate in all four dataset–method comparisons.The 2×64 architecture produces a larger reduction in certified mass and the smallest transported certificate on MNIST for FedAvg.Source: B. Simulation results and discussion · p. 8
  • Transport retention generally improves as Dirichlet concentration α increases, while increasing the number of clients consistently reduces the recovered fraction of direct certified mass.The reported ratio is 100 × Ltr/Ldir; the paper relates the client-count pattern to the difficulty of verifying transported Cartesian products.Source: Figure 3 · p. 8, B. Simulation results and discussion · p. 8
  • Fashion-MNIST lowers mean accuracy by approximately 11–14 points versus MNIST, while direct-certification decreases are typically 3–5 points for one-layer models and nearly zero for 2×64 FedAvg.Properties are evaluated only on selected correctly classified inputs, so these comparisons do not establish comparable global safety across datasets.Source: B. Simulation results and discussion · pp. 8–9
  • PoG and FedAvg have no uniform ranking across architectures; fixed post-training posterior standard deviation means these results do not compare uncertainty calibration.For 1×128, PoG has higher accuracy, direct certification, and MC–IBP on both datasets; the ordering differs for 2×64.Source: B. Simulation results and discussion · p. 9
  • Pooled statistics do not establish monotonic certification trends with client count or Dirichlet concentration; Fashion-MNIST 1×128 PoG runtime is 1593.97±2664.92 seconds.The 1×128 architecture is the most computationally expensive, and that PoG runtime has a particularly large standard deviation.Source: B. Simulation results and discussion · p. 9

V. CONCLUSION

The framework certifies one-shot federated BNN safety under the deployment rule by transporting disjoint local posterior events, with exact set propagation for FedAvg with nonnegative weights. Experiments yield non-vacuous FedAvg certificates, while exhaustive evaluation of local-event combinations scales exponentially with client count.

V. CONCLUSION

  • Across all reported dataset–architecture FedAvg aggregates, transported certificates are non-vacuous; accuracy, direct certified mass, and transported coverage need not share an ordering, and neither FedAvg nor PoG uniformly dominates across architectures.Source: §V · p. 9
  • The construction transports disjoint local posterior events through deployment aggregation and counts an event's probability only when its complete image is verified safe.For FedAvg with nonnegative weights, axis-aligned cells map exactly, so aggregation adds no geometric relaxation.Source: §V · p. 9

VI. FUTURE WORK

  • Exhaustive evaluation of the full Cartesian product grows exponentially with client count, limiting evaluation to smaller federated settings; future work proposes scalable search that preserves theoretical guarantees.Suggested directions include distributed primal-dual optimization, multi-agent reinforcement learning, and stochastic generalized Nash equilibrium methods to prune unverifiable combinations.Source: §VI · p. 9

Related papers

Loading 2609.16373v1…