PaperlayerView exact revision on arXiv

Source-linked AI summary

OPFL: Optimistic Verification of Federated Learning via Empirical Boundary

Hongxu Su, Jianzhu Yao, Xuechao Wang, Pramod Viswanath

arXiv:2609.37011v1cs.CRcs.AI

TL;DR

Federated learning keeps raw client data local, but that makes it difficult to verify that clients followed prescribed training, while privacy-preserving replay can differ numerically from local execution. OPFL uses MPC replay and an empirically calibrated gradient-discrepancy boundary, auditing sampled steps optimistically. Across evaluated settings it reports stable boundary generalization and 0% ASR against evaluated attacks, while reducing measured cost at a 1% audit rate in a LeNet workload.

  • Problem

    Federated learning keeps raw data on client devices, but limited visibility into local training makes it difficult to verify whether submitted updates came from prescribed computation.

    Source: §1 · p. 1

  • Method

    OPFL privately replays sampled client computations in MPC and compares GPU–MPC gradient discrepancies against an offline-calibrated empirical boundary.

    Source: §1 · p. 2, §2 · p. 4

  • Results

    The boundary generalizes across training stages, datasets, sequence lengths, and GPU hardware, with 0% ASR across evaluated attacks and 0% FRR across the reported settings.

    Source: §1 · p. 2, §4.1 · p. 7

  • Takeaways & Limitations

    At a 1% audit rate, OPFL was approximately 98.6× faster than full MPC-based FL and 625.5× faster than ZKSL-LP-G in the five-client LeNet workload.

    Source: §4.3 · p. 9

    OPFL verifies execution integrity, not data poisoning, and assumes existing privacy-preserving aggregation and verifier-committee mechanisms.

    Source: §2 · p. 4

Abstract

from arXiv · show

Federated learning enables multiple clients to collaboratively train models without sharing their private data. However, the lack of visibility into local training makes it difficult to verify whether clients follow the prescribed training procedure or submit malicious updates, such as model poisoning. A natural approach is to replay client training for verification. However, privacy-preserving replay produces numerical results that cannot be directly matched with local client execution because the two run in different environments. We present OPFL, an optimistic verification framework for privacy-preserving federated learning. To protect data privacy, OPFL performs replay inside secure multi-party computation (MPC). Although gradients computed on MPC and local GPUs are not bitwise identical, we observe that their absolute differences are stable and bounded. OPFL therefore calibrates an empirical boundary offline and uses it to distinguish benign numerical deviations from malicious manipulation. To reduce the cost of expensive MPC replay, OPFL adopts optimistic verification by post auditing only sampled training steps. Experiments on LeNet, BERT, and Qwen show that the boundary generalizes across datasets, input lengths, and GPUs, while achieving $0$\% ASR against model poisoning and PGD-based attacks. On a LeNet workload, at $p=0.01$, OPFL is approximately $98.6\times$ faster than full MPC-based FL and $625.5\times$ faster than ZK-based approach.

1 INTRODUCTION

Federated learning protects raw client data but leaves local execution difficult to observe, so servers cannot readily establish whether submitted updates follow prescribed training. OPFL uses privacy-preserving replay and an empirically calibrated gradient-discrepancy boundary, then audits sampled steps to reduce verification cost.

1 INTRODUCTION

  • FL keeps raw training data on client devices, but that privacy also makes it difficult to observe whether local training was prescribed or maliciously altered.Existing update-validation approaches check properties of submitted updates, while cryptographic proof approaches for training computation remain expensive and difficult to scale.Source: §1 · p. 1
  • MPC replay avoids requiring access to client data, but different execution environments can produce numerical discrepancies that make exact equality ineffective for verification.Replaying training directly can require matching the client environment and accessing its raw data, conflicting with FL's privacy goal.Source: §1 · p. 2
  • OPFL calibrates an empirical boundary from GPU–MPC gradient discrepancies to distinguish benign numerical differences from malicious deviations, and uses sampled MPC replay to lower audit cost.Calibration is performed offline once before FL begins; failed audits can trigger deposit slashing, assuming a majority of committee members are honest.Source: §1 · p. 2
  • 98.6× faster than full MPC-based verification and 625.5× faster than ZKSL-LP-G at a 1% audit rate in the five-client LeNet workload.The measurements compare verification and aggregation time; the paper cautions that different hardware environments make this a cost analysis, not a strict end-to-end speedup.Source: §1 · p. 2
  • 0% ASR and 0% false rejection rate across four representative attacks, with 0% ASR under the evaluated white-box adaptive PGD attack.The evaluated baselines were more easily bypassed under smaller adaptive perturbations.Source: §1 · p. 2
  • OPFL focuses on execution integrity and assumes an existing privacy-preserving aggregation layer rather than introducing a new aggregation protocol.Data poisoning is outside its scope and may be addressed by complementary robust aggregation or trust-based defenses.Source: §1 · pp. 2–3

2 PROBLEM FORMULATION AND THREAT MODEL

The paper formulates verification as checking whether a committed client gradient matches prescribed computation on committed private input, without revealing that input. Because honest GPU and MPC executions may differ numerically, OPFL accepts or rejects audited computations against a calibrated discrepancy boundary under stated protocol assumptions.

2 PROBLEM FORMULATION AND THREAT MODEL

  • OPFL verifies execution integrity, not whether a client's private dataset is benign; its FedSGD-style formulation uses one local training step per round and can extend to multiple steps by auditing each step and binding intermediate states.The public training configuration specifies the algorithm, hyperparameters, model and optimizer settings, data-processing rules, randomness, and replay metadata.Source: §2 · pp. 3–4
  • MPC replay lets the committee check prescribed computation on private input without revealing that input, but honest GPU and MPC executions can produce different gradients.The verification target is therefore discrepancy rather than exact equality.Source: §2 · p. 3
  • An audited pair is accepted when its discrepancy profile satisfies the empirically calibrated boundary B, and rejected otherwise.The goal is consistency between the committed gradient and prescribed computation on committed private input.Source: §2 · pp. 3–4
  • The threat model includes clients that skip computation, reuse gradients, modify intermediate computation, or submit arbitrary gradients, and permits adversaries to know and adapt to the verification procedure and boundary.The protocol assumes post-commitment auditing, binding commitments, and MPC security within the committee's privacy threshold.Source: §2 · p. 4
  • OPFL does not address data poisoning, verifier-committee construction, or privacy-preserving aggregation, relying on complementary defenses and existing mechanisms for those components.Its stated scope is verifying local training execution.Source: §2 · p. 4

3 OPFL DESIGN

OPFL fixes its training configuration and calibrated boundary before execution, commits client evidence, and verifies contributions before aggregation. It combines commitment checks for all gradients with sampled MPC replay audits that compare replay discrepancies against the boundary.

3 OPFL DESIGN

  • OPFL runs preparation and commitment, local training, then pre-aggregation verification; replay audits are performed on randomly sampled client-round pairs before eligible contributions are released for aggregation.The protocol fixes the verification rule and training configuration before execution, then commits gradients and sends them privately to the committee.Source: §3 · p. 4, Figure 1 · p. 4
  • Before FL, calibration uses honest steps to form empirical absolute and relative discrepancy distributions, with percentile quantiles summarizing their coordinate-wise values.The percentile grid can be adjusted: denser grids capture more distributional information but require more quantile computation and boundary checks.Source: §3 · p. 5
  • Because percentile profiles leave extreme coordinates unconstrained, OPFL also defines a tail discrepancy and enlarges raw bounds with predefined safety factors for deployment-time numerical variation.The relative discrepancy includes ϵ > 0 to prevent division by zero and stabilize values near zero.Source: §3 · p. 5
  • The calibrated boundary and training policy are fixed before FL, while client inputs remain private; clients commit to inputs and gradients before audit selection and secret-share gradients with the committee.OPFL relies on a public, binding commitment layer; a blockchain is offered as a natural instantiation with immutable records and deposit-and-slashing rules.Source: §3 · pp. 4–5
  • Every gradient first faces an MPC commitment check, and a failed commitment is rejected without entering aggregation.For sampled pairs, the committee additionally verifies the committed input and replays prescribed computation inside MPC.Source: §3 · p. 5
  • Only contributions passing the commitment gate and, when sampled, the replay audit are authorized for aggregation; failed contributions are excluded and trigger the predefined penalty.The committee forwards the same verified gradient shares to aggregation, and only the final eligibility decision is revealed.Source: §3 · pp. 5–6

4 EXPERIMENTAL EVALUATION

The evaluation tests OPFL against malicious training deviations, checks whether its empirical boundary generalizes, and measures audit cost and deposit trade-offs. It covers CNN and Transformer models, multiple optimizers, and full-model training and LoRA finetuning.

4.1 EFFECTIVENESS AGAINST MODEL POISONING

  • Baselines remain vulnerable to adaptive perturbations, while OPFL’s 0% FRR alongside its attack results indicates it is not simply rejecting every update.The evaluation constructs attacks separately for each verifier; the baseline rules are gradient-domain implementations, not full cryptographic-protocol reproductions.Source: §4.1 · pp. 7–8
  • OPFL achieves 0% ASR and 0% FRR across general attacks and evaluated models; PGD and tail-sparse PGD attacks also achieve 0% ASR.The tail-sparse tests are limited to the evaluated percentile grids and support sets.Source: §4.1 · pp. 7–8, Table 2 · p. 8

4.2 STABILITY AND GENERALIZATION OF THE EMPIRICAL BOUNDARY

  • With 100 calibration steps, BERT-Base, Qwen3, and LeNet achieve 0% FRR across their evaluated generalization settings.For BERT-Base, 100 steps achieve 0% FRR across four settings; 200–500 steps add no improvement and increase calibration cost.Source: Table 3 · p. 8, §4.2 · p. 8, Figure 2 · p. 8

4.3 SYSTEM OVERHEAD AND COST AMORTIZATION

  • At p = 0.01, OPFL is approximately 98.6× faster than full MPC-based FL and 625.5× faster than ZKSL-LP-G under the LeNet workload.These are cost-analysis results, not strict end-to-end speedups, because the measurements use different hardware environments.Source: Table 4 · p. 9, §4.3 · p. 9
  • Reducing the audit rate lowers verification cost but decreases detection probability; p = 0.01 requires 1.59× the deposit required at p = 0.10.The analysis illustrates a trade-off rather than prescribing a universal deposit; malicious benefits beyond saved computation can be difficult to quantify.Source: §4.3 · p. 9, Table 4 · p. 9

5 CONCLUSION

OPFL combines private MPC replay with an empirical gradient-discrepancy boundary to verify sampled client computations without requiring bitwise agreement across execution environments. Experiments show stable boundaries across diverse training settings, detection of evaluated attacks with low false rejection, and substantially reduced verification cost.

5 CONCLUSION

  • OPFL combines private MPC replay with an empirical gradient-discrepancy boundary to verify sampled client computations without requiring bitwise agreement across execution environments.The framework aims to verify clients without exposing their private training data.Source: §5 · p. 10
  • Experiments show that the boundary remains stable across diverse training settings and detects evaluated direct and adaptive attacks while maintaining a low false rejection rate.Source: §5 · p. 10
  • An optimistic verification mechanism substantially reduces verification cost.Source: §5 · p. 10

A ADDITIONAL BOUNDARY GENERALIZATION RESULTS

The deployed boundary is calibrated on the first 100 training steps and then frozen. Across generalization settings for Qwen3, LeNet, and BERT-Base, it achieves 0% FRR.

A ADDITIONAL BOUNDARY GENERALIZATION RESULTS

B ADAPTIVE PGD ATTACK DETAILS

The section describes verifier-specific adaptive PGD attacks and reports that OPFL maintains 0% attack success across three models and all tested attack strengths.

B ADAPTIVE PGD ATTACK DETAILS

  • The attacker optimizes a perturbation to maximize post-update prediction loss while satisfying a verifier-specific differentiable penalty with λ = 100.The selected candidate is then checked against the actual verification rule.Source: B ADAPTIVE PGD ATTACK DETAILS · p. 12
  • Optimization uses 20 normalized-gradient-ascent iterations, projection to a fixed-radius sphere, and three initializations.The step size is one fifth of the perturbation radius, and the soft penalty guides optimization but does not replace the final verification check.Source: B ADAPTIVE PGD ATTACK DETAILS · p. 12
  • Figure 3 shows generalization of deployed boundaries calibrated from 100 steps for Qwen3, BERT, and LeNet, with absolute and relative gradient discrepancies summarized by median and percentile ranges across evaluation steps.Source: Figure 3 · p. 13
  • Each model is evaluated on 200 instances selected to be correct after the honest update, with attack success requiring a verified prediction flip.For Qwen3, the evaluated prediction is at the final supervised token position.Source: B ADAPTIVE PGD ATTACK DETAILS · pp. 13–14
  • The comparison uses gradient-domain adaptations of RoFL-L2, RoFL-L∞, EIFFeL-NormBall, and RiseFL-Gaussian, rather than their complete cryptographic protocols.For deterministic norm-based baselines, the adaptive penalty uses the verifier's norm statistic and threshold.Source: B ADAPTIVE PGD ATTACK DETAILS · p. 13
  • Across all three models, OPFL maintains 0% ASR for every tested attack strength, while norm-based baselines remain vulnerable to adaptive attacks.The baselines are particularly vulnerable at moderate perturbation strengths.Source: B ADAPTIVE PGD ATTACK DETAILS · p. 15
  • Tail-sparse LeNet attacks restrict perturbations to the top 1% or 10% of coordinates ranked by absolute loss-gradient magnitude and retain an infinity-norm guard.These experiments use independently specified verification grids.Source: B ADAPTIVE PGD ATTACK DETAILS · p. 15, Table 9 · p. 15

Related papers

Loading 2609.37011v1…