PaperlayerView exact revision on arXiv

Source-linked AI summary

One-Shot Federated Model Editing for Device Dynamics at the Edge

Youming Tao, Weijun Yin, Huanyi Xie, Yanqing Yao, Dongxiao Yu, Xiuzhen Cheng, Di Wang, Falko Dressler

arXiv:2610.35212v1cs.LGcs.AIcs.DCcs.NI

TL;DR

One-shot FL does not readily accommodate client additions and withdrawals after training, while repeated retraining is costly in dynamic settings. F-SOSA combines curvature-aware fusion with randomized client subsampling, and F-SOSA-U/I edit client contributions while matching retraining’s joint model-and-state distribution. Only an expected O(ρ)-fraction of requests trigger editing, and F-SOSA-U provides a more favorable overall trade-off among utility, communication efficiency, and computational cost.

  • Problem

    One-shot FL does not readily accommodate client additions and withdrawals after training, while repeated retraining is costly in dynamic settings.

    Source: §I · pp. 1–2

  • Method

    F-SOSA combines curvature-aware fusion with randomized client subsampling, and F-SOSA-U/I edit client contributions while matching retraining’s joint model-and-state distribution.

    Source: §I · p. 2, A. TV-Stable One-Shot Federated… · p. 6

  • Results

    Only an expected O(ρ)-fraction of requests trigger editing, and F-SOSA-U provides a more favorable overall trade-off among utility, communication efficiency, and computational cost.

    Source: §I · p. 2, C. Scalability of F-SOSA-U and… · p. 23

  • Takeaways & Limitations

    The framework supports exact post-training additions and withdrawals across successive client-set changes while preserving model utility and editing efficiency.

    Source: §I · p. 2, §VII · p. 15

    As client counts rise, absolute accuracy decreases on MNIST, SVHN, and CIFAR-10 because each client receives less training data under the stated partitioning strategy.

    Source: C. Scalability of F-SOSA-U and… · p. 23

Abstract

from arXiv · show

Standard federated learning (FL) requires persistent device connectivity across multiple communication rounds, making it unsuitable for dynamic environments such as vehicular networks, where devices frequently join and leave. One-shot FL offers a promising alternative by completing training in a single communication round. However, one-shot FL suffers from model solidification and cannot readily adapt to subsequent client dynamics, including new client contributions and the withdrawal of previous contributions. We therefore consider exact model editing, which removes withdrawn contributions through unlearning or incorporates newly contributed ones through incremental learning without retraining from scratch. We identify that efficient editing fundamentally hinges on learning stability. To this end, we propose F-SOSA, the first stable one-shot FL algorithm that enables rapid and exact model editing. F-SOSA leverages sub-sampling of local models with carefully tuned rate for stable aggregation and performs curvature-aware model fusion to preserve utility. We prove that F-SOSA achieves vanishing training error and reasonably small generalization error bounds. Building on this stability, we further develop two single-round editing algorithms: F-SOSA-U for unlearning and F-SOSA-I for incremental learning. They ensure that the edited model, along with all intermediate algorithmic states, is statistically indistinguishable from retraining while incurring only an expected $O(\rho)$ fraction of its cost, where $\rho$ quantifies stability. Experiments demonstrate that our approach matches retraining performance while reducing cumulative time and communication overhead by at least 36% and 22%, respectively, under realistic urban mobility simulations.

I. INTRODUCTION

Dynamic edge participation motivates one-shot federated learning, but post-training client changes require exact editing rather than repeated retraining. The paper frames this goal, distinguishes it from related settings, and develops stability as a basis for efficient edits.

I. INTRODUCTION

  • One-shot FL can reduce reliance on persistent participation, but model solidification leaves client additions and withdrawals difficult to handle without exact post-training editing.Approximate successive updates may propagate history-dependent drift, while exact editing restores the retraining distribution after each change.Source: §I · p. 1

C. Federated Model Editing

  • Exact federated model editing removes a departing client’s contribution or incorporates a new client’s contribution while preserving statistical indistinguishability from retraining for the model and internal states.Matching the joint model-and-state distribution supports repeated edits without accumulating approximation error.Source: §I · p. 2, C. Federated Model Editing · p. 3

II. RELATED WORK

  • Prior work addresses approximate unlearning, fixed-client continual learning, or churn during multi-round training, not exact post-training additions and withdrawals in one-shot FL.The paper identifies repeated edits as a setting where residual deviations may propagate.Source: §II · p. 2

A. System Model

  • The system model uses a trusted coordinator and dynamic devices that may join or leave, without imposing a particular distribution on arrival or departure events.A departing device may request withdrawal before losing connectivity.Source: A. System Model · p. 3

B. Federated Learning

  • The learning setup assigns each device a local dataset and considers editing sequences whose active population stays within a constant factor of its initial size.The stated bound is M(0)/2 ≤ M(t) ≤ 2M(0).Source: B. Federated Learning · p. 3

D. Total Variation (TV) Stability

  • Total-variation stability supports editing by limiting how much the output distribution changes after a client change, allowing existing computations to be reused.The minimum expected editing cost is proportional to TV(P, Q), so only an O(TV(P, Q)) fraction of requests requires recomputation.Source: D. Total Variation (TV) Stability · p. 3

IV. OUR METHOD

F-SOSA combines curvature-aware fusion with randomized client subsampling to support stable one-shot learning and reusable edits. Its projection formulation uses client-additive statistics, while Fisher representations trade curvature fidelity against resource costs.

A. TV-Stable One-Shot Federated Learning: F-SOSA

  • F-SOSA reformulates curvature-aware fusion as a projection using client-additive statistics, then uses randomized client subsampling to control total-variation stability.The projection statistics can be updated by subtracting a departing contribution and adding an arriving one, reusing unchanged contributions.Source: A. TV-Stable One-Shot Federated… · pp. 4–5
  • The local posterior approximation assumes a negative-log-likelihood loss from an exponential-family model, including common squared-loss and cross-entropy choices.The derivation uses a flat prior so the global MAP estimator coincides with the learning objective’s minimizer.Source: A. TV-Stable One-Shot Federated… · p. 4
  • Fisher-weighted local approximations yield a global quadratic fusion objective, and a closest-to-local-optima rule selects a solution in over-parameterized regimes.The projection solves a linear system in the dual variable and recovers the fused model in closed form.Source: A. TV-Stable One-Shot Federated… · p. 5
  • Fisher representations trade curvature fidelity against communication, storage, and coordinator computation; aggregated K-FAC structure may be lost, raising projection cost.Conjugate gradient terminates in at most rank(F) iterations; a diagonal approximation has lower structural overhead than richer K-FAC curvature.Source: A. TV-Stable One-Shot Federated… · pp. 4, 6
  • Uniformly sampling P = ⌊ρM0⌋ clients makes neighboring client sets produce overlapping output distributions, provided client contributions have comparable aggregation priority.The subset size remains fixed throughout the editing sequence.Source: A. TV-Stable One-Shot Federated… · p. 6
  • F-SOSA combines Fisher-informed projection and subsampling; participating clients locally train, compute Fisher information, and send their contributions for fusion.The coordinator forms sufficient statistics and projects the subsampled model onto the affine constraint set.Source: A. TV-Stable One-Shot Federated… · p. 6

B. Efficient Editing Algorithms: F-SOSA-U and F-SOSA-I

F-SOSA-U and F-SOSA-I edit a fixed-size sampled client subset by coupling it to the updated population’s sampling distribution. When the subset changes, they update only the affected contribution and reuse the remaining fusion statistics.

B. Efficient Editing Algorithms: F-SOSA-U and F-SOSA-I

  • F-SOSA-U/I couple each client-set change to the updated uniform size-P subset distribution, retaining the current subset whenever the coupling permits.Deletion replaces a departing client only if selected; insertion includes the newcomer with probability P/(M + 1).Source: B. Efficient Editing Algorithms:… · p. 6
  • Each triggered edit changes one client contribution to the sufficient statistics and performs one projection solve, without recomputing all previously sampled clients.The coordinator updates aggregate fusion statistics by subtracting the departing contribution and adding the incoming one.Source: B. Efficient Editing Algorithms:… · p. 7
  • Editing is triggered with probability at most O(ρ), giving expected cost O(ρ) relative to full retraining per client change.For sequential requests, F-SOSA-U/I update after each request and preserve fixed-P exactness by induction.Source: B. Efficient Editing Algorithms:… · p. 7
  • For simultaneous additions and deletions, paired replacements and fixed-P couplings determine the final subset; the coordinator applies only net contribution changes and performs at most one projection.The model is reused if the final subset is unchanged, and no intermediate model is released.Source: B. Efficient Editing Algorithms:… · p. 7

V. THEORETICAL RESULTS

F-SOSA’s stability analysis bounds how much its output distribution changes when one device joins or leaves. The bound follows from uniform client subsampling and yields ρ-TV stability when the sample size is set to ρM.

A. Stability of F-SOSA

  • F-SOSA is ρ-TV-stable for any ρ ∈ (0, 1), bounding output-distribution change after a one-device population update.The proof compares the output distributions through the TV distance between their client-subset sampling measures.Source: A. Stability of F-SOSA · p. 8
  • For a departing device, the proof extends the subset probability spaces with a zero-probability event so both sampling measures can be compared on a common space.The subset measure samples P clients uniformly from the current population.Source: A. Stability of F-SOSA · p. 8
  • With fixed initialization and parameters, the algorithm is the same deterministic map from a sampled subset to the model for neighboring device sets.The proof uses the data processing inequality to bound output TV by the TV distance between subset-sampling measures.Source: A. Stability of F-SOSA · p. 8

B. Learning Utility of F-SOSA

F-SOSA’s utility analysis studies a two-layer ReLU network using over-parameterized training bounds and assumptions on the data and Gram matrix. It establishes vanishing empirical risk and a population-risk bound involving federated Rademacher complexity and a stability–utility trade-off.

B. Learning Utility of F-SOSA

  • The analysis considers a two-layer ReLU network and assumes normalized, unique data with a non-degenerate Gram matrix; it makes no bounded heterogeneity assumption.The analysis therefore applies to arbitrary non-i.i.d. client data under the stated assumptions.Source: B. Learning Utility of F-SOSA · p. 8
  • With sufficient network width and local iterations, F-SOSA’s empirical risk can be made vanishingly small in the over-parameterized regime.The proof combines local fitting-error bounds with bounds on additional error from Fisher-based fusion.Source: B. Learning Utility of F-SOSA · pp. 9–10
  • F-SOSA’s population-risk bound is governed by a federated Rademacher complexity term of order 1/√(ρN), under the theorem’s assumptions and sufficient clients and local samples.The bound also includes a target-dependent quantity involving y⊤(H∞)−1y and λ0.Source: B. Learning Utility of F-SOSA · p. 11
  • Larger ρ includes more clients and improves the utility/generalization bound, but weakens TV stability and increases expected editing cost.The analysis identifies this as a trade-off between editing efficiency and generalization.Source: B. Learning Utility of F-SOSA · p. 11

C. Exactness and Efficiency of Model Editing

F-SOSA-U/I transport the sampled subset distribution after client additions or removals, preserving exact editing. Their recomputation burden is bounded in expectation by stability, while batch editing couples simultaneous changes and minimizes subset-change measures.

C. Exactness and Efficiency of Model Editing

  • F-SOSA-U/I satisfy exact federated model editing and trigger at most O(ρR) recomputations in expectation across R requests.Their total expected runtime is bounded by O(max{ρR · Training Time, R}).Source: C. Exactness and Efficiency of Model… · p. 11
  • The batch coupling simultaneously minimizes the probability of changing the subset and the expected number of replaced members among couplings of the initial and final uniform size-P distributions.The proposition establishes exactness at batch completion.Source: C. Exactness and Efficiency of Model… · p. 12
  • For batch edits, Algorithm 4 produces the retraining subset distribution and exact model and internal state at batch completion.The proof obtains this by pairing replacements, applying fixed-P insertion or deletion couplings, and updating the net aggregate statistics.Source: C. Exactness and Efficiency of Model… · p. 12
  • For a batch of k client changes, if P ≤ ρM0 and both populations are at least M0/2, expected net replacements are at most 2ρk and projection is triggered with probability at most min{1, 2ρk}.A projection is needed only when the final subset changes.Source: C. Exactness and Efficiency of Model… · p. 12

VI. EXPERIMENTS

The experiments assess F-SOSA’s learning performance, editing accuracy and efficiency, stability trade-offs, and resistance to membership inference attacks, including a simulated vehicle-mobility case study.

VI. EXPERIMENTS

  • The evaluation covers one-shot learning performance, editing accuracy and efficiency, stability-controlled trade-offs, and resilience to membership inference attacks.A vehicle-mobility simulation provides an additional case study.Source: §VI · p. 12
  • A vehicle-mobility simulation is included to examine the approach’s practical applicability.Source: §VI · p. 12
  • The evaluation also examines resilience to membership inference attacks.Source: §VI · p. 12

1) Datasets and Models:

The experiments use four standard image datasets and vary client heterogeneity, with additional evaluations on deeper architectures and a real-world driving dataset.

1) Datasets and Models:

  • MNIST, SVHN, CIFAR-10, and FEMNIST form the main dataset set, with Dirichlet partitions used to vary heterogeneity for the first three.The Dirichlet α values are 0.1, 0.5, and 0.8, with 0.5 as default; FEMNIST users are sampled at 10% and grouped into clients.Source: 1) Datasets and Models · p. 12
  • The default classification setup uses the two-layer CNN from with a K-FAC Fisher approximation.Source: 1) Datasets and Models · p. 12
  • Additional architecture evaluations use ResNet18 on CIFAR-10 and YOLOv8s on BDD100K.Source: 1) Datasets and Models · p. 12

2) Baselines:

Across benchmark and simulated mobility settings, F-SOSA and its editing variants maintain competitive utility while reducing communication or computation costs; the conclusions report statistical exactness alongside utility and efficiency.

2) Baselines:

  • Baselines include one-shot fusion methods, editing-specific unlearning methods, and fine-tuning; experiments report accuracy, loss, wall-clock time, and communication bytes.Unless otherwise specified, learning uses 20 clients and ρ = 0.5; editing starts with 50 active clients among at most 100, with K = 50. Retrain uses the same current subset size as the editing methods.Source: 2) Baselines · p. 12, 4) Evaluation Metrics · p. 12, 3) Hyperparameters · p. 12

1) Performance of F-SOSA in Heterogeneous Settings:

3) Performance of F-SOSA-I/U:

  • On CIFAR-10, F-SOSA-I matches Retrain accuracy and reduces final cumulative communication cost by 39.73% versus Retrain.F-SOSA-U reaches accuracy comparable to Retrain and exceeds FedEraser, FedRR, and FATS-CU by up to 10%.Source: 3) Performance of F-SOSA-I/U · p. 14, Figure 3 · p. 14

4) Trade-off Between Editing Utility and Efficiency w.r.t. Stability Parameter ρ:

5) Resistance to Membership Inference Attacks:

6) F-SOSA Performance Across Model Architectures:

  • On ResNet18/CIFAR-10 and YOLOv8s/BDD100K, F-SOSA maintains competitive utility with lower communication and computation costs.The conclusions describe the framework as achieving statistical exactness while preserving utility and efficiency.Source: 6) F-SOSA Performance Across Model… · p. 15, §VII · p. 15

7) Scalability with the Number of Clients:

  • Across client populations from 20 to 500, F-SOSA-I/U maintain accuracy close to Retrain while reducing communication and computation costs.The efficiency gains remain substantial at 500 clients.Source: 7) Scalability with the Number of… · p. 15

C. Case Study Using Simulation of Urban Mobility (SUMO)

  • 36.66%–53.84% lower cumulative time and 21.97%–25.76% lower communication overhead accompany accuracy matching Retrain in the SUMO mobility case study.The reported communication cost includes model transfers and Fisher-information uploads.Source: C. Case Study Using Simulation of… · p. 15, Figure 6 · p. 15

APPENDIX A PROOFS OF AUXILIARY LEMMAS

The auxiliary proofs use convexity, concentration bounds, and gradient-descent recurrences to establish intermediate bounds.

APPENDIX A PROOFS OF AUXILIARY LEMMAS

  • A convexity argument bounds the squared distance of the averaged model from initialization by averaging local-model distances.Source: Appendix A · p. 18
  • Hoeffding’s inequality is applied after expressing a Gram-matrix entry as an average of bounded independent variables.Source: Appendix A · p. 19
  • A gradient-descent recurrence is unrolled to express prediction evolution and the final model’s distance from initialization.Source: Appendix A · p. 20

U ˜HPU

The supplied proof fragments record intermediate steps involving concentration, data normalization, and a union bound; they do not provide a complete account of the section’s argument.

  • The proof also invokes the data normalization assumption when bounding a submatrix expression.Source: U ˜HPU · p. 20
  • A proof step applies Markov’s inequality with failure probability δ/N, then takes a union bound over all (i, j) pairs.Source: U ˜HPU · p. 21
  • The proof invokes McDiarmid’s inequality because replacing one data point changes R(F) by at most c/N.Source: U ˜HPU · p. 21

APPENDIX B ADDITIONAL EXPERIMENTAL DETAILS AND RESULTS

Across architecture, client-population, and mobility settings, the experiments evaluate F-SOSA and its editing algorithms against retraining and other baselines. The reported results show utility close to retraining alongside communication and computation savings.

1) Dataset and Models:

  • The experiments use MNIST, SVHN, CIFAR-10, and FEMNIST, plus ResNet18 on CIFAR-10 and YOLOv8s on BDD100K, with Dirichlet partitioning for heterogeneity.BDD100K is a real-world driving dataset; FEMNIST is sampled from LEAF users.Source: 1) Dataset and Models · p. 21

2) Hyperparameters:

  • For ResNet18 and YOLOv8s experiments, the study uses 20 clients and a local learning rate of 0.001, with task-specific epochs, batch sizes, and BDD100K scheduling settings.BDD100K uses 20 local epochs, batch size 32, and 640×640 inputs; CIFAR-10 uses 50 epochs and batch size 64.Source: 2) Hyperparameters · p. 21

3) Metrics:

  • BDD100K detection utility is measured by mAP50-95, averaging AP across IoU thresholds from 0.50 to 0.95 and ten object categories.Higher mAP50-95 indicates better combined recognition and localization performance.Source: 3) Metrics · p. 21

B. F-SOSA Performance Across Model Architectures

C. Scalability of F-SOSA-U and F-SOSA-I Across Client Populations

  • At 500 clients, F-SOSA-I stays within 2.46 percentage points of Retrain across four datasets, with communication cost falling to 0.67× Retrain and time savings of 23.26%–49.04%.F-SOSA-U also remains close to or exceeds Retrain, saving 38.92%–43.35% of retraining time at 500 clients.Source: C. Scalability of F-SOSA-U and… · p. 22
  • As client populations grow, absolute accuracy falls on MNIST, SVHN, and CIFAR-10, which the authors attribute to less training data per client; this trend is not observed on FEMNIST.The algorithms retain best or near-best relative accuracy against their respective baselines.Source: C. Scalability of F-SOSA-U and… · p. 23

D. Real-World Driving Case Study with BDD100K and SUMO Mobility

  • After 31 BDD100K editing requests, F-SOSA-I/U reaches 20.27% mAP50-95 versus Retrain’s 20.19%, while reducing cumulative communication by 25.00% and computation time by 32.87%.Figure 7 reports mAP50-95 and cumulative cost for simulator-generated editing requests on BDD100K.Source: D. Real-World Driving Case Study with… · p. 23, Figure 7 · p. 23

E. Detailed Communication-Cost Breakdown

  • Across five datasets, F-SOSA-I/U reduce cumulative communication by 21.97%–25.76% relative to retraining, including communication for model broadcasts and selected-client uploads.The accounting includes approximate Fisher-information uploads; unselected clients do not upload models or Fisher information.Source: E. Detailed Communication-Cost Breakdown · pp. 24–25

Related papers

Loading 2610.35212v1…