PaperlayerView exact revision on arXiv

Source-linked AI summary

Global Federated Learning Strategies for Building Efficient Personalized Models

Seongyoon Kim

arXiv:2608.15107v1cs.LG

TL;DR

Federated learning must handle heterogeneous user data without sacrificing global or personalized performance. This dissertation develops feature-normalization and feature-distillation strategies for global knowledge preservation and shows that feature normalization improves federated learning performance while a single global initialization can support personalization after sufficient fine-tuning.

  • Problem

    Heterogeneous user data distributions make it difficult to maintain both global-model and personalized performance in federated learning.

    Source: Chapter 1. Introduction · p. 14, §2.1 · p. 16

  • Method

    The dissertation uses feature normalization for federated aggregation and feature distillation from global representations to preserve global knowledge during local adaptation.

    Source: §2.1 · pp. 16–17, §3.1 · p. 42

  • Results

    Greater data heterogeneity produces larger feature-norm disparities, while feature normalization delivers superior federated-learning performance.

    Source: §2.8 · p. 40

  • Takeaways & Limitations

    Feature representations are a central target for improving federated learning under heterogeneous data.

    Source: §2.1 · p. 16, §2.8 · p. 40

    Strengthening local alignment can amplify forgetting of unobserved classes and degrade global-model performance.

    Source: §3.2 · p. 43

Abstract

from arXiv · show

Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that "increasing the number of global models yields better initialization," and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.

Chapter 2. FedFN: Feature Normalization for Alleviating Data Hetero- … FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning

This chapter examines feature-norm discrepancies as a key bottleneck in federated learning under data heterogeneity and presents FedFN, which integrates feature normalization into local training. It evaluates the method across heterogeneous federated and centralized settings, model configurations, and analyses of feature and weight-norm behavior.

FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning

  • Feature representations deteriorate more than classifier weights as data heterogeneity increases, making feature collapse the primary bottleneck in FedAvg.The gap between local observed-class feature norms and global or unobserved-class norms widens, whereas classifier-weight norms show different behavior.Source: FedFN: Feature Normalization for… · p. 16
  • FedFN introduces feature normalization into federated averaging to mitigate feature-norm disparities between local and global models.The method is presented as a straightforward learning method targeting the observed discrepancy under heterogeneous data.Source: FedFN: Feature Normalization for… · p. 16
  • FedFN demonstrates superior performance through extensive experiments, including experiments with pretrained ResNet18.The chapter also confirms FedFN’s applicability to foundation models.Source: FedFN: Feature Normalization for… · p. 16

2.1 Introduction

  • Federated learning aggregates locally trained models to improve a global model, but diverse client data distributions degrade global performance.This heterogeneity motivates examining whether feature representations or classifier weights are more strongly affected.Source: §2.1 · p. 16
  • FedFN incorporates feature normalization into federated averaging, eliminating local feature-norm discrepancies and maintaining representation quality under highly heterogeneous data.The normalization technique is also applied to existing algorithms, with reported performance improvements that persist with pretrained models.Source: §2.1 · pp. 16–17
  • FedAVG feature representations are more adversely affected by data heterogeneity than classifier weights, with widening norm disparities across observed and unobserved classes and between local and global models.The analysis motivates targeting feature norms rather than classifier weights.Source: §2.1 · pp. 16–17

2.2.1 Federated Learing

  • Prior FL research addresses heterogeneity through client-drift mitigation, server aggregation, data sharing, personalized updates, loss modification, prototype communication, and feature-alignment methods.Related work also reports dimensional collapse in global representations and uses local regularization to improve feature alignment.Source: §2.2.1 · p. 17

2.2.2 Partial Model Updates in FL

  • Partial-model-update studies target classifier bias with post-calibration, restricted softmax losses, or fixed orthogonal classifiers, while other work enhances features through contrastive losses, regularization, or prototype communication.These approaches address missing-class classifier updates, local feature alignment, dimensional collapse, or personalized feature exchange.Source: §2.2.2 · p. 17

2.2.3 Feature Normalized Model in DL

  • Feature normalization has been used in face recognition, regression, and federated learning to improve feature discriminative power, sometimes normalizing both feature vectors and classifier weights.The resulting logits are based on cosine similarity with restricted element values.Source: §2.2.3 · p. 18

2.3 Preliminaries

  • The chapter introduces FL notation and describes a privacy-preserving procedure in which selected clients locally update the received global model before server aggregation over R rounds.The aggregated model is iteratively refined for performance on the entire test dataset while preserving individual client data privacy.Source: §2.3 · p. 18, §2.3.1 · pp. 18–19

2.3.2 Experimental Setup

  • Experiments simulate federated learning with 100 clients, using CIFAR-10 with VGG11 and CIFAR-100 with MobileNet, and distributing data via two distinct non-IID partition strategies; concentration parameter α controls heterogeneity strength.Source: §2.3.2 · p. 19

2.4.1 4-Factor Analysis of FedAVG

  • The FedAVG analysis compares feature representations and classifier weights using four factors, including weight similarity and inter-class similarity between feature prototypes.Weight similarity is computed from normalized classifier-weight cosine similarities, with lower similarity preferred for distinct class classifiers.Source: §2.4.1 · p. 20
  • As data heterogeneity increases, FedAvg is most adversely affected in inter-class similarity and prototype-weight alignment, while intra-class similarity improves.Weight similarity remains comparatively low across the evaluated heterogeneity settings.Source: §2.4.1 · p. 21

2.4.2 Feature Norm Discrepancy Persists in Local and Global Models

  • Local models retain higher feature norms for in-distribution classes than global models even after weight norm bias vanishes.Both biases diminish as the learning rate decreases, but feature norm bias persists in local models.Source: §2.4.2 · p. 22

2.4.3 Feature Norm Discrepancy with Increasing Heterogeneity

  • Feature norm discrepancy between local and global models is examined across IID and s ∈ {2, 5, 10} heterogeneity settings on CIFAR-10.The supplied result passage states that the discrepancy varies with heterogeneity but does not provide its direction or numerical values.Source: §2.4.3 · p. 22

2.4.4 Reducing Feature Norm Discrepancy for Improved Performance

  • L2 feature norm regularization reduces local-global norm discrepancies and improves performance, with optimal µ depending on heterogeneity.For s = 5 and s = 2, the reported optimal values are µ = 0.05 and µ = 0.005, respectively; µ = 0.005 aligns norms more closely with IID behavior.Source: §2.4.4 · p. 23

2.5 FedFN: Federated Averaging with Feature Normalization Update

  • FedFN integrates feature normalization with FedAvg to address local feature norm bias while retaining the federated averaging framework.The feature normalization update sets each input feature-vector norm to 1 and modifies the logit computation through norm scaling.Source: §2.5 · p. 23, §2.5.1 · p. 24

2.5.1 FedFN Algorithm

  • FedFN uses normalized feature vectors and a separately tuned initial learning rate of 0.03, compared with 0.01 for FedAvg.The learning rates were selected through grid search because feature normalization changes the logit scaling and learning-rate effects.Source: §2.5.1 · p. 24

2.6 Experiment Results

  • FedFN significantly improves accuracy over FedAvg while remaining compatible with existing federated learning algorithms.The section also reports comparative analysis with a pretrained model, but supplies no detailed values here.Source: §2.5.1 · p. 24, §2.6 · p. 25

2.5.2 4-Factor Analysis of FedFN

  • FedFN substantially improves inter-class prototype similarity and consistently outperforms FedAvg, despite slight decreases in intra-class similarity and prototype-weight alignment.The improved inter-class separation is associated with better discrimination and stability across heterogeneous settings.Source: §2.5.2 · pp. 24–25

2.6.1 Compatibility of FN with Existing FL Algorithms

  • FN updates consistently outperform baseline, BABU-augmented, and other evaluated algorithms across all heterogeneity settings.Scaffold generally performs well but can fail under high heterogeneity, including with BABU, whereas FN remains superior.Source: Table 2.3 · p. 25, §2.6.1 · p. 25

2.6.2 Comparative Analysis with Pretrained Model

  • FedFN consistently outperforms FedBABU and FedAVG with pretrained and non-pretrained ResNet18 models, improving rather than declining when pretrained models are used.The pretrained-model experiments use ResNet18 on CIFAR-10, with FN updates initialized at 0.1.Source: §2.6.2 · pp. 25–26, Table 2.4 · p. 26

2.7.1 Grid Search Result

  • Grid searches select 15 local epochs for CIFAR-10 and 5 for CIFAR-100, with default initial learning rates of 0.01 and 0.1, respectively.The searched learning-rate ranges are {0.01, 0.03, 0.05, 0.1} for CIFAR-10 and {0.1, 0.3, 0.5, 1.0} for CIFAR-100.Source: §2.7.1 · p. 26

FedBABU vs FedAVG vs FedFN

SphereFed

  • SphereFed (MSE) performs suboptimally in the initial η range, requiring an extended search that selects η=4.5, while SphereFed (CE) selects η=0.03.The initial range for both variants is {0.1, 0.3, 0.5, 1.0}, and the extended MSE range is {1.5, 3.0, 4.5, 5.0}.Source: Table 2.9 · p. 30, SphereFed · p. 30, Table 2.10 · p. 31

Weight Norm Does Not Matter on FedFN

  • Weight-norm disparity between local and global models increases with data heterogeneity early in training but diminishes as the learning rate decreases.In the s=2 setting, local models initially favor seen classes over unseen classes, then align with the global model’s weight-norm mean.Source: Weight Norm Does Not Matter on FedFN · p. 32, Figure 2.7 · p. 33

2.7.3 Additional GFL Results

  • For the seed used in Table 2.11, baseline algorithms exhibit notably lower performance; applying FN consistently demonstrates superior performance across all settings, with differences less pronounced and outcomes more stable.Source: §2.7.3 · p. 33

2.7.4 Personalized Federated Learning (PFL) Results

  • FedFN-FT consistently outperforms the compared one-step and two-step personalized federated learning methods.The comparison includes local models, FedPer [21], Per-FedAVG [23], FedRep [22], FedAVG-FT, FedBABU-FT [6], and SphereFed-FT [1].Source: Table 2.12 · p. 34, §2.7.4 · p. 34

2.7.6 Logit Should Be Non-Restricted

  • Relaxing logit constraints improves SphereFed(CE) up to τ = 15, surpassing FedBABU and SphereFed(MSE) there, although FedFN remains stronger.The findings recommend constructing logits through feature normalization with relaxed element-wise constraints.Source: Table 2.14 · p. 36, §2.7.6 · p. 36

2.7.7 Reproduced Result from SphereFed

  • The original SphereFed results could not be reproduced: exact reported comparisons of 71.85 vs 18.01 and 68.78 vs 37.76 fell far below, while the best configuration reached 52.69.Reproduced centralized learning achieved 68.27, below the original reported FL performances; the reproduced results used the original MobileNet settings and learning-rate searches.Source: Table 2.15 · p. 37, §2.7.7 · p. 37

2.7.8 FedFN vs FedFR

  • FedFN consistently outperforms FedFR across all tested settings, while FedFR remains superior to FedAVG and FedBABU on CIFAR-10.On CIFAR-100 with s = 10, FedFR is worse than FedBABU and FedFN across all hyperparameter candidates.Source: §2.7.8 · p. 38, Table 2.16 · p. 38, Table 2.17 · p. 38

2.7.9 FN in the Centralized Learning

  • In centralized CIFAR-100 training, the FN module significantly improves over the Full module, whereas the Body module declines slightly because only part of the model is updated.The experiments use MobileNet and replicate the settings from.Source: §2.7.9 · p. 39

2.8 Conclusion

  • FedFN consistently improves feature representations and achieves exceptional performance across diverse federated settings, including pretrained ResNet18 and foundational-model applications.The conclusion attributes the problem to increasing feature-norm disparities between global and local models and identifies feature normalization as the remedy.Source: §2.8 · p. 40

FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning

This section introduces FedDr+, which combines dot-regression for local feature alignment with global feature distillation to preserve knowledge of unobserved classes in heterogeneous federated learning. It also presents FedDr+ FT for client-specific fine-tuning and evaluates performance, computational cost, sensitivity, configurations, and procedures across federated settings.

FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning

  • FedDr+ outperforms frozen-classifier methods and other state-of-the-art approaches across diverse data distributions.The method is designed to improve both global and personalized federated learning performance.Source: FedDr+: Stabilizing Dot-regression… · p. 41

3.2 Related Work

  • Prior frozen-classifier methods address heterogeneity by fixing classifier weights, while dot-regression targets faster feature alignment by removing cross-entropy’s pushing component.This approach was motivated by limitations in aligning features with corresponding classifier weights.Source: §3.1 · p. 42, §3.2 · p. 43

3.1 Introduction

  • Dot-regression improves alignment for observed classes but can reduce global-model performance by disrupting representations of unobserved classes.The resulting forgetting reflects insufficient preservation of global knowledge during local training.Source: §3.1 · p. 42
  • FedDr+ combines dot-regression loss with feature distillation to align local features while preserving global knowledge of unseen classes.It freezes the classifier as a simplex ETF, uses dot-regression for alignment, and distills global feature representations during aggregation.Source: FedDr+: Stabilizing Dot-regression… · p. 41, §3.1 · p. 43
  • FedDr+ FT extends FedDr+ to personalized federated learning by fine-tuning its global model with the same dot-regression loss on client-specific data.The extension emphasizes that effective personalization starts from a well-generalized global model.Source: §3.1 · p. 43

3.2 Related Work

  • Prior FL methods use realignment, fixed simplex-ETF classifiers, semantic alignment, or knowledge distillation, whereas this work proposes feature distillation for simultaneous local-global performance.The section characterizes this as, to the authors’ knowledge, the first application of feature distillation in FL.Source: §3.2 · p. 44

3.3 Preliminaries

  • The preliminaries establish the FedAvg pipeline, dot-regression loss, and notation used throughout the framework.The notation summary is provided in Table 3.1.Source: §3.3 · p. 44, Table 3.1 · p. 44

3.3.1 Basic Setup of Conventional FedAvg Pipeline

  • The study follows conventional FedAvg, where participating clients train on private heterogeneous data and the server aggregates their locally updated model parameters across communication rounds.The setup defines client-specific distributions, partial client participation, global-model transmission, local updates, and server aggregation.Source: §3.3.1 · pp. 44–45

3.3.2 Dot-Regression Loss for Feature Alignment

  • LDR accelerates feature alignment by removing cross-entropy’s pushing gradient and concentrating its pulling effect on the true-class classifier direction.The loss aligns penultimate-layer features with the true-class classifier, while the classifier can remain frozen under a simplex ETF construction.Source: §3.3.2 · p. 45

3.4 When Dot-Regression Loss Meets FL

  • FedDr+ addresses LDR’s loss of unobserved-class information by incorporating feature distillation to maintain both local and global model performance.The section presents feature distillation as a novel loss component and identifies it as the proposed response to LDR’s global-generalization limitation.Source: §3.2 · p. 44, §3.4 · p. 46

3.4.1 Impact of LDR on Local and Global Models

  • LDR outperforms cross-entropy on observed-class alignment but performs worse on unobserved classes, causing lower global accuracy in the shard setting.The degradation arises because each client observes at most 10 of 100 classes, leaving LDR poorly aligned on unobserved classes.Source: §3.4.1 · pp. 46–47, Figure 3.3 · p. 47
  • LDR improves observed-class performance but degrades unobserved classes, motivating FedDr+ to retain local alignment while mitigating forgetting of unobserved knowledge.The underlying issue is a feature-alignment gap with the fixed classifier between observed and unobserved classes.Source: §3.4.1 · p. 47, §3.4.3 · p. 48

3.4.2 FedDr+: LDR With Feature Distillation for FL

  • FedDr+ combines LDR with a frozen classifier and a regularizer that distills the global model’s feature vectors to client features, enhancing generalization across all classes.Its proposed loss combines LDR and feature-distillation loss LFD as LDr+ = β · LDR + (1 − β) · LFD.Source: §3.4.2 · p. 47

3.4.3 Effect of Feature Distillation

  • Feature distillation reduces feature distance and angle distance for both observed and unobserved classes, improving alignment and global accuracy over LDR by preserving global knowledge.At round 320, LDr+ provides the best local alignment across all classes and superior global performance compared with LCE and LDR.Source: §3.4.3 · p. 48

3.4.4 Synergistic Effect with Different Types of FL Algorithms and Regularizers

  • FedDr+ achieves the strongest performance among tested algorithm–regularizer combinations, with feature distillation most effective alongside dot-regression, SphereFed, and FedBABU.Table 3.2 reports the comparison; the synergy is strongest for dot-regression, followed by SphereFed and FedBABU.Source: §3.4.4 · p. 49, Table 3.2 · p. 49

3.5 Experiments and Results

  • The experiments evaluate FedDr+ in global federated learning, then analyze elapsed time and sensitivity to local epochs, client sampling ratios, and β before proposing FedDr+ FT.FedDr+ FT fine-tunes the FedDr+ global model with LDr+ and is compared with existing personalized federated learning methods.Source: §3.5 · p. 50

3.5.1 Experimental Setup

  • The study uses 100 clients across CIFAR-10, CIFAR-100, and ImageNet-100 with VGG11, MobileNet, and ResNet-18 under sharding and LDA partitions.Smaller shard counts and Dirichlet α values increase heterogeneity; each communication round samples 10% of clients over 320 rounds.Source: §3.5.1 · p. 50
  • 3.5.1 Experimental Setup: Learning rates and local epochs were selected through grid searches, with the learning rate decayed by 0.1 at communication rounds 160 and 240.The experiments cover CIFAR-10, CIFAR-100, and ImageNet-100.Source: Implementation details · p. 50

3.5.2 Global Federated Learning Results

  • 3.5.2 Global Federated Learning Results: FedDr+ consistently outperformed all baselines in highly heterogeneous settings, achieving a 3.15% improvement.Table 3.3 reports mean and standard deviation accuracy across three seeds; SCAFFOLD failed under LDA non-IID with α = 0.1.Source: Table 3.3 · p. 51, §3.5.2 · p. 51

3.5.3 Elapsed Time Results

  • 3.5.3 Elapsed Time Results: FedDr+ required slightly more time per round than most algorithms but less than FedSOL and MOON.Using a global model during local updates generally increases computation costs and elapsed time.Source: §3.5.3 · p. 51, Table 3.4 · p. 52

3.5.4 Sensitivity Analysis

  • 3.5.4 Sensitivity Analysis: FedDr+ remained best or highly competitive across client sampling ratios and local-epoch settings, with three local epochs optimal for all baselines.Too few local epochs cause underfitting, whereas too many cause client drift; β values in {0.3, 0.5, 0.7, 0.9} performed consistently well, unlike β ∈ {0, 1}.Source: §3.5.4 · p. 52

3.5.5 Personalized Federated Learning Results

  • 3.5.5 Personalized Federated Learning Results: FedDr+ FT consistently outperformed both two-step and one-step personalized federated learning baselines across all settings.FedDr+ FT fine-tunes the FedDr+ global model using LDr+ in a two-step approach.Source: §3.5.5 · pp. 52–53, Table 3.5 · p. 53

3.6.1 Notations

  • 3.6.1 Notations: The section introduces the key notations used throughout the paper and summarizes them in Table 3.6.The notation table provides the paper-wide reference for these symbols.Source: Table 3.6 · p. 54, §3.6.1 · p. 54

3.6.2 Pseudo Code of FedDr+ and FedDr+ FT

  • 3.6.2 Pseudo Code of FedDr+ and FedDr+ FT: The algorithms are presented as two-stage procedures for global and personalized federated learning, using an ETF classifier and broadcast initialization.The pseudocode specifies total rounds, local epochs, client datasets, and server broadcasting of V.Source: §3.6.2 · p. 55

10 Server Aggregation: θg · 11 GFL output: Θg R = (θg

The supplied algorithm fragment shows an indexed loop over i = 1, . . . , N involving θg, followed by local-step and batch loops.

10 Server Aggregation: θg

  • The procedure iterates over i = 1, . . . , N while referencing θg.Source: §11 · p. 55

11 GFL output: Θg R = (θg

  • The GFL output subsection includes an assignment involving θg.Source: §11 · p. 55
  • The visible implementation combines an outer i-indexed loop with nested local-step and batch indices.Source: §11 · p. 55
  • Within the procedure, local steps are indexed by e = 1, . . . , E.Source: §11 · p. 55
  • The fragment specifies iteration over local steps before processing batches.Source: §11 · p. 55
  • The local-step loop contains a batch loop indexed by j = 1, . . . , B.Source: §11 · p. 55

16 PFL outputs: {Θi R+1 = (θi

This section examines personalized federated learning under heterogeneous and IID settings, focusing on how training objectives and global initialization affect feature alignment, personalization, and global knowledge preservation. It evaluates FedDr+ and related methods across controlled experimental settings while discussing limitations that motivate alternative alignment approaches.

3.6.3 Preliminaries: Pulling and Pushing Feature Gradients in CE

Physical Meaning of ∇f(x;θ)LDR(x, y; θ)

  • Dot-regression rotates feature vectors toward the target classifier direction, increases their norm, and produces progressively smaller updates as cosine similarity approaches its maximum.The update contains an orthogonal component toward V_y and converges as alignment improves.Source: Physical Meaning of ∇f(x;θ)LDR(x, y; θ) · p. 58

3.6.5 NTK Perspective: Why Dot-Regression in FL Fails on Unobserved Classes

  • The NTK analysis assumes sufficiently wide networks, with kernel weights determined by inputs and initialization; finite-width effects can cause deviations from ideal behavior.In the infinite-width regime, the kernel remains constant during training, while finite-width effects may introduce small variations.Source: §3.6.5 · p. 59, Gradient Flow in the NTK Regime · pp. 59–60

Limitations of Dot-Regression loss in FL under the NTK Regime

Gradient Flow in the NTK Regime

Cross-Entropy Loss and Feature-Classifier Alignment

  • Unlike dot-regression, cross-entropy continues producing meaningful updates for absent classes through softmax-weighted classifier directions and preserves feature-classifier alignment across classes.The refined softmax probabilities after aggregation further strengthen alignment, whereas dot-regression struggles without feature distillation.Source: Cross-Entropy Loss and… · p. 61

3.6.6 Experimental Setup

Non-IID Partition Strategies

  • The experiments distribute class-balanced datasets among 100 clients using sharding and LDA, with fewer shards or smaller α producing stronger client heterogeneity.Sharding gives each client disjoint data with at most s classes, whereas LDA allocates class proportions using a Dirichlet concentration parameter α.Source: Non-IID Partition Strategies · pp. 62–63

Rationale for Varying Initial Learning Rate (η)

Grid Search Process and Results

  • Grid searches find 10 local epochs optimal for every algorithm on CIFAR-10, 3 for every algorithm on CIFAR-100, and 5 fixed for ImageNet-100.Optimal learning rates vary by algorithm, and the selected settings also perform well in less heterogeneous settings.Source: Grid Search Process and Results · pp. 63–64

3.6.8 Synergy Effect Details

  • Ignoring differing loss scales while setting β = 0.9 makes several algorithm–regularizer synergies significantly inferior.Optimal β values are selected through systematic tuning and reported alongside synergy results.Source: Table 3.13 · p. 65, §3.6.8 · p. 65

3.6.9 Personalized Federated Learning Results

  • Across all heterogeneous settings, LDR fine-tuning consistently outperforms LDr+, while FedDr+ provides a stronger fixed fine-tuning initialization than dot-regression, supporting two-step personalization from well-initialized global models.The study evaluates four combinations of dot-regression or FedDr+ global training with LDR or LDr+ fine-tuning on MobileNet/CIFAR-100 under sharding and LDA heterogeneity.Source: Table 3.14 · p. 66, §3.6.9 · p. 66

3.6.10 IID Data Performance

  • Under IID CIFAR-100 data, Dot-regression and FedDr+ achieve the highest global accuracy, significantly outperforming FedAvg and FedBABU, with nearly identical performance.The comparison uses 100 clients and mean±standard deviation across five seeds.Source: Table 3.15 · p. 66, §3.6.10 · pp. 66–67
  • With IID data, FedDr+ provides no additional forgetting-mitigation benefit, while Dot-regression and FedDr+ both improve local alignment across all classes.The absence of unobserved classes in IID data explains why feature distillation adds little benefit.Source: §3.6.10 · p. 67

3.6.11 Performance in Stochastic Client Data Settings

  • In stochastic client settings, FedDr+ consistently outperforms CE and Dot-regression in global accuracy while maintaining stronger alignment for observed, unobserved, and erased classes.During final convergence, FedDr+ surpasses CE on unobserved and erased-class alignment, supporting global knowledge preservation under changing class distributions.Source: §3.6.11 · p. 67, Table 3.16 · p. 67

3.6.12 Scaling to Larger Numbers of Clients and Training Rounds

  • FedDr+ consistently outperforms FedAvg and FedBABU across client-count settings with 1,000 communication rounds, demonstrating robust scalability in large-scale FL.The comparison covers 100, 200, 500, and 1,000 clients, with results averaged over three independent seeds.Source: §3.6.12 · p. 68, Table 3.17 · p. 68

3.7 Limitations

  • FedDr+ relies on dot-regression, and further work is needed to develop more principled alignment methods that improve robustness across observed and unobserved classes.Dot-regression alone is less effective at preserving unobserved-class alignment, motivating alternative local-alignment strategies and theoretical investigation.Source: §3.7 · p. 69

3.8 Conclusion

  • FedDr+ achieves top global and personalized FL performance under non-IID data by combining dot-regression with feature distillation that preserves global class knowledge.Feature distillation regularizes local features toward global features, mitigating forgetting of unobserved classes during local training.Source: §3.8 · p. 69

Are Multiple Global Models Necessary in Federated Personalized Reward Model Learning?

This section examines whether multiple global models are necessary for personalized federated reward modeling under heterogeneous preferences by comparing single- and multi-global initialization across distributions and adaptation settings.

4.2.2 Server-Side Multi-Model Designs in Federated Learning

  • The comparison targets the common intuition that clustering clients into multiple global models should provide better personalization starting points than one shared model.Prior multi-global designs cluster clients by preference, task, or domain, but their effect on client-side personalized reward models had not been systematically examined.Source: Are Multiple Global Models Necessary… · p. 70, §4.2.2 · p. 72

4.2.1 Reward Modeling Under Preference Heterogeneity

  • The study addresses heterogeneous, decentralized preference data by training reward models with FL, then locally fine-tuning an assigned global model into each client’s personalized reward model.The fixed two-stage protocol separates server-side federated training from client-side personalization.Source: Are Multiple Global Models Necessary… · p. 70, §4.2.1 · pp. 71–72

Are Multiple Global Models Necessary in Federated Personalized Reward Model Learning?

  • A single FL-trained reward model consistently outperforms Hard and Soft multi-global designs for personalized performance and also exceeds a centrally trained model.The protocol compares these initializations after client-side fine-tuning across real-world and synthetic preference distributions.Source: Are Multiple Global Models Necessary… · p. 70

4.1 Introduction

  • The findings suggest that concentrating conflicting preferences during FL aggregation is more favorable than mixing them in every centralized batch.Averaging client models trained on preference-derived local data provides a strong personalization initialization despite preference heterogeneity.Source: Are Multiple Global Models Necessary… · p. 70, §4.1 · p. 71

4.3 Preliminaries

  • The framework keeps pairwise preference data decentralized while a coordinating server aggregates client updates to train global reward models before local adaptation.The preliminaries describe collaborative training followed by local fine-tuning, with communication proceeding over sampled clients and repeated rounds.Source: §4.1 · p. 70, §4.3 · p. 72

4.3.1 FL Procedure

  • In each FL round, the server broadcasts global parameters to sampled clients, which perform local training iterations before their updates are aggregated.This procedure is defined for pairwise preference data distributed across M clients over R communication rounds.Source: §4.3.1 · p. 72
  • The FL procedure maintains either one global parameter vector or K global models, then uses the resulting initialization for separate client-side personalization.In the multi-global design, each model is trained independently over its associated client subset before clients fine-tune locally.Source: §4.3.1 · p. 72

4.3.2 Datasets and Preference Heterogeneity Design

  • Experiments use a real annotator-preference dataset and a synthetic dataset with controlled style heterogeneity across four equally sized client groups.The real dataset contains 34 clients and 144,502 samples; the synthetic setup contains 40 clients divided across four preference-aligned clusters.Source: §4.3.2 · p. 73

4.3.3 Implementation Details

  • Experiments run 200 communication rounds, sampling five clients per round for exactly 30 local training iterations, using Qwen2-0.5B with LoRA-based parameter-efficient tuning.For multiple global models with dynamic client–model assignment, assignments refresh every T = 20 rounds.Source: §4.3.3 · p. 73

4.4 Empirical Motivation: Does Increasing Global Models Help?

  • The empirical motivation directly tests whether increasing the number of global models improves initialization for downstream personalization.The Hard design dynamically assigns each client to the global model with the lowest validation loss and aggregates updates within clusters.Source: §4.4 · pp. 73–74

4.4.1 Single Beats Multi-Global FL

  • Single consistently achieves higher personalized accuracy than Hard for K ∈{3, 4, 5} across fine-tuning budgets τ > 0 on both datasets.On the synthetic dataset, Hard initially performs better, but Single overtakes it once local adaptation is allowed.Source: §4.4.1 · p. 75

4.4.2 Why More Global Models Hurt Personalization?

  • On the synthetic dataset, Hard performs best, while merging different groups across fewer clusters reduces personalization accuracy.Source: §4.4.2 · p. 75

4.5 Why Can a Single Global Model Be Enough?

  • Across real-world and synthetic datasets, Single yields the strongest personalized reward models after fine-tuning among the broader initialization strategies evaluated.The comparison includes Single, Hard, Hard-Oracle, Soft, Base, and Centralized.Source: §4.5 · p. 75, Figure 4.3 · p. 76

4.5.1 Within FL: Single vs. Multi-Global

  • Single achieves the highest mean personalized accuracy across all fine-tuning budgets, outperforming Hard, Soft, and Hard-Oracle on both datasets.On the synthetic dataset, Hard-Oracle is strongest at τ = 0, but Single overtakes every multi-global method after only a small amount of local fine-tuning; on the real dataset, Single remains strictly best.Source: §4.5.1 · p. 76

4.5.2 Preference Conflicts: Centralized Batches vs. FL Aggregation

  • Single consistently outperforms Centralized across fine-tuning budgets, while Centralized can underperform the random-start Base baseline on the synthetic dataset.The results suggest that client-side adaptation before aggregation avoids inconsistent supervision from mixing opposing preferences within centralized batches.Source: §4.5.2 · pp. 76–77
  • Under matched compute, Single increasingly outperforms Centralized after 80 fine-tuning updates as local steps increase, showing that sufficient client-side adaptation improves initialization.The sensitivity analysis varies τ ∈{1, 5, 15, 30} and uses default versus 10× smaller learning rates; both methods begin near random accuracy before personalization.Source: §4.5.2 · p. 77, Figure 4.4 · p. 77

4.6 Algorithmic Details of Global Initialization Topologies for Federated Personalized Reward Models

  • The empirical study compares Single, hard- and soft-clustered multi-global models, and Hard-Oracle, differing in how global models are constructed and updated before local personalization.All methods first learn global initializations through federated training and then fine-tune locally; Hard is the main multi-global design, while Hard-Balance is mainly reported in Section 4.7.Source: §4.6 · p. 78

4.6.1 Single Global Model

  • Federated training uses LoRA-style parameter-efficient fine-tuning with a frozen shared base language model, transmitting and aggregating only low-rank adapters.Thus, communication and memory overhead are dominated by the small LoRA parameters, while the Single topology trains one shared global reward model before final personalization.Source: §4.6 · p. 78, §4.6.1 · p. 78

17 Phase 1: Federated training of a single global model

Phase 1 trains a single global model through repeated federated rounds: clients start from the previous global model, perform local updates, and the server applies FedAvg aggregation.

Phase 1: Federated training of a single global model

  • Across rounds r = 1, 2, . . . , R, the federated training procedure repeats its global-model update cycle.Source: §17 · p. 78
  • In each round, client m starts from θ(r−1) and runs τ local update steps on Dm.Source: §17 · p. 78
  • The server aggregates client updates using FedAvg.Source: §17 · p. 78

23 Phase 2: Final personalization from the global initialization

The methods compare single-global and hard-clustered multi-global initialization for personalized reward modeling. Single-global reuses the final global model for local adaptation, while Hard alternates validation-based client assignment with clustered federated updates before personalization.

23 Phase 2: Final personalization from the global initialization

  • The Single design reuses one final global reward model as each client’s initialization for independent local personalization.It trains the shared model federatively, then performs additional client-specific updates from θ(R).Source: §23 · p. 79

4.6.2 Hard-Clustered Multi-Global Models (Hard / Hard-Balance)

  • Hard maintains K cluster-specific global models, periodically assigns clients to the lowest-validation-loss model, and updates each cluster through clustered federated learning [94].The design warms up models independently, alternates E-step reassignment with T rounds of cluster-specific M-step updates, and includes Hard-Balance as a size-control variant.Source: §4.6.2 · p. 79

28 Phase 1: Warm-up (independent FedAvg for each model)

Phase 1 warms up each of K models independently using FedAvg over T communication rounds, with client subsets sampled for parallel local updates.

28 Phase 1: Warm-up (independent FedAvg for each model)

  • Phase 1 runs an independent warm-up loop for each model k = 1, 2, . . . , K.Source: §28 · p. 80
  • For every model, training proceeds across rounds t = 1, 2, . . . , T.Source: §28 · p. 80
  • Each round samples a client subset S_t,k ⊆ [M], whose clients perform the FedAvg update for model k in parallel.Source: §28 · p. 80

34 Phase 2: Dynamic client-driven hard assignment

Phase 2 dynamically reassigns clients according to validation loss and can optionally balance assignments before cluster-wise training. The procedure then samples globally and aggregates models with residuals from previous models.

Dynamic client-driven hard assignment

  • Clients are dynamically reassigned by validation loss during each round.The reassignment step updates client-to-model assignments in Phase 2.Source: §34 · p. 80
  • Hard-Balance optionally moves a client from the largest assignment group to the smallest when its second-best model matches the target group.This balancing step is explicitly optional and is evaluated empirically in Section 4.7.Source: §34 · p. 80
  • After assignment, the method performs single global sampling and cluster-wise training with weighted aggregation using residuals from previous models.Nonempty client subsets define disjoint cluster partitions for training.Source: §34 · p. 80

49 Phase 3: Final personalization (common to Hard/ Hard-Balance)

In Phase 3, both Hard and Hard-Balance variants become personalized federated learning models through validation-based global-model assignment followed by client-side fine-tuning. The resulting personalized models are directly comparable to the single-global baseline.

49 Phase 3: Final personalization (common to Hard/ Hard-Balance)

  • Each client selects its best global model by validation loss and locally fine-tunes it on D_m to produce a personalized model.This final personalization procedure is shared by both Hard and Hard-Balance variants.Source: §49 · p. 80
  • The resulting personalized models are directly comparable to the single-global baseline.Source: §49 · p. 80

4.6.3 Soft-Fusion Multi-Global Models

  • Soft-Fusion is described as federated training with soft-clustered global models and final personalization.Its algorithm takes the number of global models, total rounds, reassignment period, local update steps, and client set as inputs.Source: §4.6.3 · p. 81

55 Step 0: Initial soft assignments via validation accuracy · 57 Step 1: Federated multi-global training

The procedure first initializes soft assignments via validation accuracy, then performs federated multi-global training with local updates, expert-wise server aggregation, and periodic soft-assignment updates.

57 Step 1: Federated multi-global training

  • Step 1 iterates federated training rounds indexed by r = 1, 2, . . . , R.Source: §57 · p. 81
  • Each round includes local training at clients before server-side processing.Source: §57 · p. 81
  • The server performs expert-wise aggregation during federated multi-global training.Source: §57 · p. 81
  • The procedure loops over experts indexed by k = 1, 2, . . . , K.Source: §57 · p. 81

55 Step 0: Initial soft assignments via validation accuracy

  • Step 0 initializes soft assignments using validation accuracy.Source: §57 · p. 81

57 Step 1: Federated multi-global training

  • Soft-assignment weights are updated every T rounds and again at the final round r = R.Source: §57 · p. 81

68 Step 2: Final personalization phase

The final personalization phase uses soft-weighted multi-global initialization while preserving single-model-per-client communication, then converts fused models into personalized models through local fine-tuning. An oracle hard-clustered variant fixes behavior-based client assignments before training to isolate topology effects from dynamic clustering.

68 Step 2: Final personalization phase

  • Soft-Fusion maintains client-specific weights over K global models, updates experts by weighted aggregation, and produces personalized models through local fine-tuning while sending only one model per client each round.Weights are refreshed every T rounds, and the per-round communication cost matches the single-global baseline.Source: §68 · pp. 81–82
  • Because standard LoRA initialization makes all K models functionally identical, Soft-Fusion would collapse to nearly uniform weights, so mutually diverse low-rank perturbations are used instead.The perturbations keep initial global models slightly but meaningfully different and preserve non-uniform soft assignments from the beginning.Source: §68 · p. 82

4.6.4 Hard-Oracle: Oracle Hard-Clustered Multi-Global GFL

  • Hard-Oracle fixes client cluster assignments before federated training, enabling evaluation of hard-clustered multi-global topologies without confounding effects from dynamic reassignment.Oracle clusters are identified from behavioral similarity of client-specific local-only models and their predictions across test distributions.Source: §4.6.4 · p. 82

82 Phase 1: Oracle clustering from local-only models

Phase 1 builds oracle client clusters from local-only models by constructing an MCC-based distance matrix and applying K-medoids, then normalizes the round budget and initializes cluster-global models for subsequent training.

Phase 1: Oracle clustering from local-only models

  • Local-only models are trained first, providing the basis for constructing the clustering inputs.Source: §82 · p. 83
  • The method constructs a distance matrix by comparing model behavior across all test sets and defining D(i, v) = 1 − S(i, v), where S is MCC similarity.Source: §82 · p. 83
  • K-medoids is run on the distance matrix to obtain oracle hard clusters.Source: §82 · p. 83
  • The procedure matches each client’s overall training effort to Single GFL, initializes one global model per cluster, and keeps the oracle clusters fixed during subsequent cluster-wise GFL.Source: §82 · p. 83

96 Phase 3: Final personalization · Chapter 5.

The dissertation develops federated personalization strategies that stabilize global representations, preserve transferable knowledge, and test whether multiple global initializations improve local adaptation. Its final findings show that a single federated global model, locally fine-tuned, achieves the strongest personalized performance under preference heterogeneity, making additional global models unnecessary.

96 Phase 3: Final personalization

  • Hard-Oracle assigns each client one fixed cluster-global model and locally fine-tunes it without validation-based model selection, while matching the Single baseline’s overall training effort.This creates a reference point for evaluating multi-global architectures in personalized federated reward modeling.Source: §96 · p. 84

4.6.5 LoRA Initialization for Soft-Fusion

  • Standard LoRA initialization makes all K global models functionally identical with uniform soft weights, collapsing Soft-Fusion to single-global behavior.Orthogonal, norm-controlled low-rank perturbations instead create diverse initial models and non-uniform soft weights while keeping updates near the base model; experiments use ε_layer = 0.005 and jitter = 0.01.Source: §4.6.5 · pp. 84–85

4.7.1 Within FL: Single vs. Hard-Balance

  • Single achieves higher personalized accuracy than every Hard-Balance variant for K ∈ {3, 4, 5}, although K = 4 best matches the four equally sized synthetic preference groups.K = 4 assigns each group to its own cluster, whereas K = 3 merges groups and K = 5 splits a group, degrading personalization as K deviates from the true group count.Source: §4.7.1 · pp. 85–86, Figure 4.5 · p. 85

4.7.2 Single Remains Best Under Group-Size Imbalance

  • Single still achieves the highest mean personalized accuracy across fine-tuning budgets under imbalanced synthetic groups, despite hard assignments becoming inconsistent with group identity.The imbalance setting uses |G1| = |G4| = 5 and |G2| = |G3| = 15; Figure 4.6a reports the accuracy comparison and Figure 4.6b shows the assignments.Source: §4.7.2 · p. 86, Figure 4.6 · p. 86

4.8 Conclusion and Future Work

  • The reward-modeling conclusion is that local fine-tuning adapts a shared federated initialization effectively, so multi-global initializations are not required under preference heterogeneity.Future work may investigate single-global objectives and regularizers for stronger personalization under data heterogeneity.Source: §4.8 · p. 86

Chapter 5.

  • Across the dissertation, effective personalization depends on global training that stabilizes feature vectors and preserves transferable knowledge under heterogeneous data.FedFN reduces local-global feature-norm mismatch, while FedDr+ combines dot-regression with global feature distillation to improve local alignment without forgetting unobserved-class knowledge.Source: Dissertation Conclusion · p. 87
  • A single federated global model followed by local fine-tuning achieves the best personalized performance, outperforming multi-global variants and a centralized pooled-data model.Clients can effectively adapt one shared initialization using their own preference-derived data, so additional global models are unnecessary for strong personalization.Source: Dissertation Conclusion · p. 87

Related papers

Loading 2608.15107v1…