When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimation

Wenhao Yuan; Chenchen Lin; Wenhao Hu; Jian Chen; Jinfeng Xu; Shujie Li; Edith Cheuk Han Ngai

Source-linked AI summary

When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimation

Wenhao Yuan, Chenchen Lin, Wenhao Hu, Jian Chen, Jinfeng Xu, Shujie Li, Edith Cheuk Han Ngai

arXiv:2608.15639v1Aug 16, 2026cs.DCcs.AIcs.LG

TL;DR

Heterogeneous clients can make a single static split point insufficient, while repeated probing of candidate depths is costly. FedSGA estimates client-specific shallow sufficiency and harmonizes interfaces across depths, consistently achieving the best performance across evaluated datasets, partitions, and backbones while enabling earlier offloading when appropriate.

Problem
Under client heterogeneity, static split points may not preserve sufficient task-relevant information for every client and leave representation-level interface suitability underexplored.
Method
FedSGA uses private prompt states, semantic alignment, temporal stability, and interface harmonization to estimate client-specific shallow sufficiency without exhaustive depth probing.
Results
FedSGA consistently achieves the best performance across evaluated datasets, data partitions, and ResNet-18 and ViT-B/16 backbones.
Takeaways & Limitations
FedSGA enables client-specific early offloading, reducing unnecessary deeper local computation without uniformly enforcing shallow splitting across heterogeneous clients.
Takeaways & Limitations
Existing split-selection criteria can be unreliable when based only on current representations, while probing multiple depths adds repeated forward computation during training.

Abstract

from arXiv · show

\textit{Split Federated Learning} (SFL) enables distributed model training by splitting networks between the server and clients. However, under client heterogeneity, the conventional static split strategy may be suboptimal because clients can differ in data distributions, adaptation dynamics, and representation learning progress, making a single split point insufficient to accommodate client-specific training states. In this paper, we propose \textsc{FedSGA}, a \textbf{S}ufficiency-\textbf{G}uided \textbf{A}daptive split \textbf{Fed}erated learning framework that addresses this question through client-specific shallow sufficiency estimation. First, we introduce a client-specific adaptation channel based on private prompt tokens, which tracks local adaptation dynamics separately from the shared backbone and provides a lightweight signal for detecting whether client adaptation remains active. To further avoid repeated online probing over multiple candidate depths, we design a shallow sufficiency estimator that combines cross-client semantic alignment, temporal interface stability, and prompt-state variation to estimate whether the shallowest split is already sufficient. Finally, we introduce a split-compatible interface harmonization module that projects activations from different split depths into a shared semantic space, improving the comparability of heterogeneous client interfaces before server-side prediction. Extensive experiments on multiple heterogeneous benchmarks demonstrate the effectiveness of \textsc{FedSGA} in improving model performance compared with state-of-the-art methods while reducing unnecessary client-side computation.

1 Introduction

The introduction argues that static or resource-driven split points are inadequate for heterogeneous Split Federated Learning because clients differ in data distributions, learning behaviors, and adaptation states. It presents FedSGA, which estimates client-specific shallow sufficiency using adaptation signals and harmonizes heterogeneous interfaces for server-side prediction.

Motivation: Split Federated Learning partitions networks between clients and servers to reduce client computation, but static splits may be suboptimal under heterogeneous client conditions.Clients differ in data distributions, local learning behaviors, and available resources.
Open Challenges: Existing adaptive strategies rely on device or communication criteria, while online probing across candidate depths does not directly establish whether an interface preserves sufficient task-relevant information.Reliable decisions must also account for whether client-side adaptation has stabilized.
FedSGA: FedSGA uses private prompt tokens as client-specific adaptation states to track whether local adaptation remains active.The adaptation channel is designed as a lightweight signal separate from the shared backbone.
FedSGA: Its shallow sufficiency estimator jointly uses cross-client semantic alignment, temporal interface stability, and prompt-state variation without exhaustively probing all candidate split depths.A separate interface harmonization module maps activations from heterogeneous split depths into a shared semantic space before server-side prediction.

2 Related Works

Related work establishes SFL as collaborative model partitioning that reduces client computation through server-side processing, while existing methods primarily target system efficiency and robustness to heterogeneous data. However, fixed split structures and conventional representation alignment do not fully address client-specific split depths and their heterogeneous semantic granularity.

Split Federated Learning: SFL partitions models between clients and servers, exchanging intermediate activations and gradients while reducing clients’ local training burden.This preserves federated optimization while moving part of model computation to the server.
Split Federated Learning: Existing SFL studies mainly pursue system-level improvements, whereas fixed or resource-oriented split strategies leave the representation-level role of split interfaces underexplored.The cited limitation concerns how split interfaces affect representations, not only computation or resource allocation.
Client Heterogeneity: Client heterogeneity in data distributions, domain semantics, and optimization behavior can cause client drift, unstable optimization, and degraded generalization.Existing responses include domain-invariant representations, feature alignment, update regularization, and domain generalization.
Representation Alignment: Federated representation-alignment methods use feature alignment, prototypes, contrastive learning, or knowledge distillation, but adaptive split learning also requires alignment across client-specific semantic granularities.The limitation arises because different split depths induce heterogeneous semantic granularity.

3 Methodology

FedSGA treats split selection under client heterogeneity as client-specific shallow sufficiency estimation. It combines private prompt-based adaptation tracking with semantic alignment and interface stability signals to estimate when the shallowest split is sufficient without repeated online depth probing.

Adaptive split selection: FedSGA formulates adaptive split selection as estimating whether each client’s shallowest interface contains sufficient task-relevant information for server-side training.This avoids requiring every client to compare all candidate split depths online.
Client-specific adaptation channel: Client-specific prompt tokens track local adaptation separately from the shared backbone, whose updates are entangled with cross-client aggregation.Prompt parameters are updated only through the corresponding client’s local objective and are not directly aggregated.
Adaptive split selection: The framework compares the shallow reference with one fixed moderately deep reference, providing a stable and low-cost operational target rather than identifying the globally optimal split each round.Additional local forwarding is unnecessary when the shallow interface preserves sufficient information; otherwise, the client retains a deeper prefix.
Shallow sufficiency estimation: The estimator combines cross-client semantic alignment, shallow-interface temporal stability, and prompt-state variation to predict shallow sufficiency.Semantic alignment alone can be misleading when a client remains in substantial local adaptation.
Shallow sufficiency estimation: A low-capacity monotone model increases sufficiency estimates with prototype alignment and decreases them with representation drift and prompt-state variation.The conservative design requires semantic alignment, temporal stability, and limited residual client-local adaptation before accepting a shallow interface.

1 Initialize 𝒘0 𝑠, {𝒘0

FedSGA harmonizes heterogeneous client interfaces with a depth-aware projection and prototype alignment, while reducing adaptive split overhead through sparse calibration and client-specific early offloading. Its auxiliary computation remains lightweight compared with exhaustive multi-depth probing.

Interface harmonization: A depth-aware projector conditions interface representations on the selected split depth to compensate for systematic semantic shifts across split locations.Pooling first produces compact interface features, while the depth embedding supplies explicit information about the representation’s semantic level.
Interface harmonization: Server-maintained class prototypes provide cross-client semantic anchors that regularize projected representations from different depths and client states.The prototype-based alignment objective preserves consistent class-level geometry during server-side optimization.
Interface harmonization: The harmonization layer improves interface comparability and training stability without fully eliminating mismatches from heterogeneous split depths and adaptation states.It is designed as a minimal stabilization module rather than a complete solution to interface mismatch.
Efficiency: FedSGA replaces repeated exhaustive probing of K candidate depths over T rounds, which costs O(KT), with sparse two-depth calibration costing O(|Tcal|).Calibration compares p1 and pref at selected rounds, then the sufficiency estimator selects the split during ordinary training.
Efficiency: Client-specific early offloading lets clients with sufficient shallow interfaces avoid unnecessary deeper computation without uniformly enforcing the shallowest split.This preserves the distinction between adaptive computation savings and potentially harmful aggressive shallow splitting under heterogeneity.
Efficiency: The auxiliary overhead remains lightweight: prompt descriptors require O(m) operations over m prompt tokens, while server prototype alignment costs O(B|Y|) per client batch.The sufficiency estimator runs once per client in ordinary rounds, and sparse calibration compares only two depths.

4 Numerical Experiments · 4.1 Experiment Setups

The numerical experiments evaluate FedSGA under heterogeneous environments, dissect its components and hyperparameters, and assess computation and adaptive split selection. Experiments use multiple heterogeneous benchmarks and compare against established federated and split-federated learning baselines under standardized training settings.

4 Numerical Experiments: The experiments address five questions spanning overall performance, component contributions, hyperparameter sensitivity, client-side computation, and adaptive split selection.These questions compare FedSGA with state-of-the-art FL and SFL methods and alternative split strategies.
4.1 Experiment Setups: Experiments cover CIFAR-10, CIFAR-100, Tiny-ImageNet, and DomainNet, representing statistical and domain-level heterogeneity.The datasets examine adaptive split learning under different client distribution shifts.
4.1.2 Baselines.: The evaluation compares FedSGA with federated baselines including FedAvg, FedProx, FedBABU, and FedGH.These baselines represent classical federated optimization, proximal stabilization, personalization, and feature-alignment approaches.
4.1.2 Baselines.: Split-federated baselines include SplitFed and MergeSFL, covering standard split aggregation and intermediate-representation merging.SplitFed exchanges intermediate activations and gradients across the split interface, while MergeSFL reduces split-learning overhead by merging representations.
4.1.2 Baselines.: Unless otherwise specified, experiments use N=20 clients, L=5 local epochs, batch size 64, SGD, and full client participation per communication round.Results are reported as averages over multiple runs.

4.2 Main Results and Analysis (RQ1)

FedSGA achieves the best accuracy across datasets, data partitions, and both ResNet-18 and ViT-B/16 backbones. Its advantages are especially pronounced under challenging Non-IID conditions, where client heterogeneity makes static or resource-oriented split strategies less reliable.

Overall accuracy: FedSGA consistently achieves the best performance across all datasets, data partitions, and ResNet-18 and ViT-B/16 backbone architectures.This demonstrates the effectiveness of sufficiency-guided adaptive split selection under heterogeneous client conditions.
Heterogeneous settings: FedSGA shows more pronounced advantages over conventional FL and SFL baselines under Non-IID settings with stronger distributional and representational heterogeneity.The reported challenging cases include Tiny-ImageNet and DomainNet, where heterogeneous semantics and complex visual domains complicate split-interface reliability.
Method analysis: Baselines using static partitions, resource-oriented split decisions, or post hoc representation alignment suffer greater degradation on challenging heterogeneous datasets.FedSGA instead estimates whether each client’s shallow interface is sufficient and harmonizes interfaces across heterogeneous split depths.

4.3 Ablation Study (RQ2) · 4.4 Hyperparameter Sensitivity Analysis (RQ3)

The ablation results show that all three FedSGA components contribute positively, with the full model performing best and sufficiency estimation central to reliable split selection. Hyperparameter analyses identify balanced settings around ρ=0.1 and τ_a=0.5 while maintaining stable performance across datasets and backbones.

4.3 Ablation Study (RQ2): All three modules improve performance across datasets, data partitions, and backbone architectures, while the full model achieves the best results in every case.The evaluated modules are the client-specific adaptation channel, shallow sufficiency estimator, and split-compatible interface harmonization module.
4.3 Ablation Study (RQ2): Reliable adaptive split learning requires both client-local adaptation awareness and split-compatible server-side interface modeling.The ablation findings connect the adaptation channel with server-side harmonization as complementary requirements.
4.3 Ablation Study (RQ2): Removing the shallow sufficiency estimator consistently reduces performance with both ResNet-18 and ViT-B/16, indicating less reliable adaptive split decisions.The estimator is identified as central to split selection because it explicitly assesses shallow-interface sufficiency.
4.3 Ablation Study (RQ2): The client-specific adaptation channel provides stable gains, especially under Non-IID and DomainNet settings with more heterogeneous local adaptation dynamics.These gains support tracking client-local adaptation separately from shared model behavior.
4.4 Hyperparameter Sensitivity Analysis (RQ3): ρ=0.1 generally gives the best performance; smaller values offload earlier, whereas larger values shift computation deeper onto clients.The analysis covers CIFAR-100 and DomainNet with ResNet-18 and ViT-B/16, as illustrated in Figures 2 and 3.
4.4 Hyperparameter Sensitivity Analysis (RQ3): τ_a=0.5 generally gives the best performance; smaller temperatures sharpen prototype alignment, while larger temperatures weaken class-level discrimination.The results indicate that excessive alignment can over-constrain task representations, motivating balanced interface harmonization.
4.4 Hyperparameter Sensitivity Analysis (RQ3): Overall, FedSGA benefits from balanced sufficiency-estimation and interface-harmonization configurations while retaining stable performance across datasets and backbone architectures.This conclusion is reported for the CIFAR-100 and DomainNet analyses using ResNet-18 and ViT-B/16.

4.5 Computational Efficiency Analysis (RQ4)

FedSGA improves the efficiency–performance trade-off over MultiSFL by reducing client-side computation while maintaining higher accuracy. These gains are especially evident on larger, more heterogeneous datasets such as DomainNet.

4.5 Computational Efficiency Analysis (RQ4): FedSGA consistently achieves higher accuracy than MultiSFL while reducing client-side computational cost across all datasets.The comparison is presented in Table 5.
4.5 Computational Efficiency Analysis (RQ4): FedSGA requires fewer activated parameters, lower FLOPs, and shorter training time than MultiSFL.These reductions indicate that adaptive splitting avoids unnecessary deep local computation without sacrificing representation quality.
4.5 Computational Efficiency Analysis (RQ4): On larger and more heterogeneous datasets such as DomainNet, FedSGA reduces computational overhead while maintaining clear accuracy improvements.The observed behavior is consistent with the framework’s sufficiency-guided adaptive split design.

4.6 Split Strategy Comparison (RQ5)

FedSGA outperforms fixed static and random split strategies on CIFAR-100 and DomainNet under both IID and Non-IID settings. Its gains are attributed to principled client-specific split decisions guided by reliable interfaces and prompt-state variation.

Split strategy comparison: FedSGA achieves the highest accuracy on CIFAR-100 and DomainNet under both IID and Non-IID settings, outperforming fixed static and random split strategies.The comparison supports sufficiency-guided split selection over static or random alternatives.
Split strategy comparison: FedSGA’s adaptive split selection is guided by reliable interface estimates rather than unguided variation in split points.Each client offloads only when its selected interface is likely to be reliable.
Split strategy comparison: The results attribute FedSGA’s gains to principled client-specific split decisions informed by prompt-state variation.Prompt-state variation helps determine when a client’s selected interface is sufficiently reliable for offloading.

5 Conclusion

The paper proposes FedSGA, a sufficiency-guided adaptive split federated learning framework for heterogeneous clients. It reframes split selection as client-specific shallow sufficiency estimation rather than using a uniform static split or repeatedly probing candidate depths.

5 Conclusion: FedSGA targets heterogeneous clients with sufficiency-guided adaptive split federated learning.The framework is designed for client-specific split selection.
5 Conclusion: The framework formulates split selection as a client-specific shallow sufficiency estimation problem.This replaces uniform static split points and repeated probing of multiple candidate depths.
5 Conclusion: A private prompt-based adaptation channel captures whether client-local adaptation remains active.Prompt-state variation serves as a lightweight signal for this purpose.

GenAI Usage Disclosure

The authors state that the work is entirely original and contains no content generated by generative AI tools.

The authors declare that the work is entirely original and contains no content generated by generative AI tools.