PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Resource Optimization for Split Federated Learning

Wei Wei, Xianhao Chen

arXiv:2608.17849v1cs.LG

TL;DR

SFL resource optimization combines discrete model splitting with resource allocation, creating a difficult mixed-integer problem. This paper develops optimal and approximation-based algorithms, achieving efficient solutions and the optimal energy–latency tradeoff.

  • Problem

    Jointly optimizing SFL model partitioning and resource allocation is challenging because heterogeneous computing and communication factors are coupled.

  • Method

    The framework jointly optimizes model splitting, GPU frequency scaling, and transmission power using an optimal polynomial-time algorithm and a two-dimensional master-problem approximation scheme.

  • Results

    The proposed methods achieve optimal or high-quality approximate solutions efficiently, with OC+OGP outperforming baselines in cost and running time.

  • Takeaways & Limitations

    The framework provides an efficient approach for resource optimization in resource-constrained SFL edge systems.

Abstract

from arXiv · show

Split federated learning (SFL) has emerged as a powerful paradigm for model training at the edge. However, SFL inherently involves discrete decision variables for model splitting and resource allocation, resulting in a challenging mixed-integer problem. Consequently, prior optimization schemes for SFL are either \textit{heuristic} or \textit{computationally inefficient}, which cannot handle large-scale user populations. To address this limitation, this work establishes an efficient optimization framework for SFL under resource-constrained networks. Our framework jointly optimizes model splitting and resource allocation to minimize training cost, which is defined as the weighted sum of latency and energy costs. We first study the model splitting problem and develop a polynomial-time algorithm that achieves the global optimum. Then, we extend the approach to the joint model splitting and resource allocation problem. In this case, we formulate it as a two-dimensional master problem and develop an efficient approximation method with a $(1+ε)$-approximation guarantee. Extensive experiments show that the proposed approach provides efficient solutions to strike the optimal energy--latency tradeoff.

I. INTRODUCTION … B. Stages of SFL

The paper develops resource optimization for split federated learning by jointly addressing model partitioning and heterogeneous resource control. It models SFL as a five-stage computation-and-communication pipeline and provides optimal or approximate optimization methods with provable guarantees.

  • I. INTRODUCTION: SFL resource optimization must coordinate model splitting, computation, and communication under heterogeneous edge-device constraints.Slow clients can become stragglers, while GPU frequencies and transmission powers shape the energy–latency tradeoff.
  • I. INTRODUCTION: The framework jointly optimizes model splitting, GPU frequency scaling, and transmission power control.The joint problem is formulated as a two-dimensional master problem with a (1 + ϵ)-approximation guarantee.
  • I. INTRODUCTION: Model-splitting optimization is solved through an optimal polynomial-time approach.The paper identifies polynomial-time optimal model splitting as a central unresolved challenge in prior SFL work.
  • II. RELATED WORK: Prior resource-control studies improve deep-learning efficiency but generally assume fixed partitioning or do not address heterogeneous edge-device partitioning.SFL studies have separately considered workload distribution, model splitting, communication/computing allocation, and hierarchical split designs.
  • A. SFL Architecture: The SFL architecture assigns each client an initial sub-model through cut layer Lc,i and the edge server the remaining layers.The federated server periodically aggregates model parameters after local SFL training rounds.
  • B. Stages of SFL: Each SFL round uses five stages: client-side FP, activation uplink, server-side FP and BP, gradient downlink, and client-side BP.Federated aggregation occurs on a slower timescale and is treated as a constant offset after every I local training rounds.
  • B. Stages of SFL: Client-side FP computes activations through layer Lc,i, which are transmitted to the edge server over the uplink.The stage depends on client GPU frequency, computational cost, allocated uplink bandwidth, transmit power, and channel conditions.
  • B. Stages of SFL: The edge server performs the remaining FP and BP, sends gradients downlink, and the client completes BP and updates its local sub-model.Server and client stage times depend on their GPU frequencies, FLOPs-per-cycle parameters, bandwidths, transmit powers, and channel fading.

C. Per-round Latency · D. Energy Model

The per-round SFL latency model accounts for parallel client-side execution and sequential server-side processing across the five-stage pipeline. The energy model similarly quantifies each client stage and aggregates computing and communication energy for a complete SFL round.

  • C. Per-round Latency: Client-side forward propagation and uplink latency are determined by the slowest client under parallel execution.Clients execute in parallel within each SFL round.
  • C. Per-round Latency: The edge server processes server-side forward and backward propagation sequentially for all clients.The server maintains a shared server-side sub-model.
  • C. Per-round Latency: Server-side workloads therefore contribute additively to the aggregate per-round latency.The aggregate latency term reflects sequential processing across clients.
  • C. Per-round Latency: The latency formulation includes stage-specific client and server processing parameters together with transmission power and cut-layer settings.The supplied formulation fragments reference fS,j, PTX, and Lc,i.
  • D. Energy Model: Per-round energy consumption combines computing and communication energy across the five-stage SFL pipeline.Energy is defined for each client i ∈ N and stage j ∈ {1, . . . , 5} under cut layer Lc,i.
  • D. Energy Model: The energy model assigns stage-level consumption according to client and server operating parameters, including frequencies and transmission power.The supplied energy expression references fS,j and PTX under the cut-layer configuration.
  • D. Energy Model: Chip-dependent coefficients Gi and GS determine the energy model for each client and the edge server.The coefficients are measured in Watt/(cycle/s)^3 and depend on chip architecture.
  • D. Energy Model: The total energy consumption of one SFL round aggregates the stage-level energy contributions.The paper states that the total round energy follows the per-stage definitions.

E. Control Input of Each SFL Stage · IV. PROBLEM FORMULATION

The paper models each SFL stage through controllable client and server computing and transmission resources, with cut-layer choices shaping service time. It then jointly optimizes these controls and cut layers to balance per-round energy consumption and training latency.

  • E. Control Input of Each SFL Stage: Control inputs u_i,j(t) govern the evolution rate f_i,j and determine each stage’s service time s_i,j.The control variables represent adjustable resource decisions for SFL-stage operation.
  • E. Control Input of Each SFL Stage: The controls include client and edge-server GPU frequencies plus uplink and downlink transmit powers.f_i,j(t) and f_S,j(t) are instantaneous GPU frequencies, while P_TX_i,j(t) and P_TX_S,j(t) denote transmit powers.
  • E. Control Input of Each SFL Stage: The cut-layer index L_c,i determines which DNN layers execute locally versus at the edge, influencing the mapping from controls to service time.Thus, model partitioning affects how resource settings translate into stage duration.
  • E. Control Input of Each SFL Stage: Bounded GPU frequencies and transmit powers constrain feasible processing and transmission rates, implying lower and upper bounds on service times.These resource limits apply to both clients and the edge server.
  • IV. PROBLEM FORMULATION: The optimization jointly selects cut-layer decisions {L_c,i} and control variables u_i,j = {f_i,j, P_TX_i,j, f_S,j, P_TX_S,j}.The stated control vector combines client and server computing frequencies with transmission powers.
  • IV. PROBLEM FORMULATION: The objective balances per-round energy consumption and training latency through an energy–latency tradeoff objective.The formulation is posed as an optimal hybrid control problem minimizing E_r(u_i,j, s_i,j, L_c,i).
  • IV. PROBLEM FORMULATION: λ > 0 controls the tradeoff between energy costs and training latency.Changing λ changes the relative weighting assigned to the two objective components.

A. Problem Transformation

The problem is transformed by lifting stage variables and introducing epigraph variables, while binary cut-layer choices couple model splitting with service-time constraints. The resulting weighted energy–latency formulation restricts admissible cut layers, but may recover only supported Pareto-efficient points because the attainable objective set is generally non-convex.

  • Latency reformulation: Lifting {s_i,j} as high-level decision variables rewrites per-round latency as a function of the stage durations.Each s_i,j is determined by (u_i,j, L_c,i) through the stage models.
  • Epigraph reformulation: Epigraph variables T1 and T2 replace max terms, yielding the objective λ(T1 + T2) + λΣ_i s_i,3 under the stated constraints.At optimality, T1 = max_i(s_i,1 + s_i,2) and T2 = max_i(s_i,4 + s_i,5).
  • Cut-layer selection: Binary indicators y_i,ℓ select exactly one cut layer per client and constrain each stage duration to the service-time range associated with that layer.The indicator equals 1 when client i chooses cut layer ℓ and 0 otherwise.
  • Design-space restriction: The cut-layer set is restricted to L_max = {1, 2, . . . , Lmax}, focusing optimization on the system-level energy–latency tradeoff rather than explicit test-accuracy optimization.The restriction is motivated by prior observations that shallower split points generally converge faster, while overly deep split points may degrade learning performance.
  • Pareto analysis: Varying λ need not recover all Pareto-efficient solutions because binary cut-layer variables and generally non-convex stage-energy terms make the attainable objective set non-convex.Thus, the weighted-sum formulation may recover only supported Pareto-efficient points.

V. SPECIAL CASE: MODEL-SPLITTING OPTIMIZATION · A. Model-splitting Problem Formulation

The paper treats model splitting as a special case of SFL optimization and transforms its exponential integer search into an exactly solvable polynomial-time formulation. With fixed feasible resource controls, cut-layer choices become binary selections whose resulting discrete problem is solved by a polynomial-time algorithm.

  • V. SPECIAL CASE: MODEL-SPLITTING OPTIMIZATION: Model-splitting optimization is identified as an important special case of the original SFL problem P1.
  • V. SPECIAL CASE: MODEL-SPLITTING OPTIMIZATION: Auxiliary variables divide the joint optimization into client subproblems, enabling exact solution in polynomial time despite an exponential integer solution space.
  • A. Model-splitting Problem Formulation: The model-splitting problem is derived from the original energy-latency objective while fixing per-stage resource controls at prescribed feasible constants.
  • A. Model-splitting Problem Formulation: These fixed controls include client and server transmit powers and client and server GPU frequencies, making each candidate cut layer’s stage time and energy deterministic.
  • A. Model-splitting Problem Formulation: Binary variables y_i,ℓ indicate whether client i selects cut layer ℓ, and client stage time is rewritten through affine selections.
  • A. Model-splitting Problem Formulation: Problem P3 is a 0−1 discrete optimization that searches for the best cut layer across clients, with exponentially many cut-layer configurations in general.
  • A. Model-splitting Problem Formulation: The subsequent development addresses P3’s complexity by providing a polynomial-time algorithm that solves the formulation exactly.

B. Solution Approach

The solution transforms the model-splitting problem into a finite threshold search, where each client selects its minimum-cost feasible cut layer. Algorithm 1 uses a plane sweep with bound-based pruning and is proven globally optimal, with running time O(N^2L^2 + NL log(NL)).

  • Threshold-based model splitting: For each fixed threshold pair (T1, T2), client i selects the minimum-cost cut layer from its feasible set Fi(T1, T2).The candidate threshold sets have cardinality at most N(L −1).
  • Plane sweep and pruning: Algorithm 1 enumerates candidate threshold pairs, sweeps T1 for each fixed T2, updates clients’ best feasible cut layers, and prunes scans that cannot improve the objective.The pruning rule terminates the T1 scan when the threshold penalty exceeds the maximum possible objective decrease.
  • Optimality guarantee: Algorithm 1 obtains the globally optimal solution to Problem P3.Enumerating all candidate T2 values ensures that the global minimizer is not missed.
  • Complexity: The overall running time of Algorithm 1 is O(N^2L^2 + NL log(NL)).Candidate-set construction and sorting cost O(NL log(NL)), while the scan updates cost O(NL) per fixed T2.

VI. JOINT OPTIMIZATION OF MODEL SPLITTING AND RESOURCE ALLOCATION

The joint model-splitting and resource-allocation problem is reformulated as a two-dimensional master problem and solved approximately using a polynomial-time uniform grid search.

  • Two-Dimensional Master Problem: The general joint optimization is exactly reformulated as a two-dimensional master problem over synchronization variables (T1, T2).The reformulation preserves Problem P2 exactly.
  • Uniform Grid Approximation: Because the master problem is generally nonconvex, the method approximates it through uniform grid search over the (T1, T2) domain.The grid search addresses the nonconvexity of the resulting master problem.
  • Polynomial-Time Algorithm: The resulting approximation algorithm runs in polynomial time for joint model splitting and resource allocation.The method is presented as a polynomial-time approximation approach for the general joint optimization problem.

A. Two-Dimensional Bottleneck Projection

Fixing the two synchronization variables T1 and T2 decouples client optimizations, allowing the mixed-integer problem to be projected onto a two-dimensional master problem. Minimizing this master problem is equivalent to solving the original Problem P2 and yields an optimal solution.

  • Two-Dimensional Bottleneck Projection: Fixing (T1, T2) removes coupling among clients, so each client independently evaluates feasible cut layers and selects the least-cost option.Infeasible cut-layer choices are assigned value +∞.
  • Two-Dimensional Bottleneck Projection: Each client’s fixed-cut-layer objective decomposes into three independent blocks associated with server computation, uplink-side constraints, and downlink-side constraints.The resulting client value function is obtained by minimizing over cut layers.
  • Two-Dimensional Bottleneck Projection: The original mixed-integer Problem P2 is exactly reduced to a master problem over the two synchronization variables T1 and T2.The master objective sums optimal client values and adds the synchronization penalty λ(T1 + T2).
  • Two-Dimensional Bottleneck Projection: Problem P2 is equivalent to the master problem, so solving Problem P5 produces an optimal solution to the original problem.This equivalence follows from client separability after fixing (T1, T2), minimizing each client value over cut layers, and summing the resulting contributions.

B. Grid Approximation of the Master Problem

The two-dimensional, generally nonconvex master problem is approximated by uniform grid search over a bounded rectangle containing a global optimizer. The method provides a (1+ε)-approximation guarantee and runs in polynomial time under bounded search ranges.

  • Grid approximation: The master objective is generally nonconvex because each client value function minimizes over discrete cut layers.The reformulated master problem remains only two-dimensional, enabling grid-based approximation.
  • Grid approximation: Uniform grid search evaluates J(T1, T2) at every grid point in a bounded rectangle and returns the best value Jgrid.The search box is chosen to contain at least one global optimizer.
  • Approximation guarantee: Choosing Δ ≤ ε(T 1 + T 2)/2 suffices to guarantee Jgrid ≤ (1 + ε)J∗.The guarantee assumes the search region contains a global optimizer and J∗ ≥ λ(T 1 + T 2), with T 1 + T 2 > 0.

C. Recovering the Optimal Control Inputs

The original hybrid control solution is recovered after solving the high-level problem, using its optimal cut-layer and service-time decisions to solve lower-level control subproblems stage by stage. The resulting policy combines optimal resource allocation with control inputs satisfying system dynamics and boundary conditions.

  • C. Recovering the Optimal Control Inputs: The optimal hybrid control solution is recovered by solving lower-level control subproblems stage by stage after the high-level problem is solved.This procedure maps the high-level solution to the original control problem.
  • C. Recovering the Optimal Control Inputs: The high-level optimizer provides the optimal cut-layer decisions {L∗c,i} and service-time variables {s∗i,j}.These decisions determine the inputs to the lower-level recovery process.
  • C. Recovering the Optimal Control Inputs: The recovered control inputs satisfy the associated system dynamics and boundary conditions, completing the optimal policy.The final policy combines high-level resource-allocation decisions with the recovered low-level control inputs.

VII. SIMULATIONS · A. Simulation Setup

The simulations evaluate the proposed SFL optimization method on standard datasets and models under IID and non-IID settings. Experiments use heterogeneous computation and communication configurations and compare OC + OGP with five benchmark methods.

  • A. Simulation Setup: MNIST and CIFAR-10 evaluate ResNet-50 and VGG-16 under both IID and non-IID data distributions.In the non-IID setting, approximately 70% of each client’s samples come from a primary class.
  • A. Simulation Setup: N = 20 clients are considered unless specified otherwise, with mini-batch size 32 at each client.
  • A. Simulation Setup: λ = 100 balances training latency and energy consumption in the experiments.
  • A. Simulation Setup: Client computation capability is heterogeneous, with n_i uniformly selected from [0.2, 3] × 10^4 FLOPs/cycle, while the edge server uses n_S = 3 × 10^4 FLOPs/cycle.
  • A. Simulation Setup: Client and server computation-energy coefficients are G_i = 1 × 10^-26 W/(cycle/s)^3 and G_S = 2 × 10^-26 W/(cycle/s)^3, respectively.
  • A. Simulation Setup: Computing frequencies are configured within measured graphics-clock ranges for the NVIDIA Jetson Orin Nano client and NVIDIA GeForce RTX 4090 server platforms.The client range starts at f_i^min = 625 MHz, while the server range starts at f_S^min = 3.135 GHz.
  • A. Simulation Setup: Wireless experiments use B_U_i = 200 MHz uplink bandwidth, client and server transmit powers bounded within dBm, and independent Rayleigh fading uplink channels.The average uplink channel power gain is generated within [0.1, 1]×10^-6, while the fixed downlink gain is 1 × 10^-6 and N_0 = 3.98 × 10^-21 W/Hz.
  • A. Simulation Setup: OC + OGP is compared with OC + OG, OC + OP, OC, and two additional benchmarks.The proposed method is named “Optimal Cut and Optimal GPU Frequency Scaling and Power Control.”

B. Performance Comparison … VIII. CONCLUSION

The proposed OC+OGP method consistently achieves the lowest training cost across datasets, architectures, bandwidths, client populations, and extreme energy–latency settings. The paper further establishes polynomial-time optimality for model splitting and formulates the general joint optimization as a two-dimensional master problem.

  • B. Performance Comparison: OC+OGP achieves the lowest cost for a given test accuracy across MNIST and CIFAR-10 with ResNet-50 and VGG-16.Cost generally increases with test accuracy, with larger increases at higher accuracy levels.
  • B. Performance Comparison: Increasing uplink or downlink bandwidth gradually decreases total cost, while OC+OGP remains lowest across all evaluated bandwidth settings.Larger bandwidth improves transmission efficiency and shortens communication time.
  • B. Performance Comparison: Increasing the maximum allowable client-side GPU frequency continuously reduces cost for all compared methods within the tested range.The experiment indicates that enlarging the feasible client-side frequency set benefits cost reduction.
  • B. Performance Comparison: Uncertainty is modeled through Gaussian perturbations of client and server GPU capabilities and uplink and downlink channel power gains.The uncertain parameters are n_i, n_S, |h_i|^2, and |h_S|^2, with different coefficients of variation.
  • C. Running Time Comparison: As the number of clients grows, OC+OGP maintains lower cost than the baselines and OC+OGP and OC require less running time than ESFL and DSQL.OC matches ESFL in cost and consistently outperforms DSQL; the results support OC's polynomial-time optimality.
  • D. Impact of Hyperparameter λ: When λ = 0, OC+OGP achieves the lowest energy, while at λ = ∞ it achieves nearly the same latency as OC+OG, OC+OP, OC, and ESFL and remains below DSQL.Under latency-only optimization, GPU frequencies and transmit powers reach their maximum feasible values.
  • VIII. CONCLUSION: The paper formulates joint model splitting and resource allocation as a hybrid optimization problem capturing computation, communication, and model-splitting decisions.It develops an optimal polynomial-time solution for special model splitting and reformulates the general problem as a two-dimensional master problem.
Loading 2608.17849v1…