PaperlayerView exact revision on arXiv

Source-linked AI summary

DART-FL: Burst-Aware Multitask Federated Learning under Dynamic Inference Demand at the Edge

Yiming Xie, Pinrui Yu, Geng Yuan, Xue Lin, Ningfang Mi

arXiv:2608.27713v1cs.LG

TL;DR

Edge devices must serve inference while training models under limited computation and changing task demand. DART-FL jointly allocates inference and training resources, then prioritizes training for demanded tasks using queue-aware scheduling. Experiments show improved burst-period accuracy for high-demand tasks while preserving comparable long-term multitask performance.

  • Problem

    Concurrent inference and training share limited edge computation, while bursty task-specific demand requires resources and training emphasis to adapt across tasks.

  • Method

    DART-FL reserves the minimum inference capacity using backlog and service capacity, distributes remaining training capacity with a queue-aware DPP-inspired scheduler, and maps allocations to dynamic task weights.

  • Results

    DART-FL improves high-demand-task inference accuracy during burst periods while preserving training opportunities for other tasks and long-term multitask performance.

  • Takeaways & Limitations

    Inference demand can guide both the inference-training resource split and the timing of task-level training emphasis in multitask federated learning.

Abstract

from arXiv · show

Edge intelligence systems increasingly require model training and online inference to coexist on resource-constrained devices, while inference demand can vary substantially across tasks over time. This creates two coupled challenges: sufficient computation must be reserved for inference to maintain service-level objectives (SLOs), while the remaining training capacity should adapt to task-specific demand so that frequently requested tasks can improve earlier during training. We propose an SLO-aware, demand-driven multitask federated learning framework (DART-FL) that jointly adapts the inference-training resource split and task-level training emphasis. At each scheduling interval, DART-FL uses the inference backlog and profiled service capacity to determine the minimum resource allocation required for inference. The remaining training capacity is then distributed across tasks using a queue-aware DPP-inspired scheduler, and the resulting task allocations are mapped to dynamic loss weights. This allows tasks experiencing higher inference demand to receive greater training emphasis in earlier communication rounds. Clients train a shared backbone with task-specific heads, and the complete multitask model is aggregated through FedAvg. We evaluate DART-FL using Stanford Cars and Oxford Flowers 102 under both synthetic and real Alibaba trace-derived workloads. Results show that DART-FL dynamically adapts the inference-training resource split to time-varying inference demand and shifts the learning progress of high-demand tasks toward their burst periods, improving model accuracy when those tasks are frequently requested while maintaining comparable long-term multitask performance.

I. INTRODUCTION

DART-FL addresses the tension between concurrent inference and multitask federated training under bursty, task-specific demand. It jointly reserves inference capacity and shifts training emphasis toward currently demanded tasks.

  • Edge devices share limited computation between continuous model training and real-time inference, creating a resource-allocation tension.
  • Bursty task-specific arrivals can build queues, making uniform training allocation inefficient when highly requested tasks need additional model improvement.
  • Experiments on Stanford Cars and Oxford Flowers 102 with synthetic and Alibaba trace-derived workloads improve actively requested-task accuracy while maintaining overall multitask performance.
  • DART-FL formulates concurrent inference and multitask federated training as a joint scheduling problem under time-varying demand.
  • Its SLO-aware allocator uses inference queue states and profiled service capacity to split computation between inference and local training.
  • A queue-aware DPP-inspired scheduler distributes remaining training capacity across tasks and converts allocations into dynamic multitask loss weights.

B. Joint Training and Inference on Edge Devices

DART-FL connects online task-specific inference demand to both the inference-training split and the temporal allocation of multitask training. Queue states guide these decisions while shared representations and federated aggregation are preserved.

  • B. Joint Training and Inference on Edge Devices: Training and inference compete for computation on resource-constrained edge platforms, so increasing one workload reduces capacity for the other.
  • B. Joint Training and Inference on Edge Devices: Unlike prior approaches, DART-FL uses distinct, time-varying task demand to influence both resource division and task-level training emphasis.
  • B. Joint Training and Inference on Edge Devices: Aggregate inference backlog determines the SLO-aware inference-training split, while task-specific queues determine relative training emphasis.
  • B. Joint Training and Inference on Edge Devices: The scheduler uses task-specific pre-service queues as demand signals rather than directly allocating training resources to serve those queues.
  • B. Joint Training and Inference on Edge Devices: Bursty demand can advance a high-demand task’s learning progress to earlier communication rounds, enabling higher accuracy during frequent requests while preserving long-term multitask performance.
  • A. System Model and Queue Dynamics: DART-FL uses a shared multitask model with a common feature extractor and task-specific prediction heads on federated clients.
  • A. System Model and Queue Dynamics: At each interval, the system queues arrivals, computes the minimum inference ratio, allocates remaining capacity to training, and maps task allocations to dynamic weights.

B. SLO-Aware Inference–Training Resource Allocation

The first-stage scheduler determines the inference allocation needed to clear the current backlog within the target delay, preserving the remaining capacity for training when feasible. If the required ratio exceeds the maximum allowed allocation, the target SLO is temporarily infeasible.

  • SLO-aware allocation: The scheduler uses the current pre-service backlog and maximum service capacity to derive the inference ratio required by the delay target.The full-resource capacity can be expressed either in requests per second or requests per scheduling interval; these forms are equivalent.
  • SLO-aware allocation: The scheduler assigns only the required inference resources, leaving the remaining computation capacity available for local training.The first-stage decision determines total training capacity before task-level distribution.
  • SLO-aware allocation: When the required inference ratio exceeds γmax, the scheduler saturates at γmax and the target SLO becomes temporarily infeasible.This occurs when workload demand exceeds the maximum inference allocation permitted by the resource constraint.

C. Queue-Aware Training Scheduling

The second-stage scheduler distributes the training capacity left after inference allocation across tasks using queue-aware utility and temporal regularization. It prioritizes tasks with higher inference demand while limiting abrupt allocation changes.

  • Queue-aware allocation: ατ(t) denotes each task’s fraction of available training capacity, subject to normalization and minimum and maximum allocation bounds.The first-stage inference decision and second-stage task allocation operate on distinct resource quantities.
  • Temporal regularization: A switching cost penalizes large changes from the previous allocation, so workload gains must justify temporal regularization.This balances demand responsiveness against allocation stability across consecutive intervals.
  • Queue-aware allocation: The queue-aware objective increases a task’s scheduling importance as its pre-service queue grows, while logarithmic utility discourages monopolization.The scheduler responds to current inference workload rather than treating tasks uniformly.
  • Temporal regularization: A smaller V makes allocations respond more aggressively to current demand, whereas a larger V favors smoother allocations over time.ζ controls the numerical scale of the switching cost, while V controls the responsiveness–smoothness tradeoff.
  • Queue-aware allocation: The task queues guide training allocation but are not directly served by ατ(t), making the objective DPP-inspired rather than an exact Lyapunov-drift minimizer.The queue states provide workload information for the training scheduler.

D. Dynamic Multitask Training and Federated Aggregation

DART-FL converts queue-aware task allocations into dynamic loss weights for local multitask training, then aggregates each client’s complete model through FedAvg. Demand therefore changes federation-wide task emphasis without separate client scheduling states.

  • Dynamic task weighting: The framework maps the scheduler’s optimal task emphasis ατ(t)∗ to dynamic task weights wτ(t) in the local multitask objective.The base weight w0 and prioritization coefficient β control the weighting formulation.
  • Dynamic task weighting: A larger wτ(t) increases task τ’s contribution to local training, directing greater emphasis toward tasks with higher current demand.The weighting mechanism links inference workload to learning emphasis.
  • Dynamic task weighting: Normalized task weights represent each task’s relative training importance and are shared by all clients selected for the corresponding federated round.The normalized weights are nonnegative and sum to one.
  • Dynamic task weighting: Inference demand changes the federation’s relative training emphasis without requiring independent per-client scheduling states.The scheduler produces shared weights used across selected clients.
  • Federated aggregation: Each client uploads its shared backbone and all task-specific heads, and the server aggregates the complete multitask model using sample-weighted FedAvg.Nc denotes the number of local training samples at client c.

E. Overall Scheduling and Federated Training Algorithm

At each scheduling interval, the shared scheduler updates inference queues, reserves SLO-aware inference capacity, allocates the remainder across tasks, computes dynamic weights, and runs federated training. The allocation theorem guarantees minimum feasible inference resources when the target is attainable.

  • Scheduling workflow: Algorithm 1 first incorporates new requests into task-specific pre-service queues and computes the SLO-aware inference ratio γ∗(t).The remaining capacity is then determined for training.
  • Scheduling workflow: The scheduler updates queues after serving inference, solves the queue-aware training allocation problem, and converts α∗(t) into normalized dynamic task weights.These weights are used in the subsequent local multitask objective.
  • Analysis: The framework analyzes SLO-aware allocation, inference-queue stability, queue-aware training emphasis, and federated multitask convergence under dynamic weights.These are the four stated analysis dimensions.
  • SLO-aware allocation: The backlog-clearing delay is the time to serve the current backlog assuming no new arrivals, not a per-request latency guarantee.This defines the practical interpretation of the delay target used by the allocator.
  • Federated training: Selected clients perform local optimization with the weighted multitask objective before complete multitask models are aggregated.The workflow combines shared scheduling with federated model aggregation.
  • SLO-aware allocation: The SLO-aware allocation uses the minimum feasible inference resource needed to meet the backlog-clearing delay target whenever feasible.This maximizes the computation remaining for training.

B. Inference Queue Stability

The SLO-aware allocation stabilizes aggregate inference queues under the stated assumptions, but temporary workload bursts can still exceed the backlog-clearing delay target.

  • The analysis defines maximum per-interval service capacity as Smax = γmaxµmaxTs.
  • Theorem 2 establishes strong stability for the aggregate inference queue under the SLO-aware allocation.
  • When backlog demand exceeds the permitted maximum inference allocation, the scheduler saturates at γmax and the target SLO becomes temporarily infeasible.
  • Temporary bursts may exceed the backlog-clearing delay target, while long-term queue stability is preserved whenever the workload is supportable.

C. Queue-Aware Training Emphasis

The training-emphasis scheduler uses inference queues as demand signals rather than directly serving them, yielding a unique optimum whose responsiveness is controlled by Vζ.

  • The training-emphasis problem is strongly convex and admits a unique global optimum α∗(t) when its feasible region is nonempty.
  • A larger eQτ increases task τ’s training emphasis, while the quadratic term suppresses abrupt changes.
  • Vζ controls the responsiveness–smoothness tradeoff in queue-aware training emphasis.
  • Inference queues provide demand signals for training emphasis, while physical inference service remains controlled by γ(t).

D. Federated Multitask Convergence

The convergence analysis accounts for scheduler-induced objective variation in addition to standard federated optimization effects, with asymptotic disappearance of that tracking term under stated conditions.

  • The normalized task-weight mapping retains a nonzero contribution from every task, while β = 0 gives equal weights and larger β increases responsiveness to α∗,r.
  • Theorem 4 analyzes FedAvg after each participating client performs E local SGD steps with a sufficiently small learning rate η.
  • The convergence bound includes optimization, stochastic-gradient, client-heterogeneity, and scheduler-induced objective-variation terms.
  • When η = Θ(R−1/2), E is fixed, and ¯δR = o(R−1/2), the additional tracking term vanishes asymptotically.

V. EVALUATION RESULTS

DART-FL is evaluated on multitask vision workloads under controlled synthetic bursts and irregular Alibaba trace-derived demand. It improves high-demand task accuracy during bursts while balancing training across tasks.

  • Workloads and setup: The evaluation uses Stanford Cars and Oxford Flowers 102 with a shared ResNet backbone and task-specific classification heads.Experiments use 100 federated clients, selecting 30% per communication round, with full-model FedAvg aggregation.
  • Workloads and setup: Synthetic arrivals use a 3 requests/s baseline, with selected-task demand increasing fourfold during predefined burst intervals.Cars and Flowers arrivals are generated independently using Poisson processes.
  • Workloads and setup: Alibaba trace-derived workloads contain multiple irregular demand peaks, enabling evaluation under realistic workload dynamics.The workload is derived from Alibaba Cluster Trace 2018 and assigns two normalized demand streams to Cars and Flowers.
  • Accuracy results: DART-FL improves Cars inference accuracy over Round Robin by 5.17% on average during the synthetic burst, reaching 15.9% instantaneously.The increased Cars emphasis temporarily reduces Flowers accuracy during the same period.
  • Comparison: Burst-Aware FIFO achieves higher Cars accuracy but can leave Oxford Flowers with very low accuracy by assigning nearly all training opportunities to Cars.DART-FL continues reserving training capacity for Flowers, avoiding complete stalling of its learning progress.
  • Accuracy results: On three Alibaba demand peaks, DART-FL improves Cars accuracy over Round Robin from 0.618 to 0.654, 0.650 to 0.727, and 0.641 to 0.712.These correspond to relative improvements of 5.83%, 11.85%, and 11.08%, respectively.

E. Runtime Adaptation Behavior

DART-FL adapts both the inference-training resource split and task-level training priorities as demand changes. This joint behavior improves responsiveness while preserving progress for lower-demand tasks.

  • Inference-training split: During a Cars burst, DART-FL increases the inference resource ratio from approximately 0.2 to 0.5 while reducing the training ratio.The allocator reserves more computation for request processing instead of maintaining a fixed split.
  • Inference-training split: Under Alibaba demand peaks, the inference-training resource split varies more frequently, returning computation to training when demand decreases.The adaptation follows multiple irregular workload fluctuations.
  • Task-level priorities: During the synthetic Cars burst, its normalized training loss weight rises from approximately 0.5 to 0.6 while Oxford Flowers falls from approximately 0.5 to 0.4.Training priorities shift after inference resources are increased.
  • Task-level priorities: Under the Alibaba trace, task weights change irregularly with demand, keeping Cars generally higher when its workload dominates.This jointly adapts task priorities and the inference-training split under real-world dynamics.
  • Multitask balance: Oxford Flowers retains a non-zero training weight throughout the burst, unlike Burst-Aware FIFO, which can effectively stall Flowers training.DART-FL therefore avoids both Round Robin’s lack of demand response and FIFO’s starvation behavior.
  • Conclusion: DART-FL dynamically adjusts the inference-training resource split and task-level training weights using queue-aware scheduling under time-varying demand.Experiments with synthetic and Alibaba workloads report improved burst-period accuracy for high-demand tasks while preserving other-task opportunities.
Loading 2608.27713v1…