Momentum-Guided Federated Split Distillation for Personalized Temporal Edge Intelligence

• IoT edge devices face limited computation, memory, energy, and communication resources, making large temporal models challenging to train and deploy directly.Temporal reasoning is valuable for long-range dependencies and sequence behavior, but it is expensive for the edge.Source: §1 · pp. 1–2
• A server-side component that remains necessary for inference can limit autonomous edge deployment under latency, bandwidth, or connectivity constraints.Federated learning avoids sharing raw data, but typically assumes clients can train and synchronize complete models; split learning partitions the model between weak clients and a stronger server.Source: §1 · pp. 1–2
• Heterogeneous client objectives and temporal dynamics can make a single global model inadequate, while temporal learning signals may be noisy and unstable.The paper motivates update mechanisms that account for evolving client learning trajectories to balance collaboration and personalization.Source: §1 · p. 2
• TeRR-SAtt pairs reservoir-based temporal representations and lightweight student learning with temporal attention and specialized outputs for personalized, autonomous edge inference.Its training-only server teacher provides personalized knowledge guidance without requiring server-side inference at deployment.Source: §1 · p. 2
• AMGF identifies clients with compatible learning trajectories and derives cluster-aware teacher updates rather than relying on one shared server update.The framework also introduces learning-momentum clustering and anticipatory momentum-guided fusion to control specialization and momentum reinforcement.Source: §1 · p. 2

Federated Split Learning for Edge Intelligence

• Existing FSL approaches reduce computation, communication, storage, or deployment constraints, but treat the server-side model as globally shared rather than adaptively personalized.Examples include parallel split execution in EPSL, reduced gradient transmission in CSE-FSL, heterogeneous edge-assisted models in EUSFL, and pruning in MP-FSL.Source: §2 · p. 3

Federated Distillation and Personalization

• Federated distillation and personalization methods support local models or heterogeneous distributions, but do not specify how a temporal server teacher should adapt to client learning trajectories.FSMKD, PFL-DKD, DKD-pFed, and pFedVS address distinct personalization or distillation settings.Source: §2 · p. 3

Client Clustering and Gradient-Based Aggregation

• Momentum-based FL methods improve optimization for heterogeneous data, while other clustering methods group clients by distributional, stochastic, or classifier-similarity criteria.The cited approaches do not combine momentum-based future collaboration discovery with cluster-specialized personalized updates.Source: §2 · p. 3
• AMGF bridges update combination with distillation by determining which clients share a teacher update and how it is specialized and anticipated.It uses learning-path momentum to discover compatible clients and construct personalized updates.Source: §2 · pp. 3–4

3.1 Problem Statement

• A single global update can be inadequate when clients follow incompatible temporal learning directions, motivating personalized guidance under edge resource constraints.The objective is a deployable predictor for each client that preserves data locality and satisfies client-side resource constraints.Source: §3.1 · pp. 4–5

3.2 TeRR-SAtt Model Design

• TeRR-SAtt decomposes each client predictor into a lightweight reservoir representation, compact temporal student, and personalized output module.The U-shaped split-distillation design keeps input-side and output-side modules at the client while the higher-capacity teacher remains server-side during training.Source: §3.2 · p. 5
• During training, server-teacher activations return to the client as representation-level distillation targets for the local temporal student.Client-induced teacher gradients are processed by AMGF rather than naively aggregated.Source: §3.2 · p. 6
• At inference, clients use their retained predictor without the server, separating collaborative training from autonomous edge deployment.The design addresses the training–deployment mismatch of federated split learning.Source: §3.2 · pp. 6–7

3.3 Anticipatory Momentum-guided Fusion

• AMGF groups clients by compatible learning trajectories and constructs cluster-specialized teacher variants instead of relying on a single global update.The number of groups is inferred at each round as trajectories evolve.Source: §3.3 · pp. 7–9
• AMGF smooths teacher-relative client paths and gates continuous look-ahead by cluster agreement and temporal consistency.Anticipation is suppressed when either reliability signal is low; when both are high, it approaches the maximum look-ahead magnitude.Source: §3.3 · pp. 7–8

Experimental setting

• The evaluation uses 20 LBNL building-zone devices forecasting six future steps from 72-step inputs, with four configurations compared over 10 rounds.The configurations are Trad-FL/FSL, FSL-KD, AMGF without anticipation, and full AMGF.Source: §4 · pp. 9, 11

Edge computational efficiency

• 65.50% training latency reduction vs FL and 53.70% vs FSL; inference latency falls 44.70% vs FL, with TeRR-SAtt outperforming FSL end-to-end.The plotted FSL inference value excludes required server communication and server-side execution of the offloaded attention module.Source: §4 · pp. 9–10, Figure 2 · p. 10
• 18.40% training memory reduction vs FL and 11.40% vs FSL; inference CPU usage falls 33.10% vs FL and 14.50% vs FSL.TeRR-SAtt's inference memory is about 10% lower than FL and close to the locally measured FSL baseline.Source: Figure 3 · p. 10, §4 · pp. 10–11
• TeRR-SAtt does not increase training CPU usage vs FSL, while FL reaches higher CPU usage; TeRR-SAtt uses a fraction of full-model edge resources.Its fixed, non-trainable reservoir and lighter student are cited as reasons the added distillation step does not raise edge CPU pressure.Source: §4 · pp. 10, 12

Learning effectiveness

• TeRR-SAtt matches or exceeds the full-capacity model's learning accuracy on most edges and rounds.Trad-FL/FSL has the highest initial RMSE on nearly all illustrated edges and remains dominated across most rounds on z58 and z68.Source: Figure 4 · p. 11, §4 · p. 12
• AMGF collaboration groups are re-estimated each round by Affinity Propagation from momentum affinity, with the cluster count determined by client trajectory similarity.The six representative edges span shared-dynamics alignment, divergence between global and personalized learning, personalization demand, and bursty trajectories.Source: §4 · p. 12
• 35.31% maximum RMSE reduction over FSL-KD is reported for full AMGF; across 20 clients, its mean reduction is 5.14%±6.14%.The mean is averaged over 10 rounds; AMGF without anticipation achieves a mean reduction of 0.51% ± 7.48%.Source: §4 · p. 13

Scalability analysis

• AMGF's server momentum update and teacher fusion scale linearly with client count, while affinity-matrix construction and clustering scale quadratically.Edge-side inference remains autonomous, and its measured efficiency is independent of the number of participating clients.Source: §4 · p. 14

5 Conclusion

• TeRR-SAtt reduces edge training latency by up to 65.50%, inference latency by 44.70%, training memory usage by 18.40%, and inference CPU usage by 33.10%.The reported results are on real-world smart-building data and avoid the server dependency inherent to FSL inference.Source: §5 · p. 14
• AMGF achieves up to 35.31% RMSE reduction with anticipation and up to 31.49% without anticipation, improving local learning over global updates.The conclusion attributes complementary benefits to path-aware clustering and reliability-gated anticipation.Source: §5 · p. 14
• Future work will extend evaluation to additional smart-building and industrial IoT datasets and compare AMGF against personalized and clustered federated learning baselines.The authors also plan to investigate hierarchical momentum modeling and use momentum to reduce split-learning communication overhead.Source: §5 · p. 14

A Geometric Intuition: Anticipation and Personalized Collaboration

• Clients can be compatible when their recent teacher-relative trajectories indicate similar displacement tendencies, even if their instantaneous gradients differ.AMGF uses path-aware momentum to capture these persistent tendencies for trajectory-aware clustering.Source: A Geometric Intuition: Anticipation… · p. 17
• AMGF clusters clients by the evolution of their trajectories in teacher-parameter space, so collaboration groups reflect compatible update histories rather than only current gradients or static data similarity.These groups support personalized teacher specialization.Source: A Geometric Intuition: Anticipation… · p. 17
• Clients 1 and 2 have similar paths and aligned momentum directions, while client 3 follows a distinct trajectory that motivates a separate collaboration group.The figure presents these trajectories on an illustrative objective landscape.Source: A Geometric Intuition: Anticipation… · p. 17, Figure 5 · p. 18
• AMGF extends a conservative cluster update through anticipation only when intra-cluster agreement and temporal consistency indicate a sufficiently reliable direction.Anticipation is not applied as a fixed extrapolation step.Source: A Geometric Intuition: Anticipation… · pp. 17–18

B Behavioral Analysis of AMGF Coefficients

• The maximum anticipation magnitude sets the largest admissible look-ahead, while the agreement and stability conditions constrain its use.The appendix describes these control quantities as regulating the look-ahead optimization space.Source: B Behavioral Analysis of AMGF… · pp. 18–19
• AMGF activates strong anticipation only when cluster momentum directions agree and client trajectories remain stable across rounds.Low intra-cluster agreement or temporal consistency reduces the look-ahead term.Source: B Behavioral Analysis of AMGF… · p. 19
• The anticipation coefficient adapts the effective teacher displacement rather than applying a fixed anticipation step.When the coefficient approaches zero, the update reduces to conservative cluster momentum; as it approaches one, it approaches the maximum look-ahead regime.Source: B Behavioral Analysis of AMGF… · p. 19
• Cluster momentum sets the direction of teacher specialization, while agreement and temporal trajectory stability regulate how far that direction is extrapolated.These controls prevent over-specialization when the resulting group is internally inconsistent.Source: B Behavioral Analysis of AMGF… · pp. 18–20

C Stability and Bounded-Drift Properties of the Anchored Teacher Update

• Expressing each cluster-specialized teacher as a displacement from the initial teacher keeps the displacement bounded whenever path-aware cluster momentum is bounded.The cluster momentum is obtained from exponentially smoothed client displacement requests.Source: C Stability and Bounded-Drift… · p. 20
• Anchoring the update to the initial teacher avoids accumulating previous cluster-specialized displacements across rounds.A recursive update may propagate stale or cluster-incompatible directions when assignments change, especially as clients move between groups.Source: C Stability and Bounded-Drift… · p. 21
• The anchored formulation treats cluster momentum as a current smoothed displacement request relative to a common reference teacher, keeping specialized teachers comparable in one parameter coordinate system.The anticipation coefficient is gated by intra-cluster agreement and temporal consistency.Source: C Stability and Bounded-Drift… · pp. 20–21