PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Federated Learning and Digital Twin for Industrial Internet of Things

Wen Sun, Shiyu Lei, Lu Wang, Zhiqiang Liu, Yan Zhang

arXiv:2010.13058v2cs.LGcs.DC

TL;DR

The paper tackles federated learning for dynamic, heterogeneous Industrial IoT when digital-twin estimates may deviate from actual device states and synchronized aggregation can suffer from stragglers. It combines trust-based aggregation, DQN-driven adaptive frequency control, and clustered asynchronous learning under resource constraints. Numerical results report superiority over a benchmark in learning accuracy, convergence rate, and energy saving.

  • Problem

    Federated learning must adapt to digital-twin deviations, time-varying resources, nonlinear aggregation gains, and stragglers in heterogeneous Industrial IoT.

  • Method

    The framework uses digital twins, trust-weighted aggregation, DQN-based adaptive aggregation frequency, and clustered asynchronous federated learning.

  • Results

    The proposed scheme is superior to the benchmark in learning accuracy, convergence rate, and energy saving.

  • Takeaways & Limitations

    The framework supports adaptive federated learning for heterogeneous Industrial IoT under resource constraints and varying channel states.

Abstract

from arXiv · show

Industrial Internet of Things (IoT) enables distributed intelligent services varying with the dynamic and realtime industrial devices to achieve Industry 4.0 benefits. In this paper, we consider a new architecture of digital twin empowered Industrial IoT where digital twins capture the characteristics of industrial devices to assist federated learning. Noticing that digital twins may bring estimation deviations from the actual value of device state, a trusted based aggregation is proposed in federated learning to alleviate the effects of such deviation. We adaptively adjust the aggregation frequency of federated learning based on Lyapunov dynamic deficit queue and deep reinforcement learning, to improve the learning performance under the resource constraints. To further adapt to the heterogeneity of Industrial IoT, a clustering-based asynchronous federated learning framework is proposed. Numerical results show that the proposed framework is superior to the benchmark in terms of learning accuracy, convergence, and energy saving.

I. INTRODUCTION

The paper addresses privacy-preserving learning in heterogeneous, dynamic Industrial IoT, where data islands, time-varying resources, and stragglers complicate federated learning. It combines digital twins, trusted aggregation, adaptive aggregation frequency, and clustered asynchronous learning to improve efficiency under resource constraints.

  • Industrial IoT requires dynamic perception and intelligent decisions despite heterogeneous devices and complex industrial environments.
  • Digital twins capture device states and industrial-environment dynamics, but distributed data and privacy constraints limit centralized model training.
  • Existing federated-learning efforts address privacy, communication, or convergence, but asynchronous designs can impose substantial communication burdens.
  • The paper targets adaptive aggregation because aggregation gains are nonlinear, network conditions vary, and synchronized learning suffers from stragglers in heterogeneous scenarios.
  • Digital-twin deviations are incorporated into trust-weighted aggregation to quantify device contributions and enhance learned-model reliability and accuracy.
  • DQN adaptively calibrates aggregation frequency under a resource budget, while clustered asynchronous learning uses time-weighted inter-cluster aggregation to address heterogeneity.

III. SYSTEM MODEL

The system model uses a three-tier Industrial IoT architecture linking resource-constrained devices, servers, and real-time digital twins. Devices train locally and periodically upload models for curator aggregation until convergence or a target accuracy is reached.

  • The architecture contains industrial devices, servers, and digital twins that map physical device states and update in real time.
  • Each device twin represents training parameters, current training state, computational capability, and energy consumption.
  • Digital-twin mappings may differ from actual device values, so CPU-frequency deviation is modeled and calibrated using empirical deviation information.
  • Devices collaboratively train models for industrial tasks such as defective-product detection using their local datasets.
  • After initialization, each node updates its local model on local data, uploads parameters after preset local training, and receives the aggregated global model.
  • Global aggregation repeats until the global loss converges or reaches preset accuracy.

C. Trust-based Aggregation in DT-driven Industrial IoT

The trust-based aggregation strategy weights device updates using digital-twin deviation, learning quality, malicious behavior, uncertainty, and transmission reliability. This reduces the influence of inaccurate or unreliable participants during global aggregation.

  • The strategy gives greater aggregation weight to reputable nodes while considering DT deviation, learning quality, and malicious data rather than security threats alone.
  • Nodes with larger digital-twin mapping errors receive lower belief, while low-deviation nodes receive greater weight in aggregation.
  • Reputation incorporates DT deviation, malicious actions, and learning quality relative to the majority of training devices.
  • Learning quality is based on the honesty of most training devices, and higher-quality clients receive higher reputation values.
  • The curator updates reputations and aggregates participating local models into a weighted global model, improving robustness against malicious participants.

D. Energy Consumption Model in Federated Learning

The energy model separates local computational energy from communication energy for global aggregation. The associated optimization chooses local-update frequencies that minimize federated-learning loss under resource constraints while accounting for digital-twin estimates and calibration.

  • A training node’s energy consumption comprises computational energy from local training and communication energy from global aggregation.
  • Communication-resource consumption depends on model size, OFDMA subchannels, bandwidth, transmission power, channel gain, noise, and allocated time fractions.
  • The optimization seeks the best tradeoff between local updates and global aggregation in time-varying communication environments under a fixed resource budget.
  • The decision variables specify the number of local updates required before each global update, with constraints on available resource consumption.
  • Digital twins estimate training state and computational capability, while trust-based aggregation calibrates computational-energy deviations caused by mapping errors.

IV. DEEP REINFORCEMENT LEARNING FOR AGGREGATION FREQUENCY BASED ON DIGITAL TWIN

The resource-constrained aggregation-frequency problem is simplified with Lyapunov optimization, which converts long-term budgets into dynamic per-slot resource constraints. The resulting objective balances training improvement against resource consumption while adapting to changing training conditions.

  • The transformed optimization selects aggregation decisions {a_0,a_1,...,a_k} after simplifying the original nonlinear long-term resource problem.The original problem becomes increasingly complex as federated learning rounds increase.
  • Lyapunov optimization establishes a dynamic resource deficit queue to replace long-term resource constraints with available budgets for each time slot.The queue is maintained by minimizing Lyapunov drift and tracking the difference between used and available resources.
  • The k-th aggregation has available resource budget βR_m/k, while its resource deviation is (a_iE_cmp+E_com)−βR_m/k.
  • Positive control parameters v and Q(i) dynamically balance training performance against resource consumption.The parameter v increases with training rounds because later accuracy improvements are more costly.

B. Markov Decision Process (MDP) Model

The MDP models adaptive aggregation-frequency control using digital-twin states, node-specific local-update actions, and rewards that combine loss reduction with resource-queue conditions. DQN training explores actions before progressively adopting greedy decisions toward convergence.

  • Markov Decision Process (MDP) Model: The MDP state includes node training states, the resource deficit queue, and average hidden-layer outputs from each node’s neural network.These variables describe local training progress, resource status, and node-model information.
  • Markov Decision Process (MDP) Model: Each action specifies the discretized number of local updates for every node, represented by a_i.
  • Markov Decision Process (MDP) Model: The policy maps the current state to an action, A(t) = P{S(t)}, determining local-update decisions.The policy is also used to assess whether a local model update is trusted.
  • Markov Decision Process (MDP) Model: The reward seeks a tradeoff between local updating and global aggregation by relating loss decline to the resource-loss queue.Consumed computational energy and digital-twin deviation also affect the reward and can destabilize decisions.
  • Markov Decision Process (MDP) Model: Digital-twin mapping supplies the current state, while DQN predicts the next state without running the physical environment.The paper expresses this transition as S(t + 1) = S(t) + P(S(t)).
  • Markov Decision Process (MDP) Model: DQN training begins with random action selection and gradually increases the greedy coefficient until it reaches 1.Under fully greedy selection, unchanged strategies indicate convergence.

C. DQN-based Optimization Algorithm for Aggregation Frequency

The DQN-based optimizer uses digital twins as a real-time virtual environment for learning aggregation-frequency decisions. It combines node resource and loss states with replay-based training to produce decisions for federated learning.

  • Digital twins map physical Industrial IoT objects into virtual space, allowing a DRL agent to learn global aggregation-frequency decisions from twin states.The federated learning module uses the trained model and digital-twin status to make frequency decisions.
  • The digital-twin environment lets the agent achieve the same training effect as the real environment at lower cost.
  • DQN states combine each node’s initial resource value with its corresponding training loss, and each iteration updates rewards, states, and experience memory.Experience replay samples stored transitions to train the networks.

1) Training Step:

The training step updates DQN networks from stored transition experiences. It calculates target values with a separate target network, applies stochastic gradient descent, and periodically synchronizes network parameters.

  • Training Step:: After each action, the algorithm computes the reward, updates the system state, and stores state-action-reward-next-state tuples in experience replay.
  • Training Step:: The target network calculates q-target values independently of the current evaluation-network parameters.Samples from the experience relay provide the target-network inputs.
  • Training Step:: Stochastic gradient descent optimizes the objective using the evaluation-network output and target value.
  • Training Step:: Evaluation-network parameters are periodically copied to the target network while loss and experience replay are updated during training.The process repeats until the loss reaches a preset value.

2) Running Step:

The running step clusters heterogeneous devices, assigns curators, and uses DQN-selected local aggregation frequencies to support asynchronous federated learning. Trust-weighted local aggregation and time-weighted global aggregation combine cluster updates while addressing stragglers and changing conditions.

  • Asynchronous federated learning: Clusters train autonomously with different local aggregation frequencies, avoiding synchronization limited by the slowest heterogeneous nodes.
  • Node clustering: K-means classifies nodes by data size and computing power, assigning curators to form local training clusters.
  • Aggregation frequency decision: The DQN-based decision uses minimum local-update time T_m and tolerance factor α to constrain other clusters’ training time.The tolerance factor α increases with the global aggregation round because aggregation’s learning-efficiency effect decays over time.
  • Local aggregation: Curators locally aggregate parameters using trust weights, giving greater weight to nodes with low mapping deviation and high learning quality.
  • Global aggregation and outcomes: Time-weighted global aggregation distinguishes local models using update timing, while the heterogeneous trust-based framework is reported to eliminate stragglers and evade malicious-node attacks.The framework is also reported to improve convergence rate and learning quality, although DQN training requires many samples and is retrained when topology or channel conditions change.

V. NUMERICAL RESULTS

The numerical results evaluate DQN convergence, trust-weighted calibration of digital-twin deviation, adaptive aggregation under channel variation, clustering, and adaptive versus fixed aggregation frequency. The proposed mechanisms improve accuracy, convergence behavior, and energy efficiency under heterogeneous and changing conditions.

  • DQN convergence: DQN loss stabilizes after about 1200 training rounds and converges to a better result.The trained DQN is reported as suitable for heterogeneous scenarios with good convergence performance.
  • Digital-twin deviation: Trust-weighted aggregation achieves higher federated-learning accuracy after calibrating digital-twin deviation than when the deviation remains present.The calibrated approach is also superior before the two algorithms converge, while DQN with deviation does not converge.
  • Adaptive aggregation frequency: DQN adaptively avoids aggregation in poor channel conditions and makes almost all aggregations within 5 rounds.As the distribution of good channel states increases, aggregations in good channel states increase; when the good-channel probability is 0.2, total aggregation is higher than in other cases.
  • Energy consumption: Energy consumption decreases with improving channel quality and declines in all three channel states during DQN training.The paper attributes this to adaptive timing that selects local training instead of delayed, high-energy aggregation under relatively poor channel quality.
  • Clustering: More clusters produce higher accuracy within the same time and reduce the time required to reach the same accuracy.The results attribute these gains to different local aggregation times and more effective mitigation of the straggler effect.
  • Adaptive versus fixed frequency: DQN-based federated learning ultimately achieves greater accuracy than fixed-frequency federated learning.The paper relates this result to the non-linear aggregation gain and the fixed-frequency scheme missing the best aggregation opportunity.

VI. CONCLUSIONS

The conclusion presents a DQN-based strategy for balancing local and global updates under a resource budget, combined with digital-twin-assisted adaptation and clustered asynchronous federated learning. Numerical results report superiority over the benchmark in learning accuracy, convergence rate, and energy saving.

  • Adaptive aggregation: DQN explores the best trade-off between local and global updates under a given resource budget.The strategy adaptively adjusts aggregation frequency according to channel state.
  • Asynchronous clustering: An asynchronous federated-learning architecture based on node clustering is designed to eliminate the straggler effect in heterogeneous scenarios.The architecture is described as more suitable for heterogeneous Industrial IoT.
  • Overall results: The proposed scheme is superior to the benchmark in learning accuracy, convergence rate, and energy saving.
Loading 2010.13058v2…