Source-linked AI summary
Low-latency Federated Learning and Blockchain for Edge Association in Digital Twin empowered 6G Networks
Yunlong Lu, Xiaohong Huang, Ke Zhang, Sabita Maharjan, Yan Zhang
TL;DR
The paper addresses unreliable communication, limited edge resources, and distrust that hinder federated learning for IIoT. It introduces DTWN with permissioned blockchain-enabled federated learning and optimizes association and resource allocation using multi-agent reinforcement learning. Experiments on a real-world dataset report reduced learning latency, good convergence, and lower system time cost than benchmark approaches.
Problem
Unreliable communication, limited resources, and lack of mutual trust hinder effective federated learning in IIoT edge networks.
Method
The paper combines DTWN, permissioned blockchain, federated learning, and multi-agent reinforcement learning for joint edge association and communication-resource allocation.
Results
Numerical results on a real-world dataset show reduced learning latency, good learning convergence, and lower system time cost than benchmark algorithms.
Takeaways & Limitations
The proposed scheme improves the running efficiency of blockchain- and federated-learning-enabled digital-twin edge intelligence under limited resources.
Abstract
from arXiv · showhide
Emerging technologies such as digital twins and 6th Generation mobile networks (6G) have accelerated the realization of edge intelligence in Industrial Internet of Things (IIoT). The integration of digital twin and 6G bridges the physical system with digital space and enables robust instant wireless connectivity. With increasing concerns on data privacy, federated learning has been regarded as a promising solution for deploying distributed data processing and learning in wireless networks. However, unreliable communication channels, limited resources, and lack of trust among users, hinder the effective application of federated learning in IIoT. In this paper, we introduce the Digital Twin Wireless Networks (DTWN) by incorporating digital twins into wireless networks, to migrate real-time data processing and computation to the edge plane. Then, we propose a blockchain empowered federated learning framework running in the DTWN for collaborative computing, which improves the reliability and security of the system, and enhances data privacy. Moreover, to balance the learning accuracy and time cost of the proposed scheme, we formulate an optimization problem for edge association by jointly considering digital twin association, training data batch size, and bandwidth allocation. We exploit multi-agent reinforcement learning to find an optimal solution to the problem. Numerical results on real-world dataset show that the proposed scheme yields improved efficiency and reduced cost compared to benchmark learning method.
I. INTRODUCTION
The paper introduces DTWN, combining digital twins, blockchain, and federated learning to address unreliable communication, limited resources, and distrust in IIoT edge intelligence. It formulates joint association and resource-allocation optimization and solves it with multi-agent reinforcement learning.
- Digital Twin Wireless Networks: DTWN maps IoT devices to digital twins at edge servers, moving data analysis and optimization toward the edge plane.The model synchronizes physical-system information with digital twins and reduces the impact of unreliable, long-distance communication.
- Privacy-Preserving Learning: Federated learning keeps original user data at users while transmitting locally trained model parameters, reducing data leakage risks.The paper uses federated learning for collaborative edge intelligence rather than centralized raw-data transmission.
- Joint Optimization: The proposed optimization jointly considers digital-twin association, communication resource allocation, and learning efficiency to reduce time cost.The formulation responds to dynamic edge-server resources and limited base-station bandwidth during federated-learning rounds.
- Multi-Agent Reinforcement Learning: Multi-agent reinforcement learning finds an optimal solution for the edge-association and communication-resource-allocation problem.The algorithm is designed to improve the efficiency of the proposed scheme.
- Blockchain-Enabled Collaboration: Permissioned blockchain records digital-twin data and manages participating users through permission control to address limited mutual trust.The blockchain is maintained by base stations and is intended to enhance system security and data privacy.
A. Edge Association in DTWN
Edge association assigns each IoT user’s digital twin to a base station while accounting for data size, base-station computation capability, and transmission rate. The problem seeks efficient resource use and effective digital-twin operation under one-to-many base-station association constraints.
- Edge Association Definition: Edge association chooses a target base station to construct each user’s digital twin.The association variable Φ(i, j) equals the user-data size when user i is associated with base station j and zero otherwise.
- Association Constraints: A base station may associate with multiple digital twins, whereas each digital twin can be associated with at most one base station.This establishes the one-to-many association structure used in the DTWN model.
- Association Criteria: Association decisions use IoT-user datasets, base-station computation capability, and transmission rates between users and base stations.These factors capture both workload and communication conditions in the association problem.
- Optimization Objective: The edge-association objective is to improve resource utility and the efficiency of running digital twins in DTWN.Digital-twin maintenance includes synchronizing real-time data and building corresponding models under limited wireless-network resources.
B. Federated Learning Model
The federated learning model trains local models on digital-twin data, aggregates them hierarchically at base stations, and then updates a global model at the macro base station. Communication efficiency matters because participating base stations transmit model parameters over limited wireless bandwidth.
- Federated learning trains a global model from digital-twin data without transmitting the original training data.
- Client base stations train local models and transmit local gradients or models to the macro base station for global aggregation.
- Base stations first aggregate models from multiple digital twins before sending the aggregated model to the macro base station, reducing transmission load.
- The macro base station collects parameters from participating base stations and updates the global model.
- Limited wireless bandwidth makes communication efficiency vital to convergence during parameter transmission.
C. Blockchain Model
The framework uses a permissioned blockchain maintained by base stations to record and verify digital-twin data and local models during federated learning. Delegated Proof of Stake governs block production through training coins and elected producers.
- Base stations act as blockchain nodes and maintain a permissioned blockchain for securing digital twins and local models.
- Trained local models are recorded as blockchain transactions, broadcast for verification, packed into blocks, and validated through consensus.
- The consensus process uses Delegated Proof of Stake, with training coins allocated to base stations according to digital-twin data.
- Base stations earn coins when their models pass verification, while elected block producers take turns packing transactions into blocks.
- The block-verification interval is set to multiple local-training periods because verification consumes substantial resources.
D. Communication Model
The communication model uses OFDMA for uplink transmission of trained local models and downlink broadcasting of the global model. Transmission rates depend on bandwidth allocation, power, channel gain, path loss, distance, noise, and interference.
- OFDMA divides wireless transmission across C sub-channels shared by base stations uploading local-model parameters.
- The uplink rate depends on each base station’s time fraction, sub-channel bandwidth, transmission power, and uplink channel gain.
- The model defines transmission parameters for both uplink and downlink wireless links between base stations and the macro base station.
- Uplink channel conditions account for path loss, base-station distance, path-loss exponent, noise power, and interference from other base stations.
- The macro base station broadcasts the global model during the downlink phase using a rate determined by downlink power and channel gain.
III. EDGE ASSOCIATION IN DTWN: PROBLEM FORMULATION
The paper formulates edge association as a latency-minimization problem under a learning-accuracy requirement. The decision variables jointly control digital-twin association, training batch sizes, and bandwidth allocation, while the resulting combinational problem is challenging because variables and base-station resource states interact.
- The proposed optimization minimizes total system time cost while satisfying a required global learning accuracy.
- The formulation assumes Lipschitz-smooth gradients and a strongly convex loss function to derive an upper bound on global iterations.
- The latency model includes local training, model aggregation, transaction transmission, block validation, and other iteration costs.
- Aggregation time is neglected relative to other phases because model parameters are small and base-station computing capability is high.
- Increasing each digital twin’s training batch size can improve learning accuracy but also increases computation time.
- The optimization jointly considers digital-twin edge association, batch-size selection, and bandwidth allocation under dynamic computing and communication capabilities.
- Problem (18) is combinational and difficult because its objective contains products of variables and each base station’s time cost depends on other base stations’ resource states.
IV. MULTI-AGENT DRL FOR EDGE ASSOCIATION
The edge association problem is formulated as a Markov Decision Process because current network states and previous allocation policies determine system states. Multi-agent deep reinforcement learning is used to solve it.
- The edge association problem is modeled as an MDP using current network states and previous iteration allocation policies.
- Multi-agent DRL is applied to solve the formulated edge association problem.
A. Multi-agent DRL Framework
The framework treats each base station as a DRL agent interacting with digital twins in a shared environment. Agents select digital-twin allocation, training batch sizes, and bandwidth allocation to minimize system time cost using shared blockchain-recorded information.
- Each base station is a DRL agent, while base stations and end-user digital twins form the common environment.
- The state includes base-station computing capability, digital-twin counts, training-data sizes, and channel states.
- Each agent chooses digital-twin allocation, training-data batch sizes, and bandwidth allocation from the system state.
- The shared reward is based on the maximum agent time cost, so agents maximize reward by minimizing system time cost.
- Blockchain records digital-twin and base-station states, allowing each base station to retrieve other agents’ states and actions.
B. Multi-agent DRL algorithm for edge association
The proposed algorithm uses cooperative multi-agent DDPG to optimize edge association and communication resources in a large continuous state-action space. Agents train actor-critic networks with replay memory and deploy the trained models for online resource allocation.
- DDPG addresses the large continuous state and action spaces through cooperative actor-critic agents and shared rewards.
- Agents update primary actor and critic networks using replay-memory samples and update corresponding target networks.
- Each agent generates actions with an actor DNN, while exploration uses Ornstein-Uhlenbeck noise.
- Algorithm 1 initializes actor-critic networks and replay memory, then iteratively observes states, executes actions, stores transitions, and updates networks.
- The action comprises digital-twin association policies, training-data batch size, and bandwidth allocation.
- Parallel agent training has complexity O(L_a^2 · E), related to DNN size and episode count.
V. NUMERICAL RESULTS
Experiments use CIFAR10 in a wireless network with one MBS, five BSs, and 100 end users. The proposed method reduces system time cost, achieves comparable convergence to fully trained federated learning, and shows faster, lower-cost DRL training with γ = 0.9.
- The experiment uses CIFAR10 with 60,000 images and a network containing 1 MBS, 5 BSs, and 100 end users.
- The proposed algorithm significantly reduces system time cost compared with random and average edge-association benchmarks.
- The proposed method achieves comparable convergence to fully trained federated learning and improves learning loss over random edge association.
- γ = 0.9 achieves the best cumulative-average performance, with higher convergence rate and smaller system time cost.
- Training the optimization algorithm costs much more time than testing because deep reinforcement learning iteratively explores the environment.
VI. CONCLUSION
The paper combines digital twin wireless networks with blockchain-enabled federated learning for edge intelligence, then optimizes edge association to improve efficiency. Multi-agent reinforcement learning provides the solution, which reduces learning latency and supports good learning convergence on real-world data.
- VI. CONCLUSION: The proposed DTWN model combines federated learning and blockchain to improve the running efficiency of edge intelligence.The framework is designed for a wireless edge network consisting of end users and base stations.
- VI. CONCLUSION: A blockchain-empowered federated learning scheme is developed in DTWN for realizing edge intelligence.
- VI. CONCLUSION: Multi-agent reinforcement learning derives an optimal solution for edge association between digital twins and base stations.
- VI. CONCLUSION: The proposed scheme effectively reduces learning latency and achieves good learning convergence on a real-world dataset.