PaperlayerView exact revision on arXiv

Source-linked AI summary

Privacy-Preserved Task Offloading in Mobile Blockchain with Deep Reinforcement Learning

Dinh C. Nguyen, Pubudu N. Pathirana, Ming Ding, Aruna Seneviratne

arXiv:1908.07467v2eess.SP

TL;DR

Resource-constrained mobile devices struggle to perform blockchain mining, motivating MEC-assisted offloading. The paper jointly optimizes offloading and privacy with RL and DQN, and reports improved privacy, energy, latency, and system-cost performance over benchmark schemes, while assuming a stable action space.

  • Problem

    Resource-constrained mobile devices may be unable to participate in blockchain mining because mining requires intensive computation and storage, while existing blockchain offloading frameworks largely ignore user privacy.

  • Method

    The paper models MEC-based task offloading and user privacy preservation jointly, using RL and a DQN algorithm for dynamic offloading decisions.

  • Results

    The proposed schemes significantly improve user privacy and reduce energy consumption, computation latency, and system costs compared with other blockchain offloading schemes.

  • Takeaways & Limitations

    MEC-assisted RL offloading provides a privacy-aware approach for mobile blockchain systems with computationally constrained users.

  • Takeaways & Limitations

    The current method assumes a stable action space, although real scenarios may have dynamic user demands, MEC resources, and wireless-channel usage.

Abstract

from arXiv · show

Blockchain technology with its secure, transparent and decentralized nature has been recently employed in many mobile applications. However, the mining process in mobile blockchain requires high computational and storage capability of mobile devices, which would hinder blockchain applications in mobile systems. To meet this challenge, we propose a mobile edge computing (MEC) based blockchain network where multi-mobile users (MUs) act as miners to offload their mining tasks to a nearby MEC server via wireless channels. Specially, we formulate task offloading and user privacy preservation as a joint optimization problem which is modelled as a Markov decision process, where our objective is to minimize the long-term system offloading costs and maximize the privacy levels for all blockchain users. We first propose a reinforcement learning (RL)-based offloading scheme which enables MUs to make optimal offloading decisions based on blockchain transaction states and wireless channel qualities between MUs and MEC server. To further improve the offloading performances for larger-scale blockchain scenarios, we then develop a deep RL algorithm by using deep Q-network which can efficiently solve large state space without any prior knowledge of the system dynamics. Simulation results show that the proposed RL-based offloading schemes significantly enhance user privacy, and reduce the energy consumption as well as computation latency with minimum offloading costs in comparison with the benchmark offloading schemes.

I. INTRODUCTION

The paper addresses the computational limits that prevent resource-constrained mobile devices from mining blockchain transactions. It proposes MEC-assisted task offloading with RL and DQN to jointly optimize offloading, privacy, latency, energy, and cost.

  • Motivation: Mobile mining puzzles require intensive computation, limiting blockchain participation by resource-constrained IoT nodes and mobile devices.MEC is introduced as a way to offload computation to nearby, more powerful servers.
  • Research gap: Existing blockchain offloading studies optimize mining, resource allocation, or scheduling but generally ignore user privacy.Related approaches include convex optimization, RL, and DRL methods.
  • Proposed system: The proposed MEC blockchain model lets mobile users act as miners and offload IoT data-processing and mining tasks to an MEC server.The model includes a public blockchain, an MEC server, an access point, IoT devices, and multiple mobile users.
  • Approach: The paper formulates task offloading and privacy preservation jointly, then uses RL and DQN to learn offloading policies without prior system dynamics.DQN is intended to address high-dimensional state spaces as the number of blockchain users grows.
  • System model: Each task is characterized by data size, required CPU cycles, and maximum tolerable delay, while offloading decisions depend on transaction and channel states.The task model includes newly received and buffered IoT data.

B. Offloading For IoT Data Processing Tasks

The system models dynamic IoT task arrivals and lets each miner choose between local execution and MEC offloading. A binary policy represents this choice, while channel states inform privacy-aware decisions and miners may also participate in consensus.

  • Task generation: Dynamic transaction volumes can require miners to split oversized IoT data between immediate processing and buffered future processing.The blockchain task combines newly received data with data already in the miner’s buffer.
  • Computing modes: Miners use two computing modes: offloading data-processing tasks to the nearby MEC server or executing them locally.The two modes are explicitly defined for each mobile miner.
  • Offloading policy: The binary offloading variable selects MEC execution when it equals 1 and local processing when it equals 0.The decision applies to each miner’s task at a timeslot.
  • Channel model: Wireless channel power gain is modeled as a Markov chain with good and bad channel states, supporting privacy-aware offloading analysis.The policy uses channel conditions to analyze user privacy during task offloading.
  • Mining participation: Mobile users can also participate in consensus and obtain rewards, while cloud-based mining supplies hashing power for resource-constrained devices.Leased hashing power from edge cloud providers is presented as one mining option.

D. Offloading Privacy

The privacy formulation treats task offloading as exposing user information through observable wireless activity and offloading history. It evaluates offloading through latency and energy components alongside privacy concerns.

  • Privacy model: Offloading privacy concerns preserving usage patterns and user locations against eavesdroppers or curious MEC servers.Monitoring transmissions, service migrations, channel states, and task sizes can reveal personal information.
  • Latency cost: Offloading latency is decomposed into uploading, queuing, and MEC processing delays.The decomposition is used as a computation-latency performance measure.
  • Latency components: Uploading delay is determined by task data and wireless transmission rate, while queuing delay reflects CPU cycles waiting in the MEC task buffer.Processing delay depends on the MEC computational resource allocated to the task.
  • Modeling assumption: Download latency and energy are ignored because processed data is assumed small and the download rate generally high.This is an explicit modeling simplification for the processed result returned to the miner.
  • Energy cost: Total offloading energy includes task uploading, task processing, and MEC operation costs.The formulation includes transmit power, circuit power, and MEC energy efficiency.

2) Local computation cost:

For local computation, the paper models time and energy directly from the task’s data size and device-specific per-bit costs.

  • Local computation time: Local execution time is calculated using the task data size and the miner’s local execution time per data bit.The local per-bit time is expressed in seconds per bit.
  • Local computation energy: Local execution energy is calculated using the task data size and the mobile user’s energy consumption per data bit.The per-bit energy quantity is expressed in joules per bit.
  • Privacy: The privacy formulation considers usage-pattern privacy and location privacy during task offloading.These privacy dimensions address information inferred from wireless transmission activity and user behavior.

1) Usage pattern privacy:

The paper models privacy risks from task offloading through usage-pattern and location privacy, balancing these concerns against computation and mining considerations.

  • Usage pattern privacy: Good wireless channels encourage full offloading to reduce mobile-device processing time and energy cost.When the channel is good, miners tend to offload all blockchain transaction data to the MEC server.
  • Usage pattern privacy: MEC monitoring of offloaded data can reveal a miner’s data usage pattern.The server can obtain usage patterns by monitoring the data-processing task offload.
  • Usage pattern privacy: Under good channel conditions, deliberately changing locally processed data creates a difference between actual and offloaded transaction data sizes.The indicator function identifies good channel states, while the altered local amount is intended to preserve usage-pattern privacy.
  • Location privacy: Offloading decisions also expose location because wireless channel power gain is correlated with distance from the MEC server.Good transmission states favor offloading, while poor states lead to local execution or buffering.
  • Location privacy: Preserving location privacy requires maintaining some offloading under poor channel quality, while mitigating offloading under good quality.The total privacy level combines usage-pattern and location privacy, with λ weighting location privacy relative to usage-pattern privacy.
  • Mining reward: Mining success depends on relative hash power and block propagation, with larger blocks increasing propagation time and orphaning risk.The expected reward is R_n = Rµ_ne^(-ηφ(s_n)), while miners also pay the edge-cloud provider for mining service.

V. OFFLOADING OPTIMIZATION WITH RL

The paper formulates privacy-aware offloading and resource allocation as a joint optimization problem, then uses reinforcement learning to address its dynamic, non-convex structure.

  • Optimization objective: The optimization objective maximizes offloading privacy while minimizing the combined cost of computation latency and energy consumption.The cost is defined as a weighted sum of task-processing latency and energy consumption.
  • Cost model: Latency is modeled as the maximum of local processing time and MEC execution time.Energy includes both local consumption and the energy cost of task offloading.
  • Optimization formulation: The joint problem optimizes privacy, system cost, and mining reward under mining-latency and MEC-capacity constraints.The formulation includes binary offloading decisions and weighted energy-latency costs.
  • Optimization challenges: Because binary decisions, non-convex feasibility, and multiplicative objective terms make the problem mixed-integer non-convex and NP-hard, conventional optimization is difficult.The offloading decision selects MEC execution or local processing, subject to maximum task latency.
  • Optimization challenges: System states and rewards are unavailable in advance, limiting methods that rely only on current-state optimization.The problem also has high-dimensional joint decisions involving offloading, costs, mining profit, data sizes, channels, and hash power.
  • DRL solution: The proposed DRL scheme selects offloading actions from current transaction and channel states without prior system-dynamics knowledge.DRL is presented as a way to handle large state spaces and improve offloading performance in larger blockchain applications.

B. Reinforcement Learning Formulation

The reinforcement-learning formulation treats miners as agents that choose offloading actions from blockchain, channel, and mining-service states to optimize long-term system performance.

  • B. Reinforcement Learning Formulation: Each miner interacts with the blockchain environment at each timeslot and maps state s to an offloading action a through policy π.The policy seeks to maximize cumulative reward over the long run.
  • B. Reinforcement Learning Formulation: The RL formulation defines miners as agents that seek optimal policies from state, action, and reward interactions.Figure 3 presents the RL-based offloading process for mobile blockchain.
  • 1) State:: Transaction data and channel conditions provide the operational state used to select offloading actions.The formulation is designed around timeslot-based interaction with the mobile blockchain environment.
  • 1) State:: The RL state includes new and buffered transaction data, wireless channel power gain, purchased hash power, and mining payment.These quantities describe the miner’s data, communication, and mining-service conditions.
  • 2) Action:: The action vector represents binary offloading decisions for the miners’ tasks.Each task is assigned to MEC execution or local processing at the mobile device.
  • 2) Action:: The MEC computation capacity is assumed large enough to serve all miners, while hash-power demand and payment are determined around the offloading process.Users purchase hash power before data offloading, and payment is completed automatically afterward.

3) Reward:

The reward formulation combines mining reward, privacy, and time-energy cost, while Q-learning and DQN learn offloading policies for increasingly large state-action spaces.

  • 3) Reward:: The RL agent seeks high mining reward and privacy while minimizing the combined time and energy cost of offloading.The immediate reward is designed to align positively with the joint optimization objective.
  • 3) Reward:: The proposed schemes include RL-based task offloading and deep RL-based task offloading.The methods are referred to as RLO and DRLO, respectively.
  • RL algorithm: Q-learning models blockchain miners as agents that explore actions through interaction with the environment without an explicit dynamics model.The MDP formulation requires exploration because miners initially lack environment experience and information.
  • RL algorithm: The optimal policy is obtained from maximum Q-values using the Bellman optimality equation.The Q-learning formulation uses the expected immediate reward plus discounted future maximum value.
  • RL algorithm: Q-learning is stated to converge with probability one over infinitely many times and to achieve the optimal Q-value.Algorithm 1 implements the RLO procedure for task offloading.
  • RL algorithm: RLO initializes a Q-table, observes transaction and channel states, selects exploratory or greedy actions, and updates values from rewards.The procedure includes local or MEC task execution and estimates privacy and system cost at each iteration.
  • DRL limitation and solution: Q-table learning becomes infeasible in large state-action spaces because sampling becomes insufficient and convergence slows.The DRL method replaces the table with a neural network that approximates Q-values.

B. DRL-based Task Offloading Algorithm

The DRLO algorithm uses observed blockchain and channel states to select offloading actions, store experience, and iteratively train a Q-network. This procedure optimizes task-offloading policies through reward feedback and loss minimization.

  • Q-network training: The Q-network is initialized with random weights and an exploration probability, then updated by minimizing a loss function.The algorithm uses a DNN to approximate Q-values and iteratively updates its weights during training.
  • Policy execution: DRLO observes blockchain transactions and channel-gain states before selecting task-offloading actions.Actions are selected randomly with probability ϵ and otherwise by maximizing the estimated Q-value.
  • Policy execution: The algorithm evaluates privacy and system cost after observing each reward and next state.These quantities provide feedback for optimizing the offloading policy over time.
  • Q-network training: DRLO stores state transitions in replay memory and samples random mini-batches for Q-value training.Stored experiences contain the state, action, reward, and next state.

VII. SIMULATION AND PERFORMANCE EVALUATION

The evaluation combines real experiments and simulations to assess latency, energy consumption, privacy, and reward under local, edge, and learning-based offloading settings. It compares the proposed methods with fixed offloading and privacy-oriented baselines while varying system and learning parameters.

  • Simulation settings: Two hidden layers with 500 hidden units produced the best performance, so the study selected this DNN architecture.The architecture used 300 units in the first hidden layer and 200 in the second.
  • Evaluation design: The evaluation measures computation latency, mining and local-computation energy, and privacy during task offloading.These metrics are used to assess the performance of mobile blockchain offloading.
  • Evaluation design: The study compares RLO and DRLO with non-offloading, edge-offloading, CMDP-based, and RL-based benchmark schemes.The NO scheme executes all tasks locally, whereas EO offloads all tasks to the MEC server.
  • Simulation settings: The reward weights latency and energy through β, with α1 = β and α2 = 1 − β.The resulting reward is rt(s, a) = P t(s, a) −[βEt(s, a) + (1 −β)Lt(s, a)].

B. Experimental Evaluation

Experiments compare local execution with edge offloading for IoT data and evaluate learning convergence. Edge offloading reduces computation time and energy in the reported file-size range, while DRLO achieves higher long-term rewards than RLO.

  • Local and edge computation: 36% time savings occurred for 50 kB files, increasing to 43% for 150 kB files with edge offloading.The comparison concerns computation time relative to local execution.
  • Local and edge computation: Edge offloading reduced energy consumption by 15% for 50 kB files, 17.8% for 130 kB files, and 19% for 150 kB files.The reported energy advantage increased with data size in these examples.
  • Convergence performance: DRLO converged to a reward of 5.9 at β = 0.5, approximately 5.2% above RLO's 5.4 after about 2500 timeslots.The results were obtained during training over 8,000 timeslots.
  • Convergence performance: At β = 0.8, DRLO converged to about 7.1, roughly 9% higher than RLO.Overall, DRLO achieved better performance than RLO in both reported tradeoff settings.

2) Task offloading in single user scenario:

The proposed DRLO offloading scheme achieves low offloading cost in single- and multi-user simulations while improving privacy and reducing energy consumption and computation latency. Its performance is evaluated under different task loads and tradeoff factors.

  • The single-user model varies the number of equal-sized tasks from M=2 to M=12 for one miner.
  • At β = 0.5, DRLO achieves the minimum sum cost for all data processing tasks.The compared costs increase as the number of tasks grows, while NO reaches 36 at M = 12 with a 125% increase.
  • 18.7%, 57%, and 65% lower offloading costs are achieved by DRLO than RLO, EO, and NO at M = 12 and β = 0.8, respectively.The result is averaged over 50 simulations.
  • 12.7%, 52.5%, and 30.4% lower offloading costs are achieved by DRLO than RLO, EO, and NO for N = 10 in the multi-user scenario.The DRLO-based strategy also exhibits minimum power usage, mining delay, and resulting offloading cost.
  • DRLO provides the best privacy performance, with 5.2% and 12.7% higher privacy levels than RL-based and CMDP-based schemes for 10 kB transactions.For 100 kB transactions, DRLO remains 5.5% higher than RL and 13.4% higher than CMDP.
  • The proposed DRL offloading approach reduces energy consumption, computation latency, and offloading costs while improving privacy level compared with baseline methods.
Loading 1908.07467v2…