PaperlayerView exact revision on arXiv

Source-linked AI summary

Accuracy-Guaranteed Collaborative DNN Inference in Industrial IoT via Deep Reinforcement Learning

Wen Wu, Peng Yang, Weiting Zhang, Conghao Zhou, Xuemin, Shen

arXiv:2301.00130v1eess.SYcs.AIcs.LGcs.SY

TL;DR

Collaborative industrial IoT inference must balance low service delay with long-term accuracy amid channel variation and random task arrivals. The paper combines sampling-rate and offloading decisions with Lyapunov-guided deep RL, and reports lower delay while preserving accuracy requirements with high probability.

  • Problem

    Collaborative DNN inference must minimize service delay while meeting long-term accuracy requirements despite changing channel conditions and random task arrivals.

    Source: §I · pp. 1–2

  • Method

    The paper formulates the problem as a CMDP, transforms it into an MDP using Lyapunov accuracy-deficit queues, and applies deep RL with an optimization subroutine for edge resource allocation.

    Source: §I · p. 2

  • Results

    The proposed algorithm significantly outperforms benchmark schemes on average service delay and meets both services’ accuracy requirements with high probability, with maximum error probability below 0.5%.

    Source: 2) Impact of task arrival rate · p. 9

  • Takeaways & Limitations

    The algorithm optimizes service delay on the fly without requiring statistical information about network dynamics.

    Source: §VI · p. 10

Abstract

from arXiv · show

Collaboration among industrial Internet of Things (IoT) devices and edge networks is essential to support computation-intensive deep neural network (DNN) inference services which require low delay and high accuracy. Sampling rate adaption which dynamically configures the sampling rates of industrial IoT devices according to network conditions, is the key in minimizing the service delay. In this paper, we investigate the collaborative DNN inference problem in industrial IoT networks. To capture the channel variation and task arrival randomness, we formulate the problem as a constrained Markov decision process (CMDP). Specifically, sampling rate adaption, inference task offloading and edge computing resource allocation are jointly considered to minimize the average service delay while guaranteeing the long-term accuracy requirements of different inference services. Since CMDP cannot be directly solved by general reinforcement learning (RL) algorithms due to the intractable long-term constraints, we first transform the CMDP into an MDP by leveraging the Lyapunov optimization technique. Then, a deep RL-based algorithm is proposed to solve the MDP. To expedite the training process, an optimization subroutine is embedded in the proposed algorithm to directly obtain the optimal edge computing resource allocation. Extensive simulation results are provided to demonstrate that the proposed RL-based algorithm can significantly reduce the average service delay while preserving long-term inference accuracy with a high probability.

I. INTRODUCTION

The paper addresses collaborative DNN inference under changing network conditions by jointly adapting sampling, offloading, and computing resources to balance service delay and long-term accuracy. It positions sampling-rate adaptation and Lyapunov-guided deep reinforcement learning as responses to limitations in existing approaches.

I. INTRODUCTION

  • Device-only inference faces high energy use and delay, while edge-only offloading faces unpredictable delay; sampling-rate adaptation can trade accuracy for lower data volume and computation workload.The paper’s bearing-diagnosis example reports accuracy rising from 95% to 98.7% as the sampling rate increases from 18 KHz to 24 KHz.Source: §I · pp. 1–2
  • The paper formulates collaborative inference as a CMDP minimizing average service delay subject to long-term accuracy requirements, then uses Lyapunov optimization and deep RL to address the constraints.Accuracy-deficit queues guide the learning agent, and an optimization subroutine directly solves edge resource allocation.Source: §I · p. 2

II. RELATED WORK

  • Related work covers device-side DNN compression, edge-assisted inference, and resource-allocation RL, while this work adds sampling-rate adaptation and long-term accuracy constraints.The paper notes that traditional RL algorithms such as DDPG do not handle constrained long-term optimization directly.Source: §II · pp. 2–3

A. Network Model

  • The network model serves multiple industrial IoT services through compressed DNNs on devices and uncompressed DNNs at one AP, with tasks processed locally or offloaded.Devices select sampling rates according to channel conditions and computation workloads.Source: A. Network Model · p. 3

B. Service Delay Model

The service-delay model treats task arrivals as random and accounts for local execution through queueing and computation. Local queue capacity and overflow penalties constrain this part of the model.

B. Service Delay Model

  • Task arrival rate at each device is assumed to follow a general random distribution.Generated task data size is represented after the sampling rate is selected.Source: 1) Executing locally · p. 3

1) Executing locally:

  • For local execution, service delay includes both local-queue waiting and task processing by the device’s compressed DNN.Processing demand depends on device CPU frequency and the compressed DNN’s computation intensity.Source: 1) Executing locally · p. 4
  • The local computing queue tracks backlogged tasks, has finite capacity, and incurs a penalty when overflow causes task drops.The model explicitly represents the amount of dropped tasks and the overflow event.Source: 1) Executing locally · p. 4

2) Offloading to AP:

Offloading to the AP adds transmission and edge-processing delays to collaborative inference, while resource allocation and queueing determine the AP-side service time.

2) Offloading to AP:

  • Offloaded tasks incur task-offloading delay, edge-queue delay, and task-processing delay.The service-delay model also accounts for local execution and potential queue-overflow penalties.Source: 2) Offloading to AP · p. 4
  • The offloading delay depends on the device-to-AP transmission rate, which is affected by channel conditions.Channel gain varies with the channel state, modeled as a finite-state ergodic Markov chain.Source: 2) Offloading to AP · p. 4
  • The AP dynamically allocates computing resources among services, and each service’s allocation share affects processing delay.The processing model uses the AP server’s CPU frequency and the uncompressed DNN’s computation intensity.Source: 2) Offloading to AP · p. 4
  • Edge-queue delay includes backlog processing and the waiting time of newly arrived tasks; tasks may be dropped when the queue is full.Queue overflow incurs a penalty in the service-delay expression.Source: 2) Offloading to AP · p. 4

C. Inference Accuracy Model

The accuracy model combines experimentally measured effects of sampling rate and DNN type. The CMDP then specifies decisions and network state for minimizing delay subject to long-term accuracy requirements.

C. Inference Accuracy Model

  • Sampling-rate accuracy is measured experimentally with AlexNet on bearing-vibration fault diagnosis, and uncompressed DNNs achieve higher accuracy than compressed DNNs.The model represents accuracy as a function of sampling rate and distinguishes compressed and uncompressed DNN accuracy.Source: C. Inference Accuracy Model · p. 5
  • The model treats sampling-rate selection and DNN selection as independent, so their accuracy contributions are multiplied to obtain inference accuracy.This accuracy characterization can be extended to other inference methods using practical experimental accuracy values.Source: C. Inference Accuracy Model · p. 5

D. Problem Formulation

  • The CMDP jointly represents sampling-rate selection, task offloading, and edge-resource allocation, with queue backlogs, channel conditions, and task data size in the state.The objective is to minimize service delay while satisfying long-term accuracy requirements.Source: D. Problem Formulation · p. 5
  • Deep RL is proposed because transition statistics are unavailable and the state and action spaces make dynamic programming computationally intractable.The approach is intended for large-scale networks without requiring statistics of network dynamics.Source: D. Problem Formulation · p. 5

IV. DEEP RL-BASED SAMPLING RATE ADAPTION AND RESOURCE ALLOCATION ALGORITHM

The method transforms the constrained problem into an MDP using Lyapunov optimization, then applies deep RL with a resource-allocation subroutine. Accuracy-deficit queues track constraint satisfaction, and the transformed objective is drift-plus-cost minimization.

IV. DEEP RL-BASED SAMPLING RATE ADAPTION AND RESOURCE ALLOCATION ALGORITHM

  • A deep RL algorithm solves the transformed MDP, with an optimization subroutine directly obtaining optimal edge-computing resource allocation.The subroutine is embedded to reduce training complexity.Source: §IV · pp. 5–6

A. Lyapunov-Based Problem Transformation

  • Lyapunov optimization transforms the long-term accuracy constraints into accuracy-deficit queues that track whether each service meets its requirement.The queues guide the learning agent toward satisfying the long-term constraints.Source: A. Lyapunov-Based Problem Transformation · p. 6
  • The transformation replaces the CMDP with per-slot minimization of drift-plus-cost.The cost combines service-delay minimization with the status of the long-term accuracy constraints.Source: A. Lyapunov-Based Problem Transformation · p. 6
  • When an accuracy constraint is violated, improving instantaneous accuracy becomes more urgent than reducing service delay.The parameter V adjusts the tradeoff between delay minimization and accuracy-constraint satisfaction.Source: A. Lyapunov-Based Problem Transformation · p. 6

B. Equivalent MDP

The equivalent MDP incorporates accuracy-deficit queues into the state transition while retaining the CMDP action. Its reward is adjusted to minimize drift-plus-cost, and the enlarged transition remains Markovian.

B. Equivalent MDP

  • Accuracy-deficit queue backlogs are incorporated into the state, and the modified state transition includes their evolution while remaining Markovian.The action stays the same as in the CMDP; the reward is modified to minimize drift-plus-cost in each time slot.Source: B. Equivalent MDP · p. 6
  • The equivalent MDP is formed from problem P0 after modifying its action, state, reward, and state transition to account for accuracy-deficit queues.The constant term sum of service accuracy requirements is omitted from the reward for brevity.Source: B. Equivalent MDP · p. 6
  • Dynamic programming for the MDP still faces a curse of dimensionality because of its large state space, motivating a deep RL solution.Source: B. Equivalent MDP · p. 6

C. Optimization Subroutine for Edge Computing Resource Allocation

The edge-computing allocation subroutine exploits the fact that resource allocation affects one-shot delay but not inference accuracy. It computes the optimal allocation after sampling-rate and offloading decisions, reducing RL training complexity.

C. Optimization Subroutine for Edge Computing Resource Allocation

  • Edge computing resource allocation is independent of inference accuracy and affects only one-shot service delay.Once task offloading and sampling-rate choices are made, the optimal allocation is obtained by solving a separate optimization problem.Source: C. Optimization Subroutine for Edge… · p. 7
  • The allocation optimization accounts for AP task-processing and queuing delays, and convexity analysis yields the optimal allocation for each time slot.The aggregate delay across devices is equivalent to the aggregate delay across services.Source: C. Optimization Subroutine for Edge… · p. 7
  • Embedding the allocation subroutine removes the need to train neural networks to learn the optimal edge-computing resource-allocation policy, reducing training complexity.Source: C. Optimization Subroutine for Edge… · p. 7

D. Deep RL-based Algorithm

The proposed algorithm extends DDPG by embedding an optimization subroutine for edge-computing allocation. An actor selects sampling and offloading actions, while a critic evaluates them; experience replay and soft target-network updates support training.

D. Deep RL-based Algorithm

  • The proposed deep RL algorithm extends DDPG [23], embedding an optimization subroutine for edge-computing allocation to reduce training complexity.The AP can deploy the algorithm, collect network-state information, and enforce the policy across connected industrial IoT devices.Source: D. Deep RL-based Algorithm · p. 7
  • An actor selects sampling-rate and task-offloading actions from the current state, while a critic evaluates actions using reward feedback.The optimization subroutine supplies the edge-computing resource-allocation action; the joint action is executed and its transition stored in replay memory.Source: D. Deep RL-based Algorithm · p. 7
  • Training uses randomly sampled minibatches from experience replay to train the critic and actor, with target networks softly updated using ratio δ.The target-network update ratio satisfies 0 < δ ≪ 1.Source: D. Deep RL-based Algorithm · pp. 7–8

V. SIMULATION RESULTS

The simulations model two bearing-vibration fault-diagnosis services with distinct data rates and accuracy requirements. The reported figures cover training performance and service delay and accuracy across task arrival rates; the listed benchmarks are delay-myopic and static configurations.

V. SIMULATION RESULTS

  • The listed figures address training-stage performance, service delay versus task arrival rate, and inference accuracy versus task arrival rate.Source: Figure 3 · p. 8, Figure 4 · p. 9, Figure 5 · p. 9
  • Type I uses 768-kilobit tasks from 48 KHz, 16-bit bearing signals with a 0.8 accuracy requirement; Type II uses 512-kilobit, 32 KHz data and requires 0.9 accuracy.Source: A. Simulation Setup · p. 8
  • Sampling rates are 25%, 50%, 75%, and 100% of raw rate, with corresponding measured accuracies of 0.59, 0.884, 0.950, and 0.987.Source: A. Simulation Setup · p. 8
  • Task arrivals per device and time slot follow U(λ − 0.5, λ + 0.5), where λ is the average task arrival rate.The balance parameter V is set to 0.05 based on extensive simulations.Source: A. Simulation Setup · pp. 8–9
  • The benchmarks are delay-myopic decisions maximizing one-step reward and a static sampling-rate and offloading configuration that guarantees service accuracy requirements.Source: A. Simulation Setup · p. 9

B. Performance Evaluation

The proposed algorithm reduces service delay against benchmark schemes while maintaining both services’ long-term accuracy requirements with high probability. Results also show convergence during training and benefits from local execution and optimized edge resource allocation.

1) Convergence of the proposed algorithm:

  • After 1,000 training episodes, accuracy converges to the predetermined requirements for both services as average service delay decreases during training.The authors report that the delay trend validates convergence of the proposed algorithm.Source: 1) Convergence of the proposed algorithm · p. 9

3) Impact of communication bandwidth:

4) Impact of optimization subroutine:

  • The optimization subroutine’s delay-reduction gain over proposed-fixed falls from 1.98× at 1 GHz to 1.02× at 1.2 GHz.The results support its effectiveness in resource-constrained settings and show that it reduces RL training complexity.Source: Figure 7 · p. 9, 4) Impact of optimization subroutine · p. 10

2) Impact of task arrival rate:

  • The proposed algorithm reduces average service delay by 19% versus delay myopic and 25% versus static configuration schemes.The comparison is evaluated across task arrival rates at W = 20 MHz.Source: 2) Impact of task arrival rate · p. 9
  • Both services meet their long-term accuracy requirements of 0.8 and 0.9 with high probability; maximum error probability is less than 0.5%.The reported accuracy distribution is evaluated across task arrival rates.Source: 2) Impact of task arrival rate · p. 9

VI. CONCLUSION

  • The conclusion states that the algorithm optimizes service delay online without requiring statistical information about network dynamics.The authors identify device mobility’s impact on inference performance as future work.Source: §VI · p. 10

APPENDIX

The proof establishes convexity, derives an optimizer using a Lagrange function and KKT conditions, and verifies positivity and complementary slackness.

A. Proof of Theorem 1

  • The proof establishes convexity because the objective’s second-order derivative is positive and the inequality constraint is linear.The text identifies the second-order derivative as 2Λ_m/(c_m f_b).Source: A. Proof of Theorem 1 · p. 10
  • A Lagrange function without the inequality constraints and the KKT conditions are used to derive the optimal value of a.The proof substitutes the resulting expression into complementary slackness.Source: A. Proof of Theorem 1 · p. 10
  • The derived optimizer is positive, satisfying the cited nonnegativity constraint; complementary slackness is then used to complete the proof.The text concludes that the resulting c*_m values are positive.Source: A. Proof of Theorem 1 · p. 10

Related papers

Loading 2301.00130v1…