Source-linked AI summary
Deep Reinforcement Learning for Collaborative Edge Computing in Vehicular Networks
Mushu Li, Jie Gao, Lian Zhao, Xuemin Shen
TL;DR
The paper addresses energy-inefficient, time-consuming computing and the complexity of server assignment in vehicular edge networks. It develops collaborative computing with MDP-based deep reinforcement learning, reporting reduced latency, improved reliability, and adaptation to high mobility.
Problem
Vehicular computing is energy-inefficient and time-consuming, while increased computing capability complicates server assignment.
Method
The paper develops a collaborative computing framework and formulates offloading and computing as an MDP addressed with deep reinforcement learning.
Results
The collaborative computing framework reduces computing service latency, improves service reliability, and supports proactive operation under high mobility.
Takeaways & Limitations
The approach can offer low-latency and high-reliable edge computing services to vehicle users.
Takeaways & Limitations
The framework is limited by the information collected by edge servers.
Abstract
from arXiv · showhide
Mobile edge computing (MEC) is a promising technology to support mission-critical vehicular applications, such as intelligent path planning and safety applications. In this paper, a collaborative edge computing framework is developed to reduce the computing service latency and improve service reliability for vehicular networks. First, a task partition and scheduling algorithm (TPSA) is proposed to decide the workload allocation and schedule the execution order of the tasks offloaded to the edge servers given a computation offloading strategy. Second, an artificial intelligence (AI) based collaborative computing approach is developed to determine the task offloading, computing, and result delivery policy for vehicles. Specifically, the offloading and computing problem is formulated as a Markov decision process. A deep reinforcement learning technique, i.e., deep deterministic policy gradient, is adopted to find the optimal solution in a complex urban transportation network. By our approach, the service cost, which includes computing service latency and service failure penalty, can be minimized via the optimal workload assignment and server selection in collaborative computing. Simulation results show that the proposed AI-based collaborative computing approach can adapt to a highly dynamic environment with outstanding performance.
I. INTRODUCTION
MEC-enabled vehicular networks address limited vehicle computation capacity but face dynamic connectivity, service interruption, and difficult server-assignment tradeoffs. The paper proposes collaborative computing with task scheduling and CNN-based DDPG to reduce latency and improve reliability.
- Motivation: MEC lets resource-constrained vehicles offload computation-intensive tasks to capable edge servers through V2I communications.Compared with mobile cloud computing, MEC reduces offloading delay because edge servers are near vehicles.
- Challenges: Dynamic topology can move vehicles out of server coverage during computation, causing service disruption and making reliable server selection difficult.Dense urban infrastructure can also create frequent radio-association changes and increase computing service latency.
- Challenges: Cooperation among edge servers can accelerate processing through parallel task execution or forwarding, but multi-hop communication adds transmission delay and spectrum overhead.The resulting tradeoff between communication overhead and computing capability complicates server assignment.
- Proposed Framework: The proposed framework distributes workloads among edge servers and selects result-delivery servers so vehicles can receive results without mobility-induced disruption.An edge server may partially or fully distribute an offloaded workload to another edge server to reduce computing latency.
- Proposed Framework: TPSA partitions workloads and schedules task execution using a near-optimal, low-time-complexity solution given the offloading strategy.The scheme targets minimized computing time for all received tasks in the network.
- AI-Based Control: The offloading and computing problem is formulated as an MDP and addressed with CNN-based DDPG to handle stochastic transitions and high-dimensional states and actions.The approach is model-free and targets offloading strategy and MEC server assignment in a two-dimensional transportation system.
II. RELATED WORKS
Prior work addresses edge-server selection, task partitioning, scheduling, service migration, and cooperative computing, but often omits mobility-aware reliability or relies on communication-intensive result delivery. This paper combines task partition and scheduling with collaborative edge-server decisions to improve reliability while reducing communication overhead.
- Existing MEC studies formulate edge-server selection to optimize computing delay, energy consumption, or computing cost.
- Task partitioning enables computing tasks to be split across multiple servers, while prior cooperation studies generally do not address user mobility.
- Prior work also evaluates task execution ordering, commonly formulating scheduling as mixed-integer programming and using heuristic algorithms for near-optimal solutions.
- Service migration and server cooperation have been studied to mitigate disruption when vehicles leave server coverage, including trajectory prediction and vehicle-to-vehicle result dissemination.
- Unlike these approaches, the proposed framework improves reliability through collaboration and task scheduling among edge servers without cooperative transmission, reducing result-delivery communication overhead.
III. SYSTEM MODEL
The system models an MEC-enabled vehicular network in which a centralized controller assigns receiver, helper, and deliver RSUs for zone-based tasks. Each task session comprises offloading, collaborative computing, and result delivery under mobility-aware coverage constraints.
- The network contains vehicles, RSUs, vehicle trajectories, workload-offloading flows, and result-delivery flows within RSU coverage areas.
- A global controller centrally makes offloading and computing decisions for all vehicles with full knowledge of the transportation network.
- Each computing session has three stages: vehicle-to-receiver offloading, receiver/helper collaborative computing, and result delivery by a reachable RSU.
- The deliver RSU is restricted to the receiver or helper RSU to reduce delivery overhead.
- Roads are divided into equal-length zones, and vehicles in the same zone share one offloading and computing policy.
- Binary decisions identify receiver, helper, and deliver RSUs for each zone and time slot.
B. Cost Model
The cost model represents service delay for partitioned tasks processed cooperatively by receiver and helper RSUs. It accounts for offloading, forwarding, queuing, and processing delays, with task completion governed by the slower computational branch.
- Task processing partitions workload x_z,t to the receiver RSU and 1 − x_z,t to the helper RSU.
- Result-delivery transmission delay is neglected because output data is usually much smaller than input data.
- Offloading and forwarding delays depend on wireless data rates, distances, bandwidth, transmit power, noise, and signal-to-noise constraints.
- Queuing delay depends on previously scheduled tasks and the processing order assigned to tasks sharing an RSU.
- Processing delay depends on the RSU computing capability and the number of computation cycles required per input bit.
- Overall service delay is determined by the longer completion time between the receiver and helper RSUs.
2) Service Failure Penalty:
Vehicle mobility can make result delivery fail when a vehicle leaves its deliver RSU’s coverage during a service session. The formulation therefore combines delay and failure penalty while optimizing server selection, workload partitioning, and scheduling.
- Vehicle mobility introduces uncertainty in downloading computing results and can cause service failure outside the deliver RSU’s coverage.
- The model identifies the vehicle’s result-delivery zone at time t + T^Service_z,t and evaluates delivery using a signal-to-noise threshold.
- The system objective minimizes a weighted sum of computing service delay and service-failure penalty.
- The optimization jointly includes receiver, helper, and deliver-server selection, task partitioning, and task execution scheduling.
- The mixed-integer nonlinear problem is difficult for conventional optimization and too high-dimensional for direct model-free methods.
- The problem is divided into task partition-and-scheduling and edge-server-selection subproblems, with DRL used for the latter policy.
V. TASK PARTITION AND SCHEDULING
The section develops TPSA to jointly partition workloads and schedule task execution across receiver and helper RSUs, reducing service latency despite a mixed-integer, NP-hard optimization problem.
- TPSA addresses workload allocation and execution ordering for tasks offloaded to edge servers, targeting minimum service latency under a given offloading policy.The formulation includes a continuous partition ratio and integer scheduling variables.
- The scheduling problem is NP-hard because, even with workload partitioning fixed, the remaining integer problem varies from the traveling salesman problem.The mixed-integer structure combines continuous allocation with discrete execution ordering.
- Under stated assumptions and a fixed execution order, the partition ratio from Lemma 1 is optimal for the workload-allocation problem.The assumptions include identical server computing capability, distinct receiver and helper RSUs, and helper queuing time no shorter than offloading time.
- With one available RSU and offloading time proportional to task size, shortest service time first is the optimal execution order.This result applies to the single-RSU case described in Lemma 3.
- TPSA uses task-partition properties and heuristic scheduling to efficiently obtain an approximate solution rather than exhaustive optimization.The algorithm schedules the shortest-service-time task first and divides each workload between receiver and helper RSUs using the derived partition ratio.
VI. AI-BASED COLLABORATIVE COMPUTING APPROACH
The paper formulates collaborative offloading and computing as an MDP and uses CNN-based DDPG to select servers in a high-dimensional vehicular network, followed by TPSA for partitioning and scheduling.
- The server-selection problem is formulated as an MDP with state, action, transition, and cost components.The state includes zone workloads, vehicle speeds, and prior task-completion delays; actions select receiver, helper, and delivery RSUs.
- DDPG is adopted to obtain the collaborative computing policy when dynamic, high-dimensional state and action spaces make traditional dynamic programming inefficient.The critic evaluates Q values while the actor updates policy parameters in the critic-suggested direction.
- After DDPG selects servers, TPSA computes the corresponding task partition and scheduling policy and provides the state-action cost.The resulting policy is evaluated through long-term discounted cost minimization.
- The DDPG implementation uses evaluation and target actor-critic networks, experience replay, and periodic or soft target-network updates.The four networks separately support value and policy evaluation and stabilization of training.
- CNN layers are used in both actor and critic networks to exploit correlations among zones and reduce training parameters through weight sharing.Convolution and pooling layers learn relevant input features before fully connected layers.
VII. PERFORMANCE EVALUATION
The evaluation first tests TPSA for task partitioning and scheduling, then assesses the AI-based collaborative computing approach in a VISSIM-simulated vehicular network using DDPG-selected policies.
- The experiments evaluate TPSA’s efficiency in task partitioning and scheduling before evaluating the AI-based collaborative computing approach.TPSA schedules computing tasks according to the policy generated by DDPG in the vehicular-network simulation.
A. Task Partition and Scheduling Algorithm
TPSA achieves service-delay performance close to brute force while avoiding its rapidly increasing runtime as task count grows.
- The TPSA evaluation compares the proposed algorithm with brute-force and random scheduling schemes using the optimal partition ratio for workload allocation.Results are averaged over 200 Monte Carlo simulation rounds.
- As task number increases, overall service time increases, with the random scheme showing the highest increasing rate among the three schemes.This comparison is reported for the average service-delay evaluation in Fig. 5(a).
- TPSA achieves performance very close to the brute-force scheme for service delay.The brute-force scheme exhaustively searches for the optimal scheduling order, whereas random scheduling assigns task order randomly.
- As task number increases, brute-force runtime grows exponentially, whereas TPSA has imperceptible runtime while producing a scheduling result close to the optimal one.The paper summarizes TPSA as near-optimal with low computation complexity.
- The DDPG comparison includes a Greedy approach that always offloads tasks to the RSU with the highest SNR.The supplied passage identifies this baseline’s server-selection rule.
NEURAL NETWORK STRUCTURE
The proposed collaborative computing approach combines server collaboration with TPSA-based task partitioning and scheduling, while AI-based decisions select offloading, computing, and result delivery policies. Across simulations, it achieves the lowest computing cost, failure percentage, and service delay while successfully processing the most data among the compared approaches.
- Collaborative computing policies: The Random+TPSA approach randomly selects receiver, helper, and delivery RSUs, while TPSA determines task partition ratios and execution order.The proposed scheme instead uses learned decisions for collaborative computing and server selection.
- Simulation results: The proposed approach achieves the lowest weighted computing cost among the four approaches.The random approach has the highest cost because of inefficient server selection.
- Simulation results: The proposed AI-based approach achieves the lowest service failure percentage and successfully computes the highest amount of data among the four approaches.These results are reported as computing task arrival rates per vehicle vary.
- Simulation results: The proposed scheme significantly reduces service delay for 1 Mbits of successfully computed data compared with the other three approaches.The random approach's delay increases exponentially at high task arrival rates because less data is successfully computed.
- Convergence: The proposed AI-based approach converges after 10,000 episodes, corresponding to approximately 3,000 episodes or 60,000 training steps under the reported exploration procedure.The convergence plot uses a moving average from 50 samples around each point.
- Conclusion: The framework targets reliable, low-latency computing for highly mobile vehicles by learning network dynamics and supporting proactive offloading decisions in complex urban environments.The authors identify multi-agent learning under limited information collected by edge servers as future work.
APPENDIX A: PROOF OF LEMMA 2
The proof of Lemma 2 analyzes how changing a task's partition ratio affects queued tasks under the lemma's assumptions. It concludes that workload allocation must follow the prescribed partition ratio to avoid increasing overall service time.
- Conclusion: Under Lemma 2's assumptions, overall service time increases when workload allocation does not follow the partition ratio presented in Lemma 1.This establishes the optimality condition for the workload allocation used by TPSA.
- Case analysis: For subsequent tasks, the proof evaluates cases where the affected RSU is the receiver or helper RSU and derives corresponding service-time reductions.The cases distinguish whether the next task is partially or fully assigned to the receiver RSU, or uses the affected RSU as helper.
- Case analysis: When server computing capabilities are identical, the maximum service-time deduction for a queued task is ∆t_r(1)/2.The proof then bounds the total deduction across all tasks queued after task 1.
APPENDIX B: PROOF OF LEMMA 3
The proof of Lemma 3 compares task execution orders under a shortest-task-first scheduling assumption. It concludes that violating this order increases the overall service time under the stated assumptions.
- Scheduling assumption: The proof assumes tasks in an edge server are scheduled using the shortest-task-first rule.It analyzes the relative transmission and processing times of two tasks queued on the server.
- Order comparison: Switching the order of tasks 1 and 2 changes their individual service times through the corresponding queueing, transmission, and processing terms.The proof derives separate expressions for the service-time changes of task 2 and task 1 after the switch.
- Case analysis: The three cases compare task 2's total time with task 1's queueing, transmission, and processing components.In each case, the difference between the original and switched order is shown to be nonnegative.
- Conclusion: Under the assumptions, service time increases if the task execution order does not follow shortest-task-first scheduling.This is the lemma's final scheduling condition.