Source-linked AI summary
Joint Optimization of Memory and Computing Frequency for Energy-Efficient DNN Inference
Yunchu Han, Zhaojun Nan, Sheng Zhou, Zhisheng Niu
TL;DR
Mobile DNN inference is energy- and latency-intensive, while memory-frequency effects are often overlooked. The paper jointly optimizes memory and computing frequencies with communication resources, reducing energy consumption by up to 10.4% versus other methods.
Problem
Memory-frequency effects on DNN inference latency and energy are largely overlooked despite their substantial measured impact on mobile devices.
Method
The paper models DNN inference time and jointly optimizes memory frequency, computing frequency, bandwidth, and transmission power under deadline constraints.
Results
10.4% average energy savings versus No DVFS were achieved, while the near-optimal local-inference solution remained within 2.5% of optimal performance.
Takeaways & Limitations
Jointly accounting for memory and computing frequency reduces mobile-device energy consumption compared with optimizing computing frequency alone or using no DVFS.
Abstract
from arXiv · showhide
Deep neural network (DNN) inference on mobile devices often incurs high latency and energy consumption due to limited computing and memory resources. To enable energy-efficient DNN inference, most existing studies focus on dynamic voltage and frequency scaling (DVFS) for adjusting the computing frequency, while the impact of memory frequency on the inference performance has been greatly overlooked. In this paper, we consider the impact of memory frequency and computing frequency on DNN inference time, and jointly optimize these two frequencies together with communication resources for energy-efficient DNN inference. Based on a realistic inference time model, we formulate an optimization problem to minimize the energy consumption of all mobile devices under the deadline constraint. For local inference, we derive a near-optimal closed-form solution via convex optimization, while an optimal closed-form solution for transmission power is obtained for edge inference with the given bandwidth. Furthermore, we propose a low-complexity heuristic algorithm to effectively solve the overall problem with polynomial time complexity. Simulation results based on measured data show that the proposed near-optimal solution for local inference can achieve optimal performance under strict deadline constraints, with a performance gap of up to 2.5% compared with the optimal solution. Meanwhile, our proposed algorithm significantly reduces the energy consumption of devices by up to 10.4% compared to other methods.
I. INTRODUCTION
The introduction motivates jointly optimizing memory frequency, computing frequency, and communication resources for energy-efficient DNN inference on resource-constrained mobile devices. Measurements show that memory-frequency scaling can substantially reduce inference time, while the paper develops closed-form and heuristic optimization methods for local and edge inference.
- Motivation: DNN inference challenges mobile devices because intensive computation and memory access consume limited energy and computing resources.Mobile edge computing is presented as a way to support low-latency, energy-efficient inference through task offloading.
- Limitations of Existing Approaches: Existing DVFS approaches primarily adjust computing frequency, such as CPU or GPU frequency, to trade off inference latency and energy consumption.The introduction notes that prior work applies DVFS to local inference and resource allocation.
- Memory-Frequency Impact: Increasing memory frequency from 0.1 GHz to 1.6 GHz reduces average inference time by 84% for ResNet152 and 93% for VGG19 on Jetson TX1.The measurements motivate considering memory frequency, especially for memory-intensive DNN models.
- Proposed Approach: The paper jointly optimizes memory frequency, computing frequency, and communication resources to reduce mobile-device energy consumption.It considers an edge intelligence system spanning local and edge inference.
- Proposed Approach: For local inference, the paper derives optimal solutions in special cases and a near-optimal closed-form solution using an upper-bound analysis.The contribution targets low-complexity optimization of the local-inference problem.
II. SYSTEM OVERVIEW · A. System Model
The system comprises N mobile devices and an edge server for deadline-constrained DNN inference, using OFDMA and jointly optimizing memory and computing frequencies with transmission power and bandwidth. Edge inference energy and result-download latency are excluded under stated assumptions.
- A. System Model: The edge intelligence system consists of N mobile devices and one edge server.The mobile-device set is denoted N ≜ {1, 2, . . ., N}.
- A. System Model: Each mobile device must execute its DNN inference task within deadline Dn.
- A. System Model: OFDMA is adopted to reduce interference among mobile devices.
- A. System Model: Bandwidth allocated to each device is assumed not to overlap with other devices’ bandwidth.
- A. System Model: The system jointly optimizes memory and computing frequencies, transmission power, and bandwidth.This configuration is illustrated in Fig. 2.
- A. System Model: Edge-inference energy is ignored because the MEC server is usually powered by the grid, and downloading inference results is assumed to add negligible latency due to their small data volume.
B. Inference Time and Energy Consumption Model
This section develops an inference-time and energy-consumption model using measured data across memory and computing frequencies. It represents inference time as a frequency-dependent function and decomposes device power into memory and computing components.
- Inference-time model: Real-world experiments measure average DNN inference time under combinations of memory and computing frequencies.These measurements provide the basis for modeling the effects of f_n,mem and f_n,com on inference time.
- Inference-time model: Inference time is modeled as a function of memory frequency f_n,mem and computing frequency f_n,com for each device n ∈ N.The model is given by Equation (1), with parameters determined from the experimental data.
- Power model: CMOS dynamic power follows P = αCV^2f, where α, C, V, and f denote activity factor, capacitance, supply voltage, and frequency.The device power model uses this relationship as its foundation.
- Energy model: Mobile-device inference power is decomposed into memory power and computing power, and its energy consumption is then formulated from the execution power and inference duration.The detailed power and energy expressions are provided in the cited model.
C. Transmission Time and Energy Consumption Model
The model characterizes task offloading through transmission rate, latency, and energy consumption for each mobile device. Transmission rate depends on allocated bandwidth, transmission power, channel gain, and noise power, while latency also depends on task data volume.
- Transmission Time and Energy Consumption Model: Transmission rate for mobile device n is modeled using its communication parameters.The parameters include allocated bandwidth, transmission power, channel gain, and noise power.
- Transmission Time and Energy Consumption Model: Offloading latency is determined from the task’s data volume and transmission rate.The task data volume is denoted by s_n.
- Transmission Time and Energy Consumption Model: The model also defines the corresponding transmission energy consumption for each mobile device.This energy term is associated with transmitting the computation task during offloading.
III. PROBLEM FORMULATION AND SOLUTIONS
The paper formulates energy minimization over offloading, communication, memory-frequency, and computing-frequency decisions under deadlines, yielding a MINLP. It then derives tractable local- and edge-inference solutions and a heuristic for the overall problem.
- Problem formulation: The optimization minimizes total mobile-device energy by jointly selecting offloading decisions, transmission power, bandwidth, memory frequency, and computing frequency subject to deadlines.The binary offloading variables make the formulation a mixed-integer nonlinear programming problem.
- Local inference: For local inference, the frequency-allocation problem is convex when 0 ≤ b_n ≤ 2 and 0 ≤ d_n ≤ 2, enabling solution with standard convex optimization tools.The constraints are convex, while the objective is not always convex.
- Local inference: A deadline-based convex approximation yields closed-form optimal memory and computing frequencies derived from Karush-Kuhn-Tucker conditions.The resulting frequencies are restricted to their feasible ranges.
- Edge inference: For edge inference with fixed bandwidth, the optimal transmission power is obtained in closed form because the objective increases monotonically with transmission power.The solution takes the larger of the deadline-required power and the minimum allowable power.
- Heuristic algorithm: The heuristic equally allocates bandwidth among offloading devices, computes optimal transmission powers, and removes the device requiring the highest power when the power budget is violated.It then computes near-optimal local frequencies for the remaining devices and repeats until termination.
IV. SIMULATION RESULTS
Simulations show that jointly optimizing memory and computing frequencies reduces energy consumption, while the near-optimal local-inference solution closely matches optimal performance under tight deadlines. Frequency reductions eventually plateau as deadlines grow, and the proposed algorithm outperforms benchmark policies across bandwidth and device settings.
- Local inference: Under relatively small deadlines, the near-optimal local-inference solution achieves optimal energy performance.The evaluation uses Jetson TX1 with ResNet152 and VGG19.
- Local inference: As the deadline increases, near-optimal memory and computing frequencies decrease, whereas optimal frequencies remain constant beyond a threshold.Lower-frequency power savings no longer compensate for increased inference time beyond the threshold.
- Local inference: 15.4% and 13.7% are the gaps between optimal and near-optimal memory frequencies for ResNet152 at D_n = 0.32 s and VGG19 at D_n = 0.22 s, respectively.These values are reported under the specified deadline conditions.
- Overall algorithm: 3.6% and 10.4% are Algorithm 1's average energy savings versus Only Compute and No DVFS, respectively.Increasing total bandwidth offloads more tasks to the edge server and improves energy consumption for all policies.
- Overall algorithm: Algorithm 1 significantly outperforms Random, while policy gaps narrow when device counts are small or available bandwidth is sufficiently large.The Random policy fails to effectively exploit channel characteristics.
V. CONCLUSION
The paper jointly optimizes memory frequency, computing frequency, bandwidth, and transmission power for energy-efficient DNN inference. Analytical modeling and convex optimization yield optimal or near-optimal closed-form solutions for local inference and transmission power.
- V. CONCLUSION: The study jointly optimizes memory frequency, computing frequency, bandwidth, and transmission power for energy-efficient DNN inference.The approach integrates analytical modeling of DNN inference time with convex optimization methods.
- V. CONCLUSION: For local inference, it derives an optimal solution under specific cases and a near-optimal closed-form solution for general cases.
- V. CONCLUSION: With given bandwidth, the paper derives an optimal closed-form solution for transmission power.