Source-linked AI summary
A Survey of Methods For Analyzing and Improving GPU Energy Efficiency
Sparsh Mittal, Jeffrey S. Vetter
TL;DR
High GPU performance has brought substantial power consumption, creating a need for improved energy efficiency and better understanding of GPU power management. The paper surveys and classifies research on analyzing and improving GPU energy efficiency, reviews comparisons with alternative computing systems, and identifies future research directions. The survey concludes that platform efficiency depends on the application, devices, and evaluation methodology, while GPU efficiency still requires improvement for exascale systems.
Problem
GPU power consumption has increased with performance demands, while GPU power management remains less explored despite its importance for reliability, scaling, and deployment.
Method
The paper surveys and classifies GPU energy-efficiency research, compares GPUs with CPUs, Cell processors, FPGAs, and other systems, and limits coverage to evaluated application- and architecture-level techniques.
Results
Across most studies, FPGAs are more energy efficient than GPUs and GPUs are more energy efficient than CPUs, but no single platform is most efficient for all applications; GPUs also require further improvement for exascale.
Takeaways & Limitations
The survey provides state-of-the-art insights, highlights similarities and differences among techniques, and identifies trends for designing more energy-efficient GPUs.
Takeaways & Limitations
The survey excludes studies showing only performance improvement, circuit-level techniques, and a uniform presentation of results across differing platforms and methodologies.
Abstract
from arXiv · showhide
Recent years have witnessed a phenomenal growth in the computational capabilities and applications of GPUs. However, this trend has also led to dramatic increase in their power consumption. This paper surveys research works on analyzing and improving energy efficiency of GPUs. It also provides a classification of these techniques on the basis of their main research idea. Further, it attempts to synthesize research works which compare energy efficiency of GPUs with other computing systems, e.g. FPGAs and CPUs. The aim of this survey is to provide researchers with knowledge of state-of-the-art in GPU power management and motivate them to architect highly energy-efficient GPUs of tomorrow.
1. INTRODUCTION
GPU adoption in high-performance computing has increased alongside substantial power consumption, making GPU power management important. This survey reviews and classifies energy-efficiency research, compares GPUs with alternative platforms, and defines a scoped review approach.
- GPUs support high-performance computing, but their performance-oriented design has driven increased power consumption.Titan, a CPU-GPU supercomputer, consumes 8.2MW.
- GPU power consumption affects reliability, economic feasibility, architecture design, performance scaling, and deployment across applications.
- Exascale power demands make power management essential for CPUs and GPUs, while GPU power management remains less explored than CPU power management.An exascale machine built with current supercomputer technology has been estimated to consume several gigawatts.
- The survey reviews GPU energy-efficiency research, classifies techniques by important features, and compares GPUs with CPUs, Cell processors, FPGAs, and other systems.These comparisons are intended to help readers judge GPU energy efficiency against alternative computing platforms.
- The review includes studies analyzing GPU power consumption or evaluating GPU energy efficiency, covers application- and architecture-level techniques, and excludes circuit-level techniques.Because studies use different platforms and methodologies, the survey focuses on key ideas rather than presenting all results uniformly.
2. BACKGROUND
GPU power consumption arises from leakage and dynamic components, while inefficient resource use, reliability risks, cooling costs, and scaling constraints motivate power management. Energy efficiency also influences GPU adoption relative to CPUs and FPGAs and supports sustainable computing.
- GPU Terminology and Sources of Power Consumption: GPU power consists of leakage power consumed while powered and dynamic power arising from runtime transistor switching.SMs and local, global, and shared memories contribute to GPU power consumption.
- Addressing Inefficient Resource Usage: GPU over-provisioning, memory-bandwidth bottlenecks, and limited scaling of regular parallel applications can leave resources underutilized and reduce energy efficiency.
- Ensuring Reliability: A 15°C temperature increase can raise component failure rates by up to a factor of two, linking power consumption to reliability risks.Failures can have serious economic effects in GPU-dependent services.
- Providing Economic Gains: For each watt dissipated by computing equipment, cooling consumes an additional 0.5 to 1W, and cluster energy costs can exceed hardware acquisition costs.
- Enabling Performance Scaling: Thermal and leakage constraints may prevent simultaneous use of all cores, while complex cooling can increase chip complexity and offset GPU performance benefits.
- GPU Adoption and Sustainability: GPU energy efficiency affects adoption decisions because FPGAs have shown better performance and energy efficiency than GPUs for several applications.The relative outcome depends on the computing platform and application.
- Achieving the Goals of Sustainable Computing: Improving GPU energy efficiency is presented as relevant to sustainable computing amid projected growth in ICT carbon emissions.ICT carbon emissions were estimated to triple from 2002 to 2020.
3. RESEARCH WORKS ON ANALYZING GPU ENERGY EFFICIENCY
The surveyed comparisons do not identify one universally most energy-efficient platform: results depend on applications, devices, and evaluation methodology. Across studies, FPGAs often outperform GPUs in energy efficiency, while GPU-versus-CPU outcomes vary substantially.
- Overview: The survey compares GPU energy efficiency with CPUs, Cell processors, FPGAs, and other computing systems using prior studies.
- Overview: Across most studies, FPGAs are more energy efficient than GPUs and GPUs are more energy efficient than CPUs, but no platform is best for every application.The results depend crucially on the devices and evaluation methodology.
- Discussion: 20 pJ per floating-point operation is required for a 20MW exascale machine, versus 225 pJ for state-of-the-art GPUs and 1700 pJ for CPUs.The comparison indicates that GPU efficiency requires further improvement for the exascale challenge.
- Discussion: For fast multipole computation, an optimized CPU is nearly 2.4× as energy-efficient as a one-GPU system and 1.8× as energy-efficient as a two-GPU system.
- Discussion: For matched filtering, Cell and FPGA systems outperform GPUs in performance and energy efficiency, while GPUs outperform CPUs on both measures.
- Discussion: For Viola-Jones face detection, four GPUs provide comparable performance to one FPGA, but the FPGA design has orders-of-magnitude better energy efficiency.
- Discussion: For geometric algebra and k-means, GPUs are less energy efficient than FPGAs but more energy efficient than CPUs; data-transfer overhead further degrades GPU efficiency for geometric algebra.
- Discussion: For random forest classification, FPGA performance per watt is an order of magnitude better than GPU performance per watt, which is better than CPU performance per watt.The FPGA requires a multi-board system even for modest-size problems, increasing cost.
4. TECHNIQUES FOR IMPROVING GPU ENERGY EFFICIENCY
The survey classifies GPU energy-efficiency techniques and reviews methods spanning voltage/frequency scaling, CPU-GPU coordination, application behavior, and component-level management.
- Classification: The survey classifies energy-efficiency techniques by their main research ideas and discusses overlapping techniques in one category for clarity.The categories include DVFS, CPU-GPU workload division, component-level methods, and application-specific or programming-level management.
- DVFS-Based Techniques: DVFS saves power by reducing frequency and the corresponding supply voltage, but scaling must balance energy savings against performance loss.GPU and CPU frequency scaling are both included when they save energy at the system level.
- DVFS-Based Techniques: GPU power depends on application characteristics such as the ratio of global-memory transactions to computation instructions and instruction-issue rate.Processor and memory frequencies can therefore be adjusted according to workload behavior.
- Application-Specific and Programming-Level Techniques: Software prefetching combined with iterative DVFS seeks a frequency that satisfies performance constraints while minimizing energy.Prefetching can reduce memory stalls but adds instructions and power consumption, requiring a performance-energy trade-off.
- CPU-GPU Workload Division: CPU-GPU workload division balances completion times and combines processor-frequency adjustments to reduce idle energy with minimal performance degradation.Other approaches use runtime adaptation to map computations automatically between CPU and GPU.
- Saving Energy in GPU Components: Component-level methods reduce energy through cache, memory, and access management, including LLC bypassing, local storage, and adaptive memory-bank partitioning.The surveyed examples report improved energy efficiency, while GPU LLC bypassing selectively preserves cache capacity for latency-sensitive CPU applications.
5. FUTURE RESEARCH TRENDS AND CONCLUSION
The paper identifies future GPU energy-efficiency challenges across chip, architecture, system, and infrastructure levels. It highlights 3D stacking, heterogeneous memory, coordinated scheduling, and virtualization as important directions.
- Chip and Architecture-Level Directions: Future GPUs require energy-efficient throughput cores and memory designs that exploit instruction-, data-, and fine-grained task-level parallelism.The paper places these opportunities at the chip-design level.
- Chip and Architecture-Level Directions: CPU-GPU integration with unified memory and intelligent scheduling or work division is needed to combine their complementary competencies.These directions span architecture and system levels.
- Emerging Memory Technologies: 3D die stacking can mitigate GPU memory-bandwidth bottlenecks through shorter, high-bandwidth, power-efficient interconnects and denser integration.It also enables heterogeneous GPU-memory technologies.
- Emerging Memory Technologies: Non-volatile memories such as PCM and STT-RAM offer negligible leakage power and higher density, but higher write latency and energy require new device, architecture, and system solutions.The paper presents these benefits and trade-offs as a major future research opportunity.
- Large-Scale Systems: Large-scale GPU systems will require intra-node and inter-node power management that accounts for load balancing, temperature reduction, and performance targets across heterogeneous configurations.The challenge grows as GPUs enter data centers and supercomputers.
- Virtualized GPU Infrastructure: Virtualization remains an open direction for minimizing the power and total cost of ownership of GPU computing infrastructure.The paper notes that GPUs are increasingly used in cloud-computing and virtual-machine platforms.
- Conclusion: The survey synthesizes GPU energy-efficiency methods, classifies their similarities and differences, and identifies trends for future green GPU research.Its stated aim is to inform researchers about the state of GPU power management.