Source-linked AI summary
How to estimate carbon footprint when training deep learning models? A guide and review
Lucia Bouza Heguerte, Aurélie Bugeau, Loïc Lannelongue
TL;DR
Training and deploying deep-learning models has environmental costs, but practitioners face difficulties choosing and interpreting energy-estimation tools. The paper reviews and compares seven tools across models and infrastructures, finding that estimates vary with measurement scope and implementation choices and using these findings to provide tool-selection guidance. It also identifies important boundaries around carbon-intensity timing and environmental impacts beyond training energy.
Problem
Practitioners need clearer guidance on the vocabulary, use, and suitability of tools for estimating deep-learning training energy and greenhouse-gas emissions.
Method
The paper reviews seven estimation tools, explains their technical requirements, and compares them across two image-processing experiments, infrastructures, and wattmeter measurements.
Results
Estimates vary because tools measure different scopes and handle idle power, usage factors, CPUs, and GPUs differently.
Takeaways & Limitations
Tool choice should depend on the infrastructure and whether practitioners want to measure the whole node or only the process.
Takeaways & Limitations
Most tools do not use real-time carbon intensity, and the paper does not cover many life-cycle impacts such as resource depletion, ecotoxicity, and embodied emissions.
Abstract
from arXiv · showhide
Machine learning and deep learning models have become essential in the recent fast development of artificial intelligence in many sectors of the society. It is now widely acknowledge that the development of these models has an environmental cost that has been analyzed in many studies. Several online and software tools have been developed to track energy consumption while training machine learning models. In this paper, we propose a comprehensive introduction and comparison of these tools for AI practitioners wishing to start estimating the environmental impact of their work. We review the specific vocabulary, the technical requirements for each tool. We compare the energy consumption estimated by each tool on two deep neural networks for image processing and on different types of servers. From these experiments, we provide some advice for better choosing the right tool and infrastructure.
1 Introduction
This guide addresses the difficulty of estimating the environmental impact of deep-learning training by reviewing tools, infrastructures, and measurement concepts, then comparing their estimates across experiments.
- 1 Introduction: The guide aims to help practitioners overcome hardware vocabulary, tool-use, and tool-selection difficulties when measuring energy and associated greenhouse-gas emissions.Its focus is training deep-learning models on different infrastructures, although the discussion can apply more broadly.
- 1 Introduction: The paper reviews seven tools for estimating training energy consumption and explains their measurement scope, infrastructure requirements, usage factors, defaults, and information sources.The tools studied are Green-Algorithms, CodeCarbon, Eco2AI, CarbonTracker, Experiment-Impact-Tracker, MLCO2, and Cumulator.
- 1 Introduction: It compares these approaches with wattmeter measurements and tests them on MNIST digit classification and DNCNN image denoising across several infrastructures.The first experiment lasts approximately 1 minute, while the second lasts approximately 2 hours.
- 1 Introduction: Figures 1 and 2 report energy consumption for different tools across five infrastructures, using total energy when component-level CPU, GPU, and memory detail is unavailable.The first experiment is reported in Wh and the second in kWh.
- 1 Introduction: The reported energy varies substantially because tools target different scopes, such as the whole machine or a particular process, and handle idle power, usage factors, CPUs, and GPUs differently.These design choices affect how estimates should be interpreted.
2 Related works
Prior work has compared energy and carbon-estimation tools using technical, configuration, functional, and infrastructure-related criteria, while broader environmental impacts remain only partly covered.
- 2 Related works: Only recently available estimation tools have led to relatively few comparative studies of strategies for measuring deep-learning energy consumption.This motivates further comparison and analysis of existing approaches.
- 2 Related works: One review compared six tools for natural-language processing using publication, technical, configuration, and functional criteria.The criteria included availability, ease of use, documentation, carbon intensity, PUE, installation dependence, and functionality.
- 2 Related works: Another study compared server-node tools, distinguishing external and internal sensors, power-profiling software, energy-measurement packages, and online calculators.Its criteria included publication, hardware compatibility, virtualization, and software-related functionality.
- 2 Related works: Related guidelines discuss strengths and weaknesses of online calculators, embedded packages, and server-side tools, including compatibility, ease of use, and scalability.The discussion covers different hardware, programming languages, research fields, job counts, and operating periods.
- 2 Related works: Existing tool comparisons mainly address training energy, whereas broader assessments can also include embodied impacts, static infrastructure, and deployment emissions.The paper notes that these broader impacts were included in a later assessment of BLOOM.
3 Estimating greenhouse gas emissions
The paper explains how tools estimate energy use and greenhouse-gas emissions across hardware components, communications, datacenter overhead, and electricity sources. It emphasizes that methodological choices—including usage factors, measurement frequency, PUE, and carbon-intensity data—affect interpretation and comparability.
- Energy estimates may cover the entire machine or a particular process, with differing treatment of idle power, usage factors, CPUs, and GPUs.
- Green-Algorithms and CodeCarbon are the only reviewed Python tools that report energy or emissions separately for memory, CPU, and GPU.
- Measurement frequency trades measurement quantity for approximation: CodeCarbon samples every 15 seconds, while Eco2AI, CarbonTracker, and Experiment-Impact-Tracker sample every 10 seconds.Cumulator estimates energy without querying sensors or making intermediate measurements.
- CPU and GPU energy are estimated from manufacturer TDPs or software tools, while memory models differ in whether they use available or process-allocated memory.GPU software measurement through pynvml works only for Nvidia GPUs, and some tools report whole-machine rather than process-specific consumption.
- Datacenter overhead is represented by PUE, while emissions depend on regional carbon intensity; most tools use historical averages rather than real-time values.The paper notes that execution time, energy-source distribution, datacenter location, and dataset-transfer impacts can affect emissions, and summarizes configurable defaults in its comparison tables.
4 Infrastructure
Infrastructure choice determines which energy-measurement tools can run, because tools require different hardware access, operating systems, permissions, and resource information. The guide compares these constraints across servers, clusters, personal computers, and Google Colab.
- 4 Infrastructure: Tool availability depends on infrastructure resources, including physical or virtual servers, cloud environments, supercomputers, and personal computers.Table 8 summarizes tool requirements and hardware compatibility.
- 4.1 Access to information and resources: Virtual environments may report emulated CPU information, preventing users from identifying the real processor used during execution.This makes CPU-based estimation less reliable when a tool requires the CPU model.
- 4.1 Access to information and resources: RAPL-based tools require administrator permission, Intel CPUs, and Linux, while Power Gadget is also Intel-exclusive and must be installed.These requirements restrict access on other processors, operating systems, or managed systems.
- 4.1 Access to information and resources: GPU usage factors are difficult to obtain from command-line tools, and no reviewed tool measures GPU time for non-NVIDIA GPUs.GPU utilization measured with nvidia-smi may include other processes rather than only the measured process.
- 4.2 Infrastructure types: Wattmeters measure an entire node, so concurrent jobs can make readings difficult to attribute to one process.On Grid5000, requesting only part of a node can also prevent privileges needed by some tools and alter how CPU consumption is estimated.
- 4.2.4 Colab: Google Colab restricts tool access because users lack administrator privileges, RAPL access, and knowledge of the real resources being used.Green-Algorithms, CodeCarbon, Eco2AI, and Cumulator rely on average consumption there, while CarbonTracker requires GPU runtime and omits CPU and memory energy.
5 Experiments and results analysis
Experiments across two image-processing tasks and multiple infrastructures show that tool estimates vary substantially, while hardware scale and configuration materially affect energy use. The comparisons also identify practical trade-offs involving measurement strategy, batch size, checkpointing, and data loading.
- 5.2 Tool comparison: 400% higher consumption was reported by MLCO2 than Eco2AI on the Gemini-1 node of Grid5000.
- 5.2 Tool comparison: Process-focused tools such as Eco2AI and EIT reported lower consumption than full-machine estimators CodeCarbon and CarbonTracker, so their values were not directly comparable with wattmeters.
- 5.2.2 Comparison between software tools and wattmeter: CodeCarbon most closely matched wattmeter measurements, while CarbonTracker showed greater variability across infrastructures.For the first experiment, CodeCarbon was almost exactly aligned with the wattmeter; for the second, it exceeded 80% of the wattmeter value on all infrastructures.
- 5.3 Influence of infrastructures: 1.69 kWh was reported for Denoiser training on Gemini-1 versus 1.12 kWh on the smaller Rosenblatt server, despite Gemini-1 finishing faster.The runs took 2 hours on Gemini-1 and 3 hours 16 minutes on Rosenblatt; GPU usage factors were 14.3% and 54%, respectively.
- 5.5 Influence of data loading: Only 0.5% of energy was used for loading the already-local 6 GB ImageNet validation split, excluding downloading and storage impacts.
- 5.6 Influence of batch size: An intermediate batch size of 64 provided the better energy-runtime compromise in the denoising experiment.Larger batches ran faster, but batch sizes 32 and 128 had the highest energy use in this test.
- 5.7 Influence of checkpointing: Checkpointing had no impact on energy consumption or runtime, and epoch-level measurements appeared sufficiently constant for short-run extrapolation.The checkpointing comparison used CodeCarbon and a wattmeter on Gemini-1, while the constant-epoch observation was also reported by CarbonTracker.
6 Discussions
The discussion translates the tool comparison into practical guidance for measuring and reducing training-related energy use, while clarifying that the estimates cover only part of AI’s lifecycle footprint.
- Practical measurement guidance: Online tools can estimate impacts before training, albeit less precisely, while software tools add a small measurement load and can extrapolate total training energy from a few epochs.Green-Algorithms supports estimation before or after training; measuring only selected runs or early epochs can reduce measurement overhead.
- Which tools to use: Green-Algorithms offers broad infrastructure compatibility but requires manual inputs and may be less precise, whereas CodeCarbon best estimates total machine consumption and Eco2AI or EIT target process-level measurement.CodeCarbon is more accurate when RAPL files are available; Eco2AI avoids that requirement, while EIT requires RAPL access and code modification.
- Which infrastructure to use: Because idle consumption can be substantial, practitioners should limit reserved resources, share supercomputers when suitable, and switch off computers or servers after computation.For cloud infrastructure, the paper recommends low-PUE, low-emission data centers and lower-emission execution hours.
- Scope and limitations: Training-only estimates understate complete AI-service emissions because deployment, repeated runs, equipment life-cycle impacts, data movement, and storage are also excluded or only partly modeled.The paper notes that deployment consumption can exceed training consumption and that embodied emissions have been estimated at half of training emissions under assumptions.
7 Conclusion
The paper presents and analyzes seven tools for estimating energy consumption during deep-learning training, explaining their measurement details and practical trade-offs. Its experiments focus on regular CNN image-processing workloads, but the authors expect the main findings to extend to other architectures based on prior studies.
- 7 Conclusion: The paper reviews seven energy-estimation tools, explains their technical specificities, and derives recommendations for AI practitioners.The study emphasizes both tool advantages and limitations.
- 7 Conclusion: The two experiments use regular CNNs for image processing, although prior studies suggest similar estimator behavior across other applications and workloads.The authors therefore expect the main results to hold for other architectures, while the direct experiments remain limited in scope.
A Methodologies to estimate energy consumption of CPUs and GPUs
The appendix describes two approaches for estimating CPU and GPU energy: manufacturer-rated power multiplied by training duration, and hardware or software interfaces that monitor processor power.
- A Methodologies to estimate energy consumption of CPUs and GPUs: The TDP method estimates energy in kWh by multiplying the CPU or GPU manufacturer’s rated power by training duration.TDP represents power dissipation under maximum theoretical load.
- A Methodologies to estimate energy consumption of CPUs and GPUs: RAPL and Power Gadget provide alternative power-monitoring approaches for processor components such as CPU cores, memory controllers, and GPUs.RAPL exposes hardware energy counters through Linux powercap files, while Power Gadget uses Intel’s standalone monitoring software.
B Bugs fix of some software tools
Some software tools require modifications before use because they contain unresolved bugs, and the appendix lists the necessary changes.
- B Bugs fix of some software tools: Several tools must be modified before use because their authors have not fixed identified bugs.The appendix provides tool-specific changes.
B.1 Experiment-Impact-Tracker
The Experiment-Impact-Tracker setup requires package and import corrections, along with changes to logging and log-folder handling.
- The PyPi package is outdated and does not correspond to its documentation, requiring correction.
- Long runs require reducing the INFO log level to ERROR and restricting results to the determined logs folder.
- The package requires correcting imports in base.py because its file structure differs from the package structure.
B.3 CarbonTracker
CarbonTracker requires correcting a deprecated Python 3.10 decode function in its NVIDIA GPU component.
- CarbonTracker requires correcting the deprecated decode function in Python 3.10.
- The required correction is located in carbontracker/components/gpu/nvidia.py.
- The Python 3.10 compatibility issue concerns CarbonTracker's NVIDIA GPU component.
C Neural network architectures of experiments
The experiments use two neural-network architectures: a fully connected network for Experiment 1 and the DnCNN architecture for Experiment 2.
- Experiment 1: Experiment 1 uses a fully connected network with one hidden layer of 32 neurons and an output layer of 10 neurons.
- Figure 4 shows the architecture used for Experiment 1, while Figure 5 shows the DnCNN architecture used for Experiment 2.
- Experiment 2: Experiment 2 uses the DnCNN network presented in [37], specifically the architecture proposed in the original paper.