Source-linked AI summary
Measuring the Carbon Intensity of AI in Cloud Instances
Jesse Dodge, Taylor Prewitt, Remi Tachet Des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A. Smith, Nicole DeCario, Will Buchanan
TL;DR
AI’s growing computational demands create a need for accessible, reliable measurements of greenhouse-gas impacts. The paper measures operational software carbon intensity using location-based, time-specific marginal emissions data and evaluates cloud-region, timing, and scheduling strategies, finding that geography has the largest reduction impact among the examined approaches.
Problem
Reliable, accessible measurements of AI’s greenhouse-gas impact are limited, hindering actionable emission-reduction tactics.
Method
The paper measures operational cloud-instance carbon emissions with location-based and time-specific marginal emissions data per energy unit and evaluates region, time-of-day, and dynamic-pausing strategies on Microsoft Azure.
Results
The study measures operational software carbon intensity across natural-language-processing and computer-vision models and finds that geographic region significantly affects carbon intensity, with region choice having the largest reduction impact.
Takeaways & Limitations
Machine-learning practitioners can use software carbon-intensity information to select lower-emission regions and times and schedule workloads to reduce environmental impact.
Takeaways & Limitations
The tool focuses on GPU computation for a single cloud instance and leaves embodied emissions from specialized machine-learning hardware for future measurement and accounting.
Abstract
from arXiv · showhide
By providing unprecedented access to computational resources, cloud computing has enabled rapid growth in technologies such as machine learning, the computational demands of which incur a high energy cost and a commensurate carbon footprint. As a result, recent scholarship has called for better estimates of the greenhouse gas impact of AI: data scientists today do not have easy or reliable access to measurements of this information, precluding development of actionable tactics. Cloud providers presenting information about software carbon intensity to users is a fundamental stepping stone towards minimizing emissions. In this paper, we provide a framework for measuring software carbon intensity, and propose to measure operational carbon emissions by using location-based and time-specific marginal emissions data per energy unit. We provide measurements of operational software carbon intensity for a set of modern models for natural language processing and computer vision, and a wide range of model sizes, including pretraining of a 6.1 billion parameter language model. We then evaluate a suite of approaches for reducing emissions on the Microsoft Azure cloud compute platform: using cloud instances in different geographic regions, using cloud instances at different times of day, and dynamically pausing cloud instances when the marginal carbon intensity is above a certain threshold. We confirm previous results that the geographic region of the data center plays a significant role in the carbon intensity for a given cloud instance, and find that choosing an appropriate region can have the largest operational emissions reduction impact. We also show that the time of day has notable impact on operational software carbon intensity. Finally, we conclude with recommendations for how machine learning practitioners can use software carbon intensity information to reduce environmental impact.
1 INTRODUCTION
The paper addresses how to measure operational carbon costs of AI workloads and whether computation can shift across locations and times to mitigate emissions. It introduces a cloud-instance measurement tool, evaluates model-training carbon intensity, and proposes region, timing, and scheduling strategies for reducing impact.
- The paper asks how operational carbon costs of AI workloads should be measured and reported, and whether computation can shift spatially and temporally to mitigate emissions.
- The tool estimates real-time CO2 emissions for cloud instances using location-based and time-specific marginal emissions data per energy unit.
- The authors measure carbon intensity for machine-learning models spanning natural language processing and computer vision on Microsoft Azure.
- Users can reduce CO2 by changing compute regions or the time of day when models run, with time-based shifting presented as a novel focus.
- The tool can automatically schedule jobs to exploit carbon-intensity differences across time and geographic location.
- The paper provides guidance on measurement and reporting according to the Green Software Foundation’s Software Carbon Intensity guidelines.
2 RELATED WORK
Prior work has estimated machine-learning emissions, but carbon accounting and reporting remain limited and existing estimates may not represent true training emissions. This paper extends the area toward fine-grained cloud-instance tracking while situating it among broader sustainability and accounting efforts.
- Earlier studies estimated emissions from AI training and neural architecture search using energy consumption, server carbon intensity, or coarse regional measures.
- The paper’s tool enables fine-grained carbon tracking on cloud instances, extending prior training-emissions measurement work.
- Recent findings indicate that existing results vary significantly and are not fully representative of emissions incurred during training.
- Related tools compare machine-learning energy use across hardware and physical locations, while other studies examine hardware manufacturing, sustainable-AI impacts, and assessment methodologies.
- Cloud providers’ carbon-neutrality claims commonly rely on renewable-energy credits and offsets, whereas deployment decisions should use grid emissions signals because purchasing clean energy is not equivalent to consuming it.
3 REPORTING AI CARBON INTENSITY
This section presents Software Carbon Intensity as a framework for reporting operational carbon emissions from cloud-based AI workloads. It defines the measurement components, explains the focus on GPU electricity consumption, and identifies practical and methodological boundaries of the approach.
- Motivation: The paper argues that systematic information about ML models’ environmental footprint remains limited, despite increasing carbon accounting and reporting.The authors position accessible emissions information as necessary for more informed workload decisions.
- Contribution: The proposed tool is intended to report cloud-instance emissions and help developers choose computation conditions with lower carbon intensity.The authors specifically recommend that cloud providers disclose SCI to support workload decisions involving geographic regions and times of day.
- Methodology: Software Carbon Intensity measures carbon emissions per functional unit, combining operational and embodied emissions in the paper’s general framework.The functional unit is defined here as one machine learning training job, though the framework is extensible.
- Methodology: Operational emissions are calculated from energy consumption multiplied by location-based, time-specific marginal grid carbon intensity.The paper uses GPU energy in kilowatt-hours and carbon intensity in gCO2eq/kWh.
- Scope and limitations: The paper focuses on operational emissions and leaves embodied emissions from specialized ML hardware, such as GPUs, to future work.Its estimates also concern electricity used by a single cloud instance rather than all shared data-center components.
- Scope and limitations: In a BERT-base training experiment, the GPU accounted for 74% of the total electricity consumption from the measured server components.The result supports focusing the tool’s single-instance electricity measurement on GPUs, which are typically not shared across instances.
4 ELECTRICITY CONSUMPTION FOR AI WORKLOADS
The paper measures electricity consumption across 11 NLP and computer-vision models, spanning brief single-GPU jobs to multi-day, multi-GPU training runs. Consumption varies substantially with model and scale, from negligible DenseNet workloads to an estimated 103.5 MWh for completing the 6.1-billion-parameter Transformer.
- Scope of workloads: 11 models spanning NLP and computer vision were measured, with experiments lasting from less than an hour on one GPU to more than 8 days on 256 GPUs.The measurements provide the electricity inputs later used to calculate software carbon intensity.
- NLP: 37 kWh powered BERT-small pretraining for approximately 36 hours on 8 NVIDIA V100 GPUs.The reported run consumed over 37 kWh.
- NLP: 3.2 kWh powered BERT-small finetuning for approximately 6 hours on 4 NVIDIA V100 GPUs, less than one tenth of its pretraining electricity.The comparison concerns the reported BERT-small finetuning and pretraining runs.
- NLP: 13.8 MWh was consumed training a 6.1-billion-parameter Transformer for 8 days on 256 NVIDIA A100s.The run reached only 13% of completion; the paper estimates approximately 103.5 MWh for full training.
- NLP: 103.5 MWh was the estimated electricity required to train the 6.1-billion-parameter Transformer to completion.The estimate scales the 13.8 MWh consumed over 8 days by 60/8, based on the stated 60-day full-training duration.
- Computer vision: 20–38 Wh powered DenseNet training jobs lasting 20–25 minutes, described as negligible compared with the other models.The reported range equals 0.02 to 0.04 kWh.
- Computer vision: Vision Transformer training consumed approximately 1.7 kWh for ViT tiny and approximately 237 kWh for ViT huge.The smallest experiment used one V100 for around 19 hours; the largest used 4 V100s for more than 9 days.
5 EMISSIONS BY REGION AND TIME OF DAY
Operational emissions vary substantially by cloud region and, to a lesser extent, by time of day. The paper measures these differences across models and shows that experiment duration and regional carbon intensity shape the potential impact of scheduling choices.
- Measurement framework: The framework estimates cumulative emissions by combining workload energy consumption with location- and time-specific marginal carbon intensity.The measurements use time-series marginal emissions data for cloud datacenters.
- Region: 7k grams vs. 26k grams separates BERT training in the most- and least-carbon-intensive examined regions.The comparison uses 8 V100 GPUs for 36 hours across 16 regions.
- Experiment scale: The 6 billion parameter transformer can emit more than a full year of average US-home emissions even when only partially trained.The model was trained for about 13% of the time needed to converge, and the estimate exceeds 8.3 metric tons of CO2.
- Experiment scale: 21 to 78 metric tons of CO2 is the estimated full-training range for the 6 billion parameter model across regions.Even the most efficient examined region exceeds the emissions of a full barrel of oil for the partial run.
- Time of Day: Up to 8% more carbon emissions can result from starting BERT finetuning at midnight instead of 6:00 in Central US.Diurnal variation differs by region and time of year, while individual regions can remain relatively similar across months.
6 OPTIMIZING CLOUD WORKLOADS
The paper evaluates two carbon-aware scheduling strategies on Azure: choosing a lower-emissions start time, and pausing and resuming workloads during high-intensity periods. Their benefits depend on workload duration, regional variability, and the accuracy of carbon-intensity forecasts.
- Scheduling methods: Flexible Start selects the lowest-emissions launch time within a future window, while Pause and Resume intermittently stops jobs during high-emissions periods.The latter can extend total job duration by a specified absolute or relative amount.
- Flexible Start: Greater than 30% savings, reaching up to 80% in West US, are possible for very short jobs when start time shifts by up to 24 hours.For an 8-day, 6 billion parameter language-model run, the best reduction is below 1.5% in any region.
- Regional effects: West US enables larger Pause and Resume reductions because its emissions vary frequently between high and very low periods.Regions with less intraday variance yield smaller reductions.
- Pause and Resume: About 25% savings are possible for very long runs when Pause and Resume doubles the workload duration.Short experiments such as DenseNet 201 obtain minimal reductions under the same relative-duration allowance.
- Algorithm comparison: Flexible Start generally provides a lower bound for Pause and Resume because it permits one optimized start but no interruptions.The comparison is made across the 11 analyzed models and different duration increases.
- Forecasting limits: The reported optimization gains are upper bounds for longer workloads because the evaluation uses historical data as if carbon intensity were predicted perfectly.Forecasts extend up to 24 hours, so short-workload results better reflect currently available forecasts.
7 CONSIDERATIONS FOR MODEL DEVELOPMENT AND DEPLOYMENT
The paper frames carbon measurement as a practical input throughout model development and deployment, while emphasizing that carbon-aware choices must be balanced against time, cost, data transfer, and lifecycle considerations.
- Reporting: Researchers and practitioners should record and report emissions from training workloads across the ML project lifecycle.The proposed tool is intended to support use from exploratory training through hyperparameter tuning and deployment.
- Carbon-aware tooling: Carbon-aware configurations can guide model training and let cloud users opt in to lower-intensity regions or times.The paper identifies carbon-informed optimization and cloud-based recommendations as possible uses.
- Decision-making: The measurements should guide informed decisions alongside other considerations rather than serve as a single optimization target.The paper explicitly rejects a one-size-fits-all approach to eliminating emissions.
- Trade-offs: Strict project deadlines can make nighttime scheduling impractical and may shift emissions elsewhere in the project.Engineering time spent improving workload efficiency should also be weighed against moving computation to a lower-emissions region.
- Lifecycle scope: Upstream hardware manufacturing and downstream model uses or misuses can eclipse direct training emissions.The paper therefore situates operational training emissions within a broader ML lifecycle analysis.
- Economic trade-offs: Lower-emissions regions may have higher operating or data-transfer costs, potentially offsetting their environmental benefit.The energetic, environmental, or monetary cost of moving data can exceed the gains from changing regions.
8 FUTURE DIRECTIONS
The paper identifies future work needed to broaden emissions accounting, improve transparency and emissions-rate data, and develop AI-specific sustainability standards. It also highlights downstream environmental effects that extend beyond operational training emissions.
- Emissions scopes: The current work focuses on Scope 2 electricity-use emissions and does not cover Scope 1 or Scope 3 emissions.The authors describe these omitted categories as relevant to the total environmental impact of ML.
- Accounting standards: GHGP Scope 2 guidance is incompatible with the paper’s marginal-emissions method and may need revision toward consequential accounting.The current guidance primarily emphasizes broad generation-based average rates.
- Accounting standards: The field lacks a concrete structure for categorizing, quantifying, and mitigating its different emissions scopes.Developing the required tooling and reporting would require active participation from relevant stakeholders.
- Green AI certification: AI-specific certification or badge systems could help developers and users make more informed sustainability choices.The paper anticipates such systems spanning research and academia.
- Transparency: Carbon estimation and reporting remain relatively rare in ML publications and technical reports despite existing tools and optional conference reporting.The authors call for broader adoption to improve carbon transparency.
- Data availability: Better electric-system data are needed, especially in regions where hourly marginal emissions estimates cannot currently be produced.The paper identifies improved provider inputs and broader geographic data availability as future needs.
- Downstream impacts: Assessing AI’s environmental impact should include whether downstream applications incentivize harmful or beneficial environmental practices.Examples discussed include oil exploration, deforestation, and mining.
B ADDITIONAL TABLES
The additional tables report emissions reductions from Flexible Start and Pause and Resume optimizations across 11 models, averaged over a year and 16 regions. They examine progressively larger allowed increases in job duration and also report Pause and Resume pause frequency.
- Tables 6–13 report percentage CO2-emissions decreases for Flexible Start and Pause and Resume across all 11 models.The gains are averaged across the 16 considered regions and over the year.
- The tables evaluate Flexible Start and Pause and Resume while allowing job duration to increase by 25%, 50%, 75%, or 100%.
- Additional tables evaluate the same optimizations with allowed job-duration increases of 6h, 12h, 18h, and 24h.
- The Pause and Resume results include the average number of pauses performed per hour.