Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning

• Training and deploying ML models requires computational resources such as GPUs and energy to power them.Energy generation can emit greenhouse gases including CO2.Source: §1 · p. 1
• Existing information is limited about the field’s overall energy consumption, carbon footprint, evolution, and relationship with performance across tasks.Source: §1 · p. 1
• The paper examines how energy sources, CO2 emissions, time, and model performance shape the carbon footprint of ML training.It asks what energy sources are used, the emissions magnitude, how emissions evolve, and whether more energy improves performance.Source: §1 · p. 1
• The paper proposes a centralized hub for reporting and tracking machine-learning carbon footprints.Source: §1 · p. 2

Empirical studies on carbon emissions

• Prior model-specific work includes estimating 284,019 kg of CO2 for training and fine-tuning a large Transformer with Neural Architecture Search.Subsequent studies examine neural-network architectures and the relative efficiency of methods.Source: §2 · p. 2
• Empirical emissions studies remain sparse and biased toward certain research areas, motivating broader and more multifaceted analysis.The authors identify a need to better understand the scale and variation of carbon emissions in the ML community.Source: §2 · p. 2

Tools and approaches for measuring carbon emissions

• Carbon-estimation tools exist for training and post-training use, but their estimates vary significantly and can consistently under-report emissions.No single accepted estimation approach makes standardized reporting and comparisons difficult.Source: §2 · p. 2

Broader impacts of ML models

• Environmental impacts are not yet consistently tracked and reported alongside other documented model impacts, with a few language-model exceptions.Source: §2 · p. 2

Efficient algorithms and hardware

• Research on efficient architectures and hardware seeks faster training and inference with less energy use and indirectly fewer training emissions.Energy-efficient benchmarks also enable model comparisons using computational and environmental criteria.Source: §2 · pp. 2–3

Other aspects of the carbon impact of ML

• The field’s carbon footprint includes conference attendance, hardware manufacturing, development and deployment life cycles, and production deployment.The relative contribution of these factors remains unclear and requires further disentangling.Source: §2 · p. 3

3.1 Data collection

• The authors sampled 500 papers equally across five tasks and contacted first authors to obtain missing model-training details.The collected models were, on average, within 8% of state-of-the-art performance at publication.Source: §3.1 · p. 3
• The dataset contains 95 models from 77 papers across five tasks, spanning nine years and representing the largest ML-training carbon-footprint sample to date.The tasks are Image Classification, Object Detection, Machine Translation, Question Answering, and Named Entity Recognition.Source: §3.1 · p. 3

3.2 Estimating carbon emissions

• Carbon emissions are decomposed as C = P×T×I = E×I, combining hardware power or energy, training time, and grid carbon intensity.A 300 W GPU trained for 100 hours emits 15 kg CO2eq at 500 gCO2eq/kWh but 3 kg at 100 gCO2eq/kWh.Source: §3.2 · p. 4
• Grid carbon intensity is estimated from training location using public sources, yearly averages, and company information for internal or commercial cloud infrastructure.Available geographic resolution varies from sub-state data in the United States to country-level data in China.Source: §3.2 · p. 4
• Hardware power is approximated using Thermal Design Power, although real-time consumption requires measurement during training.The TDP-based approach is commonly used and considered a fair approximation for many hardware models.Source: §3.2 · p. 4
• Training time is measured in total hardware hours rather than wall time, so parallel hardware usage is accumulated across units.For example, 16 GPUs used for 24 hours equal 384 GPU hours, equivalent to 8 GPUs used for 48 hours.Source: §3.2 · pp. 4–5

4.1 What are the main sources of energy used for training ML models?

• Most sampled models used high-carbon energy sources: 61 relied primarily on coal or natural gas, versus 34 using low-carbon sources; the sample average was 372 gCO2eq/kWh.The sampled average was below the global average of 475 gCO2eq/kWh, but the study identifies switching to renewable energy as an opportunity for improvement.Source: Table 2 · p. 5, §4.1 · p. 5
• Training locations were highly concentrated: 48 models were trained in the United States and 18 in China, while 76 models were trained in countries, including neither Africa nor South America.Source: §4.1 · pp. 5–6

4.2 What is the order of magnitude of CO2 emissions produced by training ML models?

• Hydroelectricity-trained models emitted about two orders of magnitude less carbon than coal- or gas-trained models at similar energy use, while emissions generally scaled linearly with energy.Energy use ranged from just about 10 kWh to more than 10,000 kWh; hardware variation was comparatively small, with TDPs of 180 W to 300 W.Source: §4.2 · pp. 6–7, Figure 2 · p. 7

4.3 How do the CO2 emissions produced by training ML models evolve over time?

• Overall carbon emissions per model increased by a factor of about 100 from 2012 to recent years, although individual tasks showed no consistent increase and machine translation emissions peaked in 2019 before declining.Recent Image Classification and Question Answering models included records of about 10^4 and 10^5 kilograms of CO2eq.Source: Figure 3 · p. 8, §4.3 · pp. 8–9
• The study observes emissions across recent years but notes that disentangling the influence of different factors on overall carbon emissions and pre-training versus fine-tuning contributions requires further work.The subsection focuses on the evolution of emissions among the sampled papers rather than isolating each factor’s effect.Source: §4.3 · p. 9

4.4 Does more energy and CO2 lead to better model performance?

• ImageNet was the only task where better performance systematically coincided with more CO2, although the relationship was weak and included both low-performing high emitters and high-performing low emitters.The two highest-performing models were also the highest emitters, while the third most carbon-intensive model achieved 76 % accuracy and some low-emitting models performed highly.Source: §4.4 · p. 9
• WMT translation showed no clear relationship between CO2 emissions and BLEU score for either English-French or English-German; English-French emitted more partly because its dataset was almost 4 times larger.The larger dataset can require longer training and higher energy consumption.Source: §4.4 · p. 9
• For SQuAD and CoNLL, the connection between emissions and accuracy was very unclear, with high- and low-emission models often achieving comparable performance.The analysis had fewer data points for these tasks: 10 for question answering and 11 for named entity recognition.Source: §4.4 · p. 9
• Across ImageNet, WMT translation, SQuAD, and CoNLL, greater energy consumption or CO2 emissions generally did not necessarily yield higher accuracy.The comparison covered 35 ImageNet models, 30 WMT models, 10 SQuAD models, and 11 CoNLL models.Source: §4.4 · pp. 9–10

5.1 Discussion of Results

• The analyzed models emitted about 253 tons of CO2eq in total, while average emissions increased from 487 tons for 2015–2016 models to 2020 tons for models trained in 2020–2022.The sample is only a small portion of the entire Machine Learning field.Source: §5.1 · p. 11
• Carbon intensity and training time are the main sources of emissions variance, whereas hardware power consumption has a smaller influence.Training times range from about 15 minutes to more than 400,000 hours, with a median of 72 hours.Source: §5.1 · p. 11
• Using the lowest-carbon-intensity grid would have reduced the longest-training model’s emissions by about 30 times, but emissions would still have exceeded 1 ton of CO2eq.Very long training times can therefore produce substantial emissions even with lower-carbon energy sources.Source: §5.1 · p. 11
• Higher performance is not generally achieved with more energy, and the study reports that good performance can be achieved with limited carbon emissions.Image classification showed the strongest performance–emissions correlation, although small carbon increments there were associated with large top-1 accuracy increments.Source: §5.1 · p. 11

5.2 Limitations

• The study estimates only training emissions, omitting data processing, transfer, storage, hardware manufacturing and maintenance, model development, and inference.It also lacks information such as data-center PUE and does not account for carbon offsets or power purchase agreements.Source: §5.2 · p. 12

5.3 Future Work

• Future work should improve reporting standardization, examine sustainability–fairness trade-offs, expand empirical analysis, and cover upstream and downstream ML lifecycle emissions.The authors propose more transparent reporting of computational needs and energy use, alongside broader accounting of deployment and equipment impacts.Source: §5.3 · p. 12

• The information request supported a broader effort to estimate CO2 emissions from recent state-of-the-art machine-learning papers.Source: §A.1 · p. 17
• They asked where training occurred, including cloud provider and server region when applicable.Source: §A.1 · p. 17
• The authors requested hardware and total training time to gather missing information about models’ computing infrastructure.Source: §A.1 · p. 17

• The appendix summarizes the five most-used GPU and TPU models, their usage counts, quantity ranges, and thermal design power.Source: Table 3 · p. 17, §A.2 · p. 17
• The Tesla V100 was the most popular hardware, representing almost a third of the papers, followed by the TPU v3.Source: §A.2 · p. 17
• Hardware thermal design power ranged from 180W for the GTX 1080 to 450W for the TPU v3.Source: §A.2 · p. 17
• Up to 1024 TPU v3s were used for some models, while others used a single GTX 1080, creating large differences in total energy consumption.Source: §A.2 · p. 17

• Energy-consumption trends were largely similar to the performance-related trends described earlier.Source: §A.3 · p. 18
• Better performance on machine translation and image classification was not necessarily contingent on higher energy consumption.Source: §A.3 · p. 18
• Figure 5 compares machine-translation BLEU scores and image-classification top-1 accuracy with energy consumed.Source: Figure 5 · p. 18, §A.3 · p. 18

• Models trained with coal increased starkly over time.Source: §A.4 · p. 19
• Average carbon intensity did not decrease, nor did the number of models trained with cleaner energy sources.Source: §A.4 · p. 19
• Figure 6 plots each model’s energy-grid carbon intensity over the years alongside the number of models using each energy source.Source: Figure 6 · p. 19, §A.4 · p. 19