Towards Climate Awareness in NLP Research

1 Introduction

• NLP’s expanding computational requirements increase energy use and environmental concerns, despite growing attention to efficiency and sustainability.Training a single BERT base model is reported to require as much energy as a trans-American flight, while compute requirements double every 10 months.Source: §1 · p. 1
• Existing measurement tools have not been adopted broadly, and most NLP researchers remain unaware of the environmental impact of training and running large models.The authors argue that climate awareness should become part of mainstream NLP rather than remain a niche concern.Source: §1 · pp. 1–2
• The authors survey environmental impact statements in recent NLP literature across five dimensions that directly influence environmental impact.Source: §1 · p. 2
• They propose an efficiency taxonomy and ten reporting dimensions to support transparent descriptions of research-related environmental impact.Source: §1 · p. 2
• The paper proposes a climate performance model card designed for transparent reporting with limited information about experiments and hardware.The model card is intended to simplify climate performance reporting and increase awareness.Source: §1 · p. 2

2 Background

• Existing tools estimate emissions, energy, and compute impacts, generate standardized reports, and in some cases support intervention or code and infrastructure optimization.Examples include calculators and Python packages that estimate carbon footprints, generate Carbon Impact Statements, or stop training when predicted costs exceed a threshold.Source: §2.1 · p. 2

2.2 Greenwashing

• The authors warn that framing NLP and AI as green through efficiency alone may make efficiency appear to solve the broader environmental problem.Source: §2.2 · p. 2

2 Background

• The paper aligns its motivation with initiatives requiring standardized methods to substantiate environmental-footprint claims.Source: §2.2 · pp. 2–3

2.2 Greenwashing

• Efficiency reductions do not eliminate resource consumption or greenhouse-gas emissions, while scaling AI performance can ignore environmental externalities.Source: §2.2 · p. 3

2 Background

• Carbon offsets are presented as a controversial approach because the carbon prices required to meet climate goals remain highly debated.Source: §2.3 · p. 3

Quantitative Analysis

• The automatic survey analyzed 2016–2022 *ACL papers by matching five dimensions: model weights, training duration, energy, computation location, and GHG emissions.Only deep-learning-related papers were included because climate-related issues were considered more relevant for them.Source: §3.1 · pp. 3–4
• From 2017 to 2022, papers publishing model weights rose from about 1% to more than 4%, while energy and emissions reporting also increased but remained low.These findings come from pattern matching in deep-learning-related *ACL papers.Source: Figure 2 · p. 3, §3.1 · p. 4

Manual Annotation

• In a manually annotated EMNLP 2021 sample, 13% reported model weights and 28% reported training or optimization duration, though both proportions remained low.Borderline cases were counted as reported, producing an optimistic estimate relative to the automatic approach.Source: §3.2 · p. 4, Table 1 · p. 4
• Manual annotation produced higher estimates for model-weight and duration reporting because pattern matching could not judge borderline cases, while other dimensions aligned with the automatic results.The manual sample contained 100 EMNLP 2021 papers.Source: §3.2 · p. 4

Qualitative Survey

• Many papers reported code for reproducibility but not model weights, and often disclosed computation time only for selected steps.A few examples evaluated accuracy alongside training or inference efficiency or assessed carbon footprint across an entire project.Source: §3.3 · p. 4
• The findings motivate greater awareness and a simple, transparent reporting approach, while incentives are left for future work.The paper focuses on intrinsic motivation rather than extrinsic incentives.Source: §3.3 · p. 4

Towards Actionable Awareness

• Efficiency should be understood as resources consumed to achieve a specified output, with standardized measures needed for comparable climate-performance evaluation.Different input and output definitions correspond to different notions of efficiency and climate impact.Source: §4 · p. 4
• Existing work shows that efficiency measures differ, FPOs are hardware-independent but insufficient alone, cloud-region and time-of-day affect carbon intensity, and hardware development matters.These findings support a broader reporting framework rather than reliance on a single metric.Source: §4.1 · pp. 4–5

Adopting Principles from Finance

• The framework adapts the Greenhouse Gas Protocol’s principles while broadening reporting from direct GHG emissions to climate-related performance assessments.The proposed principles include relevance, completeness, consistency, transparency, and accuracy.Source: §4.2 · p. 5
• Reporting should cover training, evaluation, and deployment, including model factors such as parameters and external factors such as the energy mix.Completeness requires disclosing exclusions, missing information, and what data would be needed to address gaps.Source: §4.2 · p. 5
• Transparent reporting requires coherent documentation of assumptions, calculation methods, data sources, and confidence in the reported quantification.Consistency also requires documenting methodological changes and assumptions about energy mix and conversion factors.Source: §4.2 · p. 5

Actions Towards Improvement

• The climate performance model card is proposed as a practical tool for reflection and immediate improvement using limited experimental and hardware information.The broader goal is climate-resilient NLP and algorithms that support long-term positive impacts.Source: §4.3 · pp. 5–6

• The model card extends impact statements and model cards to report climate performance, mitigation measures, alternative-solution comparisons, and expected impacts from reusing models.Source: §5 · p. 6
• It contains eleven elements covering publicly available artefacts, computation duration, hardware power, location, energy mix, emissions, inference, positive impact, and comments.Source: §5 · pp. 6–7
• The minimum card records basic model distribution, purpose, and approximate computational work, while the extended card adds energy mix information for calculating CO2eq emissions.Source: §5 · pp. 6–7, Table 2 · p. 6
• The card recommends rough estimates rather than maximal precision, including peak hardware power and automated tools for calculating energy use and emissions.Source: §5 · p. 7
• It also asks authors to report expected positive environmental impact and allows comments about uncertainty, overestimation, underestimation, and possible improvements.Source: §5 · pp. 6–7

Responsibility and accountability

• NLP research lacks best practices specifying what information supports accurate environmental reporting and informed sustainability decisions.Source: §6 · p. 8

Extrapolation to indirect impact

• Indirect impacts from reuse and deployment are complex and can only be estimated, so environmental impact statements will be more abstract and harder to assess.Source: §6 · p. 8
• The proposed scope framework distinguishes emissions from the paper’s experiments, effects on other researchers and practitioners, and later positive environmental applications.Source: §6 · p. 8

Multi-objective optimization

• Performance evaluation should consider both model performance and climate performance, including climate performance in optimization objectives and benchmarking.Source: §6 · p. 8, Table 3 · p. 8

Do increase transparency

• The model card should provide concrete but flexible guidelines for reporting energy consumption and CO2eq emissions across different hardware setups.Source: §7 · p. 9

Do use the model cards

• Researchers and institutions should use model cards to report climate performance and road-test them for institutional greenhouse-gas reporting, where harmonized approaches remain lacking.Source: §7 · p. 9

Do not use our model card for assessing research quality

• The model card should not assess research quality because contribution value and emissions depend on publication timing and hardware, respectively.Source: §7 · p. 9

Do not report your voluntary financial climate protection contributions as emission offsetting

• Relative efficiency improvements should not be communicated as absolute climate-performance improvements or as emission offsetting.Source: §7 · p. 9

Do not use this model card to assess net climate-related impacts

• The model card does not assess net climate-related impacts, instead focusing on transparency about each model’s first-order effects.Source: §7 · p. 9

• Climate awareness should be promoted in mainstream NLP rather than treated as a niche concern.Source: §8 · p. 9
• The paper surveys climate discussion, proposes actionable awareness measures, and introduces a model card for transparent climate-performance reporting.Source: §8 · p. 9
• The discussion, survey, and recommendations are designed for NLP but are applicable to other AI fields.Source: §8 · p. 9

• Climate awareness is necessary but insufficient for changing researchers’ and practitioners’ behavior.The authors state that values do not necessarily determine behavior and that awareness is only the first step toward climate-responsible practice.Source: §9 · p. 10
• The model card requires less precise information than existing reporting tools, which may limit informed decision making.The authors suggest that quantifying uncertainty could mitigate over-reliance on the less precise information.Source: §9 · p. 10
• Mandatory climate reporting could be used for greenwashing when incentives exist without mechanisms checking researchers’ honesty.Source: §9 · p. 10

• The patterns identify climate-impact terms including CO2, GHG, carbon footprint, emissions, and offsetting.Source: A Patterns for Quantitative Analysis · p. 13
• Training or optimization duration is detected through time expressions such as seconds, minutes, hours, days, weeks, months, and hours of computation.Source: A Patterns for Quantitative Analysis · p. 13
• Energy-use patterns cover energy, power, electricity consumption, watt values, kilowatt values, and PUE.Source: A Patterns for Quantitative Analysis · p. 13
• Location patterns identify data centers, clouds, virtual or GPU machines, computer clusters, HPC, and cloud-provider regions.Source: A Patterns for Quantitative Analysis · p. 13
• The analysis applies patterns to full paper text for papers identified as deep-learning-related.The text includes abstracts, main contents, and appendices, while ignoring capitalization.Source: A Patterns for Quantitative Analysis · p. 13

• ClimateBert’s example model card documents model availability, computation time, total computation duration, hardware, location, and energy mix.ClimateBert is a language model fine-tuned on climate-related text, and the model card follows the paper’s reporting guidelines.Source: B Example Model Card · p. 13
• 12 days is the estimated duration for all computations, although the estimate is likely pessimistic.The authors state that the model card prioritizes transparency over accuracy.Source: B Example Model Card · p. 13
• 2 x NVIDIA RTX A5000 GPUs, each using 230 watts, were the main training hardware, with 120 watts added for remaining hardware.Source: B Example Model Card · p. 13
• 470 gCO2eq/kWh is the reported energy mix for servers located in Germany.Source: B Example Model Card · p. 13
• 2.63kg CO2eq emissions are reported for the example.Source: §7 · p. 14

• 94.75kg CO2eq emissions are reported for the described computation.Source: §8 · p. 14
• A pass of 100,000 samples took 0.187 hours on the same server using a batch size of 512.Source: §8 · p. 14
• The calculation derives the emissions associated with inference of one sample.Source: §8 · p. 14
• Block pruning could reduce CO2eq emissions by substantially decreasing the number of model weights while slightly reducing performance.Source: §8 · p. 14
• The GHG Protocol provides emission factors by fuel source for calculating greenhouse-gas emissions from energy consumption.Source: D GHG Protocol Information… · p. 14
• The reporting approach uses GHG Protocol guidance on information to disclose, presented through the paper’s model-card approach.Source: D GHG Protocol Information… · p. 14, Figure 4 · p. 15