Source-linked AI summary
Sustainable AI: Environmental Implications, Challenges and Opportunities
Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga Behram, James Huang, Charles Bai, Michael Gschwind, Anurag Gupta, Myle Ott, Anastasia Melnikov, Salvatore Candido, David Brooks, Geeta Chauhan, Benjamin Lee, Hsien-Hsin S. Lee, Bugra Akyildiz, Maximilian Balandat, Joe Spisak, Ravi Jain, Mike Rabbat, Kim Hazelwood
TL;DR
AI’s super-linear growth raises environmental concerns that cannot be assessed from training alone. The paper examines the ML development cycle and hardware life cycle across operational and embodied carbon, then identifies optimization opportunities and challenges. It reports substantial footprint reductions from data, algorithm, hardware-software, utilization, and energy strategies, while emphasizing that embodied carbon remains important.
Problem
AI’s growth across data, models, and infrastructure requires environmental assessment spanning the full ML process and hardware life cycle, not training alone.
Method
The paper analyzes industry-scale Facebook use cases across data, algorithms, systems, datacenters, and on-device computing, including manufacturing and operational emissions.
Results
The paper identifies end-to-end optimization opportunities and reports large reductions in operational and overall carbon footprints through data efficiency, co-design, utilization, and renewable energy.
Takeaways & Limitations
Sustainable AI requires combining algorithm, platform, infrastructure, hardware, telemetry, and carbon-accounting practices rather than optimizing model quality alone.
Abstract
from arXiv · showhide
This paper explores the environmental impact of the super-linear growth trends for AI from a holistic perspective, spanning Data, Algorithms, and System Hardware. We characterize the carbon footprint of AI computing by examining the model development cycle across industry-scale machine learning use cases and, at the same time, considering the life cycle of system hardware. Taking a step further, we capture the operational and manufacturing carbon footprint of AI computing and present an end-to-end analysis for what and how hardware-software design and at-scale optimization can help reduce the overall carbon footprint of AI. Based on the industry experience and lessons learned, we share the key challenges and chart out important development directions across the many dimensions of AI. We hope the key messages and insights presented in this paper can inspire the community to advance the field of AI in an environmentally-responsible manner.
I. INTRODUCTION
AI is expanding rapidly across research, data, models, and infrastructure, while its environmental implications require an end-to-end view of the ML pipeline and hardware life cycle. The paper frames sustainable AI as a holistic design challenge involving operational and embodied carbon.
- AI growth and societal benefits: AI research and development are growing rapidly across scientific and industrial applications.The paper describes major investment and expanding applications in science, medicine, finance, education, and product development.
- Super-linear scaling: 1000× model growth, 2.4×–1.9× data growth, and rising infrastructure demand characterize AI’s super-linear scaling.Facebook recommendation data reached exabyte scale, ingestion bandwidth demand increased 3.2×, and training and inference capacity increased 2.9× and 2.5×.
- Environmental implications: Higher model quality and AI scaling create significant energy and environmental footprint implications.The paper argues that focusing only on model training is insufficient for understanding AI’s overall environmental impact.
- Holistic scope: Sustainable AI requires accounting for data, experimentation, training, inference, and hardware manufacturing and operation.The paper emphasizes both operational and embodied carbon across the ML development process and system hardware life cycle.
- Paper contribution: The paper characterizes AI’s environmental footprint across industry-scale Facebook use cases and charts directions spanning data, algorithms, systems, metrics, standards, and practices.Its stated goal is to support environmentally responsible advancement of AI through end-to-end analysis and hardware-software design.
II. MODEL DEVELOPMENT PHASES AND AI SYSTEM HARDWARE LIFE CYCLE
AI model development proceeds through Data Processing, Experimentation, Training, and Inference across the life cycle of AI system hardware. These phases have distinct computational and infrastructure demands, from data ingestion and algorithm exploration to large-scale training and deployment.
- The AI development cycle comprises Data Processing, Experimentation, Training, and Inference, considered alongside the life cycle of AI system hardware.
- Data Processing extracts features and applies feature weights, while Experimentation designs and evaluates algorithms, architectures, techniques, and training methods.The model exploration process is computationally-intensive.
- Training evaluates promising solutions on larger, newer, richer production data and often includes additional hyper-parameter tuning.Training or retraining frequency depends on the ML task requirement.
- 2.96 GPU days is the p50 duration for a production model training workflow, while p99 workflows can take up to 125 GPU days.
- Inference deploys the best-performing model for trillions of daily predictions serving billions of users, with expected compute cycles exceeding those of training.
B. Machine Learning System Life Cycle
The analysis measures AI’s carbon footprint across operational use and hardware manufacturing, covering model training, inference, and system life cycle. It finds that both operational and embodied emissions matter, with manufacturing becoming dominant when carbon-free energy reduces operational emissions.
- Life Cycle Analysis: Life Cycle Analysis covers manufacturing, transport, product use, and recycling, while this study focuses on manufacturing and product use.Manufacturing represents embodied emissions from AI infrastructure; product use represents operational emissions from using AI.
- Production Models: Facebook’s production models include six representative systems: one language model and five recommendation or ranking models.LM denotes Facebook’s Transformer-based Universal Language Model for text translation; RM1–RM5 represent distinct production recommendation and ranking models.
- Embodied Carbon Footprint: 50% is the approximate manufacturing carbon cost relative to location-based operational carbon for large-scale ML tasks.With carbon-free energy, operational emissions can fall substantially, leaving manufacturing as the dominant source of AI’s carbon footprint.
- Operational Carbon Footprint: Training and inference both contribute significantly to operational emissions, but their relative contribution varies across machine-learning use cases.The operational footprint is divided into offline training, online training, and production inference.
- Operational Carbon Footprint: 1.8× is the average operational carbon footprint of Facebook’s models relative to Meena, while Facebook’s footprint is one-third of GPT-3’s training footprint.The estimate uses total energy consumption, location-based carbon intensities, and a data-center PUE of 1.1.
- Optimization: 20% is the average operational energy-footprint reduction achieved every six months through iterative hardware-software optimization.The reported reductions span the machine-learning hardware-software stack.
B. Carbon Footprint Optimization from Hardware-Software Co-Design
Cross-stack optimization reduces operational energy footprints by combining model, platform, infrastructure, and hardware changes. An iterative process delivers substantial aggregate savings, while the language-model case demonstrates the value of coordinated caching, acceleration, precision reduction, and algorithmic optimization.
- Optimization Over Time: 28.5% is the operational power-footprint reduction achieved over two years despite continued AI infrastructure scale-out.The result reflects iterative reductions across the machine-learning hardware-software stack alongside Jevon’s Paradox.
- Optimization Over Time: 20% average operational power-consumption reduction occurs every six months across model, platform, infrastructure, and hardware optimizations.The optimization areas include resource-efficient models, quantization support, data-center optimization, low-precision hardware, and domain-specific acceleration.
- Language-Model Optimization: More than 800× reduction in LM infrastructure resources results from platform-level caching, GPU acceleration, low-precision formats, and model optimization.The LM example applies these steps to a Transformer-based language-translation workload.
- Language-Model Optimization: 6.7× power-efficiency improvement comes from application-level caching that precomputes and shares frequently accessed embeddings.The caching design uses DRAM and Flash storage devices as shared caches.
- Language-Model Optimization: 10.1× additional energy-efficiency improvement comes from deploying the LM on GPU-based specialized AI hardware.This improvement is reported in addition to the caching step.
- Language-Model Optimization: 12× additional energy-efficiency reduction comes from algorithmic optimization, including lower precision and custom Transformer operators.Halving precision from 32-bit to 16-bit provides a 2.4× GPU efficiency improvement, while custom operators provide another 5× gain.
- Recommendation-Model Optimization: Recommendation models face rapidly rising memory capacity and bandwidth demands because sparse embedding networks can exceed 95% of total model size.Embedding operations dominate inference execution time for several important recommendation and ranking use cases.
C. Machine Learning Infrastructures at Scale
At-scale infrastructure efficiency depends on accelerator utilization, workload-specific server customization, and fleet-wide resource management. Carbon-free energy further reduces operational emissions, while improved utilization lowers both embodied and operational carbon in the analyzed setting.
- ML Accelerators: GPU under-utilization during ML experimentation and training makes accelerator utilization important for amortizing upfront embodied carbon costs.The paper frames maximizing accelerator utilization as necessary to amortize the embodied emissions of deployed accelerators.
- Efficiency of Scale: Higher performance density reduces the number of processors deployed and improves amortization of shared infrastructure overheads.Higher operational power efficiency also reduces the inherited carbon cost from IT infrastructure and data-center buildings.
- At-Scale Optimization: Facebook customizes server SKUs across compute, memcached, storage tiers, and ML accelerators to maximize performance and power efficiency.The fleet’s internal-workload focus enables at-scale resource-management design and optimization.
- At-Scale Optimization: Up to 25% of web-tier machines can be freed during off-peak hours through auto-scaling in fleets with diurnal utilization patterns.Large-scale deployment enables measurement and optimization tools aimed at maintaining high infrastructure utilization.
- Carbon-Free Energy: Matching data-center energy consumption with 100% renewable energy, together with offsets, reduces Facebook’s operational carbon footprint.The paper describes this as part of the path toward net-zero emissions.
D. Going Beyond Efficiency Optimization
AI sustainability requires more than operational efficiency: embodied carbon, edge computation, and infrastructure constraints must also be addressed. Improving utilization and renewable-energy use can reduce emissions, but embodied carbon can remain dominant.
- Domain-specific architectures can reduce operational training energy by more than 90%, yet their greater resource requirements increase embodied carbon.
- Carbon-free energy may reduce emissions, but its infrastructure requires rare materials, economic resources, and substantial construction time.
- Federated learning and on-device computation can emit substantial carbon, while the edge has more limited access to renewable energy than datacenters.
- As GPU utilization rises to 80%, overall carbon footprint decreases by 3×, while renewable energy provides an additional 2× reduction.
- Embodied carbon becomes the dominant source after efficiency and renewable-energy improvements, motivating complementary strategies to reduce hardware footprint.
IV. A SUSTAINABILITY MINDSET FOR AI
A sustainability mindset should guide AI across data, experimentation, algorithms, and systems. Measurement, efficient data practices, resource-aware optimization, and utilization improvements are central to reducing AI’s environmental footprint.
- AI research should evaluate efficiency alongside accuracy to counter computational-resource and environmental-footprint growth.
- Data Utilization Efficiency: Data scaling can increase embodied and operational carbon through larger storage, ingestion, and training requirements.
- Data Utilization Efficiency: Intelligent sampling with 10% of data preserved recommendation-algorithm rankings while achieving 5.8× average execution-time speedup.
- Experimentation Efficiency: Grid-search NAS can incur over 3000× environmental-footprint overhead, motivating sample-efficient search and early stopping of under-performing workflows.
- Algorithmic Efficiency: Multi-objective optimization can incorporate energy and carbon footprint alongside model quality and system-resource trade-offs.
- System and Algorithm Efficiency: Resource-efficient architectures, efficient optimization, communication compression, and pipelining can reduce memory, training, or communication costs.
- System Utilization: More than tens of thousands of training workflows use GPUs at only 30–50% utilization, leaving room for efficiency improvement.
C. Efficient, Environmentally-Sustainable AI Infrastructure and System Hardware
Environmentally sustainable AI infrastructure requires lifecycle-aware hardware and software design, higher resource utilization, carbon-aware scheduling, and attention to edge-computing emissions. Operational efficiency alone does not determine total carbon footprint.
- Accelerator and system utilization should be maximized and infrastructure lifetimes prolonged to amortize embodied carbon.
- Accelerator Utilization: GPU virtualization and workload consolidation can improve accelerator utilization in machine-learning experimentation.
- Sustainable System Design: System design must consider operational and embodied footprints across the hardware and model-development lifecycle, not performance and power efficiency alone.
- Hardware Choices: The best hardware choice depends on algorithm and model evolution, use-case diversity, software maturity, deployment lifetime, operational efficiency, and embodied carbon.
- Carbon-Efficient Scheduling: Carbon-aware scheduling can exploit intermittent renewable generation across datacenters, though flexibility may require server over-provisioning.
- On-Device Learning: Federated learning emits non-negligible edge carbon through computation and wireless communication, despite avoiding raw-data collection.
- On-Device Learning: On-device AI can accumulate substantial emissions across billions of devices, where renewable energy is more limited than in datacenters.
V. CALL-TO-ACTION
The paper calls for accessible carbon measurement, lifecycle-aware comparison metrics, and routine disclosure of AI’s operational and embodied footprint. These practices would make environmental costs more visible in research and system development.
- Carbon-accounting telemetry should be easy to adopt so the community can quantify AI environmental footprints and progress.
- Standard AI carbon metrics should cover model stages from data through deployment and hardware stages from manufacturing through use.
- Environmental impact should complement efficiency measures in leaderboards and comparisons of AI data, models, algorithms, and hardware.
- Research papers should disclose operational and embodied carbon, while model cards should describe training and inference carbon footprints.
VI. KEY TAKEAWAYS
AI’s rapid growth requires assessing environmental impact across the full ML pipeline and hardware life cycle. Cross-stack optimization can substantially reduce operational energy and carbon footprints, while sustainable AI requires responsible development across multiple dimensions.
- Growth of AI: AI data, model, and infrastructure capacity have grown rapidly, increasing demands on ingestion bandwidth and training and inference capacity.Facebook’s recommendation data grew 2.4×, while training and inference capacity increased 2.9× and 2.5×, respectively.
- A Holistic Approach: Sustainable AI analysis must cover data collection, experimentation, training, optimization, inference, and system hardware manufacturing and operation.The paper frames environmental accounting across the end-to-end ML pipeline and the hardware life cycle.
- Efficiency Optimization: Cross-stack optimization reduced the Transformer-based universal translation model’s operational carbon footprint by 810× and operational energy footprint by 25.8%.The reported reductions span algorithms, platforms, infrastructures, and hardware.
- Future Directions: The paper charts research directions spanning data, algorithms and models, experimentation, system hardware, and telemetry for datacenter and edge AI.These directions support achieving competitive accuracy at fixed or reduced computational and environmental cost.
- Responsible Development: AI development should deliberately account for environmental implications so that advances in machine intelligence do not worsen climate change.The paper calls for environmentally sustainable AI technology development.
APPENDIX
The appendix describes measurement, scaling, pipeline, hardware, and edge-learning considerations that shape sustainable AI. It emphasizes Pareto-efficient scaling, disaggregated ML stages, longer hardware use, and accounting for embodied and operational carbon.
- Research Incentives: Over 90% of ML publications focus only on model accuracy improvements, indicating limited incentives for efficiency-oriented research.The paper identifies this lack of incentives as one factor behind underinvestment in minimizing AI costs.
- Measurement Challenges: There is no standard telemetry for accurate, reliable energy and carbon-footprint measurement, and datacenter, hardware, and energy-source factors can perturb results.The paper identifies measurement methodology as complex and sensitive to infrastructure choices.
- Normalization: Algorithmic progress is often reported without normalizing model accuracy by CPU, GPU, or TPU hours, energy use, or carbon footprint.The paper argues that resource requirements should accompany conventional quality metrics.
- Platform Fragmentation: Platform fragmentation makes efficiency portability difficult because implementation details can significantly affect real-world performance.Best practices for portable performance and efficiency remain elusive.
- Data and Model Scaling: Scaling data and model sizes together can reduce energy: the 2× data, 2× model setting used roughly 4× less energy than the 8× data, 16× model setting, with 0.004 Normalized Entropy degradation.Independent scaling deviates from the energy-optimal Pareto trend.
- Pipeline Disaggregation: Disaggregating data ingestion and preprocessing from model training increased overall training throughput by 56% and can lower embodied carbon through more efficient resource use.Checkpointing also improves fault tolerance by avoiding full experiment reruns after ingestion or preprocessing failures.
- Hardware Reliability: Extending hardware lifetime can amortize embodied carbon, but aging hardware may cause silent corruption, accuracy degradation, nondeterministic execution, or system failure.The paper discusses differential reliability as one architectural response to these risks.
- On-Device Learning: Federated learning and optimization can create non-negligible edge carbon emissions through both computation and wireless communication.The estimate used 90-day federated-learning log data and a methodology similar to prior work.
C. Efficiency and Self-Supervised Learning
Self-supervised learning reduces dependence on human labels but can require substantially more pre-training computation than supervised learning. Semi-supervised methods and foundation-model reuse offer potential ways to improve this cost-benefit trade-off.
- Self-Supervised Learning: Self-supervised learning trains neural networks without explicit human-annotated labels and is commonly used to pre-train foundation models.Foundation models can later be fine-tuned with smaller amounts of labeled data for downstream tasks.
- Efficiency Trade-off: 69.3% top-1 validation accuracy required 1000 SSL pre-training epochs, whereas supervised ResNet-50 training typically reached at least 76.1% after 90 epochs.In this example, supervised training reduced training effort by roughly 10×.
- Semi-Supervised Pre-Training: With labels for 10% of ImageNet images, PAWS achieved 75.5% top-1 accuracy after 200 epochs and roughly 16 hours on 64 V100 GPUs.The passage presents this as evidence that limited labels can narrow the efficiency gap.
- Amortization: A single expensive foundation-model pre-training run can potentially be amortized across many tasks through inexpensive fine-tuning.The paper states that the cost-benefit trade-off of this paradigm requires substantial additional research.