Stratospheric Aerosol Injection as a Deep Reinforcement Learning Problem

Christian Schroeder de Witt; Thomas Hornigold

Source-linked AI summary

Stratospheric Aerosol Injection as a Deep Reinforcement Learning Problem

Christian Schroeder de Witt, Thomas Hornigold

arXiv:1905.07366v1May 17, 2019cs.LGphysics.ao-phstat.ML

TL;DR

SAI offers a way to offset greenhouse-gas forcing, but naive deployment can produce severe regional effects and current control studies remain low-dimensional. The paper proposes high-dimensional DRL control with a deep-learning HadCM3 emulator and context-sensitive rewards. It presents this as a proposed first application of DRL to climate sciences, while recognizing model-response and aerosol-representation limitations.

Problem
SAI control is studied with low-dimensional approximations despite sensitive regional effects and potentially serious geostrategic consequences.
Source: §1 · p. 1, §2 · p. 2
Method
The paper formulates SAI as a high-dimensional MDP, trains DRL policies with context-sensitive rewards, and proposes a deep-learning emulator of HadCM3.
Source: §1 · p. 1, §3 · p. 2, §4 · p. 2
Results
The paper proposes a fast GCM-emulation and DRL framework for discovering non-trivial or catastrophic SAI policies and identifies it as the first proposed DRL application to climate sciences.
Source: §1 · p. 1, §3 · p. 2, §5 · p. 3
Takeaways & Limitations
DRL may become an important tool for studying SAI and other geoengineering approaches such as marine cloud brightening.
Source: §5 · p. 3
Aerosol models have substantially different responses to identical injection strategies, and the emulator assumes zonal uniformity, annual decay, and a 4ρ concentration cap.
Source: §2 · p. 2, §3 · p. 2

Abstract

from arXiv · show

As global greenhouse gas emissions continue to rise, the use of stratospheric aerosol injection (SAI), a form of solar geoengineering, is increasingly considered in order to artificially mitigate climate change effects. However, initial research in simulation suggests that naive SAI can have catastrophic regional consequences, which may induce serious geostrategic conflicts. Current geo-engineering research treats SAI control in low-dimensional approximation only. We suggest treating SAI as a high-dimensional control problem, with policies trained according to a context-sensitive reward function within the Deep Reinforcement Learning (DRL) paradigm. In order to facilitate training in simulation, we suggest to emulate HadCM3, a widely used General Circulation Model, using deep learning techniques. We believe this is the first application of DRL to the climate sciences.

1. Introduction

SAI is increasingly considered to offset greenhouse-gas forcing, but naive deployment can produce adversarial regional effects. The paper frames optimal SAI as a high-dimensional control problem for deep reinforcement learning and proposes GCM emulation to make training feasible.

SAI reflects incoming solar radiation and is considered a technically and economically feasible way to offset greenhouse-gas forcing.Source: §1 · p. 1
Naive SAI can produce highly adversarial regional climate effects, including in India and West Africa.Source: §1 · p. 1
Unilateral injections could disrupt the ITCZ and monsoons, while premature cessation could cause rapid warming, making SAI control geostrategically important.Source: §1 · p. 1
The paper models optimal SAI control as a high-dimensional MDP and proposes DRL to discover time-varying policies or catastrophic ones.Source: §1 · p. 1
A deep-learning emulator of a GCM is proposed to reduce DRL’s sample inefficiency; the authors present this as the first proposed DRL application to climate sciences.Source: §1 · p. 1

2. Related work

Existing SAI-control studies use low-dimensional parameterizations or manual searches despite sensitive regional and super-regional responses. Fine-grained injection may be possible, but model disagreement limits confidence in simulated strategies.

Deep-learning emulation of GCMs and sub-grid processes suggests that predicting regional precipitation and surface-temperature responses to aerosol forcing may be feasible.Source: §2 · p. 1
Current SAI-control studies are constrained to low-dimensional injection patterns or manual searches over selected edge cases.Source: §2 · p. 1
Regional climate responses are sensitive to SAI policy, with demonstrated impacts extending to El Niño/Southern Oscillation.Source: §2 · pp. 1–2
Changing injection altitude, latitude, season, or particle type could tailor SAI, but different aerosol models produce substantially different responses to identical strategies.Source: §2 · p. 2

3. GCM emulation

The paper uses HadCM3 as the climate simulator and seeks a millisecond-scale deep-learning emulator because even one simulated year is computationally expensive. The emulator predicts climate fields from aerosol-forcing states, with preliminary results motivating auxiliary cloud-cover tasks.

HadCM3 is selected because it avoids flux adjustments for large-scale climate drift and remains an IPCC baseline model.Source: §3 · p. 2
The aerosol representation assumes zonal uniformity by latitude band, annual decay, and a 4ρ concentration cap.Source: §3 · p. 2
A HadCM3 year still requires about 15 CPU hours, motivating an emulator that predicts next states in milliseconds.Source: §3 · p. 2
The emulator represents sea ice fraction, surface temperature, depth layer-weighted ocean heat content, and stratospheric aerosol optical depth, predicting next-step sea ice, ocean heat content, temperature, and local precipitation on 73 × 96 scalar grids.Source: §3 · p. 2
A UNet-like encoder-decoder is fine-tuned on 2000 HadCM3 rollout samples generated from randomized 73-dimensional aerosol distributions under a coagulation cap.Source: §3 · p. 2
Preliminary simulations suggest that emulator training would benefit from auxiliary cloud-cover prediction tasks.Source: §3 · p. 2

4. Reinforcement learning setting

The proposed control setting treats aerosol injections across 73 latitude bands as actions in an MDP and learns policies using factorized deep Q-learning. Rewards penalize regional precipitation and temperature departures, while learned policies require cross-verification in GCMs.

Each yearly decision selects aerosol injections across 73 latitude bands, and the environment returns a scalar reward for policy learning.Source: §4 · p. 2
Off-policy deep Q-learning discretizes each latitude-band action into 10 bins, creating a 10^73 joint action space that is handled by value-function factorization.Source: §4 · p. 2
The reward discourages extreme regional climate changes using precipitation and surface-temperature departures from pre-industrial averages, with tunable weights.Source: §4 · p. 2
More advanced rewards could incorporate population density, resilience, socioeconomic interests, climate adaptability, and regional climate characteristics.Source: §4 · p. 2
Policies learned in the emulator are subsequently cross-verified in HadCM3 or other GCMs for physical consistency and robustness.Source: §4 · p. 2
Figure 1 organizes full-HadCM3 cooling, radiative fluxes, and aerosol-optical-depth distributions, with the latter three views taken from the same run and time.Source: Figure 1 · p. 3

5. Conclusion and Outlook

The paper proposes studying optimal SAI control as a high-dimensional problem using a fast GCM emulator and deep reinforcement learning. It suggests DRL could become an important tool for SAI and other geoengineering approaches.

The paper proposes combining a fast GCM emulator with deep reinforcement learning to study optimal SAI control as a high-dimensional control problem.Source: §5 · p. 3
This approach formulates optimal SAI control as a high-dimensional control problem using a fast GCM emulator and deep reinforcement learning.Source: §5 · p. 3
DRL may become an important tool for studying SAI and other geoengineering approaches, including marine cloud brightening, over the next decade.Source: §5 · p. 3

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning2020