PaperlayerView exact revision on arXiv

Source-linked AI summary

GCAM-Gym: A Benchmark for Agent-Based Climate Policy Search

Tianyu Zhang, Zhenghan Tai, Xinyu Wang, Irina Rish, Yoshua Bengio

arXiv:2610.23569v1cs.CY

TL;DR

Comparing climate-policy search methods requires accounting for their controls, observations, feedback timing, and simulator budgets. GCAM-Gym supports full-policy and stepwise evaluation and compares seven systems in three settings; among four systems sharing an attempt budget, CMA-ES achieves the highest best-return IQM in every setting.

  • Problem

    Comparisons of climate-policy search methods must account for differences in policy control, available observations, feedback timing, and simulator budgets.

    Source: §1 · p. 2

  • Method

    GCAM-Gym provides a shared interface for complete carbon-price trajectories and stepwise control, exposes reward components, and evaluates seven systems across three settings.

    Source: §1 · p. 2

  • Results

    Among four systems with 24 attempts per seed, CMA-ES achieves the highest IQM of best returns from completed policy evaluations in all three settings.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The archive supports inspection and reconstruction of recorded comparisons, but validating policy outcomes and method rankings requires a qualified runtime and new evaluations with clean adaptive search histories.

    Source: §7 · p. 9

    The experiments cover global carbon-price control through 2060 under one GCAM configuration and fixed proxy objectives, while runtime issues leave outcome validation and method rankings unresolved.

    Source: §6 · p. 9, §5.4 · p. 9

Abstract

from arXiv · show

Comparing climate-policy search methods requires accounting for differences in policy control, available observations, feedback timing, and simulator budgets. We introduce GCAM-Gym, a Gymnasium-compatible environment that supports full-policy evaluation and stepwise control through a common interface and exposes the components of the policy objective. We compare seven search and control systems in three settings: target attainment and cost, cost-temperature trade-offs, and one-step delayed feedback. Each system is evaluated with ten seeds per setting, for a total of 3,210 attempted GCAM evaluations. Four systems share a budget of 24 attempts per seed; among these, CMA-ES achieves the highest interquartile mean (IQM) of the best returns from completed policy evaluations in all three settings. We release the evaluation archive and analysis tools to reconstruct the reported comparisons without running GCAM.

1 INTRODUCTION

Comparing climate-policy search requires accounting for policy control, observations and feedback timing, and simulator budgets. GCAM-Gym provides a shared environment for full-policy and stepwise evaluation; among four equally budgeted systems, CMA-ES has the highest best-return IQM in all three settings.

1 INTRODUCTION

  • A policy-search benchmark must specify what systems control, which observations they receive and when, and how many evaluation attempts they use, including failures.It should link proposed policies to executed trajectories, feedback histories, and evaluation outcomes.Source: §1 · p. 2
  • GCAM-Gym provides a Gymnasium-compatible environment for full-policy evaluation and stepwise control through a shared action decoder and GCAM backend.It returns structured observations, cost and distributional proxies, and reward components; a wrapper supports one-step delayed feedback.Source: §1 · p. 2
  • The study compares seven systems across three settings, with ten seeds per system in each setting.The systems span fixed policies, schedule search, and observation-driven control.Source: §1 · p. 2
  • CMA-ES achieves the highest IQM of best returns from completed policy evaluations among four systems sharing 24 attempts per seed in all three settings.The comparisons cover target attainment and cost, cost–temperature trade-offs, and one-step delayed feedback.Source: §1 · p. 2

2 RELATED WORK

GCAM-Gym is positioned as a resource for evaluating single-agent global carbon-price trajectories in a coupled climate simulator. The related work distinguishes its evaluation scope and records from existing climate-policy environments, optimizer benchmarks, feasibility methods, language-model approaches, and baseline-statistics practices.

Climate-policy environments

  • GCAM-Gym focuses on single-agent global carbon-price control, evaluating a policy trajectory through a shared simulator interface with time-indexed responses retained for inspection.Its three evaluation settings differ from Shared Socioeconomic Pathways and robust decision making; the AI Economist instead studies taxation with coadapting agents and a planner.Source: §2 · p. 2

Reusable optimization evaluation

  • Each attempted evaluation records actions, observations, reward components, and execution status, supporting endpoint comparisons and inspection of when feedback reaches a controller.This extends the resource-oriented approach of reusable optimizer benchmarks to policy trajectories in a coupled climate simulator.Source: §2 · pp. 2–3

Search objectives and feasibility

  • The adapters use penalty scores for failures rather than inheriting explicit constraint-modeling or safe-exploration guarantees, and attained cost–temperature pairs are not produced by a Pareto-search algorithm.The related work contrasts these choices with constrained optimization and NSGA-II.Source: §2 · p. 3

Language models and delayed interaction

  • LLM+BO uses language-generated initialization followed by conventional GP search, while the delayed-feedback setting applies a fixed one-step lag.This differs from reproducing LLAMBO and from work that explicitly models randomly delayed interaction.Source: §2 · p. 3

Baseline methods and statistics

  • The suite uses CMA-ES, Gaussian-process optimization, PPO, and a ReAct adaptation, with configurations including initialization, pretraining, and failure feedback recorded.Aggregate reporting uses IQMs and seed-bootstrap intervals, with independent-unit and aggregation conventions specified in the study.Source: §2 · p. 3

3 THE GCAM-GYM INTERFACE

GCAM-Gym provides a shared interface for evaluating full policy schedules and stepwise control, with structured observations, rewards, and execution records. Its evaluation settings vary outcome summaries and feedback timing while sharing the simulator configuration, horizon, and scalar objective.

3.1 STATE, OBSERVATION, AND ACTION SPACES

  • Observations contain twelve climate, energy, cost, and distributional fields that flatten to 22 scalars; their Markov sufficiency is not assumed.The solver-iteration field is a constant-zero placeholder, and execution status is returned separately.Source: §3.1 · p. 3
  • The API exposes four normalized controls, but only carbon price affects the native simulator in this study.Schedule optimizers choose a complete trajectory before execution, whereas stepwise controllers choose actions from available feedback; search-domain restrictions differ by system.Source: §3.1 · pp. 3–4

3.2 OBJECTIVES AND RECORDED OUTCOMES

  • The undiscounted episode reward combines climate, cost, equity, and electricity-price stability components, with fixed parameters across settings.The climate term penalizes temperatures above Tref, while the stability term penalizes relative electricity-price increases above δE.Source: §3.2 · pp. 4–5
  • The accumulated cost measure K is a clipped revenue-to-output proxy, not discounted consumption loss or calibrated welfare.Its monetary units follow the native convention, with no currency-year adjustment; the input datasets’ monetary units have not been independently validated.Source: §3.1 · p. 4, §3.2 · p. 5
  • Evaluation records link proposed policies to executed actions, feedback, and execution status; failed attempts consume budget and preserve available trajectory prefixes.The reported incumbent is the best complete-episode return, while CMA-ES and BO adapters assign penalized scores to failures in subsequent proposals.Source: §3.2 · p. 6

3.3 THREE EVALUATION SETTINGS

  • The three settings assess target attainment and cost, cost–temperature outcomes, and one-step delayed feedback; schedule-search systems precommit actions in the delayed setting.For target and cost, the minimum-cost target-reaching policy may differ from the best-return policy; target-reaching cost is undefined when no qualifying policy is found.Source: §3.3 · p. 6

4 EXPERIMENTAL PROTOCOL

The protocol configures seven systems with different policy representations and feedback rules, then compares them across three separately executed settings using attempt-level budgets and defined endpoint readouts.

4.1 SEVEN CONFIGURED SYSTEMS

  • CMA-ES and both Bayesian-optimization systems search complete schedules, while PPO and LLM-ReAct choose actions from stepwise observations.The comparison also includes random search and a fixed heuristic; the two BO systems use the same GP/log-expected-improvement implementation after different initial designs.Source: §4.1 · pp. 6–7
  • PPO uses fixed mock-trained models, whereas LLM-ReAct receives recent action, reward, and status entries when choosing actions.PPO models remain fixed across repeated evaluations; LLM-ReAct receives the latest three action/reward/status entries.Source: §4.1 · p. 7

4.2 ALLOCATION AND STATISTICAL READOUTS

  • The three settings comprise 210 system–setting–seed cells and 3,210 attempted evaluations, with four systems sharing 24 attempts per cell.Budgets count failed attempts; mock pretraining, LLM inference, and some preempted work or optimizer overhead are outside the attempt counts.Source: §4.2 · p. 7
  • The seed-level incumbent is the best complete return among completed evaluations within the first b attempts, and the IQM is reported only when all ten incumbents are defined.The IQM uses the middle 50% of empirical mass with fractional weights at quartile boundaries; uncertainty intervals use 10,000 bootstrap resamples of the ten incumbents.Source: §4.2 · p. 7
  • Target attainment requires at least one complete policy with TH ≤ Tgoal, while the cost endpoint is the minimum accumulated K among qualifying policies.An all-seed cost summary is undefined unless every seed reaches the target.Source: §4.2 · p. 7
  • Matched-seed delayed-feedback contrasts summarize ten differences, but they do not identify a causal effect of delay.Matching labels does not control all variation in policy generation, simulator execution, or interruptions.Source: §4.2 · p. 7
  • Table 3 reports best-return IQM and 95% seed-bootstrap intervals, completion as completed/all attempts, and target-reaching seeds.Higher best-return values are better.Source: Table 3 · p. 8

5 EVALUATION RESULTS

Across the three settings, CMA-ES leads the four equal-budget systems on best-return IQM, while target attainment, cost–temperature outcomes, completion, and runtime measurement issues qualify the comparisons.

5.1 RECORDED RETURNS AND SEARCH PROGRESS

  • CMA-ES has the highest best-return IQM among the four 24-attempt systems in all three settings: −0.708, −0.724, and −0.708, respectively.LLM+BO's IQM exceeds GP-BO's in each setting, but rank-based tests do not detect a difference between their seed-level endpoints after Holm correction (adjusted p = 0.7099 in each setting); the tests address endpoint distributions, not the IQM difference specifically.Source: §5.1 · p. 8

5.2 TARGET COST AND COST–TEMPERATURE OUTCOMES

  • CMA-ES, GP-BO, and LLM+BO reach TH ≤ Tgoal in all ten seeds, with minimum qualifying-cost IQMs of 0.333, 0.458, and 0.434 proxy-fraction years, respectively.Random search reaches the criterion in eight seeds and LLM-ReAct in six; the heuristic and PPO do not reach it in any seed.Source: §5.2 · p. 8
  • The pooled cost–temperature point set contains 15 distinct nondominated coordinates, including zero-cost PPO and negative-temperature points from GP-BO and LLM+BO.Runtime defects documented in Section 5.4 can change the dominance relations.Source: §5.2 · p. 9

5.3 COMPLETION AND TARGET ATTAINMENT

  • Of 3,210 attempts, 2,493 completed; the other 717 comprise 714 numerical failures and three infrastructure failures.PPO completes all evaluations without meeting the temperature criterion; in target-and-cost, CMA-ES completes 194 of 240 attempts and reaches the target in every seed.Source: §5.3 · p. 9
  • Under delayed feedback, the median seed-level contrast is −0.014 for LLM-ReAct and +0.016 for LLM+BO, while PPO retains zero carbon-price actions.The appendix reports ranges, sign counts, and executed-sequence comparisons.Source: §5.3 · p. 9

5.4 MEASUREMENT SCOPE

  • Runtime diagnostics found state-mutating residual checking, a residual-scale error, and recovery/read paths that bypassed required validation.Omitting the checking path changed one recorded endpoint temperature from −0.6625°C to 1.964°C in a diagnostic, not a validated replacement runtime; effects on relative performance remain unresolved, and validation requires new evaluations with clean adaptive search histories.Source: §5.4 · p. 9

6 LIMITATIONS

The experiments cover global carbon-price control through 2060 under one GCAM configuration, fixed proxy objectives, and system-specific attempt budgets.

6 LIMITATIONS

  • The evaluation scope is global carbon-price control through 2060 under a single GCAM configuration, fixed proxy objectives, and system-specific attempt budgets.Runtime measurement issues are documented separately in Section 5.4.Source: §6 · p. 9

7 CONCLUSION

The paper presents GCAM-Gym and an archive for inspecting recorded comparisons, while distinguishing those comparisons from validated policy outcomes and rankings.

7 CONCLUSION

  • GCAM-Gym combines a shared policy-evaluation interface, configured search and control systems, and attempt-level records with numerical replay.In archived runs, CMA-ES has the highest best-return IQM among four equal-attempt-budget systems, and execution completion does not guarantee target attainment; validating outcomes and rankings requires a qualified runtime and new evaluations with clean adaptive search histories.Source: §7 · p. 9

AI USE STATEMENT

The authors report using generative AI across coding, experiment management, manuscript editing, and draft figure illustration, with authors reviewing the outputs and taking responsibility for the final materials.

AI USE STATEMENT

  • Generative AI assisted with coding, experiment management, manuscript editing, and draft figure illustrations; the authors reviewed the outputs and take responsibility for the final manuscript and artifacts.Source: AI USE STATEMENT · p. 10

A SPACE BOUNDS, UNIT CONVENTIONS, AND QUERY FALLBACKS

The appendix documents the archived environment’s observation bounds, unit and query conventions, fallbacks, and baseline configurations. It distinguishes declared API behavior from archived observations and identifies provenance limits on reproducing optimizer implementations.

A SPACE BOUNDS, UNIT CONVENTIONS, AND QUERY FALLBACKS

  • Declared observation bounds apply coordinatewise, but archived observations violating those bounds are retained.Regional arrays and electricity-share coordinates follow specified group orders; flattening follows Gymnasium dictionary-space order, while LLM serialization sorts field keys.Source: A SPACE BOUNDS, UNIT CONVENTIONS, AND… · p. 12, Table 5 · p. 12
  • Reported emissions and sequestration convert native query sums to GtCO2/yr, while the cost proxy uses native emissions and GDP demand sums.Interpreting the proxy as a revenue-to-output fraction requires compatible monetary units; independent dataset-level validation remains outstanding.Source: A SPACE BOUNDS, UNIT CONVENTIONS, AND… · p. 13
  • The temperature channel selects the last matching non-spinup global_tas entry for the requested year, and its reward and target use an unresolved reference-period convention.Re-execution requires the corresponding Hector configuration.Source: A SPACE BOUNDS, UNIT CONVENTIONS, AND… · p. 13
  • On reported solver failure, scaled running component minima replace usual reward components, while incomplete trajectories have no complete-policy incumbent value in the reported analysis.The interaction penalty uses the successful-period reward weights and differs from optimizer training scores.Source: A SPACE BOUNDS, UNIT CONVENTIONS, AND… · p. 13
  • Query fallbacks can substitute zeros or previous-year climate observations, so these values are implementation behaviors rather than independent zero observations.Missing current electricity prices can produce NaN, and the stability expression has no added guard for a zero previous index.Source: A SPACE BOUNDS, UNIT CONVENTIONS, AND… · p. 13

B BASELINE CONFIGURATION AND FEEDBACK

  • CMA-ES uses recorded bounds [0, 1]7, initial standard deviation 0.2, and population size six; 24 attempts cover four populations.The inspected adapter sets its initial center to (0.25, . . . , 0.25), and coordinates use the shared nonlinear price map.Source: B BASELINE CONFIGURATION AND FEEDBACK · p. 13
  • GP-BO uses three recorded Sobol initial points and log expected improvement; LLM+BO uses ten manifest-based initial candidates before the same GP/log-EI search.LLM+BO’s initial candidates are nondecreasing integer price schedules, while later search uses the full normalized coordinate box.Source: B BASELINE CONFIGURATION AND FEEDBACK · p. 13
  • The exported package preserves environment and worker definitions but omits native binaries and optimizer dependencies, and not every inspected adapter detail is bound to contemporaneous implementation bytes.The README distinguishes these provenance levels from portable arithmetic replay.Source: B BASELINE CONFIGURATION AND FEEDBACK · p. 14

C STATISTICAL DETAILS AND ADDITIONAL VIEWS

This section reports the statistical and plotting conventions for evaluation progress and delayed-feedback contrasts. The contrasts are descriptive: return shifts and target discovery can differ, and comparisons do not isolate a causal effect of delay.

C STATISTICAL DETAILS AND ADDITIONAL VIEWS

  • At prefixes where any seed lacks a complete policy, the all-seed return IQM is undefined; minimum target-cost IQMs likewise require all ten seeds to have target-reaching policies.Degenerate observed bootstrap intervals do not imply certainty about another training run or simulator configuration.Source: C STATISTICAL DETAILS AND ADDITIONAL… · p. 14
  • The archived analysis uses 10,000 percentile seed-bootstrap resamples and Holm adjustment across an 18-comparison family; GP-BO versus LLM+BO has adjusted rank-test p = 0.7099 in each view.The portable replay reconstructs input summaries but does not rerun the resampling or permutation procedures.Source: C STATISTICAL DETAILS AND ADDITIONAL… · p. 14
  • Figure 3 plots target-and-cost best complete return against attempted evaluations using ten-seed IQMs and pointwise 95% seed-bootstrap intervals.Failed attempts count toward budgets, estimates appear only when all ten seeds have complete returns, and the axis excludes pretraining and LLM inference.Source: Figure 3 · p. 15
  • Figure 5 shows best-return IQMs and the fraction of seeds with a complete policy across all three settings, with traces ending at each system’s actual budget.PPO traces evaluate mock-pretrained policies, not native learning; pretraining and LLM inference costs are excluded.Source: Figure 5 · p. 16

Descriptive delayed-feedback contrasts

  • LLM-ReAct’s median paired-label return difference is −0.014, yet its target-reaching count rises from six to eight seeds; return change and target discovery answer different questions.LLM+BO’s median is +0.016, but its schedule search does not use stepwise feedback, so that contrast is not evidence of a within-episode delay response.Source: C STATISTICAL DETAILS AND ADDITIONAL… · p. 16

C STATISTICAL DETAILS AND ADDITIONAL VIEWS

  • For seven-step delayed-feedback episodes, rewards from steps one to five arrive one step late, while the terminal call returns queued rewards from steps six and seven.Paired action-array and decoded-price-sequence differences can also reflect shorter failed prefixes, not deliberate changes to full proposed schedules.Source: C STATISTICAL DETAILS AND ADDITIONAL… · pp. 16–17
  • PPO’s matched-cell action-array difference changes only inactive controls, because its carbon-price coordinate is zero in all 150 native evaluations.The recorded comparisons describe behavior rather than isolate a causal effect of delay.Source: C STATISTICAL DETAILS AND ADDITIONAL… · p. 17
  • The 834 cost–temperature points include every successful evaluation, while failed attempts contribute no complete coordinate pair and repeated policies remain separately recorded.Overlapping markers do not make duplicate points independent physical-policy observations.Source: C STATISTICAL DETAILS AND ADDITIONAL… · p. 17

D RUNTIME EVIDENCE AND STATUS SEMANTICS

The runtime audit documents status counts, out-of-bounds observations, and diagnostic evidence of measurement defects in archived runs. The archive supports arithmetic replay, but physical interpretation and new comparisons require validated runtime inputs and, for adaptive search, rerunning proposal generation.

D RUNTIME EVIDENCE AND STATUS SEMANTICS

  • The 3,210 attempts comprise 2,493 successes, 716 solver failures, and one API failure under original worker labels; a post-collection taxonomy instead records 714 numerical and three infrastructure failures.The infrastructure sensitivity excludes three complete cells but does not replace the main endpoint summaries.Source: D RUNTIME EVIDENCE AND STATUS SEMANTICS · p. 17
  • Among 2,493 recorded successful trajectories, 57 violated the declared observation space across 43 method–view–seed cells.The audit preserved these records without clipping temperatures or deleting them; the violations were not new physical rejection thresholds.Source: D RUNTIME EVIDENCE AND STATUS SEMANTICS · p. 17
  • With the archived residual-checking path omitted diagnostically, the zero-price policy’s recorded 2060 temperature changed from 2.256°C to 2.291°C, and the selected GP-BO policy’s changed from −0.6625°C to 1.964°C.For the selected GP-BO policy, CO2 also changed from 113.7 to 467.5 ppm; the diagnostic did not establish a replacement convergence test.Source: D RUNTIME EVIDENCE AND STATUS SEMANTICS · p. 17
  • The audit found an extra quantity-scale factor in the old residual ratio and fail-open recovery/read paths, including substitution of previous-year climate observations.Rescoring selected policies cannot reconstruct adaptive search histories because earlier feedback affects later proposals.Source: D RUNTIME EVIDENCE AND STATUS SEMANTICS · p. 17

E REUSE REQUIREMENTS

  • Replay verifies arithmetic from selected records but does not rerun GCAM; physical interpretation of a new comparison requires runtime and measurement validation.The archive omits native binaries, initialization artifacts, and trained PPO checkpoints; broader controls or horizons beyond carbon-price study through 2060 require interface and evaluation-design extensions.Source: E REUSE REQUIREMENTS · p. 18

Related papers

Loading 2610.23569v1…