PaperlayerView exact revision on arXiv

Source-linked AI summary

From Weather to Enhanced Weathering: Weather Foundation Models for Carbon Dioxide Removal at Operational Scale

Sasankh Munukutla, Felipe Oviedo, Danrong Zhang, Kenji Takeda, Juan M Lavista Ferres

arXiv:2610.28973v1cs.LGcs.AI

TL;DR

Enhanced Rock Weathering has the potential to remove gigatons of CO2, but quantifying removal requires dense sampling and laboratory analysis. This study evaluates five weather Foundation Models on operational ERW sequences, finding that their representations transfer to ERW and that TerraNova-ERW positively orders measurements in unseen fields, while unseen-field mean prediction remains limited.

  • Problem

    Quantifying ERW removal requires dense physical ground-truth sampling and laboratory analysis, while carbon dioxide removal is needed in nearly every mitigation pathway to balance hard-to-abate residual emissions.

    Source: §1 · pp. 1–2

  • Method

    The study evaluates five weather Foundation Models as frozen encoders with lightweight decoders on operational ERW sequences from 264 fields across Brazil and the US.

    Source: §1 · p. 2, §2.1 · p. 2

  • Results

    Weather-FM representations transfer across backbones, and TerraNova-ERW positively ranks held-out measurements with pooled point Spearman +0.29, while unseen-field mean error is only marginally below the shuffled-ERA5 reference.

    Source: §3 · pp. 3–4

  • Takeaways & Limitations

    TerraNova-ERW complements rather than replaces physical sampling, supporting sparser, targeted sampling designs and reducing sampling by 28–98% on the evaluated fields.

    Source: §5 · p. 5

    Deployment data covers only two countries and two feedstock types, monitoring horizons are limited to date, and LOFO does not cover entirely new regions.

    Source: §5 · p. 5

Abstract

from arXiv · show

Enhanced Rock Weathering (ERW) is a carbon removal solution with the potential to remove gigatons of CO$_2$ per year, but quantifying that removal today requires dense soil sampling and laboratory analysis. We evaluate five weather Foundation Models (FMs) as frozen encoders feeding lightweight decoders, under a leave-one-field-out protocol on 5,192 chronosequences from 264 fields across Brazil and the US. To our knowledge, this is the first holistic evaluation of weather FMs for ERW and the largest evaluation on operational ERW data. We find that across weather FMs, representations transfer to ERW: our best model, TerraNova-ERW, built on Aurora 1.5, positively orders measurements within fields it has never seen. Zero-shot prediction of a new field's mean removal is not yet feasible, and we attribute this to data scale: the same architecture excels on $\sim$$10^5$ natural-weathering sequences, anchoring the scale we expect ERW data must reach. Few-shot prediction, blending the model with a few samples per field, is supported today: TerraNova-ERW recovers over 60% of the highest-removal fields with two samples per field, roughly halves the samples needed to estimate a field's mean removal, and can cut sampling on our densest fields by 28-98%.

1 Introduction

Enhanced Rock Weathering could contribute gigatons of annual carbon removal, but measuring its removal requires dense sampling and laboratory analysis. This work evaluates weather Foundation Models on operational ERW data and investigates their use in reducing sampling requirements.

1 Introduction

  • Nearly every climate-mitigation pathway requires carbon dioxide removal to balance hard-to-abate residual emissions, with about 10 gigatons required annually by 2050.Source: §1 · p. 1
  • ERW could remove gigatons of CO2 annually at global scale while providing agronomic co-benefits such as improved crop yields and soil health.It accelerates natural weathering by applying finely crushed silicate rocks, such as basalt, to agricultural soils.Source: §1 · p. 1
  • Dense physical ground-truth sampling and laboratory analysis are required to quantify ERW carbon removal, making measurement, reporting, and verification a key scaling obstacle.Process-based models also require explicit process definitions and can be difficult to parameterize and computationally expensive to scale.Source: §1 · pp. 1–2
  • Weather Foundation Models pretrained on decades of reanalysis data provide representations of atmospheric drivers, including precipitation, temperature, and soil moisture, that govern weathering rates.These representations are considered alongside process-based models and deep learning approaches for ERW.Source: §1 · p. 2
  • Building on Munukutla et al., this work evaluates five weather Foundation Models on operational ERW data from Brazil and the US and examines how TerraNova-ERW could reduce sampling.The authors describe this as the first holistic evaluation of multiple weather FMs for ERW and the largest evaluation on operational ERW data.Source: §1 · p. 2

2 Methods

The methods formulate ERW as sequence regression, pairing weather representations and site-specific features with a later carbon-removal rate. The study compares five weather-model backbones using lightweight decoders and evaluates performance with field-disjoint protocols and metrics.

2.1 Task and Dataset

  • The task predicts the next-time-point CDR rate from monthly historical atmospheric representations and site-specific features; the dataset contains 5,192 operational ERW sequences from 264 fields and 192,511 global natural-weathering sequences.Main ERW results use the ERW sequences; natural-weathering data is used only for joint training and a separate scale study.Source: §2.1 · p. 2

2.2 Model Architecture

  • Five pretrained, frozen weather FMs provide pooled local, regional, and global monthly latents, which are integrated over each chronosequence and fused with site features by a lightweight decoder.TerraNova-ERW uses Aurora 1.5 with a light-wide decoder and 64d latent projection; the selected configuration uses unstandardized location inputs and validation-RMSE early stopping.Source: §2.2 · pp. 2–3

2.3 Training and Evaluation

  • Leave-One-Field-Out evaluation merges nearby fields into components to limit spatial leakage, tests point-level ordering and unseen-field means, and compares single-design-choice experiments using MAE, RMSE, and Spearman correlation.Uncertainty is bootstrapped over held-out fields; point-level % best measures the share of closable error removed relative to a fold’s treated-training-mean constant, with each field’s median or mean defining the best case.Source: §2.3 · p. 3

3 Results

Weather FM representations transfer to ERW prediction across architectures, with Aurora 1.5 selected as the strongest and stable model. TerraNova-ERW positively ranks measurements in unseen fields, but its absolute estimates of unseen-field means remain close to a no-information reference.

3 Results

  • Aurora 1.5 with a light-wide decoder achieved the best all-ERW metrics: point MAE 3.06, RMSE 4.20, Spearman +0.29; site MAE 1.78, RMSE 2.57, Spearman +0.26.It was stable on every seed and selected among about a dozen decoder and training variants evaluated on the same 68 folds.Source: Table 1 · p. 3, §3 · p. 4
  • Across weather FM backbones, point MAE was 3.06–3.26 and Spearman +0.12 to +0.29, while shuffled weather yielded MAE 3.32 and Spearman −0.01.TerraNova-ERW removed 41% of MAE and 54% of RMSE error, versus 19% and 18% for raw ERA5; FM encodings also held positive ranking across seeds while raw ERA5 was unstable.Source: §3 · pp. 3–4
  • TerraNova-ERW positively orders held-out measurements within unseen fields with pooled Spearman +0.29, but unseen-field site MAE is 1.78 ±0.18 versus 1.96 for Shuffled ERA5.The results support model-and-measure MRV, but not fully model-based quantification of unseen fields.Source: §3 · p. 4
  • Trained and evaluated exclusively on natural-weathering chronosequences, TerraNova-ERW reached validation MAE 0.014 and RMSE 0.017, roughly 90% lower error than the deployable constant predictor.The evaluation used ∼105 diverse sequences; this result anchors expectations for the data scale needed for fully model-based MRV for ERW.Source: §3 · p. 4

4 Operational Deployment of TerraNova-ERW

TerraNova-ERW can help prioritize sites and reduce sampling when paired with physical measurements. A sampling-budget simulation finds gains in recovering high-removal fields, estimating field means, and saving samples on the three densest fields.

4 Operational Deployment of TerraNova-ERW

  • With two samples per field, the hybrid recovered 63% of the ten highest-removal fields on average, versus 49% for sampling alone; the model alone recovered 50%.The figure’s panel (a) shows recovery among the ten highest-removal fields when ranking 52 fields.Source: Figure 1 · p. 4, §4 · p. 5
  • At every budget below about 10 points per field, the hybrid had lower median field-mean error than sampling alone; at one point, errors were 1.13 versus 2.18 t CO2e/ha/yr.The hybrid at three points matched sampling alone at five to ten points.Source: Figure 1 · p. 4, §4 · p. 5
  • On the three densest fields, the hybrid saved 28–98% of samples, while Aurora + Amendment matched full-input point-level error and achieved site Spearman +0.32 versus +0.13 for Aurora 1.5 (base).Aurora + Amendment uses weather latents and planned amendment specification without site characterization; its ranking advantage supports prioritizing sites rather than quantifying them.Source: §4 · pp. 4–5

5 Discussion

TerraNova-ERW is presented as a complement to physical sampling that could support targeted, sparser MRV. The evidence is bounded by the deployment data and monitoring horizons available so far.

5 Discussion

  • Sampling reductions of 28–98% target the largest MRV cost component, since sampling and laboratory analysis dominate MRV spend.MRV accounts for a third to over half of ERW cost, which can reach $275/tCO2.Source: §5 · p. 5
  • The model complements rather than replaces physical sampling, supporting sparser, targeted sampling designs and site prioritization.The marginal modeling cost of a new site is near zero because latents can be precomputed globally and the decoder is lightweight.Source: §5 · p. 5
  • Deployment data covers only two countries and two feedstock types, monitoring horizons extend only to date, and LOFO does not cover entirely new regions.Further validation across lithologies, climates, regions, and longer horizons, together with more ERW training data, is identified as needed for more robust deployment.Source: §5 · p. 5

A1 Architecture Diagram

TerraNova-ERW processes weather-model latents at local, regional, and global scales, combines them with site-specific features, and predicts CDR rate. The weather encoder is frozen; only downstream components are trained.

A1 Architecture Diagram

  • A frozen weather foundation-model encoder maps ERA5 inputs to monthly latents, with Aurora 1.5 as the default and four alternative backbones available.The alternatives are Pangu-Weather, SFNO, FCN3, and GraphCast.Source: Figure 2 · p. 8
  • Local, regional, and global latent features are pooled and aggregated across the observation window using a time-weighted trapezoidal mean.The local feature is the field-location cell, regional is a 3×3 mean, and global is the map mean.Source: Figure 2 · p. 8
  • The aggregated 3072-d vector is projected to 64-d and concatenated with five 64-d site-feature encodings before a three-layer MLP predicts CDR rate.Only the projection, site-specific feature encoders, and prediction head are trained.Source: Figure 2 · p. 8

A2 Feature Variables

The predictor uses weather-model latents alongside site, amendment, spatial, and temporal information, with measured operational ERW CDR rates as its target. The feature table distinguishes model inputs from sources and processing used for unavailable or normalized variables.

A2 Feature Variables

  • Aurora 1.5 receives 120 ERA5 fields at two monthly history timesteps, while its pooled backbone latent—not the ERA5 variables directly—enters the predictor.The input set comprises 18 surface fields plus computed insolation, 36 static fields, and five atmospheric variables at 13 pressure levels.Source: Table 2 · p. 8
  • Site-specific inputs cover soil, spatial and temporal variables, and amendment properties, with external sources used when original dataset variables are unavailable.Spatial and temporal variables are normalized before use; the listed categories include static and dynamic soil features and amendment composition and application properties.Source: Table 2 · p. 8, A2 Feature Variables · p. 9
  • Operational ERW labels are annualized CDR rates measured through soil sampling and geochemical analysis, with near-field removal defined from cation release in the sampled soil horizon.Labels are an upper bound on net durable removal because downstream loss and reversal factors are not applied; project emissions are accounted for at issuance rather than in the spatial rate surface.Source: A2 Feature Variables · pp. 10–11
  • Pangu-Weather, SFNO, and FCN3 use subsets of Aurora 1.5’s inputs, whereas GraphCast additionally requires vertical velocity from a separate ERA5 source.GraphCast's declared 6-hourly total precipitation is target-only for the operational checkpoint and is zero-filled rather than used as a network input.Source: Table 3 · p. 11

A3 Backbone Latent Extraction Details

Backbone latents are pooled at local, regional, and global spatial scales to form a monthly embedding for each weather foundation model. The embedding dimension depends on the backbone, and Table 4 records extraction layers and dimensions.

A3 Backbone Latent Extraction Details

  • Each monthly embedding concatenates local, regional, and global pooled latents, using fixed spatial scales because fields are smaller than a latent-grid cell.The regional feature averages a surrounding 3×3 window, approximately ±112 km, while global pooling averages the whole map.Source: A3 Backbone Latent Extraction Details · p. 11
  • Monthly embedding dimensions vary by backbone: Aurora 1.5 uses 3072, Pangu and SFNO 1152, GraphCast 1536, and FCN3 168.The embedding dimension is three times the backbone's native latent dimension.Source: A3 Backbone Latent Extraction Details · p. 11
  • Table 4 reports each backbone's latent extraction layer, native latent dimension, and resulting per-month embedding dimension..Source: A3 Backbone Latent Extraction Details · p. 11, Table 4 · p. 12

A4 Temporal Aggregation Methods

Monthly weather embeddings must be reduced to a fixed-size representation for the sequence model. The paper compares three aggregation strategies and selects trapezoidal integration for final experiments.

A4 Temporal Aggregation Methods

  • Monthly embeddings spanning each chronosequence interval are reduced to a fixed-size representation.The sequence contains T monthly embeddings, {z1, . . . , zT}.Source: A4 Temporal Aggregation Methods · p. 12
  • Trapezoidal integration is a parameter-free aggregation that approximates the latent trajectory’s time integral and assumes every month contributes.It was selected for all final experiments after experimentation.Source: A4 Temporal Aggregation Methods · p. 12
  • The alternatives are supervised LTAE, which learns month-importance weights with the decoder, and self-supervised LTAE, trained to reconstruct masked months without weathering-rate labels.The examples motivate weighting wet-season months for dissolution and learning temporal structure without labels.Source: A4 Temporal Aggregation Methods · p. 12

A5 LOFO Protocol Details

Across the reported protocols, the study assesses field-disjoint transfer, input choices, and sampling-budget strategies for operational ERW. Its results distinguish stable point-level ranking from field-mean estimation, and show how added measurements and amendment inputs affect the evaluated tasks.

A5 LOFO Protocol Details

  • The 68-fold protocol assigns folds to 68 location components with at least 10 measurements, covering 5,062 of 5,192 rows; treated-row metrics cover 64 components.The folds are field-disjoint, and scalers are fit on training data only.Source: A5 LOFO Protocol Details · p. 12

A6 Full Results

  • Across seeds, raw ERA5 ranking collapses to Spearman −0.02 on seed 43, while Aurora variants, SFNO, FCN3, and GraphCast remain between +0.16 and +0.28 on every seed.Pangu shares the collapse on two seeds; shuffled ERA5 stays near zero.Source: Table 6 · p. 14, A6 Full Results · p. 14
  • On location-disjoint natural-weathering validation, the TerraNova-ERW architecture scores about 90% lower MAE and RMSE than the training-mean constant predictor.The evaluation uses 19,441 validation sequences and no ERW data during training or validation.Source: Table 7 · p. 14

A7 Input Ablation: Site Selection with Limited Features

  • Adding amendment variables to Aurora latents changes point Spearman from −0.12 to +0.11 and site Spearman to +0.32, while site MAE remains near 1.8–2.0 t CO2e/ha/yr across configurations.Aurora + Amendment has site MAE 1.80 versus 2.04 for the full stack, supporting site prioritization rather than model-based quantification.Source: A7 Input Ablation: Site Selection with… · pp. 14–15, Table 8 · p. 15

A8 Sampling-Budget Simulation

  • The simulation blends revealed measurements with TerraNova-ERW predictions, with separate estimators for error and ranking objectives and ranking evaluated on a fixed set of 52 fields.Error tiers use varying field sets as the number of revealed points increases.Source: A8 Sampling-Budget Simulation · p. 15
  • For ranking 52 fields, the rank-weighted hybrid leads on Spearman and leads or ties on top-10 recovery at every measurement budget.The error-weighted hybrid is strongest on the top-quartile screen.Source: Table 9 · p. 16
  • The hybrid beats sampling alone through ten points per field and matches or beats shrinkage without a model through three; one measured point reduces the poor-accuracy share from 13% to 5%.The poor tier means error greater than 4 t CO2e/ha/yr; within-field noise is 64% of total variance.Source: A8 Sampling-Budget Simulation · p. 15, Table 10 · p. 17
Loading 2610.28973v1…