Source-linked AI summary
LoRA Adaptation Strength in Aurora-WRF: Trade-offs in Regional Heavy-Rainfall Forecasting
Boyan Liu, Xiaoyuan Zhang
TL;DR
The paper asks how the strength of low-rank adaptation affects downstream rainfall after a fixed regional model integrates atmospheric forecasts. It compares five strengths of one precipitation-weighted Aurora LoRA checkpoint in a controlled WRF coupling experiment. Unadapted Aurora has the highest pooled-window CSI, while an intermediate strength improves precipitation error and frequency behavior; GFS retains the lowest whole-domain MSE.
Problem
The effect of learned low-rank atmospheric-state adjustments on rainfall after integration by a fixed regional model is uncertain, because lower atmospheric-state error does not guarantee better rainfall error or threshold-event detection.
Source: §1 · p. 1Method
The study trains one precipitation-weighted LoRA checkpoint and couples Aurora forecasts at five adaptation strengths to the same WRF configuration.
Source: §1 · p. 1, §3.1 · p. 3Results
Takeaways & Limitations
Within this controlled experiment, adaptation strength adjusts the balance among event detection, frequency calibration, and intensity error rather than improving all criteria simultaneously.
Source: §6 · p. 9ERA5 precipitation is the only validation reference, and training and verification on the same reanalysis family limit independence despite distinct years.
Source: §5 · p. 8
Abstract
from arXiv · showhide
We investigate how parameter-efficient adaptation of an atmospheric foundation model affects downstream regional precipitation in a controlled Aurora-WRF coupling experiment over the Beijing-Tianjin-Hebei region. A single-step, precipitation-weighted LoRA adaptation of AuroraPretrained is trained on May-September 2020-2021 and selected using 2022 validation data. Five inference-time adaptation strengths are coupled to an unchanged 9-km WRF configuration and compared with a GFS-driven control under common ERA5 initialization and auxiliary-field rules. Evaluation uses 17 initializations in 2023, hourly ERA5 precipitation, and conservative remapping to a fixed 356-cell, 0.25-degree verification grid. Heavy rainfall is defined exclusively as at least 50 mm in a continuous 24-hour window, advanced hourly. For window starts from 0 to 48 hours, unadapted Aurora attains the highest mean critical success index (CSI), 0.216, compared with 0.144 for the GFS control. An intermediate LoRA strength of 0.75 reduces the Aurora-driven 24-hour precipitation mean squared error by 30.7% and brings pooled frequency Bias from 1.510 to 1.024, but lowers CSI to 0.182. GFS retains the lowest whole-domain error. These results characterize a trade-off between rainfall detection, frequency calibration, and intensity error, showing the importance of evaluating adaptation strength against multiple downstream criteria within a common coupling framework.
1 Introduction
The paper asks how the strength of a learned low-rank atmospheric-state adjustment affects rainfall after integration by a fixed regional model. It studies one LoRA checkpoint through controlled Aurora-WRF comparisons and a verification protocol that distinguishes hourly precipitation errors from moving 24-hour heavy-rainfall events.
1 Introduction
- Aurora’s shared pretrained representation supports adaptation across atmospheric prediction tasks, while WRF resolves additional dynamics and parameterized processes absent from coarse forecasts.The coupling provides a route to regional precipitation prediction without requiring Aurora to predict precipitation directly.Source: §1 · p. 1
- Atmospheric-state error reduction does not necessarily reduce rainfall error or improve threshold-event detection after WRF integration.The paper distinguishes the adaptation target from downstream rainfall verification.Source: §1 · p. 1
- The study compares one trained LoRA checkpoint at five strengths, including unadapted and fully adapted endpoints.The controlled boundary-forcing comparison records auxiliary-field provenance.Source: §1 · p. 1
- The verification protocol separates hourly precipitation errors from hourly moving 24-hour heavy-rainfall events and examines multimetric results across strengths and cases.The study reports a single-step adaptation experiment, not multi-step training, spectral objectives, ensemble forecasting, or a new WRF dynamical core.Source: §1 · p. 2
2 Data and experimental design
The experiment trains and selects local LoRA adaptation using ERA5 data from 2020–2022, then evaluates Aurora and GFS-driven WRF configurations on 2023 forecast starts. The cases are event-enriched and retrospectively inspected, and the analysis distinguishes principal from extended forecast-window starts.
2.1 Training, validation, and forecast cases
- Training uses 1,224 single-step samples from 2020–2021 and validation uses 612 samples from 2022, drawn from May–September ERA5 fields.Each sample uses two atmospheric states six hours apart to predict the subsequent six-hour state; the data cover 33–45°N and 108–123.75°E on a 0.25° grid.Source: §2.1 · p. 2
- The 17 evaluation initializations in 2023 include event-enriched, nonrandom dates, and overlapping forecast periods are not treated as independent meteorological events.Five dates were selected after a reference-only check found at least one cell with 24-hour precipitation reaching 50 mm.Source: §2.1 · p. 2
- Although local LoRA training and checkpoint selection exclude 2023, the forecast cases were repeatedly inspected during development and are diagnostic rather than an untouched final test.The local year split does not establish the complete training-data history of the upstream foundation checkpoint.Source: §2.1 · p. 2
2.2 Forecast configurations
- Six drivers—GFS and Aurora at α values 0, 0.25, 0.50, 0.75, and 1—supply future atmospheric states to the same WRF configuration.The principal analysis uses window starts from 0 to 48 hours; a supplementary analysis extends starts through 72 hours and forecast hours through 96.Source: §2.2 · p. 2
3 Methods
The methods train one precipitation-weighted, single-step LoRA checkpoint, then vary its inference strength in a shared Aurora–WRF coupling setup. A fixed spatial and heavy-rainfall verification protocol separates pooled scores from descriptive case dispersion.
3.1 Precipitation-weighted, single-step adaptation
- One six-hour LoRA prediction is trained with precipitation-weighted state loss, then run autoregressively; WRF receives no backpropagated precipitation loss.The checkpoint is selected using 2022 validation state loss, not a 2023 rainfall score.Source: §3.1 · pp. 2–3
3.2 Common initialization and boundary construction
- All drivers share ERA5 initialization and auxiliary-field rules, while the GFS control is restricted to Aurora-matched fields rather than its full operational variable set.The comparison is controlled but does not establish equivalence to either model’s native operational coupling.Source: §3.2 · p. 3
3.3 WRF configuration and spatial matching
- WRF rainfall is conservatively remapped by grid-cell overlap to a fixed 356-cell ERA5 verification mask, so reported performance applies to that support.Spatial averaging can smooth extremes and alter threshold exceedances; it is not an increase in effective resolution.Source: §3.3 · p. 4
3.4 Verification definitions
- Heavy rainfall means at least 50 mm in a continuous 24-hour window advanced hourly, with pooled classification counts and area-weighted continuous errors.Continuous errors include all valid cells, not only reference heavy-rainfall cells.Source: §3.4 · pp. 4–5
3.5 Dispersion, case composition, and reproducibility
- Dispersion summaries are descriptive, not inferential: overlapping forecast periods are not treated as independent events, and bootstrap estimates are not confidence intervals.The paper also retains archive identifiers and hashes and reproduces the 17-case scores from exported precipitation arrays.Source: §3.5 · p. 5, Figure 1 · p. 6, Table 2 · p. 6
4 Results
Across 0–48-hour window starts, unadapted Aurora leads in mean CSI but has higher Bias and whole-domain 24-hour MSE than GFS. Strength 0.75 reduces Aurora’s 24-hour MSE, while detection skill declines; initialization-level distributions and extended-range results add important context.
4.1 Detection skill and precipitation error favor different strengths
- Aurora’s mean CSI is 0.216 versus 0.144 for GFS, while Bias is 1.510 versus 0.896 and whole-domain 24-hour MSE is 302.20 versus 167.88 mm2.All drivers have 49 defined pooled CSI windows for the main 0–48-hour comparison.Source: §4.1 · p. 6
- Strength 0.75 reduces Aurora’s 24-hour MSE by 30.7% to 209.57 mm2 and brings pooled Bias to 1.024, but CSI falls to 0.182.GFS retains the lower continuous error; no strength dominates detection, frequency calibration, and intensity error.Source: §4.1 · p. 6
- Moving-window detection generally weakens at later starts, while continuous errors show event-dependent peaks; none of the six mean CSI values reaches 0.25 in the main comparison.Figure 2’s first three panels use 24-hour window starts from 0–24 h through 48–72 h.Source: §4.1 · p. 6, Figure 2 · p. 7
4.2 Initialization-level distributions
- Initialization-level 24-hour MSE is 302.20 ± 541.73 mm2 for unadapted Aurora and 209.57 ± 273.28 mm2 at strength 0.75; the distributions are strongly skewed.Table 3 reports initialization-level means ± sample SD, not the headline pooled estimators.Source: §4.2 · pp. 6–7, Table 3 · p. 8
4.3 Spatial examples and the extended range
- Spatial examples compare rainfall placement, extent, and intensity across drivers, but do not provide controlled attribution of physical mechanisms.The displayed windows are selected by reference precipitation, not forecast skill.Source: §4.3 · p. 7
- For window starts 0–72, mean CSI is lower for every driver; this extended diagnostic adds windows ending at 73–96 hours and is not skill confined to an ending lead of 72 hours.GFS has 70 valid CSI windows and each Aurora strength has 73; zero-denominator windows remain undefined.Source: §4.3 · p. 7
5 Discussion and limitations
The discussion separates rainfall detection, frequency calibration, and intensity error, while emphasizing that upstream adaptation objectives do not determine downstream WRF rainfall skill. Its conclusions are bounded by the reference data, case sample, control configuration, and verification scale.
5 Discussion and limitations
- Rainfall scores alone cannot identify which changed atmospheric variable, boundary imbalance, or physical process underlies the non-monotonic response to adaptation strength.The fitted state loss weights heavy-rain environments but neither predicts rainfall directly nor optimizes WRF rainfall skill.Source: §5 · p. 8
- Lower whole-domain error and higher CSI measure different outcomes, so GFS’s lower MSE does not conflict with its lower CSI.Conservatively weak rainfall may reduce squared error while missing threshold exceedances; a broader rain footprint may increase both detections and false alarms.Source: §5 · p. 8
- ERA5 is the sole validation reference, and using the same reanalysis family for training and verification limits independence despite distinct years.Validation against CMA, CPC, and CHIRPS is planned but not performed; daily products cannot establish hourly moving-window truth by temporal interpolation.Source: §5 · p. 8
- The aggregate Aurora–GFS CSI ordering can reverse with sample composition, making the reported ordering specific to this diagnostic sample.Broader generalization requires verification on more independent events.Source: §5 · p. 8
- The GFS control uses retrospective initialization and a restricted interface, so it does not establish superiority over a fully configured operational NWP system.Auxiliary-field rules need separate testing before deployment, and the exact WRF version matters; no cross-version or alternative-physics causal claim is made.Source: §5 · p. 8
- Verification on a coarse common grid does not demonstrate 1-km rainfall skill, gauge-scale extremes, ensemble reliability, or an end-to-end optimized forecasting system.The discussion prioritizes independent precipitation references, predeclared calibration/test separation, and physical diagnostics of boundary-state changes before claiming robust skill gains.Source: §5 · pp. 8–9
- Figure 3 retains zeros and extremes in initialization-level distributions, while its dotted line marks Bias=1 and counts are valid cases, not independent events.Boxes show medians and quartiles; whiskers extend to observations within 1.5 interquartile ranges.Source: Figure 3 · p. 9
6 Conclusion
A single precipitation-weighted LoRA adaptation changes WRF rainfall differently across strengths and metrics. In this controlled experiment, adaptation adjusts the balance among detection, frequency calibration, and intensity error rather than improving all criteria simultaneously.
6 Conclusion
- Across 17 retrospective starts, unadapted Aurora has the highest pooled-window mean CSI, an intermediate strength reduces continuous precipitation error and frequency overprediction, and GFS retains the lowest whole-domain MSE.These results use one trained LoRA adaptation under common coupling rules.Source: §6 · p. 9
Data and code availability
The ancillary files support reproduction of the reported analysis, but do not include the materials needed to reproduce model training or forecast integration.
Data and code availability
- Ancillary files include verification-grid precipitation arrays, analysis and plotting code, numeric tables, bootstrap statistics, archive hashes, and reproduction instructions.They reproduce the reported analysis, not training or forecast integration; trained weights, full training data, and native WRF histories are not included.Source: Data and code availability · p. 9
- ERA5 and GFS data are available from their providers, and Aurora code and pretrained checkpoints are available from Microsoft.ERA5-derived arrays are credited to Copernicus Climate Change Service and ECMWF, with source and licence details in the ancillary data.Source: Data and code availability · p. 9
A Evaluation sample
Table 4 describes reference statistics for the 17 evaluation initializations using a fixed spatial mask and rolling 24-hour windows.
A Evaluation sample
- Table 4 covers 17 evaluation initializations at 00 UTC in 2023, with reference statistics over a fixed 356-cell mask and 49 rolling windows starting at 0–48 hours.Event counts use the 50-mm threshold and include overlapping windows; zero means no threshold exceedance, not necessarily no rain.Source: Table 4 · p. 11
B Extended window-start range
The supplementary analysis extends window starts through 72 hours, requiring precipitation forecasts through 96 hours; Table 5 reports defined pooled CSI windows out of 73 and retains all false alarms in Bias.
B Extended window-start range
- Window starts extend from 0 to 72 hours, requiring rain forecasts through 96 hours.Source: Table 5 · p. 11
- The reported n counts defined pooled CSI windows out of 73 possible windows.Source: Table 5 · p. 11
- Pooled Bias retains false alarms, including those in windows without reference events.Source: Table 5 · p. 11
C Representative precipitation fields
Three representative precipitation fields are shown for distinct 2023 initializations and accumulation windows, using common verification-grid and color-scale conventions.
C Representative precipitation fields
- Figure 4 shows the 20 July 2023 initialization accumulated from +25 to +49 hours, with the window selected using reference precipitation only.Fields are conservatively mapped to the verification grid; red contours mark 50 mm.Source: Figure 4 · p. 12
- Figure 5 shows the 30 July 2023 initialization accumulated from +0 to +24 hours, selected by the same reference-only rule.It uses the common verification grid, color scale, and 50-mm contour convention.Source: Figure 5 · p. 12
- Figure 6 displays all six drivers for the 10 August 2023 initialization accumulated from +30 to +54 hours.The fields use the common verification grid and color scale.Source: Figure 6 · p. 13