Source-linked AI summary
PrecipJEPA: JEPA-Regularized Future-State Prediction with Motion-Source Rendering for Precipitation Nowcasting
Yufeng Zhu, Dan Niu, Qiliang Wu, Weiwei Huang, Yixiao Liang, Yongchao Feng, Chunlei Shi
TL;DR
Long-term precipitation nowcasting must track evolving radar echoes while preserving local structure, and forecast-only encoder training lacks an explicit objective for relationships within observed history. PrecipJEPA combines structured future-state prediction and motion–source rendering with masked-history supervision of the shared encoder. It improves highest-threshold CSI over the strongest baselines on both SEVIR and MeteoNet and maintains the highest mean CSI across evaluated lead times.
Problem
Long-term precipitation nowcasting must model evolving radar echoes while preserving local structure, and forecast-only encoder training lacks an explicit objective for relationships within observed history.
Source: §1 · p. 1Method
PrecipJEPA combines a structured forecasting path that constructs future states and renders motion and source–sink fields with H-JEPA masked-history prediction that supervises the shared encoder.
Source: §1 · p. 1Results
118.6% on SEVIR and 35.1% on MeteoNet: PrecipJEPA improves highest-threshold CSI over the strongest baselines and maintains the highest mean CSI across evaluated lead times.
Source: §1 · p. 1, §3.2 · p. 4Takeaways & Limitations
Ablation studies support the complementary roles of PrecipJEPA’s three components in overall, high-intensity, and long-term forecasting performance.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Long-term precipitation nowcasting requires modeling radar-echo evolution while preserving localized high-intensity structures. Recent radar-specific studies motivate location-aware prediction and separating echo displacement from intensity change. However existing encoders learn historical representations mainly from final forecast errors. We propose PrecipJEPA, which couples a structured forecasting path with an auxiliary path that enriches its encoder from observed radar history. In the forecasting path, an online encoder first converts the observations into spatiotemporal tokens. The Task-Driven Future-State Predictor (TFP) combines these tokens with a recent-dynamics summary and spatiotemporal queries to construct future radar states. The Parallel Motion-Source Renderer (PMSR) decodes these states into motion and source-sink fields that transform the latest observation into future frames. During joint training, the History-Masked JEPA (H-JEPA) operates on the auxiliary path to predict masked historical features from visible context, directly supervising the same online encoder from the observed sequence. Experiments on SEVIR and MeteoNet show that PrecipJEPA improves highest-threshold CSI by 118.6% and 35.1%, respectively, over the strongest baselines, while maintaining the highest mean CSI throughout the 3-hour forecast.
1. INTRODUCTION
Long-term precipitation nowcasting must model evolving radar echoes and preserve local structure, while forecast-only training provides no explicit objective for relationships within observed history. PrecipJEPA pairs structured future-state forecasting and motion–source rendering with masked-history supervision of the shared encoder.
1. INTRODUCTION
- Longer lead times make echo motion, deformation, intensification, and dissipation harder to predict, requiring coherent future radar fields.Reliable long-term nowcasting must summarize observed echo evolution and translate it into those fields.Source: §1 · p. 1
- Radar-specific studies motivate preserving local spatial structure while distinguishing echo displacement from intensity change and making future states location-specific.These requirements motivate a decomposition of radar evolution and future states specific to time and location.Source: §1 · p. 1
- Forecast-only encoder training learns historical features indirectly from output errors, without an explicit objective for predictive relationships within observed history.JEPA predicts masked target representations from visible context in embedding space rather than reconstructing every pixel.Source: §1 · p. 1
- PrecipJEPA’s TFP constructs location-specific future states, PMSR renders them through motion and source–sink fields, and H-JEPA predicts masked history features during training.The auxiliary objective guides the shared online encoder using observed evolution alongside future radar targets.Source: §1 · p. 1, Figure 1 · p. 2
- 118.6% on SEVIR and 35.1% on MeteoNet: PrecipJEPA improves highest-threshold CSI over the strongest baselines.These comparisons are reported in the introduction’s contribution summary.Source: §1 · p. 1
2. METHOD
PrecipJEPA maps observed radar sequences to future frames through location- and time-specific latent states, rendered with separate motion and intensity-change fields. A masked-history objective jointly trains the shared online encoder alongside forecast supervision.
2.1. Problem Formulation and Overview
- The forecasting path encodes observed radar frames as spatiotemporal tokens, predicts future states with TFP, and renders those states into a radar sequence with PMSR.H-JEPA predicts masked features within the observed history and shares the online encoder with the forecasting path.Source: §2.1 · p. 2
2.2. Task-Driven Future-State Predictor (TFP)
- TFP builds a distinct latent state for each future time–location pair by combining historical tokens, an observation-dependent dynamics summary, and temporal and spatial queries.The dynamics summary represents the latest historical state, its most recent change, and average activity over the observed period.Source: §2.2 · p. 2
2.3. Parallel Motion–Source Rendering (PMSR)
- PMSR decodes future states into separate motion and source–sink fields, warping the latest observation and then adding local intensity changes to form each forecast frame.The rendered prediction is clipped to the valid radar range; the source field models growth or decay that displacement alone cannot represent.Source: §2.3 · pp. 2–3
2.4. Historical Representation Learning with H-JEPA
- H-JEPA predicts masked historical features from visible context, using an EMA target encoder on the complete history to supply target representations.Its target encoder is updated from the online encoder by exponential moving average rather than backpropagation.Source: §2.4 · p. 3
2.5. Joint Objective and Inference
- Joint training combines future-frame supervision with the historical latent loss, while inference uses the online encoder, TFP, and PMSR without the mask, H-JEPA predictor, or target encoder.The shared encoder is trained from both masked historical prediction and future forecast supervision.Source: §2.5 · p. 3
3. EXPERIMENTS
Experiments on SEVIR and MeteoNet compare PrecipJEPA with baselines and ablate its components and H-JEPA training strategies. Results favor PrecipJEPA overall, especially at the highest precipitation thresholds, and support the roles of its forecasting components and joint history learning.
3.1. Experimental Setup
- The evaluation uses chronological splits and forecasts 36 frames from 12 inputs at 5-minute intervals on SEVIR and MeteoNet.Frames are resized to 128 × 128; CSI, HSS, CSI-p4, CSI-p16, and LPIPS are evaluated, with baselines retrained under the same protocol and checkpoints selected by validation CSI.Source: §3.1 · pp. 3–4
3.2. Comparison with State-of-the-Art Methods
- 118.6% and 35.1%: PrecipJEPA exceeds the strongest baseline in highest-threshold CSI by these margins on SEVIR and MeteoNet, respectively.It also obtains the best value for every reported metric on both datasets.Source: §3.2 · p. 4
- PrecipJEPA has the highest mean CSI across evaluated lead times and CSI-last values of 0.1493 on SEVIR and 0.1734 on MeteoNet.Later forecasts retain more complete high-intensity echo structures.Source: §3.2 · p. 4
3.3. Ablation Study
- Ablations associate TFP with final-hour CSI recovery, PMSR with high-intensity gains, and H-JEPA with higher CSI-last on both datasets.Replacing TFP with an MLP hurts final-hour accuracy; removing H-JEPA lowers CSI-last from 0.1493 to 0.1374 on SEVIR and from 0.1734 to 0.1402 on MeteoNet. Against the residual head, PMSR improves SEVIR CSI219 by 178.5% and HSS219 by 166.0%; on MeteoNet, CSI32 rises from 0.1126 to 0.1409 and HSS32 from 0.1868 to 0.2268.Source: §3.3 · p. 4, Table 2 · p. 4
- Joint H-JEPA training outperforms encoder pretraining followed by forecast-only training on both datasets.Figure 3 compares training without H-JEPA, pretraining with H-JEPA, and joint training; on SEVIR, joint training improves CSI-last and CSI219 by 5.1% and 9.0%, respectively.Source: §3.3 · p. 4, Figure 3 · p. 4
4. CONCLUSION
The conclusion presents PrecipJEPA as a deterministic long-term precipitation nowcasting method combining history-masked latent prediction, future-state construction, and motion/source-sink rendering. Experiments on SEVIR and MeteoNet support improved overall, high-intensity, and long-term performance, while probabilistic forecasting is left for future work.
4. CONCLUSION
- PrecipJEPA combines history-masked latent prediction, lead-specific future states, and motion and source-sink rendering for deterministic precipitation nowcasting.Experiments on SEVIR and MeteoNet show improved overall, high-intensity, and long-term forecasting performance, and ablations support complementary component roles.Source: §4 · p. 4
Compliance with Ethical Standards
The study uses public meteorological radar datasets and reports no human or animal subjects, conflicts of interest, or funding.
Compliance with Ethical Standards
- The study uses public meteorological radar datasets, requires no ethical approval, and reports no conflicts of interest or funding.Source: Compliance with Ethical Standards · p. 5