Source-linked AI summary
Probabilistic Machine Learning for Short-Term Atmospheric Seeing Forecasting
Mary-Joe Medlej, Rahul Srinivasan, Simon Prunet, Aziz Ziad, Christophe Giordano
TL;DR
Rapid changes in atmospheric turbulence affect astronomy and optical communications, and operational decisions can benefit from forecasts that include uncertainty. This study benchmarks deterministic and probabilistic models for Maunakea seeing up to two hours ahead. LSTM achieves the strongest deterministic accuracy, while FloTS combines competitive accuracy with flexible uncertainty estimates.
Problem
Rapid turbulence changes can affect astronomical observations and optical communications, while operational decisions may require forecast confidence as well as point predictions.
Source: §1 · pp. 1–2Method
The study benchmarks RNN, LSTM, GP, and conditional normalizing-flow FloTS models on Maunakea seeing, evaluating accuracy and probabilistic calibration.
Source: §1 · p. 2Results
LSTM achieves the strongest overall deterministic accuracy, while FloTS remains competitive with richer non-Gaussian predictive densities and more informative uncertainty estimates; temperature calibration improves forecast reliability.
Source: §5 · p. 6Takeaways & Limitations
FloTS combines competitive seeing forecasts with flexible uncertainty quantification, supporting its potential as a framework for atmospheric seeing prediction.
Source: §5 · p. 6GPs assume Gaussian predictive distributions and can become computationally expensive for large datasets.
Source: §3.2 · p. 4
Abstract
from arXiv · showhide
Accurate short-term forecasting of atmospheric seeing is of significant interest for astronomical observations and free-space optical communication systems, where rapid variations in optical turbulence can affect system performance and operational planning. In this work, we benchmark machine learning models for predicting atmospheric seeing up to two hours in advance, with a particular focus on probabilistic forecasting and uncertainty quantification. We compare two probabilistic approaches, Gaussian Processes (GPs) and a normalizing-flow- based model (FloTS), against deterministic recurrent neural network (RNN) and long short-term memory (LSTM) baselines. All models are trained and evaluated on seeing measurements from the Maunakea summit. We assess the models using both deterministic metrics, including the root mean square error and the Pearson correlation coefficient, and probabilistic metrics that evaluate the quality and calibration of predictive uncertainty estimates. The results show that deep learning approaches outperform standard recurrent networks for short-term forecasting, with LSTM and FloTS achieving the best overall predictive accuracy. While Gaussian Processes provide a useful probabilistic baseline, their predictive distributions are limited by the Gaussian assumption. FloTS combines competitive forecasting accuracy with flexible uncertainty quantification and produces the most informative probabilistic forecasts.
1. INTRODUCTION
Atmospheric turbulence degrades astronomical image quality and can impair free-space optical links, while reactive adaptive optics and rapid atmospheric changes complicate operations. This study benchmarks deterministic and probabilistic machine-learning models for seeing forecasts, emphasizing uncertainty quantification.
1. INTRODUCTION
- Atmospheric turbulence distorts optical-wave propagation, degrading astronomical image quality and potentially reducing free-space optical link reliability and performance.Atmospheric seeing is a common measure of turbulence’s effect on astronomical image quality.Source: §1 · p. 1
- Seeing forecasts could help optimize telescope scheduling, observation strategies, adaptive-optics operation, and optical-communication resource management.Adaptive-optics systems correct wavefront distortions in real time but rely on measurements of the current atmospheric state.Source: §1 · p. 1
- Prior seeing-forecast studies used ARIMA, Random Forests, and neural networks, with most efforts focused on deterministic prediction accuracy.The cited work also includes longer-term turbulence forecasting and comparisons of data-driven methods.Source: §1 · p. 2
- Probabilistic forecasts estimate a full predictive distribution, addressing the need for forecast confidence in risk-aware decisions.GPs provide uncertainty estimates, while normalizing flows can learn complex, non-Gaussian predictive distributions.Source: §1 · p. 2
- The study benchmarks RNN, LSTM, GP, and FloTS models on Maunakea seeing measurements using deterministic metrics and probabilistic calibration diagnostics.Its emphasis is comparing probabilistic models and their uncertainty quantification.Source: §1 · p. 2
2. DATASET
The study uses DIMM-derived seeing measurements from the Maunakea summit, collected by the Maunakea Weather Center. The series is segmented, gap-handled, and resampled to a uniform 10-minute grid.
2. DATASET
- The dataset uses DIMM-derived seeing measurements from the Maunakea Weather Center’s combined MASS-DIMM monitoring device.The measurements represent integrated optical turbulence across the atmospheric column.Source: §2 · p. 2
- The observations span September 2009 to November 2024, with seeing values ranging from 0.04′′ to 5′′.Source: §2 · p. 2
- Raw 90-second samples were segmented, checked for missing data, and linearly interpolated onto a uniform 10-minute grid.The resampling reduces high-frequency variability considered less relevant to operational forecasting timescales.Source: §2 · p. 2
3. METHODS
The methods compare deterministic recurrent models with probabilistic GPs and conditional normalizing flows for seeing forecasts. Each model uses a two-hour history to predict the next two hours, with the probabilistic approaches designed to provide uncertainty estimates.
3. METHODS
- All models use a two-hour seeing history to forecast the subsequent two hours, a window selected because longer inputs did not improve performance in preliminary experiments.Source: §3 · p. 3
3.1 Recurrent Neural Network (RNN)/Long Short-Term Memory (LSTM)
- The encoder–decoder RNN and LSTM generate future seeing values autoregressively, but provide point forecasts without predictive uncertainty.The LSTM replaces recurrent units with memory cells and gates that retain relevant information over longer timescales; both architectures use an encoder and two decoder layers.Source: §3.1 · p. 3
3.2 Gaussian Process (GP)
- GPs produce predictive means and variances using a mean function and covariance kernel, but assume Gaussian predictive distributions and can be costly for large datasets.The kernel controls learned temporal structure; hyperparameter estimation used a random subset of 500 training pairs because matrix-inversion costs scale cubically with sample count.Source: §3.2 · pp. 3–4
3.3 Normalizing Flows (FloTS)
- FloTS models the conditional distribution of future seeing given past measurements, using a context encoder to condition a masked autoregressive flow.The explored encoder architectures included LSTMs, which gave the best performance and provide temporal context to the flow.Source: §3.3 · p. 4
- Compared with GPs, FloTS can represent flexible, non-Gaussian predictive distributions and has more favorable computational scaling for larger datasets.The flow learns complex distributions from observations and can generate point predictions as well as full predictive probability distributions.Source: §3.3 · p. 4
4. RESULTS
Across forecast horizons, LSTM has the strongest combined point-forecast performance, with FloTS close behind; temperature calibration adjusts FloTS’s full predictive density to improve its reliability while retaining flexible, non-Gaussian structure.
4.1 Forecasting performance
- The composite score equally weights inverted min–max-normalized RMSE and normalized correlation, with 1 representing the best joint performance and 0 the worst.It evaluates prediction accuracy and temporal consistency together.Source: §4.1 · p. 5
- LSTM leads combined forecast performance across nearly all horizons, while FloTS follows closely and the standard RNN has the weakest combined scores.Performance worsens with longer horizons: RMSE rises and correlation falls.Source: §4.1 · p. 5, Figure 1 · p. 5
4.2 Uncertainty quantification
- Temperature-based post-hoc calibration adjusts FloTS’s predictive density using the mismatch between nominal and empirical credible-region coverage on validation data.The calibration temperature is a function of nominal coverage; values above 1 broaden the density to correct overconfidence, while values below 1 contract it to correct underconfidence.Source: §4.2 · p. 6
- Figure 2 contrasts uncalibrated and calibrated FloTS density maps across an August case study’s two-hour forecast, alongside observed seeing and GP, RNN, and LSTM predictions.The density is normalized independently at each forecast timestep for visualization.Source: §4.2 · p. 6
- After calibration, probability mass is redistributed rather than shifting the central tendency, concentrating likelihood around observed seeing and producing a sharper, better-calibrated forecast.Unlike Gaussian calibration that mainly rescales variance, FloTS can preserve non-Gaussian structures such as asymmetry or multimodality.Source: §4.2 · p. 6
5. CONCLUSIONS
On Maunakea seeing measurements, LSTM provides the strongest deterministic accuracy, while FloTS combines competitive accuracy with flexible probabilistic forecasts. The conclusions point to additional meteorological and turbulence inputs and multi-site generalization as future work.
5. CONCLUSIONS
- Across forecast horizons up to two hours, LSTM consistently achieves the lowest RMSE and highest correlation among deterministic approaches across most horizons.Forecasting performance degrades with increasing lead time for all models.Source: §5 · p. 6
- FloTS combines competitive deterministic performance with richer non-Gaussian predictive densities and more informative uncertainty estimates, whereas the Gaussian assumption limits GP distributions.Post-hoc temperature calibration improves reliability by aligning predicted and empirical coverage.Source: §5 · p. 6
- The authors identify normalizing-flow models as a promising framework that combines strong seeing-prediction accuracy with flexible uncertainty quantification.The benchmark uses historical seeing measurements from the Maunakea summit.Source: §5 · p. 6
- Future work will add meteorological and turbulence-related parameters and explore multi-site generalization for operational deployment.Figure 2 illustrates an August case study comparing FloTS before and after calibration over a two-hour forecast.Source: §5 · p. 7, Figure 2 · p. 7