PaperlayerView exact revision on arXiv

Source-linked AI summary

The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting

Enrico Camporeale

arXiv:1903.05192v2physics.space-phphysics.data-an

TL;DR

Space-weather forecasting needs better use of machine learning, especially for uncertainty assessment and integration with physical knowledge. The paper reviews prior forecasting applications, introduces relevant machine-learning concepts, and identifies gray-box methods as a promising direction. It concludes that gray-box approaches may become more predominant than conventional physics-based forecasting models.

  • Problem

    Space-weather forecasting lacks agreed benchmarks, and common single-point predictions do not reliably assess uncertainty.

  • Method

    The paper reviews machine-learning applications, introduces core concepts, and discusses gray-box approaches combining data-driven methods with physics-based models.

  • Results

    The review concludes that gray-box approaches may slowly become more predominant than conventional physics-based models.

  • Takeaways & Limitations

    Future research should pursue probabilistic forecasting, uncertainty quantification, and better integration of machine learning with physical understanding.

  • Takeaways & Limitations

    The review is necessarily incomplete and somewhat biased.

Abstract

from arXiv · show

The numerous recent breakthroughs in machine learning (ML) make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in space weather. The purpose is twofold. On one hand, we will discuss previous works that use ML for space weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the space weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray-box.

1 Artificial Intelligence: is this time for real?

The paper places current AI advances in a recurring cycle of expectations and disillusionment, while arguing that present industrial adoption makes this a timely moment for space-weather scientists to engage with AI.

  • AI history alternates between periods of high expectations and investment and periods of disillusionment, pessimism, and reduced funding.
  • Some experts regard the current period as a new AI spring, although another winter remains possible.
  • Industrial deployment has made AI part of everyday technologies, distinguishing the current period from earlier cycles.
  • The author argues that this is an appropriate time for space-weather scientists to develop realistic expectations about adopting AI systematically.
  • The paper presents segmentation as an example of machine-learning capability progressing from science fiction toward real applications.

2 The Machine Learning renaissance

The machine-learning renaissance is attributed to the combined availability of large public datasets, GPU computing, and accessible open-source software, motivating a review tailored to space weather.

  • Space-weather data and associated software are largely released publicly, providing an unusually accessible big-data resource.
  • GPU computing enabled the pioneering advances of machine learning and accelerated training for large models.
  • Large IT investments advanced machine-learning software while making tools open-source and relatively easy to use.
  • The combination of big data, GPU computing, and accessible software lets non-specialists access large datasets, computing resources, and machine-learning training.
  • The paper reviews prior machine-learning work in space-weather forecasting and introduces machine-learning concepts for the space-weather community.

3 Machine Learning in Space Weather

Machine learning offers fast, data-driven space-weather prediction, but the paper emphasizes uncertainty quantification and gray-box methods that combine observations with physical knowledge.

  • Space weather has extensive observational data from in-situ and remote missions, including multi-decade measurements near L1.
  • Machine-learning models typically execute quickly but may be expensive to train and cannot extrapolate outside their training range.
  • Physics-based forecasting is limited by multiscale processes, short causal time lags, and the computational cost of first-principles models.
  • Black-box machine learning seeks data-driven empirical relationships, whereas gray-box methods combine data with prior physical information.
  • Gray-box approaches can estimate uncertain physical parameters from observations using Bayesian parameter estimation and data assimilation.
  • The paper proposes learned ensembles that combine black-box and white-box models, with weights adapted to geomagnetic conditions.
  • Regression commonly produces single-point predictions, creating a need for probabilistic methods that quantify forecast uncertainty.
  • Classification can convert continuous quantities into discrete classes, with logistic regression interpreting outputs as event probabilities.

4 Machine Learning workflow

The workflow begins by formulating a physically well-posed space-weather problem, selecting appropriate task types, and preparing data with domain-aware preprocessing. Physical causality, data quality, gaps, and outliers all constrain whether machine-learning results will be meaningful and generalizable.

  • Problem formulation: A well-posed problem defines the objective, task type, physical inputs, causality, data sufficiency, and whether the input-output mapping is justified.
  • Problem formulation: Regression predicts quantities, classification uses known labels, clustering discovers structure without labels, and dimensionality reduction finds patterns or anomalies.
  • Data selection and pre-processing: Data quality strongly affects algorithm quality, so domain experts and data managers should guide data selection and exploratory analysis.
  • Data selection and pre-processing: Preprocessing includes descriptive statistics, low-dimensional visualization, cleaning, gap handling, and deciding whether outliers represent physical events or noise.
  • Data selection and pre-processing: Information-theoretic causal analysis can rank physical drivers and identify a maximum prediction horizon, helping exclude irrelevant inputs from forecasting models.

4.3 Algorithm selection

Algorithm selection depends on practical and modeling considerations rather than a universally superior method. The parametric–non-parametric distinction highlights a central trade-off between scalability and flexibility.

  • Algorithm selection: Algorithm choice depends on the model’s internal workings and practical behavior, even though optimization theory does not guarantee one algorithm’s superiority.
  • Algorithm selection: Selection should consider retraining requirements, prediction speed, scalability with dataset size, accuracy, and computational cost.
  • Algorithm selection: Parametric models use a fixed number of parameters and train faster on large datasets, whereas non-parametric models grow with training data and are more flexible.

4.4 Overfitting and model selection

Model development must balance fit, generalization, complexity, and uncertainty-aware evaluation. Overfitting lowers training error while degrading performance on unseen data, motivating validation procedures, held-out testing, and baseline-relative metrics.

  • Overfitting and model selection: Hyper-parameter tuning can materially affect both model accuracy and training cost after the algorithm is selected.
  • Overfitting and model selection: A 9th-order polynomial can fit 10 training points exactly with ε = 0, yet become highly oscillatory and fail to generalize beyond the observed data range.
  • Overfitting and model selection: As polynomial order increases, training error decreases until l = 9 fits the data exactly, while generalization to unseen data worsens.
  • Overfitting and model selection: Overfitting occurs when the model fits noise rather than signal, whereas an overly simple model may fail to capture nonlinear behavior.
  • Overfitting and model selection: Separating data into training, validation, and test sets supports tuning, model selection, and final evaluation on fresh unseen data.
  • Performance evaluation: Performance metrics should differ from the cost function, and skill scores compare a model against a baseline such as persistence or a zero-cost forecast.
  • Bias-Variance decomposition: Bias measures systematic deviation from the desired outcome, whereas variance measures sensitivity to the particular dataset used.
  • Performance evaluation: Probabilistic metrics include CRPS, which collapses to MAE for deterministic predictions, while HSS has a perfect value of 1 and a false-forecast baseline of 0.

5 Review of machine learning in Space Weather

The review surveys machine-learning applications in space weather, concentrating on geomagnetic indices, relativistic electrons at geosynchronous orbit, and solar eruptions. It presents this literature as broad but explicitly non-comprehensive and potentially shaped by personal bias.

  • The review focuses on forecasting geomagnetic indices, relativistic electrons at geosynchronous orbits, and solar eruptions including flares and coronal mass ejections.
  • The authors state that the literature review is not comprehensive and is subject to personal bias.

5.1 Geomagnetic indices

Geomagnetic-index forecasting has served as a testbed for diverse machine-learning methods, spanning neural networks, NARX, SVMs, neurofuzzy models, self-organizing maps, Bayesian methods, and Gaussian processes. Studies report strong predictive performance in several settings, while newer work explores probabilistic forecasts and architectures adapted to temporal dependence and class imbalance.

  • Forecasting Kp: TSS ∼0.8 was reported for Kp values from 2 to 8 using feed-forward and recurrent neural networks.The models used one hidden layer with 4–20 hidden neurons and data from 1975–2001.
  • Forecasting Kp: LSTM models retained information from the past and automatically selected relevant time lags, while separating storm and quiet Kp cases through classification and two regression sub-models.Storms were defined as Kp ≥5, quiet conditions as Kp <5, and their typical ratio was close to 1:30.
  • Methods: Neural networks dominate geomagnetic-index forecasting, but studies also use SVMs, NARX, neurofuzzy models, self-organizing maps, Bayesian methods, and Gaussian processes.The reviewed approaches include both supervised mappings to Dst and unsupervised storm-category classification.
  • Forecasting Kp: A probabilistic Kp forecast constructed conditional probabilities over almost 40,000 three-hour events after testing more than 1,200 models.The task was formulated as classification using combinations of three conditional parameters selected from 23 possible inputs.
  • Forecasting Dst: cc ∼0.95 was achieved by an SVM model trained and tested on 80 geomagnetic storms using 13 solar-wind inputs and five-fold cross-validation.Each fold left 16 storms for testing before averaging results.
  • Forecasting Dst: PE ∼0.975 for one-hour and PE ∼0.9 for three-hour Dst forecasts were reported with linear regression using 150–200 regressors.Other reviewed Dst results included cc ∼0.87 and RMSE ∼12 at four hours, and RMSE ∼12 with cc ∼0.9 at twelve hours.
  • Probabilistic methods: Gaussian processes offer few adjustable hyper-parameters but scale non-optimally with dataset size, motivating sparse approximate formulations.This creates a computational trade-off between flexible Bayesian prediction and dataset scalability.

5.2 Recapitulation - Geomagnetic indices

Geomagnetic-index prediction has hosted a wide range of machine-learning techniques for two decades, but comparisons are difficult because modeling choices vary substantially. The review highlights broad methodological freedom, overfitting concerns, and a shift toward more complex models enabled by GPUs and open-source libraries.

  • Recapitulation: Geomagnetic-index prediction has served as a testbed for many machine-learning techniques for roughly 20 years, although this review is necessarily incomplete.The section presents a selective overview rather than a comprehensive comparison of all published work.
  • Recapitulation: Machine-learning studies vary in regressors, time histories, forecast horizons, data-gap handling, dataset splits, cross-validation, metrics, and model complexity.These degrees of freedom make methodological choices central to interpreting reported results.
  • Recapitulation: Unconstrained input selection can associate predictive variables with causal information they do not physically represent and can produce models that generalize poorly to unseen data.The concern is especially tied to selecting inputs without regard for the physics behind them.
  • Recapitulation: Early models commonly justified input choices physically and favored parsimonious designs that traded model complexity against accuracy.The review links this tendency partly to concerns about overfitting and limited generality.
  • Recapitulation: GPU advances and accessible machine-learning libraries have shifted geomagnetic forecasting toward increasingly complex models, including deep neural networks.The easier availability of open-source software makes training large networks more accessible but is described as double-edged.

What has not been done yet?

Longer-horizon geomagnetic forecasting remains constrained by reliance on solar-wind drivers and recent geomagnetic states, while direct solar-image prediction poses unresolved data, modeling, and class-imbalance challenges. The review identifies integrating physical knowledge with machine learning into gray-box methods as a central future direction.

  • Motivation: Geomagnetic indices provide storm warnings and parameterize physical quantities in computational models, including radiation-belt diffusion coefficients through Kp.Their forecasting therefore has both operational and modeling relevance.
  • Open challenges: Most prior models use solar-wind drivers and past geomagnetic information, limiting physically based prediction horizons to about 12 hours.Future work is expected to explore solar inputs that could support prediction horizons on the order of days.
  • Open challenges: Direct use of solar images and magnetograms must address several gigabytes of observations and the design of accurate machine-learning methods.The review specifically raises questions about whether solar images contain enough information and whether one-step or multi-step Sun-to-Earth modeling is preferable.
  • Open challenges: Severe imbalance between quiet periods and relatively rare storms, especially large storms, remains an unresolved challenge for direct image-based forecasting.The paper identifies empirical or simulation-based data augmentation as one possible direction.
  • Future direction: Combining physics knowledge with machine learning, or machine learning with physics-based models, is presented as the ultimate gray-box goal for space-weather forecasting.The review expects many of the outstanding questions to be addressed within the next decade.

5.3 Relativistic electrons at geosynchronous orbit

Research on forecasting relativistic electrons at GEO spans neural networks, autoregressive models, statistical methods, and deep learning, but reported performance varies with energy, horizon, data, and evaluation design.

  • GEO electron forecasting is challenging because MeV fluxes reflect multiple loss and local-acceleration processes.
  • 0.58 to 0.67 PE was observed for one-day-ahead forecasts when the training period increased from six months to one year.The sensitivity of performance to training-set size highlights the dependence of reported metrics on data choices.
  • Comparisons among FluxPred, SWPC-REFM, and a semi-empirical model were inconclusive, with model quality varying across solar-cycle phases.All three models performed slightly better than persistence at one day and much better at two days, while forecasts were better during solar minimum.
  • NARX, NARMAX, Kalman-filter autoregression, and LSTM models provide alternatives or extensions to conventional feed-forward neural networks.These approaches incorporate nonlinear autoregression, physical-driver analysis, or time-series sequence modeling.
  • 0.73–0.84 correlation coefficients were achieved by a two-hidden-layer neural network for 1.8–7.7 MeV electrons, with performance generally decreasing at higher energies.The model used approximately 188,000 five-minute SYMH-index data points from the Van Allen Probes.
  • 0.96 PE at one hour and approximately 0.7 at 24 hours were reported for >2 MeV electrons, but evaluation used validation rather than an independent test set.The same study found lower efficiency at decreasing energies and dependence on magnetic local time.
  • A probabilistic GEO electron forecast remains unusual: Miyoshi and Kataoka produced probabilistic rather than single-point predictions.The review identifies probabilistic prediction as an important recurring challenge for future space-weather research.

5.4 Recapitulation - Relativistic electrons at geosynchronous orbit

The review finds no straightforward winner among GEO electron models because results are difficult to compare and uncertainties enter through both solar-wind drivers and magnetospheric dynamics. It therefore points toward wider-orbit gray-box models, longer horizons, and explicit uncertainty propagation.

  • 5.4 Recapitulation - Relativistic electrons at geosynchronous orbit: No straightforward conclusion or consistent superiority emerges because studies use different datasets and lack standardized benchmarks.Metrics reported across papers cannot easily be transferred between models trained and tested on different data.
  • 5.4 Recapitulation - Relativistic electrons at geosynchronous orbit: Forecast inaccuracies are attributed mainly to uncertainty in predicted solar-wind parameters used as drivers of future electron fluxes.Internal magnetospheric dynamics, including substorm cycles, may also be difficult for black-box models to capture.
  • 5.4 Recapitulation - Relativistic electrons at geosynchronous orbit: Gray-box modeling could extend electron predictions across a wider range of orbits by combining physics-based diffusion equations with machine-learned or Bayesian parameterizations.The review cites learned boundary conditions and Bayesian treatment of diffusion coefficients and electron time loss as examples.
  • 5.4 Recapitulation - Relativistic electrons at geosynchronous orbit: Longer forecast horizons require coupling particle forecasts to solar-wind predictions and modeling uncertainty propagation between models.

5.5 Solar images

Solar-image studies in space-weather forecasting mainly target solar flares and CME arrival time, usually using engineered or automatically extracted features rather than raw images. Results include promising flare skill, modest gains from temporal context, and persistent limitations in feature combination and CME arrival-time accuracy.

  • Solar images are used mainly for solar-flare prediction and coronal-mass-ejection propagation-time prediction.
  • A temporal sliding window of 45 observations improved Decision Tree and Learning Vector Quantization performance by about 10%.The window used 96-minute sampling intervals.
  • TSS was approximately 0.8 for M-class and 0.6 for C-class flare prediction on the whole 2015 test year.
  • FLARECAST compared neural networks, support vector machines, random forests, and statistical methods for binary and probabilistic >M1 and >C1 flare forecasts.Its codes were released as open access, supporting continued community use and development.
  • Combining physics-based, flare-history, and automatically generated features performed worse than using the previous two feature groups separately, with results only slightly better than Bobra and Couvidat (2015).The review suggests testing automatically generated features with a nonlinear classifier.
  • The Huang et al. study directly fed 100×100-pixel active-region patches from SOHO/MDI and SDO/HMI into a convolutional neural network without prior hand-crafted feature extraction.
  • Coronal Mass Ejections and solar wind speed: CME arrival-time mean absolute error ranged from 11.2 hours in 2013 to 22.6 hours in 2018, with typical standard deviations of ±20 hours, and forecasts did not substantially improve over six years.

5.6 Recapitulation - Solar Images

Across solar-flare, CME, and solar-wind forecasting, solar images are rarely used as direct model inputs. The review identifies feature extraction, class imbalance, limited image-source diversity, and the potential value of gray-box models as central considerations.

  • Except for Huang et al. (2018), reviewed studies use magnetograms or EUV images to create hand-crafted or automatically extracted features.
  • FLARECAST showed that machine-learning techniques can outperform statistical methods, motivating more advanced forecasting approaches.
  • The imbalanced solar-flare dataset makes the generality of reported results difficult to assess.
  • The reviewed studies rely almost exclusively on SDO images, limiting the diversity of observational inputs considered.
  • Machine-learning models might be successfully integrated with simulations in a gray-box approach.

5.7 Other space weather related areas

Machine learning has also been applied to additional space-weather problems, including plasmaspheric electron-density estimation and ionospheric scintillation modeling. These applications are described as promising but less systematic.

  • Plasmaspheric electron-density estimation has been addressed with machine-learning methods.
  • Machine learning has been used to model ionospheric scintillation in the ionosphere–thermosphere region.
  • These additional applications are characterized as less systematic but promising for data-driven approaches.

6 New trends in machine learning

The review highlights emerging machine-learning directions that extend beyond routine automation toward scientific discovery. These include gray-box and physics-informed methods, automated architecture search, adversarial robustness, and generative data augmentation.

  • Machine learning can support both automated task execution and knowledge discovery by uncovering relationships and patterns not readily identifiable by humans.
  • The paper presents these emerging ideas as potential tools for scientific discovery in physics during the coming decade.
  • Physics-informed neural networks: Physics-informed neural networks implement a gray-box strategy that combines data-driven machine learning with physics-based simulations.
  • Automatic machine learning: Auto-ML searches for effective neural-network architectures and may help encode interactions among space-weather subsystems.The review specifically mentions radiation belts, the ring current, and solar wind.
  • Adversarial training: Adversarial examples expose a major neural-network weakness because small input perturbations can cause image misclassification.
  • Adversarial training: Adversarial training can make models more robust and generalizable by accounting for perturbations that increase the loss.
  • Generative adversarial networks can generate artificial inputs for data augmentation or gap filling, including total-electron-content maps in space physics.

7 Conclusions

The review concludes that machine learning has gained recognition in space weather, while emphasizing gray-box modeling and several unresolved challenges. These include uncertainty assessment, physically informative inputs, surrogate modeling, imbalanced data, and extracting scientific knowledge from black-box models.

  • Current outlook: Machine-learning forecasting is not yet mainstream in space weather, but the field has gained recognition and may become more predominant over the next decade.The review personally predicts that gray-box approaches may gradually replace conventional physics-based models.
  • Future challenges: Machine-learning applications remain constrained by whether selected inputs contain enough physical information to formulate a meaningful cause–effect forecasting task.The review presents solar images from magnetograms and EUV channels as an example of an unresolved information question.
  • Current outlook: Gray-box models aim to combine physical understanding with large Sun–Earth-system datasets and approximated forecasting models.The review identifies this combination as a central challenge because operational models use approximations and often unobservable parameters.
  • Future challenges: Other open challenges include deciding where black-box surrogate models provide an acceptable accuracy–speed trade-off and addressing imbalanced datasets dominated by quiet conditions.The review also suggests combining high- and low-accuracy simulations and using simulation data in machine-learning pipelines.
  • Future challenges: Reliable space-weather forecasting requires uncertainty assessment rather than only single-point predictions, but propagating uncertainty through the full space-weather chain is computationally demanding.The challenge spans solar images, L1 measurements, magnetospheric observations, and ground-based observations.
  • Future challenges: Opening machine-learning black boxes could support both forecasting and scientific understanding, but users face a tension between making models work and making them understandable.The review connects this knowledge-discovery challenge with the question of how to trust algorithms that are not fully understood.
Loading 1903.05192v2…