Source-linked AI summary
70 years of machine learning in geoscience in review
Jesper Sören Dramsch
TL;DR
Machine learning in geoscience has developed over seven decades, but its applications and validation practices remain uneven. This review synthesizes historical co-development across geoscience and machine learning, covering shallow and deep methods with emphasis on validation, statistics, explainability, and subject-matter expertise. It concludes that machine learning is an established computational-geoscience tool, while applications remain incompletely automated and require careful validation and expertise.
Problem
The review addresses how machine-learning methods and applications developed together in geoscience and why recent enthusiasm should be understood in historical and disciplinary context.
Method
The paper reviews seven decades of machine-learning developments and geoscience applications across shallow methods, deep neural networks, validation practices, and explainability.
Results
Machine learning has matured into an established tool in computational geoscience, although properly validated models take time to develop and applications are not fully automated.
Takeaways & Limitations
Modern tools enable widespread geoscience applications, but scientific use depends on model validation, baseline comparisons, and prevention of leakage and overfitting.
Takeaways & Limitations
Geoscience applications face uncertain, expensive ground-truth labels, uneven reporting of caveats, and limited transfer of trained models to out-of-distribution data.
Abstract
from arXiv · showhide
This review gives an overview of the development of machine learning in geoscience. A thorough analysis of the co-developments of machine learning applications throughout the last 70 years relates the recent enthusiasm for machine learning to developments in geoscience. I explore the shift of kriging towards a mainstream machine learning method and the historic application of neural networks in geoscience, following the general trend of machine learning enthusiasm through the decades. Furthermore, this chapter explores the shift from mathematical fundamentals and knowledge in software development towards skills in model validation, applied statistics, and integrated subject matter expertise. The review is interspersed with code examples to complement the theoretical foundations and illustrate model validation and machine learning explainability for science. The scope of this review includes various shallow machine learning methods, e.g. Decision Trees, Random Forests, Support-Vector Machines, and Gaussian Processes, as well as, deep neural networks, including feed-forward neural networks, convolutional neural networks, recurrent neural networks and generative adversarial networks. Regarding geoscience, the review has a bias towards geophysics but aims to strike a balance with geochemistry, geostatistics, and geology, however excludes remote sensing, as this would exceed the scope. In general, I aim to provide context for the recent enthusiasm surrounding deep learning with respect to research, hardware, and software developments that enable successful application of shallow and deep machine learning in all disciplines of Earth science.
1 Historic Machine Learning in Geoscience
Machine learning in geoscience has evolved alongside broader cycles of enthusiasm, from early statistical and neural methods to modern Gaussian-process and deep-learning applications. Kriging remained productive through an AI winter, while neural networks gained traction as training methods and computational tools developed.
- 1950s–1960s: 1950s–1960s optimism brought k-means, Markov models, and decision trees into early geoscience applications.K-means described sediment-deposit cyclicity, while Markov-chain methods were applied to geological problems.
- 1970s: The 1970s AI winter reduced machine-learning developments and adoption in geoscience, but kriging continued expanding across mining, oceanography, and hydrology.Because kriging was not considered an AI technology, it was largely unaffected by the cultural shift.
- 1980s–1990s: Backpropagation and automatic differentiation helped drive neural-network adoption in geophysics during the late 1980s, including seismic deconvolution and earthquake discrimination.These developments enabled error-correcting training for geophysical applications.
- 1980s–1990s: The 1990s shifted machine learning from knowledge-driven expert systems toward data-driven methods, including Support-Vector Machines, Random Forests, and LSTMs.Neural-network reviews also documented applications in automated well-log analysis and seismic trace editing, while noting training, retraining, precision, optimization, and interpretability concerns.
- Neural networks: Neural networks progressed from perceptrons and single-layer systems to differentiable, multilayer architectures trained by gradient descent.Deep learning combines multiple neuron layers that learn representations at multiple abstraction levels; sigmoid and tanh activations support continuous outputs.
- Kriging and Gaussian Processes: Gaussian Processes extend kriging into a general machine-learning framework in which kernel choice shapes predictions and generalization.In the illustrated classification problem, the linear kernel achieved 71 % accuracy on unseen test data, whereas the RBF kernel achieved 90 %.
2 Contemporary Machine Learning in Geoscience
Contemporary geoscience machine learning combines established algorithms with accessible software, explicit validation, and neural-network regularization. The examples show how these practices support prediction and generalization on unseen data.
- Applications: Machine learning enabled SVMs to identify geomorphological features without modelling the underlying processes.
- Software: Open-source tools replaced difficult proprietary and university software, while scikit-learn introduced a unified application programming interface.
- Model validation: Hold-out validation tests generalization on unseen samples and prevents mistaking training-data memorization for useful predictive performance.
- Support-Vector Machines: An SVM classifier is trained by minimizing hinge loss, then applied to new data containing the same input features.
- Modern Deep Learning: Neural networks use supervised error-correcting training and regularization such as Dropout, while continued training can eventually cause overfitting.
ReLU
Deep learning architectures have expanded geoscience applications, especially seismic interpretation, but successful use still depends on validation, domain expertise, and careful attention to data and model limitations.
- ResNet: Residual blocks use convolutional shortcuts to support networks with up to 152 layers that still generalize well.ResNets popularized skip connections to address degradation as networks become deeper.
- Geoscience applications: Convolutional neural networks transferred from image-classification advances to seismic tasks including salt identification, horizon picking, and broader interpretation workflows.Applications progressed from single-image classification toward segmentation and time-series classification relevant to geoscience.
- Scope: The review covers many network types but necessarily provides only a broad overview, leaving newer architectures such as transformers and attention outside detailed treatment.Siamese networks, transformers, and attention are identified as additional developments.
- Data limitations: Ground-truth data in geoscience is often expensive, uncertain, siloed, proprietary, or stored in formats that are not machine-learning ready.Expert-derived labels can contain label noise that harms accurate and generalizable models.
- Domain requirements: Geoscience applications face expensive decisions, sparse and noisy data, heterogeneous environments, and physical constraints, making subject-matter collaboration and explainability essential.Simpler models are usually easier to interpret, particularly when diagnosing failure modes.
- Validation: Model validation and baseline comparisons can overturn apparently complex solutions: a two-parameter surrogate logistic regression outperformed a roughly 13,000-parameter deep network and overfit less.The simpler model also increased calculation speed, which was described as essential for aftershock forecasting and disaster response.