Source-linked AI summary
Machine Learning and Deep Learning -- A review for Ecologists
Maximilian Pichler, Florian Hartig
TL;DR
ML and DL remain difficult to understand because their principles and inner workings are perceived as opaque, while the field contains many algorithms. This paper reviews their history, algorithmic ideas, and general properties, concluding that they are powerful tools whose use for causal inference remains disputed.
Problem
ML and DL are difficult to understand because their principles and inner workings are perceived as opaque, and the field contains a wealth of algorithms.
Method
The paper reviews the history, algorithmic ideas, and general properties of ML and DL algorithms.
Results
ML and DL algorithms are powerful tools for predictive modeling and data analysis, with superior performance linked to higher flexibility and automatic data-dependent complexity optimization.
Takeaways & Limitations
ML is expected to become an indispensable tool in E&E, comparable to traditional statistical tools such as linear regression models.
Takeaways & Limitations
Use of ML and DL for causal inference remains disputed, and traditional statistical methods depend on the quality of their models.
Abstract
from arXiv · showhide
1. The popularity of Machine learning (ML), Deep learning (DL), and Artificial intelligence (AI) has risen sharply in recent years. Despite this spike in popularity, the inner workings of ML and DL algorithms are often perceived as opaque, and their relationship to classical data analysis tools remains debated. 2. Although it is often assumed that ML and DL excel primarily at making predictions, ML and DL can also be used for analytical tasks traditionally addressed with statistical models. Moreover, most recent discussions and reviews on ML focus mainly on DL, missing out on synthesizing the wealth of ML algorithms with different advantages and general principles. 3. Here, we provide a comprehensive overview of the field of ML and DL, starting by summarizing its historical developments, existing algorithm families, differences to traditional statistical tools, and universal ML principles. We then discuss why and when ML and DL models excel at prediction tasks and where they could offer alternatives to traditional statistical methods for inference, highlighting current and emerging applications for ecological problems. Finally, we summarize emerging trends such as scientific and causal ML, explainable AI, and responsible AI that may significantly impact ecological data analysis in the future. 4. We conclude that ML and DL are powerful new tools for predictive modeling and data analysis. The superior performance of ML and DL algorithms compared to statistical models can be explained by their higher flexibility and automatic data-dependent complexity optimization. However, their use for causal inference is still disputed as the focus of ML and DL methods on predictions creates challenges for the interpretation of these models. Nevertheless, we expect ML and DL to become an indispensable tool in E&E, comparable to other traditional statistical tools.
Introduction
ML and DL have been widely adopted in ecology and evolution, but their principles, inner workings, and relationship to classical statistical tools remain debated. This section motivates a comprehensive overview spanning algorithm families, general principles, prediction, analytical modeling, and ecological applications.
- Applications: ML and DL are increasingly used in ecology and evolution for species recognition, trait extraction, predictive modeling, and approximating mechanistic equations.Applications include video and audio analysis, network link prediction, species distribution models, and neural-network approximations of differential equations.
- Motivation: Despite their popularity, ML and DL algorithms are often perceived as opaque, and their relationship to classical statistical models remains debated.The opacity concerns both their principles and inner workings.
- Motivation: Recent reviews have focused mainly on DL, overlooking the broader range of ML algorithms and their differing behaviors and advantages.Algorithms such as k-nearest-neighbor methods and boosted regression trees do not necessarily share DL-specific properties.
- Algorithm differences: DL is advantageous for image-based tasks because it processes spatial patterns effectively, whereas traditional ML often performs better with fewer observations or structured tabular data.These differences mean that claims about DL cannot automatically be generalized to ML algorithms overall.
- General principles: The review examines shared principles such as regularization and model averaging while explaining where principles change between classical ML and DL.It highlights the contrast between the classical bias-variance trade-off and the overparameterization used by DL models for low generalization error.
- Scope: The paper provides a broad overview of ML and DL history, differences from traditional statistics, predictive and explanatory modeling, limitations, and ecological applications.It also discusses mechanisms behind predictive performance and how these tools can support analytical tasks.
History of ML and DL and its relation to statistics
Statistical learning developed from classical probability-based statistics toward machine learning approaches that use flexible algorithmic structures trained on data. This shift supports prediction and varied learning tasks, while classical statistical inference remains conditional on model assumptions.
- Classical statistics: Classical statistical methods such as MLE, NHST, and MCMC formed the historical foundation of modern statistical learning.
- Computational development: Computers and numerical algorithms enabled more complex statistical and machine learning models, including flexible approaches difficult to handle analytically.
- Relation to statistics: Traditional statistical inference depends on model assumptions, whereas flexible ML methods emphasize predictive generalization and may require computationally intensive resampling or validation.
- General ML principles: ML training adjusts parameters to data while optimizing the bias-variance trade-off so models generalize beyond their training data.
- Learning tasks: Learning tasks include supervised classification and regression, unsupervised learning without observed outcomes, and reinforcement learning based on interactions and consequences.
- Rise of machine learning: Machine learning emerged as an alternative modeling approach that often achieves lower predictive errors by training generic algorithmic structures directly on data.
Important ML and DL algorithms in more detail
The paper surveys major supervised ML and DL architectures, emphasizing their differing data structures, flexibility, computational demands, and interpretability. Ensemble methods often perform well on smaller tabular datasets, while neural architectures support flexible representations across specialized inputs.
- Supervised algorithms: Supervised algorithms include neural networks, regression and classification trees, distance-based methods, and ensemble approaches such as random forests and boosted regression trees.
- Ensemble methods: Random forests fit trees on bootstrap samples with randomized feature selection, offering flexibility and robustness but scaling poorly.
- Support-vector machines: SVMs classify by separating classes with a hyperplane and can use the kernel trick to make nonlinear problems linearly separable.
- Ensemble methods: Boosting trains weak models sequentially to correct errors or optimize a differentiable objective, with gradient boosting described as state-of-the-art.
- Ensemble methods: BRT and RF remain widely used for structured tabular data because they cope better with smaller datasets than comparable DL models.
- Neural architectures: Deep learning uses many hidden layers, while CNNs process images, RNNs process sequences, and GNNs operate directly on graph nodes and edges.
Why does ML work?
ML models can generalize despite high formal flexibility because their effective complexity is shaped by algorithmic and optimization mechanisms. Deep learning departs from the classical bias-variance picture: beyond interpolation, larger networks can reduce generalization error, although out-of-distribution prediction remains challenging.
- Generalization limits: In-distribution ML predictions commonly generalize well, whereas out-of-distribution prediction is challenging for ML and other approaches.For extrapolation, bias may not vanish with increasing model complexity and the optimal complexity is usually lower.
- Deep-learning generalization: Deep neural networks exhibit double descent: after a threshold, increasing parameters can decrease generalization error, making overparameterization beneficial.Deeper and wider networks can reduce generalization error, while parameter count is a poor measure of effective complexity.
- Complexity optimization: ML algorithms use internal and external complexity optimization to limit effective flexibility and avoid overfitting.Internal mechanisms include algorithmic self-adaptation; external mechanisms optimize hyperparameters using cross-validation or train-development-test splits.
- Complexity optimization: Hyperparameters, including architecture and regularization settings, are tuned to control model flexibility and predictive error.Nested cross-validation separates predictive-error estimation from hyperparameter tuning, while regularization imposes constraints on flexibility.
- Deep-learning generalization: Deep-learning generalization can improve after training loss is nearly zero, partly because networks favor simpler functions and contain sparse subnetworks.More than 90% of trained-network parameters can sometimes be set to zero with little or no loss of generalization accuracy.
Emerging Trends in ML (in E&E)
ML and DL use in ecology and evolution has increased sharply, with DL gaining attention while classical ML remains important in practical applications. Emerging uses extend beyond prediction toward unstructured-data analysis, simulation-based inference, machine-assisted data collection, and more interpretable or responsible AI.
- Trends in algorithm use in E&E: Use of ML and DL methods in E&E increased sharply over the last decade, while classical ML methods still dominate practical applications.SVMs were among the prominent classical methods, and publication trends show continued importance for classical ML.
- Trends in algorithm use in E&E: Classical ML is expected to remain competitive for limited structured tabular data and many ecological prediction, network, and gap-filling tasks.The passages report little evidence that DL outperforms classical ML on supervised learning with limited structured data.
- Trends in algorithm use in E&E: DL is expected to grow for complicated, unstructured ecological data such as aerial and camera-trap images or genomic sequences.DL also expands clustering and dimension reduction to images and sequences.
- Emerging responsible and explainable AI: ML and DL face a trade-off between predictive accuracy and interpretability, with DL described as potentially most accurate but least interpretable.Explainable AI is presented as a response to increasing interpretive difficulty and concerns about predictions being correct for the wrong reasons.
- New applications for ML in E&E: Future ML and DL applications will extend beyond classical prediction and classification to hypothesis generation, simulation-based inference, and model emulation.ML can generate summary statistics, accelerate or emulate process-based models, and calibrate complex stochastic models without likelihood evaluation.
- New applications for ML in E&E: Unsupervised learning can reduce dimensionality, detect patterns and trends, handle high-dimensional data, and identify anomalies before or within downstream analyses.These capabilities are motivated by the growing dimensionality of data from cheap sensors and other collection sources.
- Rethinking the data collection process in the light of the new methods: Machine-assisted data collection could create larger reusable datasets by reducing image-recognition labor, but models should be accessible without expert retraining.The authors advocate publicly available models and simple interfaces usable by nonexperts.
Conclusion
ML and DL are powerful tools for predictive modeling and data analysis, with performance advantages attributed to flexibility and data-dependent complexity optimization. Their use requires caution for extrapolation, causal interpretation, and task selection, but they are expected to become indispensable in E&E.
- Conclusion: ML and DL provide powerful tools for predictive modeling and data analysis, while classical ML remains important for structured ecological data.The paper notes that deep learning has conquered image-based tasks, whereas classical ML algorithms such as RF and BRT still excel on structured data.
- Conclusion: Higher flexibility and automatic data-dependent complexity optimization help explain ML and DL performance advantages over statistical models.RF and BRT balance complexity through ensembles, while DL uses overparameterization and stochastic gradient descent as indirect regularization.
- Conclusion: ML methods are optimized more for prediction than classical statistical tools, so causal interpretations require caution.The paper states that prediction-focused methods create interpretation challenges and that causal inference remains disputed.
- Conclusion: Extrapolation beyond the training feature space is a fundamental limitation of data-driven prediction approaches.Predictions can fail when models rely on non-causal proxies or when relationships change outside the observed data range; ecological or mechanistic constraints may improve generalizability.
- Conclusion: Simpler models may be preferable when they provide greater interpretability, statistical power, and lower computational costs.The paper advises considering whether ML or DL is necessary or promising for a task before using them.
- Conclusion: ML is expected to become an indispensable E&E tool alongside established methods such as linear regression and analysis of variance.The paper also highlights explainable AI and Bayesian neural networks as efforts to bridge ML and statistical tools, including uncertainty and prediction intervals.
Authors’ Contributions
The authors jointly conceived and designed the study, contributed equally, and jointly prepared the manuscript.
- MP and FH jointly conceived and designed the study.
- Both authors contributed equally to writing and manuscript preparation.
- The manuscript was jointly prepared by both authors.
Data Availability
Supporting materials provide trend-analysis results and example code for different ML algorithms in R, Python, and Julia.
- Example code chunks for different ML algorithms are provided in the Supporting Information.
- The supporting materials combine trend analysis with algorithm-specific example code.
- The example code is available for R, Python, and Julia.
Supporting Information Trend analysis
The supporting trend analysis searches PubMed and Medline from 1920 to 2021, normalizes yearly hits, and examines ML and DL algorithms in ecological and evolutionary research.
- Trend analysis: The global trend analysis searched PubMed and Medline NLM databases from 1920 to 2021 using the R package europepmc.
- Trend analysis: Queries represented deep learning, machine learning, and classical statistical approaches, with yearly hits normalized by total hits.
- Trend analysis: Ecology and evolution results were identified by adding ecology, ecological, or evolution terms to the queries.
- Algorithm queries: The searches also covered RF, BRT, kNN, ridge, lasso, elastic-net, and SVM algorithms.
- Ecological applications: Ecological keywords included species distribution, species interaction, mortality, remote sensing, invasive species, biodiversity, and camera traps.
- Word-cloud analysis: Word clouds were created from abstracts and titles using the R packages tm, wordcloud, and wordcloud2.
- Implementation: A quick guide with code chunks for common ML algorithms is available online in R, Python, and Julia.