Uncovering ecological state dynamics with hidden Markov models

Brett T. McClintock; Roland Langrock; Olivier Gimenez; Emmanuelle Cam; David L. Borchers; Richard Glennie; Toby A. Patterson

Source-linked AI summary

Uncovering ecological state dynamics with hidden Markov models

Brett T. McClintock, Roland Langrock, Olivier Gimenez, Emmanuelle Cam, David L. Borchers, Richard Glennie, Toby A. Patterson

arXiv:2002.10497v2Feb 24, 2020q-bio.QMstat.AP

TL;DR

Ecologists often need to infer changing system states from incomplete or indirect observations. This review introduces and synthesises hidden Markov models as a framework for separating state and observation processes, illustrating their applications and practical use. It concludes that HMMs offer broad inferential potential, while requiring careful interpretation and attention to data and modelling constraints.

Problem
Ecological states across individuals, populations, communities, and ecosystems are often hidden, making reliable inference from indirect observations difficult.
Method
The review explains HMM structure and terminology, synthesises applications, presents a flexible customisation template, and discusses implementation and interpretation.
Results
The review establishes HMMs as a broadly applicable framework for ecological inference by linking diverse applications through shared state and observation processes.
Takeaways & Limitations
HMMs can support ecological inference across hierarchical levels and observation types when their formulation is tailored to the system of interest.
Takeaways & Limitations
HMMs are not a panacea: simplifying assumptions may misrepresent some ecological processes, and model states can be crude proxies for ecological states.

Abstract

from arXiv · show

Ecological systems can often be characterised by changes among a finite set of underlying states pertaining to individuals, populations, communities, or entire ecosystems through time. Owing to the inherent difficulty of empirical field studies, ecological state dynamics operating at any level of this hierarchy can often be unobservable or "hidden". Ecologists must therefore often contend with incomplete or indirect observations that are somehow related to these underlying processes. By formally disentangling state and observation processes based on simple yet powerful mathematical properties that can be used to describe many ecological phenomena, hidden Markov models (HMMs) can facilitate inferences about complex system state dynamics that might otherwise be intractable. However, while HMMs are routinely applied in other disciplines, they have only recently begun to gain traction within the broader ecological community. We provide a gentle introduction to HMMs, establish some common terminology, and review the immense scope of HMMs for applied ecological research. We also provide a supplemental tutorial on some of the more technical aspects of HMM implementation and interpretation. By illustrating how practitioners can use a simple conceptual template to customise HMMs for their specific systems of interest, revealing methodological links between existing applications, and highlighting some practical considerations and limitations of these approaches, our goal is to help establish HMMs as a fundamental inferential tool for ecologists.

1 Introduction

Ecological state dynamics span hierarchical levels but are often difficult to observe directly, leaving ecologists to infer hidden processes from incomplete or indirect evidence. This review presents HMMs as a flexible framework for disentangling state and observation processes across diverse ecological applications.

Ecological state dynamics range from individual births and deaths to population, community, and ecosystem changes, including regime shifts.
HMMs jointly model underlying state dynamics and state-dependent observations to detect changes in noisy time-dependent phenomena.
Applications include reconstructing historical regime shifts, inferring long-term biodiversity dynamics, and determining species identity from environmental DNA.
HMMs can connect species occurrence to habitat, density, land use, host-pathogen, and predator-prey variation, while capture-recapture data can inform survival, dispersal, reproduction, disease status, and habitat use.
New data streams, greater computing power, and user-friendly software have increased HMM adoption, but their broad ecological potential remains underrecognised.
The review introduces HMM structure and terminology, synthesises applications, proposes a customisation template, and discusses implementation challenges and technical tutorials.

2 Hidden Markov models

HMMs model sequential data using a finite, temporally dependent hidden state process and state-dependent observations. Their formulation separates these processes into components that support ecological inference while allowing extensions and connections to related latent-state models.

Basic model formulation: HMMs describe sequential systems through a finite set of hidden states whose future states depend on the current state.The hidden state at time t takes one of N values, distinguishing HMMs from models with continuous state spaces.
Basic model formulation: HMM indices can represent regular time intervals, positions in sequences, or ordered events such as marine mammal dives.Continuous-time formulations also exist, although they have been less frequently applied in ecology.
Basic model formulation: The basic formulation assumes the Markov property for states and conditional independence of observations given the current state.These assumptions simplify dependence by making the current state sufficient for transitions and observations, but their ecological adequacy requires careful consideration.
Basic model formulation: A basic N-state HMM is specified by initial-state probabilities, state-transition probabilities, and state-dependent observation distributions.The observation distributions can accommodate categorical, count, or non-negative continuous data through appropriate distributional families.
Inference: A blue-whale example uses a two-state HMM to infer interspersed bouts of low and high feeding activity from 53 consecutive dives.The estimated initial distribution indicated greater probability of beginning in the low-activity state, while high-activity bouts tended to span fewer dives.

3 Ecological applications of hidden Markov models

HMM applications span ecological hierarchy levels, linking partially observed data to hidden states and their dynamics. The review shows how established capture–recapture models and community-level occupancy models fit this framework.

3 Ecological applications of hidden Markov models: HMMs are applied from individual organisms through populations and communities to identify ecological states that are difficult or impossible to observe directly.Applications include existential, developmental, and spatial states across the ecological hierarchy.
3.1.1 Existential state: The Cormack–Jolly–Seber model is a two-state HMM with alive and absorbing dead states, estimating survival from binary capture–recapture observations.Its observation process is Bernoulli, with detection probability p when alive and probability zero when dead.
3.1.1 Existential state: Extending the model with an unborn state allows birth and death to be inferred from the same binary encounter histories.The three-state formulation uses α1, αt, and βt to describe initial membership, birth timing, and entry conditional on prior non-entry.
3.1.1 Existential state: Multi-state models accommodate uncertain developmental traits and support hypotheses about disease, reproduction, and social dominance.These models address states that may be partially or completely unobservable.
3.2.3 Spatial state: Community-level HMMs extend occupancy models across species to estimate site-level and metacommunity richness and derive α, β, and γ diversity metrics.They use presence–absence data for species that could occupy a community and can construct Jaccard similarity as a β-diversity measure.

4 Implementation, challenges, and pitfalls

HMM implementation requires choosing software and model structures that fit the data-generating process, while accounting for computational and interpretive challenges. Their simplifying assumptions, finite state spaces, and numerical estimation procedures can make applications difficult or inappropriate.

4.1 Software: HMM software packages differ in flexibility, including support for multivariate, hierarchical, semi-Markov, covariate, constrained, missing-data, and custom-distribution models.The available packages vary in their state-dependent distributions and other capabilities.
4.2 Challenges and pitfalls: HMM appropriateness depends on whether hidden dynamics are Markovian and whether the system can be represented by a feasibly finite set of latent states.Residual diagnostics and expert knowledge help assess these case-dependent assumptions.
4.2 Challenges and pitfalls: Large or coarse-grained state spaces can make fully parameterised transition matrices computationally expensive and may mask or misclassify meaningfully distinct states.Choosing the number of states has no foolproof automatic solution and generally requires system-specific expertise.
4.2 Challenges and pitfalls: HMM parameter estimation relies on numerical procedures vulnerable to local maxima, label switching, poor mixing, and additional challenges as model complexity increases.These issues arise because HMMs lack an analytical solution for parameter estimation.
4.2 Challenges and pitfalls: Unsupervised HMM states are data-driven modal patterns that may only be crude proxies for biologically meaningful ecological states.Movement states labelled “foraging” may indicate area-restricted search without proving when or where feeding occurred.

5 Future directions

The review anticipates broader ecological use of HMMs as they provide a tractable way to connect sequential observations with complex, partly unobservable dynamics. Future applications may integrate multiscale biotic and abiotic data, but remain constrained by data, modelling, and accessibility hurdles.

5 Future directions: A Web of Science search suggests rapidly increasing awareness of HMMs within the ecological community over the past two decades.The search covered Biology, Ecology, Marine Freshwater Biology, and Zoology categories.
5 Future directions: HMMs can capture serial correlation, non-linearity, non-normality, and non-stationarity through discrete states and basic dependence structures.The authors present these capabilities as a tractable synthesis for formulating domain-specific models from sequential data.
5 Future directions: Future HMM applications may integrate biotic and abiotic observations at large spatio-temporal scales to investigate complex ecosystem-level processes.The state process may represent ecosystem states or lower-level unobservable dynamics within a larger non-Markovian model.
5 Future directions: Hierarchical formulations, nonparametric approaches, and coupled HMMs offer methodological developments for multiresolution, flexible, and interacting state processes.These developments address different temporal resolutions, restrictive distributional assumptions, and interacting dynamics.
5 Future directions: Widespread implementation remains hindered by unfamiliarity, limited data availability and compatibility, and the need for faithful discrete-state conceptualisations of ecosystem dynamics.These hurdles particularly affect long-term, broad-scale ecological inference.

— Supplementary Material

The supplementary material provides author and institutional information for the contributors to the paper.

— Supplementary Material: Brett T. McClintock is affiliated with the NOAA National Marine Fisheries Service in the United States.
— Supplementary Material: Roland Langrock is affiliated with the Department of Business Administration and Economics at Bielefeld University.
— Supplementary Material: Olivier Gimenez is affiliated with CNRS Centre d’Ecologie Fonctionnelle et Evolutive in France.
— Supplementary Material: Emmanuelle Cam is affiliated with the Laboratoire des Sciences de l’Environnement Marin and partner institutions in France.
— Supplementary Material: David L. Borchers and Richard Glennie are affiliated with the School of Mathematics and Statistics at the University of St Andrews.
— Supplementary Material: Toby A. Patterson is affiliated with CSIRO Oceans and Atmosphere in Australia.

Appendix A: Dynamic species co-existence HMM

The appendix formulates an HMM for species co-existence using presence–absence observations. Four latent states represent occupancy combinations, while transition and observation probabilities encode colonisation, extinction, replacement, and detection.

Appendix A: Dynamic species co-existence HMM: The model uses four latent states: species A occupied, species B occupied, both species occupied, and an unoccupied site.The appendix denotes these as A, B, AB, and U.
Appendix A: Dynamic species co-existence HMM: Presence–absence observations are coded from 0 to 3 according to which species are detected at a site.The supplied passage begins the coding scheme with 0 for neither species and 1 or 2 for only A or only B.
Appendix A: Dynamic species co-existence HMM: The transition matrix P(xt) is organised by next-state rows and current-state columns, with diagonal elements identified separately.
Appendix A: Dynamic species co-existence HMM: Transition probabilities represent joint or species-specific colonisation, extinction, and replacement between consecutive time points.Parameters include γAB, γA, γB, ηA, ηB, ωA, ωB, ϵAB, ϵA, ϵB, νA, and νB.
Appendix A: Dynamic species co-existence HMM: Observation probabilities distinguish true occupancy from imperfect detection, including detecting only one species or both species when both are present.Parameters include ψA, ψB, ψAB, pA, pB, rAB, rAb, and raB.

Appendix B: HMM software

HMM implementation is supported by numerous R packages and standalone programs, ranging from general frameworks to tools specialised for ecological applications. The supplementary tutorial illustrates a complete, reproducible analysis cycle using a striated caracara VDBA time series.

HMM software: R hosts 26 packages whose descriptions include “hidden Markov,” although they differ in supported distributions and capabilities.Most include simulation, parameter estimation, and state decoding for arbitrary numbers of states.
HMM software: General-purpose R packages provide simulation, parameter estimation, and state decoding, but many specialise in categorical sequences or particular HMM applications.Specialised tools include capture-recapture and animal-movement packages.
HMM software: Standalone programs such as MARK and E-SURGE support ecological HMM applications including capture-recapture and presence-absence data with observation-process error.These frameworks can represent non-detection and state uncertainty.
Illustrative tutorial: The tutorial’s caracara example demonstrates formulation, parameter estimation, candidate-model selection, goodness-of-fit checking, latent-state inference, and interpretation.The accompanying caracaraExample.R script reproduces and extends the example with momentuHMM.

S2.1 State process

The HMM state process models latent ecological states as a finite-state Markov chain, while the observation process links current states to data through state-dependent distributions. Model specification requires ecological judgment about states, transitions, initial conditions, and observation assumptions.

State process: The state process is a Markov chain over N states, where the next state depends probabilistically on the current state rather than earlier states.Its evolution is determined by an N × N transition-probability matrix.
State process: Specifying the state process requires choosing the number of states and any structural restrictions on transitions based on the latent ecological process.Statistical model-selection criteria alone may be insufficient for this choice.
State process: In the caracara example, four states are intended to represent behavioural modes, and transitions are allowed between every pair of behaviours.The initial state probabilities are set to the stationary distribution of the Markov chain.
State process: State-transition probabilities can be constrained or made time-dependent when ecological context requires additional structure.For example, Cormack–Jolly–Seber models forbid transitions from “dead” to “alive.”
Observation process: Observations are conditionally independent given the states, so serial dependence must be represented through the state process.This assumption can be unrealistic when observations remain dependent within a single state.
Observation process: HMMs accommodate discrete, continuous, and multivariate observations with state-dependent distributions that may depend on covariates.For positive continuous VDBA, the example uses gamma distributions with freely estimated, data-driven parameters.

S3.2 Likelihood evaluation using the forward algorithm

The forward algorithm evaluates an HMM likelihood recursively by propagating state-specific information through the observation sequence. This avoids summing explicitly over all possible state sequences and makes likelihood evaluation computationally feasible for long series.

Likelihood evaluation: The algorithm updates α_t step by step across the time series, using the Markov and conditional-independence assumptions to obtain the likelihood.The initial calculation is α_1 = δP(x_1).
Likelihood evaluation: The likelihood can be written as a sum over all possible state sequences, but direct evaluation requires N^T summands.This becomes infeasible even for moderate observation counts.
Likelihood evaluation: The forward algorithm defines α_t(j) from the observations through time t and the event that state j is active at time t.These quantities retain both cumulative likelihood information and state-probability information.
Likelihood evaluation: Likelihood evaluation costs O(T), so recursive computation is typically feasible for sequences containing millions of observations.The efficiency follows from exploiting the model’s dependence structure.

S3.3 Model fitting

HMM fitting estimates model parameters after a formulation is selected, and the caracara example uses numerical likelihood maximisation for a four-state model. Reliable fitting also requires attention to optimisation, candidate-model comparison, and goodness of fit.

Model fitting: For the VDBA data, numerical likelihood maximisation estimates the initial distribution, transition matrix, and state-dependent distributions for N = 4 states.The resulting state-dependent distributions are shown in Figure 2.
Model fitting: Complex HMMs can have problematic local likelihood maxima, so numerical optimisation should generally be repeated from several random initial parameter vectors.The EM algorithm may sometimes yield an easier maximisation problem but can be more cumbersome to implement.
Model fitting: Numerical likelihood maximisation, the EM algorithm, and Bayesian MCMC are three common approaches to fitting HMMs.All can exploit recursive schemes such as the forward algorithm.
Model selection: Candidate models may differ in state number, state-dependent distribution family, or covariates affecting transition probabilities.Exploratory analysis and relevant theory should help keep the candidate set small.
Model selection: Model-selection criteria should guide rather than determine the choice, which should also consider goodness of fit and whether study-relevant data patterns are captured.Poorly captured patterns can indicate which model components need modification.

S4.1 Model selection

Model selection combines biologically informed limits on the number of states with criteria such as AIC or BIC. In the VDBA example, both criteria selected four states, while data-driven state definitions can favour biologically implausible complexity.

Biologically informed candidates: The VDBA analysis restricted candidate models to four or fewer states based on expert knowledge of species behaviour.The authors judged five or more genuinely different behavioural modes unlikely within one hour.
Criterion-based selection: Both AIC and BIC were minimised by the model with N = 4 states.
Model-complexity caveat: Data-driven state characteristics can lead AIC or BIC to support more states than seem biologically reasonable.Additional states may compensate for insufficient structure or flexibility in the model formulation.
Model-complexity caveat: Biological judgement may therefore justify restricting model complexity despite poorer statistical fit.

S4.2 Model checking

HMM model checking uses pseudo-residuals and simulation-based comparisons to assess whether fitted models reproduce observed distributions and dependence. In the four-state VDBA example, pseudo-residuals showed minor tail deviations and some lack of fit, while their interpretation can be technically demanding.

Pseudo-residuals: Pseudo-residual model checking uses a conditional distribution of each observation given all other observations in the sequence.The conditional distribution is obtained from forward and backward variables.
Pseudo-residuals: A probability integral transform places observations on the standard normal scale, facilitating detection of extreme observations and lack of fit.
VDBA assessment: The four-state VDBA model showed tail deviations from normality, including fewer low-acceleration observations than expected under the fitted model.The deviation was relatively minor unless the analysis focused specifically on very small VDBA values and associated behaviours.
VDBA assessment: Figure 3 displays the VDBA pseudo-residual QQ-plot with point-wise 95% confidence envelopes.
Practical considerations: Pseudo-residual calculation requires both forward and backward variables and may involve numerical underflow issues.Specialised HMM software can perform these calculations.
Simulation checks: Simulation-based checking compares relevant patterns or summary statistics between real observations and sequences simulated from the fitted model.Candidate summaries include empirical quantiles and dependence measures such as autocorrelation.

S5.1 Overview

State decoding infers hidden states after model fitting, either separately at each time point or as one globally most likely sequence. Local decoding provides uncertainty information, whereas Viterbi decoding yields a hard sequence without state-misclassification probabilities.

Overview: State decoding is the inference of hidden states s_1, . . . , s_T after parameter estimation and model checking.
Local decoding: Local decoding selects the most likely state independently at each time point.It uses Pr(S_t = s_t | x_1, . . . , x_T).
Local decoding: Locally decoded states can ignore serial dependence, so individually most likely states need not form the jointly most likely sequence.
Global decoding: Global decoding seeks the complete state sequence with the highest joint probability given all observations.
VDBA example: In the VDBA example, local and global decoded states agreed over 95% of the time.Figure 5 shows the globally decoded state sequence colour-coded across 3,600 acceleration observations.
Global decoding: The Viterbi algorithm efficiently solves the global decoding optimisation problem through a recurrence followed by backward tracing.
Comparing approaches: Only local decoding supplies uncertainty quantification for decoded states; Viterbi decoding provides a hard most-likely sequence without misclassification probabilities.

S6 Interpretation

HMM interpretation must account for data-driven state definitions and model assumptions rather than treating inferred states as direct substitutes for ecological theory. In the VDBA example, four states were qualitatively linked to behaviours, but dependence-structure shortcomings may matter for questions about time spent in states.

Interpretation cautions: HMM state parameters and transition probabilities are data-driven, so inferred states may not match the ecological meanings initially intended.
Interpretation cautions: Interpreting HMM results requires examining assumptions about state number, state-dependent distributions, Markov dependence, and conditional independence.
VDBA interpretation: The VDBA model’s four states were qualitatively associated with resting, minimal activity, moderate activity, and flying.
VDBA interpretation: Global decoding assigned 31%, 27%, 27%, and 14% of one hour’s time steps to resting, minimal activity, moderate activity, and flying, respectively.
VDBA interpretation: Minor marginal-distribution and dependence-structure issues may be acceptable for broad behaviour classification but require attention when estimating time spent in each state.More sophisticated HMMs or alternative frameworks may be needed when the full empirical dependence structure matters.
Overall perspective: HMMs combine computational efficiency with a structure applicable across a wide range of ecological applications, while retaining assumptions and limitations.