Source-linked AI summary
Markov switching negative binomial models: an application to vehicle accident frequencies
Nataliya V. Malyshkina, Fred L. Mannering, Andrew P. Tarko
TL;DR
The paper addresses whether accident frequencies should be modeled with multiple, changing roadway-safety states rather than a single process. It specifies two-state Markov switching negative binomial models and estimates them with Bayesian inference and MCMC. The models fit the Indiana interstate accident data better than the standard negative binomial model, with safety states associated with weather conditions.
Problem
Single-process accident models do not represent roadway-safety states that vary with changing, partly unobserved conditions over time.
Method
The paper estimates two-state Markov switching negative binomial models for five-year Indiana interstate accident frequencies using Bayesian inference and MCMC.
Results
258.4 and 299.2 improvements in log-marginal-likelihoods are obtained by the restricted and full MSNB models, respectively, versus the standard NB model.
Takeaways & Limitations
The more frequent state is safer, while the less frequent state is less safe; the states are correlated with better and adverse weather conditions, respectively.
Takeaways & Limitations
Further empirical studies and models with more than two roadway-safety states are needed to demonstrate the approach’s broader potential.
Abstract
from arXiv · showhide
In this paper, two-state Markov switching models are proposed to study accident frequencies. These models assume that there are two unobserved states of roadway safety, and that roadway entities (roadway segments) can switch between these states over time. The states are distinct, in the sense that in the different states accident frequencies are generated by separate counting processes (by separate Poisson or negative binomial processes). To demonstrate the applicability of the approach presented herein, two-state Markov switching negative binomial models are estimated using five-year accident frequencies on Indiana interstate highway segments. Bayesian inference methods and Markov Chain Monte Carlo (MCMC) simulations are used for model estimation. The estimated Markov switching models result in a superior statistical fit relative to the standard (single-state) negative binomial model. It is found that the more frequent state is safer and it is correlated with better weather conditions. The less frequent state is found to be less safe and to be correlated with adverse weather conditions.
1 Introduction
The paper motivates two-state Markov switching count models for accident frequencies because roadway safety varies with changing, partly unobserved conditions. Unlike single-process models and zero-inflated models, the approach allows roadway segments to switch between distinct safety states over time.
- Motivation: Accident frequencies are non-negative integer counts, making Poisson and negative binomial models reasonable statistical approaches.Standard models assume a single accident-generating process and relate observed frequencies to roadway-segment characteristics.
- Proposed approach: Two-state Markov switching models represent unobserved roadway-safety states and permit switching between them over time.The two-state formulation is used for illustration, although more than two states may exist and could be tested after extending the model.
- Motivation: Weekly accident frequencies fluctuate widely, suggesting that roadway segments can become considerably more or less safe under different conditions.Potential influences include environmental conditions, driver reactions, and other factors unavailable to the analyst.
- Relation to prior models: Markov switching extends zero-inflated models by allowing transitions over time and by permitting two distinct unsafe states rather than assuming a safe state.Both states may generate nonzero accident frequencies.
2 Model specification
The model introduces a latent two-state Markov chain governing roadway safety over time, with negative binomial accident processes specified separately for the two states. State transitions are estimated alongside state-specific regression and dispersion parameters.
- Latent state process: A latent state variable assigns all roadway segments to one of two roadway-safety states during each time period.The state follows a stationary two-state Markov chain with time-independent transition probabilities.
- Latent state process: Transition probabilities p0→1 and p1→0 describe movement between states and are unknown parameters estimated from accident data.The stationary unconditional state probabilities are derived from these transition probabilities.
- Latent state process: State 0 and state 1 are labeled the more frequent and less frequent states, respectively, with state 0 constrained to occur at least as often as state 1.This labeling avoids ambiguity when switching state labels.
- State-specific count processes: Each state has a separate negative binomial process for accident counts on roadway segments during each time period.The model uses roadway characteristics such as segment length, curves, grades, and pavement properties as explanatory variables.
- State-specific count processes: The two states have distinct coefficient vectors and over-dispersion parameters, with the first coefficient serving as each state’s intercept.The likelihood is defined under independent accident events, and latent states are included among the estimable parameters.
- Model definition: Equations (1)–(7) define the two-state Markov switching negative binomial models considered in the study.The state vector contains all state values across the T observed time periods.
3 Model estimation methods
Because the safety states are unobserved and the parameter space is large, the paper uses Bayesian inference with MCMC and compares models using Bayes factors based on marginal likelihoods.
- Bayesian estimation: Bayesian inference is used because unobserved state variables make traditional maximum likelihood estimation of Markov switching models of limited use.The posterior combines the observed data with prior knowledge about the model parameters.
- Model comparison: The harmonic mean formula is used to calculate each model’s marginal likelihood from posterior expectations.The marginal likelihood is the probability distribution of the observed data given the model.
- Bayesian estimation: MCMC simulations sample from the posterior distribution when direct integration over the many-component parameter vector is infeasible.The posterior is known up to a normalization constant, which makes MCMC a practical computational method.
- Implementation check: The estimation code was tested on artificial accident data generated from known models and successfully reproduced the underlying MSNB models.The test used MATLAB simulation code and artificial data sets.
- Model comparison: Bayes factors compare models through ratios of marginal likelihoods under equal model prior probabilities.The approach penalizes models with too many parameters and guards against overfitting.
4 Model estimation results
The study estimates standard and two-state Markov switching negative binomial models for weekly accident frequencies, finding strong empirical support for switching and distinct roadway-safety states. The less frequent state is less safe, more volatile, and associated with adverse weather conditions.
- Data and models: 5769 accidents across 335 Indiana interstate segments and 260 weeks support estimation of four accident-frequency models.The data cover 1995–1999, with one common weekly state for all roadway segments.
- Data and models: The restricted MSNB model switches the intercept and over-dispersion parameter, whereas the full MSNB model allows all estimable parameters to switch.Both switching models are estimated using Bayesian-MCMC methods; the standard NB model is estimated by both MLE and Bayesian-MCMC.
- Model comparison: 258.4 and 299.2 improvements in log-marginal likelihood favor the restricted and full MSNB models, respectively, over the standard NB model.Their posterior probabilities relative to the standard NB model are larger by e258.4 and e299.2, respectively.
- Model comparison: 294.6 and 336.4 improvements in maximum log-likelihood favor the restricted and full MSNB models, respectively, over the standard NB model.AIC and BIC also strongly favor the MSNB models, despite only modest increases in continuous model parameters.
- State interpretation: The full MSNB model estimates the less frequent state as about four times as rare as the more frequent state and about two times higher in weekly accident rate.The less frequent state is also less safe and has greater accident-rate volatility, with α = 0.443 in state 0 and α = 1.16 in state 1.
- State interpretation: The less frequent, less safe state is positively correlated with extreme temperatures, precipitation, snowfall, fog, frost, and low visibility.Adding weather variables leaves the two states intact, and the new and old posterior probabilities correlate at around 90%.
5 Summary and conclusions
The study finds that roadway safety is represented by multiple states whose accident rates and covariate effects differ, with states correlated with changing weather and other conditions.
- Two roadway-safety states exist, and their accident frequencies are correlated with weather conditions.The authors relate transitions to drivers’ and possibly maintenance services’ adjustment to adverse and changing conditions.
- The less frequent state is significantly less safe than the more frequent state.The full MSNB model estimates about twice as many weekly accidents in the less frequent state.
- Covariates can have different magnitudes or directions of influence across the two safety states.The paper reports significant state differences for several roadway and traffic variables, including pavement quality, ramps, traffic, bridges, and trucks.
- Unlike zero-inflated models, Markov switching models allow transitions between states and do not assume one state is safe or accident-free.Accident frequencies may be nonzero in both states.
- Additional empirical studies and models with more than two roadway-safety states are identified as future research directions.The authors propose testing the approach on other accident samples and extending it to multi-state models.
Appendix: MCMC simulation algorithm
The appendix describes a hybrid Gibbs sampler that combines Gibbs and Metropolis-Hastings updates to generate posterior draws for the model parameters and latent states.
- The prior specifies normal distributions for state-specific regression and dispersion parameters and beta distributions for transition probabilities.The transition-probability prior also imposes the model’s stated restriction.
- The hybrid Gibbs sampler combines Gibbs and Metropolis-Hastings sampling to obtain posterior draws.Gibbs sampling is used when conditional posteriors are known, while Metropolis-Hastings handles distributions known up to normalization constants.
- The algorithm begins from an arbitrary parameter value with positive joint density and iteratively updates parameter components.Each iteration generates a new parameter vector from conditional posterior distributions.
- The first 3×10^5 draws are typically discarded as burn-in from runs containing 3×10^6 draws.Remaining draws are thinned and stored for Bayesian inference.
- Metropolis-Hastings proposes candidate parameter values from a fixed jumping distribution and accepts them using a calculated ratio.The jumping distribution remains unchanged across post-burn-in draws, and its normalization constant cancels in the acceptance calculation.
- The sampler updates state-specific regression parameters and over-dispersion terms with Metropolis-Hastings, transition probabilities with Gibbs sampling, and latent states with Gibbs sampling.Latent-state subsections are sampled jointly to address strong neighboring-state correlations.
- State subsections of length 10 to 14 are typically sampled jointly to speed MCMC convergence.Joint enumeration over 2^τ possible subsection values permits normalization of the conditional posterior.
- The appendix notes that Cauchy jumping distributions produced similar results to the normal jumping distributions.