Source-linked AI summary
Lognormal and Gamma Mixed Negative Binomial Regression
Mingyuan Zhou, Lingbo Li, David Dunson, Lawrence Carin
TL;DR
Bayesian count regression has lacked simple, efficient posterior computation. This paper proposes a lognormal-and-gamma mixed negative-binomial model and closed-form Gibbs and variational Bayes inference; its LGNB model outperforms Poisson and NB models in Pearson-residual goodness of fit.
Problem
Bayesian count regression has been limited by inefficient posterior computation, while maximum-likelihood methods do not incorporate prior information and NB dispersion estimates can be unreliable in some settings.
Source: §1 · pp. 1–2Method
Results
The LGNB model clearly outperforms Poisson and NB models in goodness of fit measured by Pearson residuals.
Source: §5.2 · p. 7Takeaways & Limitations
The proposed closed-form Bayesian inference can be implemented routinely and is readily generalizable to multivariate dependence structures.
Source: §1 · p. 2For lognormal-Poisson regression, marginalizing the random effect gives no analytical count distribution, making maximum-likelihood estimation less straightforward.
Source: §2.2 · p. 2
Abstract
from arXiv · showhide
In regression analysis of counts, a lack of simple and efficient algorithms for posterior computation has made Bayesian approaches appear unattractive and thus underdeveloped. We propose a lognormal and gamma mixed negative binomial (NB) regression model for counts, and present efficient closed-form Bayesian inference; unlike conventional Poisson models, the proposed approach has two free parameters to include two different kinds of random effects, and allows the incorporation of prior information, such as sparsity in the regression coefficients. By placing a gamma distribution prior on the NB dispersion parameter r, and connecting a lognormal distribution prior with the logit of the NB probability parameter p, efficient Gibbs sampling and variational Bayes inference are both developed. The closed-form updates are obtained by exploiting conditional conjugacy via both a compound Poisson representation and a Polya-Gamma distribution based data augmentation approach. The proposed Bayesian inference can be implemented routinely, while being easily generalizable to more complex settings involving multivariate dependence structures. The algorithms are illustrated using real examples.
1. Introduction
Count regression must accommodate overdispersion, while conventional Bayesian inference has been hindered by inefficient posterior computation. The paper proposes a mixed negative-binomial model with closed-form Bayesian methods based on two data-augmentation approaches.
1. Introduction
- The NB model accommodates overdispersed counts: its variance exceeds its mean, unlike the Poisson model’s equal mean and variance.The NB distribution arises by mixing a Poisson rate with a gamma distribution.Source: §1 · p. 1
- Maximum-likelihood estimation provides point estimates rather than prior-informed uncertainty, and estimates of NB dispersion r can be biased or fail to converge in some settings.The cited problem settings include small samples, small means, or large r.Source: §1 · p. 1
- Bayesian count analysis has been limited by inefficient posterior computation, including gamma-function approximation for r and computationally intensive Metropolis-Hastings for regression coefficients.These methods also make it difficult to incorporate prior information such as sparsity in β.Source: §1 · p. 2
- The proposed model combines lognormal and gamma random effects in negative-binomial regression and supports efficient Gibbs sampling and variational Bayes inference.The gamma prior on dispersion r is conjugate under a compound Poisson representation; a Polya-Gamma augmentation connects a lognormal prior to the logit of p for inference on β and σ^2.Source: §1 · p. 2
2. Regression Models for Counts
Count regression models use random effects to accommodate overdispersion, which violates the Poisson model’s equal-dispersion assumption. The section describes NB and lognormal-Poisson alternatives, including a trade-off in tractability for the latter.
2. Regression Models for Counts
- Poisson regression assumes equal conditional mean and variance, while a nonnegative multiplicative random effect yields variance at least as large as the mean.The random effect models individual heterogeneity; gamma and lognormal distributions are possible priors for it.Source: §2 · p. 2
2.1. The Negative Binomial Regression Model
- The NB regression model places a gamma prior on the multiplicative random effect, producing a negative-binomial count model parameterized by mean μ_i and inverse dispersion φ.The random effect has mean 1 and variance r^-1; maximum-likelihood estimates of β and φ can be found numerically with Newton-Raphson.Source: §2.1 · p. 2
2.2. The Lognormal-Poisson Regression Model
- The lognormal-Poisson model places a lognormal prior on the multiplicative random effect, but marginalizing that effect yields no analytical distribution for the count.Consequently, its maximum-likelihood estimation is less straightforward, and the model is less commonly used.Source: §2.2 · p. 2
3. The Lognormal and Gamma Mixed Negative Binomial Regression Model
The LGNB model combines a lognormal multiplicative random effect with a gamma prior on the negative-binomial dispersion parameter. Its two free parameters let it incorporate both kinds of random effects, while its logit link supports Bayesian inference for regression coefficients.
3. The Lognormal and Gamma Mixed Negative Binomial Regression Model
- LGNB places a lognormal prior on the multiplicative random effect and a gamma prior on the NB dispersion parameter r.The model connects the regression coefficients to the NB probability parameter through a logit link.Source: §3 · p. 3
- Marginalizing the coefficient-specific prior yields a Student-t prior on β_p, which promotes sparsity in the regression coefficients.The model uses a logit link between β and p to derive efficient Bayesian inference.Source: §3 · p. 3
3.1. Model Properties and Model Comparison
- The quasi-dispersion κ is the coefficient of the mean-quadratic term in the variance.The model’s variance is derived using total expectation, total variance, and the moments of the negative-binomial distribution.Source: §3.1 · p. 3
- With two free parameters, LGNB can reduce to the NB model when σ² = 0 or the lognormal-Poisson model when φ = r^-1 = 0.The extra degree of freedom lets the model incorporate both kinds of random effects, with their proportions inferred automatically.Source: §3.1 · p. 3
4. Default Bayesian Analysis Using Data Augmentation
The paper develops closed-form Bayesian inference for the LGNB model through two data-augmentation approaches. A compound-Poisson representation supports inference for dispersion r, while Polya-Gamma augmentation supports inference for regression coefficients; Gibbs sampling and variational Bayes updates are provided.
4. Default Bayesian Analysis Using Data Augmentation
- Two data-augmentation approaches yield closed-form updates for Gibbs sampling and variational Bayes inference: compound Poisson for r and Polya-Gamma for β.The methods are presented as default Bayesian analysis for the LGNB model.Source: §4 · p. 3
4.1. Inferring the Dispersion Parameter Under the Compound Poisson Representation
- Under the compound-Poisson representation, the gamma prior is conjugate for the NB dispersion parameter r.This representation writes the NB count as a random sum of logarithmic-distribution variables.Source: §4.1 · p. 4
4.1.1. Gibbs Sampling for r
- For Gibbs sampling of r, the latent count L_i is sampled conditionally on y_i using the probabilities Pr(L_i = j | −) = R_r(y_i, j).When y_i = 0, L_i is identically zero; the values of F used in the calculation can be iteratively computed, with each row summing to one.Source: §4.1.1 · p. 4
4.1.2. Variational Bayes Inference for r
- The variational-Bayes inference for r uses equations (29)–(30), with its posterior mean given by ⟨r⟩ = ˜a/˜h.The supplied passage identifies this as the VB inference for the NB dispersion parameter.Source: §4.1.2 · p. 4
4.2. Inferring the Regression Coefficients Using the Polya-Gamma Distribution
- Polya-Gamma augmentation provides conditional posteriors for ψ and ω_i given the other variables and the specified priors.The passage identifies ω_i as Polya-Gamma distributed and gives the conditional posterior setup for both variables.Source: §4.2 · pp. 4–5
4.3. Gibbs Sampling Inference
- Gibbs sampling draws latent L_i and r, then samples ω_i from PG(y_i + r, ψ_i) and ψ and β from normal conditional posteriors.The passage also gives the conditional normal parameters in terms of the design matrix, latent variables, and prior precisions.Source: §4.3 · p. 5
4.4. Variational Bayes Inference
- The variational-Bayes approximation factorizes across ψ, β, r, h, ϕ, α_p, and the latent L_i and ω_i variables.The approximation assigns normal, gamma, and Polya-Gamma factors; Monte Carlo integration is used to calculate ⟨ln(1 + e^ψ_i)⟩.Source: §4.4 · p. 5
5. Example Results
The univariate example estimates the NB dispersion parameter and contrasts Bayesian posterior inference with point estimators. Across two count-regression examples, LGNB improves Pearson-residual fit over Poisson and NB, while Bayesian inference also exposes posterior uncertainty.
5.1. Univariate Count Data Analysis
- Gibbs samples showed rapidly decreasing autocorrelation, and the VB lower bound converged quickly even from an initialization with r twice its converged value.After convergence, lower-bound variations are expected because Monte Carlo integration estimates non-analytical expectation terms.Source: §5.1 · p. 5, Figure 1 · p. 6
- Gibbs and VB posterior means for r were 1.0812 and 0.9988, versus MME 1.1667, MLE 1.0246, and MQLE 0.99472.Unlike the comparison estimators’ point estimates, the Bayesian algorithm provides a full posterior distribution and accommodates prior information.Source: §5.1 · pp. 5–6
5.2. Regression Analysis of Counts
- Table 1 reports NASCAR estimates for σ2, r, κ, and β across Poisson, NB, and LGNB models, using MLE, VB, or Gibbs estimation.The LGNB intercept differs considerably from those in Poisson and NB, while β0 + σ2/2 + ln r plays a similar role to their intercepts.Source: Table 1 · p. 6, §5.2 · pp. 6–7
- LGNB clearly outperforms Poisson and NB on Pearson-residual goodness of fit in the NASCAR and MotorIns examples.The comparison evaluates extra-Poisson variation modeled with both gamma and lognormal distributions.Source: Table 2 · p. 6, §5.2 · pp. 6–7
- Bayesian inference provides posterior distributions for σ2, r, and β; VB tends to concentrate near the mode, while Gibbs explores wider tails.The stated explanation is VB’s product-of-independent-Q-functions approximation, compared with Gibbs sampling’s use of conditional independence.Source: §5.2 · p. 7
- For NASCAR, Laps and track length coefficients were highly positively correlated, while their covariates had correlation −0.9006.The estimated coefficient correlation matrix provides information typically unavailable from MLE.Source: §5.2 · p. 7
6. Conclusions
The conclusion presents LGNB as a model for overdispersed count regression with closed-form Bayesian inference. It identifies multivariate count regression and a log-Gaussian NB process as possible extensions, with applications under investigation.
6. Conclusions
- The proposed LGNB model targets regression analysis of overdispersed counts.Its closed-form Gibbs and VB inference uses compound Poisson and Polya-Gamma data augmentation.Source: §6 · p. 7
- The framework may be modified for multivariate count regression, potentially retaining closed-form Gibbs sampling and VB inference.The conclusion frames this as a possibility, not an established result.Source: §6 · p. 7
- Linking a log Gaussian process to the NB probability logit could yield a log Gaussian NB process with tractable closed-form Bayesian inference; applying the framework to these problems remains under active investigation.Source: §6 · pp. 7–8