Source-linked AI summary
Parsimoniously Fitting Large Multivariate Random Effects in glmmTMB
Maeve McGillycuddy, Gordana Popovic, Benjamin M. Bolker, David I. Warton
TL;DR
Large unstructured covariance matrices make high-dimensional multivariate random effects difficult to estimate because their parameter count grows quadratically. The paper implements reduced-rank factor-analytic random effects in glmmTMB, enabling more parsimonious fitting and applications to ecological abundance and random-slopes data.
Problem
Unstructured covariance matrices require q(q + 1)/2 parameters, creating estimation difficulties for large multivariate random effects.
Method
The paper adds glmmTMB functionality that represents multivariate random effects as linear combinations of d < q latent variables.
Results
The approach enabled practical fitting in examples with random-effect dimensions 9 and 15 that were already too large to estimate practically without reduced rank.
Takeaways & Limitations
Reduced-rank models broaden glmmTMB to include more parsimonious large-dimensional multivariate random effects and generalized latent variable models.
Takeaways & Limitations
For larger data sets, especially when species outnumber samples, fitting may require multiple parameter starts and selecting the fit with the highest log-likelihood.
Abstract
from arXiv · showhide
Multivariate random effects with unstructured variance-covariance matrices of large dimensions, $q$, can be a major challenge to estimate. In this paper, we introduce a new implementation of a reduced-rank approach to fit large dimensional multivariate random effects by writing them as a linear combination of $d < q$ latent variables. By adding reduced-rank functionality to the package glmmTMB, we enhance the mixed models available to include random effects of dimensions that were previously not possible. We apply the reduced-rank random effect to two examples, estimating a generalized latent variable model for multivariate abundance data and a random-slopes model.
1 Introduction
Large multivariate random effects require covariance structures whose parameter count grows quadratically, making estimation difficult as dimension increases. The paper addresses this with reduced-rank latent-variable representations implemented in glmmTMB.
- Correlated multivariate random effects are needed for random-slopes models and for inducing correlation across responses in multivariate data.
- q(q + 1)/2 parameters are required for an unstructured q × q covariance matrix, so estimation quickly becomes challenging as q grows.
- Three or more species can produce convergence issues with the conventional approach, although two species yield reasonable fits and matching glmer and glmmTMB results.
- Reduced-rank models express multivariate random effects as linear combinations of d < q latent variables, reducing the problem dimension.
- The new glmmTMB functionality extends mixed models to random effects with dimensions in the hundreds or potentially thousands, including generalized latent variable models.
2 Methods
The methods formulate high-dimensional multivariate random effects through factor-analytic reduced-rank structures and estimate them using Laplace-approximated marginal likelihood. glmmTMB exposes this approach through the rr interface, with rank selection and data-driven initialization supporting model fitting.
- 2.1 Models: A GLMM models responses from the exponential family conditional on random effects, with an unstructured covariance matrix requiring q(q + 1)/2 parameters.
- 2.1 Models: Reduced-rank random effects use d independent standard-normal latent variables and a q × d factor-loading matrix Λ.
- 2.1 Models: dq − d(d − 1)/2 covariance parameters are required, so the parameter count increases linearly with q when d is fixed.
- 2.2 Estimation: The latent variables are integrated out to obtain a marginal likelihood, which is generally approximated for non-normal responses.
- 2.2 Estimation: glmmTMB maximizes a Laplace approximation using TMB automatic differentiation, while the rr formula specifies the covariance rank.
- 2.3 Software interface: The rank defaults to d = 2 and can be selected using cross-validation or information criteria; a data-driven initializer helps address multimodal likelihoods.
3 Application
The applications demonstrate reduced-rank random effects for multivariate fish abundance and country-varying school effects, enabling models that would otherwise face convergence or parameter-count problems.
- 3.1 Wind farm data: The wind farm analysis models cross-species abundance correlations using a multivariate random effect in a before-after-control-impact design.Fish abundance was measured in 2003 and 2010 across the wind farm and two reference zones.
- 3.1 Wind farm data: Nine species make an unstructured covariance difficult: estimating 45 variance parameters produces convergence problems, whereas the rank-two reduced-rank model yields interpretable residual correlations.The correlations are conditional on covariates and other random effects, and are measured on the log-scale linear predictor.
- 3.1 Wind farm data: The wind farm results show a positive Oxsimpa North–Wind Farm contrast in 2010, but no corresponding South–Wind Farm effect.The North contrast is interpreted as likely reflecting increased Oxsimpa abundance in the North Zone rather than a wind-farm effect.
- 3.1 Wind farm data: Ordination separates samples by zone and year before adjustment, while controlling for Zone, Year, their interaction, and Station removes those clustering patterns.Species positions also represent correlation: negatively correlated species lie on opposite sides, while positively correlated species are neighbours.
- 3.2 PIRLS data: The PIRLS model uses rank-three country-varying random intercepts and slopes for the joint Size_lib and Eco_disad effects, requiring 42 rather than 136 covariance parameters.AIC selected rank three; parameter initialization is required to avoid convergence issues.
- 3.2 PIRLS data: Bulgaria generally has higher literacy scores than Georgia, whereas Georgia shows little disadvantage effect and library-size patterns opposite to those expected.In Georgia, larger libraries appear beneficial in economically advantaged schools but make little difference in strongly disadvantaged schools.
4 Discussion
The paper adds reduced-rank random effects to glmmTMB, enabling parsimonious modeling of large multivariate random effects and generalized latent variable models in complex designs. Applications show practical use, while rank selection and larger datasets remain important considerations.
- The rr structure writes a large multivariate random effect as a linear combination of d latent variables, providing a more parsimonious structure for large dimensions.
- The factor analytic term extends glmmTMB so generalized latent variable models can be fitted to complex study designs through its familiar interface.
- The two applications used random-effect dimensions of 9 and 15, which were already too large to estimate practically without reduced-rank models.
- The approach contributes another model-simplification option alongside diagonal, compound-symmetry, and other structured covariance assumptions.
- Rank selection depends on the analysis goal: d = 2 supported wind-farm correlation visualization, whereas information criteria guided the PIRLS analysis.
- Fixed-effect estimates and confidence intervals were robust to different ranks, while qualitatively important interpretive changes occurred only for d < 2 in the authors' experience.
- Reduced-rank models assume singularity, circumventing warnings that other correlated-random-effect methods can produce near singularity.
- Potential applications include multilevel factor analysis, repeated measures, and genotype-by-environment interaction analysis.
A Model summaries for the wind farm example
The wind-farm model summaries show comparable Poisson mixed-model specifications fitted with lme4 and glmmTMB. The displayed output includes variance, standard deviation, correlation, and grouping information for the random effects.
- The lme4 wind-farm model uses a Poisson family with a log link and the formula abundance ~ Zone + Year + (Species + 0 | ID).
- The glmmTMB summary uses the same Poisson log-link formula for abundance, Zone, Year, Species, and ID.
- The lme4 summary reports 284 observations grouped by ID across 142 groups.
- The reported random-effect variances are 0.7603 for SpeciesTorsk and 0.9839 for SpeciesTanglake.
- The reported standard deviations are 0.8719 for SpeciesTorsk and 0.9919 for SpeciesTanglake, with correlation -0.74.
B Parametric bootstrap analysis for the wind farm example
The wind-farm interaction terms are tested with a parametric bootstrap by comparing an observed likelihood-ratio statistic with a null-model simulation distribution.
- The parametric bootstrap estimates the P-value by comparing the observed likelihood-ratio statistic with the simulated null distribution of the test statistic.
- Bootstrap replications that fail to converge are ignored when estimating the test result.
- The bootstrap procedure fits null and alternative models and computes their likelihood-ratio difference for each simulated dataset.
C Sensitivity analysis
Sensitivity analyses compare fixed-effect estimates and 95% confidence intervals across reduced-rank choices. Wind-farm estimates remain similar from d = 0 to 4, while PIRLS estimates are less similar for d = 0.
- Wind-farm fixed-effect estimates and 95% confidence intervals are similar as rank d varies from zero to four.
- PIRLS fixed-effect estimates and 95% confidence intervals are similar for ranks d = 1 through 4.
- With a random intercept replacing the reduced-rank effect at d = 0, PIRLS fixed-effect estimates are less similar and standard errors may be smaller.
D Reduced-rank random effect estimates from the PIRLS model
Figure 7 presents conditional estimates of school variables by country from the reduced-rank random effect in the PIRLS model.
- Conditional estimates summarize school variables for countries in the PIRLS model.
- The estimates are derived from the model’s reduced-rank random effect.
- The figure organizes these estimates by country for comparison.