Source-linked AI summary
Multiple imputation of multilevel missing data: An introduction to the R package pan
Simon Grund, Oliver Lüdtke, Alexander Robitzsch
TL;DR
Multilevel missing-data treatment remains challenging, while deletion, single-level MI, and ad hoc methods for clustered data continue to prevail. The paper presents pan as a tool that preserves multilevel-data features and examines the consequences of ignoring that structure.
Problem
Multilevel missing-data treatment is challenging, and deletion, single-level MI, and ad hoc methods for clustered data remain prevalent.
Method
The paper uses pan to treat multilevel missing data while preserving special features of multilevel data through posterior-predictive draws.
Results
Ignoring the multilevel structure in the imputation model can distort parameter estimates in subsequent multilevel analyses.
Takeaways & Limitations
pan is presented as a powerful tool for treating multilevel missing data because it preserves features of typical research data.
Takeaways & Limitations
Many interesting features of multilevel models may be distorted or lost when the imputation model ignores the multilevel structure, and multilevel MI remains difficult when missing data occur.
Abstract
from arXiv · showhide
The treatment of missing data can be difficult in multilevel research because state-of-the-art procedures such as multiple imputation (MI) may require advanced statistical knowledge or a high degree of familiarity with certain statistical software. In the missing data literature, pan has been recommended for MI of multilevel data. In this article, we provide an introduction to MI of multilevel missing data using the R package pan, and we discuss its possibilities and limitations in accommodating typical questions in multilevel research. In order to make pan more accessible to applied researchers, we make use of the mitml package, which provides a user-friendly interface to the pan package and several tools for managing and analyzing multiply imputed data sets. We illustrate the use of pan and mitml with two empirical examples that represent common applications of multilevel models, and we discuss how these procedures may be used in conjunction with other software.
Multilevel Modeling: An Empirical Example
Multilevel models represent persons nested within groups and can model both group-level differences and effects that vary across groups. The empirical example uses German PIRLS data to illustrate random-intercept and random-slope models with planned and unplanned missingness.
- Multilevel models account for dependency among persons nested within groups and allow predictor effects to vary across groups.
- Approximately 99.3% of students had available reading-achievement and cognitive-ability scores, while disciplinary-problem ratings in mathematics were missing for half the sample by design.
- Example 1: Random-Intercept Model: The first model estimates between- and within-group effects of disciplinary problems and SES on mathematics achievement using group-mean centering.
- Example 1: Random-Intercept Model: The random-intercept model separates class-level residual variation from student-level residual variation.
- Example 2: Random-Slope Model: The second model allows the association between cognitive ability and mathematics achievement to vary across classes through a random slope.
Multiple Imputation of Incomplete Multilevel Data
Multiple imputation replaces missing values across completed data sets and combines analyses, but multilevel applications require imputation models that preserve the analysis structure. The article contrasts appropriate multilevel modeling with ad hoc single-level strategies that can distort estimates and standard errors.
- Listwise deletion can reduce power and produce biased results, motivating multiple imputation for incomplete data.
- Multiple imputation draws replacements from the missing-data distribution conditional on observed data, analyzes each completed data set, and combines results.
- The article assumes MCAR for some planned missingness and MAR for remaining missingness conditional on observed auxiliary variables.
- An imputation model should be at least as complex as the analysis model, including interactions, clustered structure, and random slopes when relevant.
- Single-level dummy-indicator methods can distort parameter estimates and standard errors in multilevel analyses.
- pan follows the joint-modeling approach, imputing incomplete variables simultaneously with one statistical model.
The Multivariate Linear Mixed-Effects Model
pan uses a multivariate linear mixed-effects model that represents multiple dependent variables simultaneously while incorporating fixed effects, random effects, and residual covariance. Its imputation model fills incomplete responses conditional on completely observed predictors.
- The pan model is a multivariate extension of regular multilevel models and represents multiple dependent variables simultaneously.
- The model expresses each response vector as fixed-effects contributions, random-effects contributions, and residuals: yij = xijβ + zijbj + eij.
- Random effects are assumed normally distributed across groups, while individual residuals are normally distributed across persons.
- The MLMM imputes left-hand-side variables conditional on right-hand-side variables containing fixed and random effects.
- Only response variables may contain missing values; predictor variables must be completely observed.
Multivariate empty model.
The multivariate empty model places all variables on the response side and uses only an intercept on the predictor side. It decomposes variances and covariances into between-group and within-group components.
- The multivariate empty model uses only an intercept on the predictor side while placing all variables of interest on the response side.
- For each response variable, the model separates variance and covariance into between-group Ψ and within-group Σ portions.
- The model is a multivariate version of the null or intercept-only multilevel model, with dependent variables decomposed across levels.
- Its covariance matrices allow relationships among dependent variables at both group and person levels.
- The empty model is useful for estimating individual- and group-level relationships, including settings with group-level predictors.
Full mixed-effects model.
The full mixed-effects model includes both fixed and random effects, allowing multilevel imputation to represent random intercepts and slopes when the analysis model requires them.
- Full mixed-effects model: The model estimates fixed and random effects for variables included as predictors in the multivariate linear mixed-effects model.Dependent variables may contain missing data, whereas right-hand-side predictors must be completely observed.
- Full mixed-effects model: The full mixed-effects model includes both fixed and random effects for the intercept and observed predictors.It can represent random intercepts and slopes in the imputation model.
- Full mixed-effects model: Variables are not decomposed in the full model, so fixed and random effects represent their overall effects on the dependent variables.Separate within- and between-group effects require decomposing the predictor into between- and within-group portions before MI.
- Full mixed-effects model: Random slopes in the analysis model should be represented in the imputation model through corresponding slope variances.This is why the full mixed-effects model is particularly useful for models with random slopes.
Software Alternatives
Several software packages provide alternatives for multiple imputation of multilevel data, including procedures for simpler two-level models, complex random-slope models, joint modeling, and fully conditional specification.
- Software Alternatives: Mplus implements a two-level procedure similar to pan’s empty model, denoted H1, and a more complex procedure, denoted H0.H0 is intended for models such as random-slope models.
- Software Alternatives: Joint modeling approaches for multilevel multiple imputation are available in SAS, REALCOM, and the R package jomo.
- Software Alternatives: The R package mice provides a fully conditional specification approach to multiple imputation.
- Software Alternatives: Some alternative packages can support analysis steps similar to those outlined for pan.The article indicates that this possibility is discussed later.
Example Applications with Multilevel Missing Data
The examples demonstrate how mitml exposes pan’s multilevel imputation workflow, including model specification, MCMC-based imputation, convergence checks, and analysis across multiple completed data sets.
- Example Applications with Multilevel Missing Data: mitml provides a convenient interface to pan and tools for managing and analyzing multiply imputed data sets.The examples also compare pan with Mplus, single-level MI, and listwise deletion.
- Example Applications with Multilevel Missing Data: The examples used lme4 to fit analysis models to the imputed data sets and Rubin’s rules to combine parameter estimates.mitml also provides functions for extracting completed data sets and performing the analysis workflow.
- Example Applications with Multilevel Missing Data: Multilevel missing-data analysis specifies an imputation model, generates multiple imputed data sets, analyzes each separately, and combines the estimates.The imputation model should include variables and auxiliary variables relevant to the analysis model.
- Example Applications with Multilevel Missing Data: pan uses MCMC to simulate parameters and missing-value replacements from the posterior predictive distribution.Burn-in precedes the imputation phase, and saved imputations are separated by iterations to reduce autocorrelation.
Intraclass correlations.
The examples show that multilevel MI affects estimated intraclass correlations and model effects, while supporting analyses of within-group, between-group, contextual, and varying-slope relationships.
- Intraclass correlations: Multilevel MI produced larger ICC estimates than single-level MI, especially for variables with substantial missing data.Listwise-deletion estimates were closer to multilevel MI without an obvious systematic pattern.
- Intraclass correlations: Higher SES was associated with higher math achievement, whereas disciplinary problems were associated with lower math achievement.Class-level estimates were roughly twice as large as student-level estimates.
- Intraclass correlations: Single-level MI yielded similar within-group estimates but consistently larger between-group estimates than multilevel MI.
- Intraclass correlations: Listwise deletion produced larger standard errors, especially at the student level, and smaller class-level effect estimates.
- Intraclass correlations: The contextual effect of SES was statistically significant at p < .001 (F = 15.297, df 1 = 1, df 2 = 1293.0).Classes with higher SES tended to have higher math achievement after controlling for student-level SES.
- Intraclass correlations: The latent-group-mean and manifest-group-mean analysis models led to essentially the same conclusions in the example.
Convergence diagnostics.
Convergence diagnostics indicated that the pan MCMC chain had become stationary, while autocorrelation persisted mainly for group-level variance components. All parameters showed negligible autocorrelation by lags of 15,000 to 20,000 iterations.
- The largest ˆR was 1.001, indicating that the MCMC chain had become stationary for all parameters.
- Diagnostic plots showed that parameters related to the variables of interest converged quickly and were not greatly affected by autocorrelation.
- Autocorrelation was quite persistent for some parameters, especially the group-level variance components.
- Autocorrelation vanished for all parameters with a lag of 15,000 to 20,000 iterations.
Model of interest.
The multilevel model examined whether student-level cognitive ability predicted mathematics achievement after controlling for SES, while allowing effects to vary across groups. A pooled likelihood-ratio test found statistically significant slope variance, indicating that the association differed by class.
- The model examined student-level variables and included predictors centered around their group means using within.
- Students with higher cognitive ability than their class average tended to score higher on the math achievement test after controlling for SES.
- The relation between cognitive ability and math achievement appeared to vary substantially across groups.
- Compared with single-level MI, the multilevel estimates differed slightly, whereas Mplus H0 imputation produced results similar to pan4.
- The pooled LRT was statistically significant at p = .006 (F = 5.119, df 1 = 2, df 2 = 10386.2), indicating that slope variance differed from zero.
- Students with different cognitive ability may differ more or less strongly in their math achievement scores depending on their class.
- The LRT approach may have low statistical power for testing variance components, and few alternatives are available for multiply imputed data.
Analyzing Imputations Generated by Alternative Software
mitml can analyze multiply imputed data sets regardless of their software origin when they are represented as a list of data sets in R. Its wrapper and conversion functions also provide access to convergence diagnostics, while other software may be preferable for categorical, ordinal, or group-level missing variables.
- Researchers may prefer jomo, mice, or Mplus when the analysis includes categorical variables with missing data.
- The analysis steps presented can be carried out on multiply imputed data sets irrespective of their origin.
- mitml’s analysis functions require multiply imputed data sets to be represented as a list of data sets in R.
- mitml includes wrapper functions for pan and jomo, plus conversion functions for data sets generated by mice.
- For other software packages, imputed data must be converted manually, using functions such as long2mitml.list or as.mitml.list.
- Using wrapper functions is recommended because they allow access to mitml’s convergence diagnostics.
Discussion
The discussion presents pan and mitml as accessible tools for multilevel missing-data analysis, while emphasizing that simpler methods can distort estimates and that important challenges remain for complex models and multiply imputed-data analysis.
- Motivation: Multilevel missing-data methods remain underused in applied research, with listwise deletion, single-level MI, and ad hoc approaches still prevalent.The paper attributes this partly to the limited accessibility of state-of-the-art procedures and software.
- Empirical illustrations: The two examples illustrate pan and mitml for random-intercept models, between- and within-group effects, random slopes, and slope-variance estimation and testing.Example 1 focuses on random-intercept models; Example 2 addresses random slopes.
- Practical contribution: pan preserves multilevel features, uses available and auxiliary information, and supports reliable estimation without altering the model of interest.The discussion contrasts these capabilities with simpler methods that may distort or lose multilevel-model features.
- Empirical illustrations: Multilevel MI using pan produced substantially different parameter estimates from single-level MI, while listwise-deletion estimates were sometimes closer to the multilevel results.These findings illustrate that ignoring clustering can distort estimates in subsequent multilevel analyses.
- Open challenges: Important limitations remain for missing predictors in models with random slopes or interactions, and for estimating multilevel-model goodness-of-fit measures after MI.The paper notes that clear recommendations are still lacking and that some advanced methods are not available in standard software.
- Practical contribution: mitml provides tools for combining parameter estimates, model comparisons, and model constraints across multiply imputed data sets, including Rubin’s rules and D1–D3 procedures.The package addresses the challenge of analyzing multiply imputed data when software lacks tools for combining results.