Source-linked AI summary
A principal components method to impute missing values for mixed data
Vincent Audigier, François Husson, Julie Josse
TL;DR
Mixed-data imputation has relatively few established approaches, and simple dummy coding can impose unsuitable continuous-variable assumptions. This paper applies FAMD to iterative imputation and reports favorable comparisons with random forests, especially for categorical values and linear relationships.
Problem
Mixed-data imputation has less extensive prior literature, and applying continuous-data methods to dummy-coded categories is unsatisfactory when their assumptions do not hold for dummy variables.
Source: §1 · p. 2Method
The method iteratively imputes mixed data with FAMD, which balances continuous and categorical variables and uses dimensions capturing their links.
Source: §1 · p. 3, §2.1 · pp. 3–4Results
The method gives particularly good predictions for categorical values, rare categories, and linear relationships, with good imputation quality and computational time compared with random forests.
Source: Conclusion · p. 17Takeaways & Limitations
FAMD-based imputation can account simultaneously for similarities between individuals and links between continuous and categorical variables.
Source: Conclusion · p. 17As a single-imputation method, it does not account for prediction uncertainty, so analyses on the completed data can underestimate estimator variability.
Source: Conclusion · p. 18
Abstract
from arXiv · showhide
We propose a new method to impute missing values in mixed datasets. It is based on a principal components method, the factorial analysis for mixed data, which balances the influence of all the variables that are continuous and categorical in the construction of the dimensions of variability. Because the imputation uses the principal axes and components, the prediction of the missing values are based on the similarity between individuals and on the relationships between variables. The quality of the imputation is assessed through a simulation study and real datasets. The method is compared to a recent method (Stekhoven and B\"uhlmann, 2011) based on random forests and shows better performances especially for the imputation of categorical variables and when there are highly linear relationships between continuous variables.
1 Introduction
Imputation can make incomplete datasets usable, but predictions carry uncertainty, and methods for mixed continuous and categorical data remain limited. The paper introduces an FAMD-based method and compares it with random-forest imputation.
1 Introduction
- Imputation can yield a complete dataset for subsequent analysis, but uncertainty in predicted values means results require caution.Many statistical methods cannot be applied directly to incomplete datasets.Source: §1 · p. 1
- Existing categorical-data approaches include nonparametric methods and models such as log-linear models, whose parameter count can grow rapidly with the number of category levels.Latent-class and log-linear hierarchical models are also described as alternatives.Source: §1 · p. 2
- Mixed-data imputation remains less developed; coding categories as dummies and applying continuous-data methods is unsatisfactory because continuous-variable assumptions may not hold for dummy variables.Chained equations can be extended to mixed data by defining a model for each variable, but require many models.Source: §1 · p. 2
- Random-forest imputation was reported to outperform competing methods across many simulations and real datasets, serving as a reference for comparison.The cited method iteratively predicts missing values variable by variable until predictions stabilize.Source: §1 · pp. 2–3
- FAMD-based imputation is proposed for mixed data and assessed on simulations and real datasets against the method of Stekhoven and B¨uhlmann (2011).The comparison is presented as a test of the new method’s competitiveness.Source: §1 · p. 3
2 Imputation for mixed type-data thanks to FAMD
FAMD balances continuous and categorical variables when constructing dimensions of variability. Iterative FAMD adapts this analysis to incomplete mixed data by reconstructing and updating the data until imputed values stabilize.
2.1 FAMD in the complete case
- FAMD weights continuous and categorical variables so that each contributes equivalently to the dimensions of variability.Categorical variables are represented by indicator columns, while continuous variables are standardized before the weighted analysis.Source: §2.1 · pp. 3–4
- FAMD’s weighting gives rare categories greater inertia and greater influence on the dimensions of variability.The distance weighting also makes individuals taking different categories farther apart when one has a rare category.Source: §2.1 · p. 4
- The first principal component is the synthetic variable most linked to continuous variables by R2 and categorical variables by the squared correlation ratio η2.Subsequent components maximize the same criterion subject to orthogonality to earlier components.Source: §2.1 · p. 4
2.2 The iterative FAMD algorithm
- Iterative FAMD initializes missing entries, performs FAMD, reconstructs values using the first S dimensions, updates data-dependent weights and means, and repeats until the imputed matrix stabilizes.The method first encodes categorical variables with indicator columns; a missing categorical value becomes a row of missing entries in that encoding.Source: §2.2 · pp. 5–6
- Imputed categorical entries are real-valued membership degrees that sum to one for each individual and variable, and the most plausible category can be selected.The sum-to-one property follows from the initialization constraint and the specific weighting.Source: §2.2 · p. 6
- Alternating estimation and imputation can overfit, so the method regularizes iterative FAMD by thresholding singular values, shrinking smaller values more strongly.When noise is very high, the regularized algorithm imputes continuous variables with their means and categories with their proportions.Source: §2.2 · pp. 6–7
3 Properties of the imputation method
The section tests iterative FAMD against random forests across simulated mixed-data settings. It examines cross-type information, relationship structure, rare categories, and dimension choice.
3.1 Relationships between continuous and categorical variables
- Using both variable types lowers continuous-variable NRMSE and categorical-variable PFC across missingness rates.The improvement holds at each tested missingness rate, including 30%.Source: Figure 2 · p. 8, §3.1 · p. 9, Figure 3 · p. 9
3.2.1 Linear and nonlinear relationships
- With linear links, iterative FAMD has smaller NRMSE and PFC than RF; under nonlinear links, it trails RF, though differences remain reasonable.When nonlinear relationships are suspected, the authors suggest creating categorical variables by cutting continuous variables into categories.Source: Figure 4 · p. 10, §3.2.1 · pp. 10–11, Figure 5 · p. 11
3.2.2 Taking into account interactions between categorical variables
- Iterative FAMD performs poorly with interacting variables, while RF does better; adding a variable encoding the interaction can restore good imputation quality.The tested variables are pairwise independent despite having interactions.Source: §3.2.2 · p. 11, Figure 7 · p. 12
3.3 Imputation of rare categories
- FAMD recovers rare categories well, and its advantage over RF grows as category frequency falls.The evaluation recovers a rare category over 1,000 simulations, varying sample size and rare-category frequency.Source: Table 1 · p. 13, §3.3 · p. 13
3.4 Choice of the number of dimensions
- Cross-validation is used to select S, but q-category variables need at least q−1 dimensions and unrelated category dimensions may remain unrecoverable.Too few dimensions can discard relevant information; too many can treat noise as signal and destabilize imputations.Source: §3.4 · pp. 13–14
- At high SNR, categorical error minimizes at S=4 and continuous error at S=3; at low SNR, smaller S regularizes better, especially with more missingness.The simulations use a dataset with underlying S=4; smaller S can be appropriate when the data are very noisy.Source: Figure 8 · p. 14, §3.4 · pp. 14–15
4 Comparison on real datasets
Across the real datasets, iterative FAMD was slightly better overall and particularly effective for categorical variables, while random forests more often favored continuous-variable imputation.
4 Comparison on real datasets
- Real-data configurations used randomly added missing values at three rates, 200 simulations each, and cross-validation to choose reconstruction dimensions.The selected dimension was held fixed across each set of 200 simulations to save computational time.Source: §4 · p. 15
- Overall, iterative FAMD was slightly better and more efficient for categorical variables, with a 5% difference on Tips and BMI.Figure 9 reports NRMSE and PFC distributions across four mixed datasets and three missingness rates.Source: Figure 9 · p. 16, §4 · p. 16
- Random forests more often favored continuous-variable imputation; on Ozone and BMI, the error difference reached 5%.The reported comparison concerns continuous-variable errors.Source: Figure 9 · p. 16, §4 · p. 16
- On Parkinson, random forests had about 10% lower error, whereas Credit errors were lower with iterative FAMD at every missingness rate.Figure 10 contrasts the two methods on these continuous and categorical datasets, respectively.Source: Figure 10 · p. 17, §4 · p. 17
Conclusion
The method performs well against random-forest imputation, particularly for categorical variables and linear relationships, while using similarities between individuals and links between variables. Its practical scope is qualified by dimension-selection cost, dataset-specific effects of missingness, and the uncertainty limitations of single imputation.
Conclusion
- The method gives particularly good categorical imputations, including for rare categories, and performs well against random forests in imputation quality and computational time.Its predictions are especially good when variables have linear relationships.Source: Conclusion · p. 17
- Iterative FAMD imputes mixed data using similarities between individuals and links between continuous and categorical variables.The algorithm requires choosing the number of dimensions used to reconstruct the data; cross-validation can select it but is time-consuming.Source: Conclusion · p. 17
- Imputation quality deteriorates as missingness increases, but the effect depends on dataset structure and no universal acceptable missing-data percentage can be specified.When variables are unlinked, even one missing value can be problematic; strong links can make little observed data per individual sufficient.Source: Conclusion · p. 18
- As a single-imputation method, FAMD does not account for prediction uncertainty, so analyses of the completed data can underestimate estimator variability.Multiple imputation is described as a way to reflect missing-value uncertainty in the variability of estimates.Source: Conclusion · p. 18
- Iterative FAMD could be a first step toward multiple imputation for mixed data, which could provide confidence intervals for imputed values and support analyses from incomplete mixed datasets.The implementation returns imputed mixed data and a table combining imputed continuous variables with the imputed indicator matrix.Source: Conclusion · p. 18