Source-linked AI summary
MisGAN: Learning from Incomplete Data with Generative Adversarial Networks
Steven Cheng-Xian Li, Bo Jiang, Benjamin Marlin
TL;DR
GANs typically require fully observed training data, whereas many real datasets contain arbitrary missing entries. MisGAN jointly learns a complete-data generator and a mask generator, and experiments show effective distribution learning and high-quality imputation under MCAR. Theoretical analysis characterizes recoverability, while broader missingness mechanisms remain a future direction.
Problem
GANs normally require fully observed training data, but fully observed datasets may be unavailable and arbitrary missingness challenges existing machine learning models.
Method
MisGAN learns a complete-data generator together with a mask generator and trains generated data after masking them like real incomplete observations.
Results
MisGAN effectively learns complex, high-dimensional data distributions from highly incomplete data and generates high-quality imputations when the generator incorporates suitable priors.
Takeaways & Limitations
The framework provides a flexible way to use standard GAN data generators with incomplete observations and can be extended to model MAR and NMAR mechanisms.
Takeaways & Limitations
The paper focuses on MCAR, and formal learnability analysis for MAR and NMAR remains unresolved because the presented analysis no longer holds under dependence between the transition matrix and data distribution.
Abstract
from arXiv · showhide
Generative adversarial networks (GANs) have been shown to provide an effective way to model complex distributions and have obtained impressive results on various challenging tasks. However, typical GANs require fully-observed data during training. In this paper, we present a GAN-based framework for learning from complex, high-dimensional incomplete data. The proposed framework learns a complete data generator along with a mask generator that models the missing data distribution. We further demonstrate how to impute missing data by equipping our framework with an adversarially trained imputer. We evaluate the proposed framework using a series of experiments with several types of missing data processes under the missing completely at random assumption.
1 INTRODUCTION
MisGAN addresses the challenge of learning high-dimensional data distributions when GAN training data are incomplete and arbitrarily missing. It models missingness explicitly and provides empirical evidence for learning distributions and imputing missing values.
- GAN training normally requires fully observed data, but arbitrary missingness is prevalent and challenges existing machine learning models.
- Under MCAR, incomplete observations can be modeled through observed values and a binary mask indicating which entries are revealed.The generative process distinguishes observed and missing components of the complete data vector.
- Under MCAR or MAR, the observed-data likelihood factorizes into a data distribution and a missingness mechanism, enabling separate estimation of the data parameters.The cited passage notes that this decoupling permits the missingness mechanism to be ignored when learning the data-generating model.
- MisGAN introduces an auxiliary GAN to learn the mask distribution alongside a complete-data generator.The learned masks represent the missing-data process.
- The framework builds on AmbientGAN by treating missing data as a corruption process while additionally observing the missing pattern.AmbientGAN handles corrupted samples under known or parameterized measurement processes.
- Experiments provide evidence that MisGAN learns complex, high-dimensional distributions from highly incomplete data and generates high-quality imputations when suitable generator priors are used.
2 MISGAN: A GAN FOR MISSING DATA
MisGAN uses separate GANs for masks and complete data, then trains the data generator by comparing similarly masked generated and real incomplete samples. Its design preserves standard complete-data discriminator architectures while relaxing binary masks for gradient-based training.
- Incomplete examples are represented as partially observed vectors paired with binary masks, forming a dataset of independently sampled data-mask pairs.The mask indicates which entries are observed and which values should be ignored.
- The masking operator fτ fills missing entries with a constant value τ using the mask complement and element-wise multiplication.
- MisGAN explicitly models missingness with a mask generator and trains the complete-data generator on outputs masked like real incomplete data.The masked generated samples are compared adversarially with similarly masked real samples.
- The framework uses mask and data generator-discriminator pairs, with independent generators and noise distributions under the MCAR assumption.
- The data discriminator receives masked samples without needing missing-location indicators or knowledge of the masking constant, allowing standard complete-data architectures.A standard convolutional network can therefore serve as the data discriminator.
- Binary mask-generator outputs are relaxed to [0, 1] using a low-temperature sigmoid so gradient-based GAN training remains possible while encouraging near-binary masks.
3 THEORETICAL RESULTS
The theoretical analysis characterizes MisGAN’s identifiability under masking, showing that filled-in values do not affect recoverability while sparsity and missingness can determine uniqueness.
- The analysis asks whether the fill value τ and missing-location information affect recovery of the true data distribution.
- MisGAN with a fixed mask generator recovers the unknown data distribution by solving a non-negative linear system relating masked and unmasked distributions.The masking process is represented by a transition matrix Tq,τ, with masked samples following py = Tq,τp_x.
- All transition matrices using fill values τ ∈ P have the same null space, so uniqueness of recovery is independent of the chosen fill value.This yields the stated corollary that true-distribution recoverability does not depend on τ.
- Recovery is unique when the non-negative linear system has a unique solution, including under a sufficient sparsity condition on the true distribution.One stated condition uses indices s1 and s2 with p_x(s2) = 0 and opposite-sign null-space components v(s1) and v(s2).
- High missing rates enlarge the transition matrix’s null space, making non-unique recovery more likely even when natural-image distributions are sparse.Application-specific priors, such as convolutional architectures or low-dimensional latent codes, can regularize ill-posed problems.
- Knowing the locations of missing values cannot restore unique recovery when the associated linear system already has multiple non-negative solutions.The identifiability limitation is independent of the algorithm used to solve the system.
4 MISSING DATA IMPUTATION
MisGAN extends its incomplete-data framework with an adversarially trained imputer that estimates p(x_mis|x_obs) while preserving observed entries. The imputer and complete-data generator can be trained jointly or separated when the generator is pretrained.
- Imputer objective: The imputer Gi targets p(x_mis|x_obs) using the incomplete example and random noise, while keeping the observed part of x unchanged.Its corresponding discriminator Di supports adversarial training of the imputation model.
- Joint training: MisGAN jointly learns the data-generating process and imputer with an additional imputer loss alongside the framework’s existing objectives.With β = 0.1, the complete-data generator is encouraged to match imputed real data as well as masked real data.
- Alternative training: A stand-alone imputer can be trained using only its imputation objective when a pretrained complete data generator Gx is available.The imputer can also target a different missing distribution pm without access to the original incomplete training data.
- Imputer construction: Gi combines the observed entries with imputed values produced from a masked input and noise, ensuring the output retains the original observations.The masking outside bGi preserves observed values, while masking its input scales injected noise with the number of missing dimensions.
- Training configurations: The architecture supports either joint training of Gx and Gi with all components or independent imputer training after Gx has been pretrained.The latter configuration removes the faded components shown in Figure 2.
5 EXPERIMENTS
Experiments evaluate MisGAN across datasets, missingness patterns, architectures, and imputation settings. MisGAN generally outperforms ConvAC and competing imputers, while explicit mask modeling and convolutional structure improve robustness in difficult or ill-posed cases.
- Experimental setup: Experiments use MNIST, CIFAR-10, and CelebA under square observation, independent dropout, and variable-size rectangular observation processes.Evaluation includes missing rates from 10% to 90% and uses FID for generative-model quality.
- Generative samples: Conv-MisGAN produces visually better samples than FC-MisGAN, while independent dropout yields worse samples than square observation at the same missing rate.The study compares convolutional and fully connected architectures under multiple missing-data processes.
- Quantitative evaluation: MisGAN generally outperforms ConvAC, and FC-MisGAN degrades significantly relative to Conv-MisGAN as independent-dropout missing rates increase.High-rate dropout reduces overlapping pixel co-occurrences, weakening the signal for learning overall structure.
- Ill-posed missingness: Under non-overlapping square quadrants, FC-MisGAN produces boundary discontinuities, whereas convolutional structure makes the artifact less severe.The setting is ill-posed because correlations between pixels in different quadrants cannot be uniquely determined without additional assumptions.
- Ablation study: The mask discriminator prevents degenerate AmbientGAN solutions that reproduce masked outputs while learning an incorrect mask distribution.AmbientGAN can learn masks with all ones or otherwise interchange the roles of data and masks when only masked outputs are considered.
- Missing-data imputation: MisGAN imputation produces varied completions for the same incomplete input and remains more stable than GAIN across the evaluated scenarios.The imputer uses random noise to generate different results, while training without the mask generator and discriminator fails most of the time.
6 DISCUSSION AND FUTURE WORK
The work presents MisGAN for learning GAN data generators with missing data, focusing on MCAR and outlining extensions to MAR and NMAR. Formal learnability analysis and evaluation of the modified framework remain future work.
- MisGAN learns standard GAN data generators in the presence of missing data.
- The paper focuses on MCAR, while passing generator outputs to the mask generator could extend the framework to MAR and NMAR.
- Learnability under MAR and NMAR requires further investigation because the Section 3 analysis no longer applies when transition matrices depend on the data distribution.
- Experiments with the modified architecture showed results similar to original MisGAN, but formal evaluation is left for future work.
A PROOF OF THEOREM 1 AND THEOREM 2
The appendix proves that transition distributions encode the relevant observed-data marginals and that these marginals determine the transition output for every fill value. This establishes the theorem's invariance and supports MisGAN's recovery of p(xobs, m).
- The feature space is I = P^n, masks belong to M = {0, 1}^n, and Sq denotes the support of the mask distribution q.
- For each mask, [v]m groups feature vectors sharing the same observed entries under entry-wise multiplication.
- Proposition 1 states that the marginals {x([v]m)} determine Tq,τx for every τ ∈ P.
- Proposition 2 proves the converse: Tq,τx determines the collection of marginals through induction on the number of masks consistent with each v.
- Because the marginals are independent of τ, equality of Tq,τ0 outputs for one τ0 implies equality for all τ, yielding Theorem 1.
- MisGAN therefore learns p(xobs, m), since x([v]m) corresponds to p(xobs|m) and Tq,τx represents the filled-sample distribution under q = p(m).
B PROOF OF COROLLARY 2
The appendix augments the feature space with a special missingness symbol so that missing entries can be identified, converting missing-data generation into an augmented linear-system problem.
- Adding ψ to the feature-value set lets the masking operator mark missing entries explicitly.
- The corresponding transition matrix T′q,ψ turns the generative model for missing data into solving an augmented linear system.
- The true distribution is represented on the original feature space and assigned zero mass to states containing ψ.
- If the original system is non-unique, the augmented non-negative system with the extra constraint is also non-unique.
C EVALUATION OF IMPUTATION USING ROOT MEAN SQUARE ERROR
The appendix argues that RMSE can misrepresent imputation quality for multimodal conditional distributions, motivating FID as a collective distribution-based metric. On MNIST with block missingness, the metrics often rank methods differently.
- RMSE compares imputed missing values with their ground truth, but a ground-truth sample need not be the most probable conditional completion.
- When p(xmis|xobs) is multimodal, plausible samples from other modes can receive large RMSE despite having high conditional density.
- Figure 11 compares MisGAN, GAIN, and matrix factorization using observed pixels inside red boxes and generated pixels outside them.
- FID compares completed-data and fully-observed-data distributions collectively to assess distributional closeness.
- On MNIST with block observation missingness, FID and RMSE rankings are inconsistent across most assessed missing rates.
- At 90% missingness, MisGAN performs worse than GAIN and matrix factorization on RMSE but significantly better on FID.
G RESULTS OF CONVAC
ConvAC was trained with square observation missing patterns on MNIST, and Figure 18 presents the generated samples.
- Figure 18 presents samples generated by ConvAC trained with square observation missing patterns on MNIST.
H MISSING DATA IMPUTATION WITH GAIN
The section compares GAIN and MisGAN imputation under square observation missingness and presents results across CIFAR-10 and CelebA. GAIN’s imputation behavior can deteriorate with over-training, so the best-FID checkpoint is used for comparison.
- For square observation comparisons, GAIN models are selected using the best FID during training; CIFAR-10 uses the 500th epoch.
- GAIN imputation results are shown for both CIFAR-10 and CelebA.
- The experiments include MisGAN results for CIFAR-10 and CelebA under block observation and independent dropout missingness.
- MisGAN imputation results are presented separately for CIFAR-10 and CelebA.
- Over-trained GAIN gradually becomes similar to constant imputation under 20×20 square observation missingness.