PaperlayerView exact revision on arXiv

Source-linked AI summary

VIGAN: Missing View Imputation with Generative Adversarial Networks

Chao Shang, Aaron Palmer, Jiangwen Sun, Ko-Shin Chen, Jin Lu, Jinbo Bi

arXiv:1708.06724v5cs.CVstat.ML

TL;DR

When samples lack an entire view, existing imputation methods may have no information from that view to use, limiting missing-view analysis. VIGAN combines unpaired-data domain mapping with paired-data view correspondences through a jointly optimized GAN and denoising autoencoder. Experiments on MNIST and two healthcare datasets report effective missing-view recovery, with VIGAN performing best among the compared image-imputation methods.

  • Problem

    Some samples lack an entire view, a setting that classic imputation methods handle poorly when that view provides no information for the sample.

    Source: §I · p. 1

  • Method

    VIGAN learns domain mappings from unpaired data with a cycle-consistent GAN and view correspondences from paired data with a denoising autoencoder, then jointly optimizes both.

    Source: §I · p. 2, §III · p. 3, 1) Training procedure · p. 5

  • Results

    Across MNIST and two healthcare datasets, experiments indicate VIGAN effectively recovers missing views; it performed best among the compared methods in the image benchmark.

    Source: A. Image benchmark data · p. 7, §V · p. 9

  • Takeaways & Limitations

    The authors report that VIGAN integrates domain mappings and view correspondences to recover missing views across an image dataset and two healthcare datasets.

    Source: §V · p. 9

Abstract

from arXiv · show

In an era when big data are becoming the norm, there is less concern with the quantity but more with the quality and completeness of the data. In many disciplines, data are collected from heterogeneous sources, resulting in multi-view or multi-modal datasets. The missing data problem has been challenging to address in multi-view data analysis. Especially, when certain samples miss an entire view of data, it creates the missing view problem. Classic multiple imputations or matrix completion methods are hardly effective here when no information can be based on in the specific view to impute data for such samples. The commonly-used simple method of removing samples with a missing view can dramatically reduce sample size, thus diminishing the statistical power of a subsequent analysis. In this paper, we propose a novel approach for view imputation via generative adversarial networks (GANs), which we name by VIGAN. This approach first treats each view as a separate domain and identifies domain-to-domain mappings via a GAN using randomly-sampled data from each view, and then employs a multi-modal denoising autoencoder (DAE) to reconstruct the missing view from the GAN outputs based on paired data across the views. Then, by optimizing the GAN and DAE jointly, our model enables the knowledge integration for domain mappings and view correspondences to effectively recover the missing view. Empirical results on benchmark datasets validate the VIGAN approach by comparing against the state of the art. The evaluation of VIGAN in a genetic study of substance use disorders further proves the effectiveness and usability of this approach in life science.

I. INTRODUCTION

The missing-view problem arises when samples lack an entire view, a case that limits multiview analysis and is not addressed by methods assuming every view is available. VIGAN combines GAN-based domain mapping with an autoencoder to impute the absent view using paired and unpaired data.

I. INTRODUCTION

  • Multiview data pair records across views, whereas multimodal data may come from different samples and lack a one-to-one correspondence.The paper says VIGAN can handle missing data in both settings.Source: §I · p. 1
  • Some samples lack an entire view, while many existing multiview methods assume all views are available and only some variables are missing.The paper identifies this as a significant challenge for multiview analysis, particularly in large, heterogeneous datasets.Source: §I · pp. 1–2
  • Existing imputation approaches focus mainly on missing entries or use information from another complete view, leaving whole-view absences a challenge.The paper also notes that classic multiple-imputation and matrix-completion methods are hardly effective when there is no information in the specific view.Source: §I · pp. 1–2
  • VIGAN combines a multi-modal autoencoder trained on paired data with a cycle-consistent GAN trained on unpaired data, then jointly re-optimizes them to refine GAN estimates.The GAN supplies an initial estimate through domain translation; the autoencoder denoises that estimate using view correspondences.Source: §I · p. 2
  • The proposed approach is described as scalable, able to use paired and unpaired data together, and extendable to more than two views.The authors report empirical evaluation on synthetic and real-world datasets, with superior imputation performance and computational efficiency.Source: §I · p. 2

II. RELATED WORKS

The related work covers matrix completion, autoencoders, GANs, and unsupervised domain mapping. It highlights limitations of existing approaches for whole-view missingness and the different data settings their methods use.

A. Matrix Completion

  • Matrix completion targets missing entries under assumptions such as low rank, but block-structured missing views can violate randomness assumptions and prevent recovery guarantees.Some approaches can also be computationally prohibitive for large datasets; SVD-based methods may have O(N^3) complexity.Source: A. Matrix Completion · pp. 2–3

B. Autoencoder and RBM

  • Multimodal autoencoders learn cross-view relationships from paired data, but methods trained only on paired samples leave incomplete-view samples out of training.The cited work describes reconstructing a removed view from its counterpart.Source: B. Autoencoder and RBM · p. 3

C. Generative Adversarial Networks

  • GANs use a generator to produce samples and a discriminator to compare generated samples with real data in a minimax game.The paper describes GANs as having potential for data imputation.Source: C. Generative Adversarial Networks · p. 3

D. Unsupervised Domain Mapping

  • Unsupervised domain-mapping methods learn between modalities from unpaired data; CycleGAN translates images through a cycle-consistent mapping [17], while pix2pix uses paired data [21].Other approaches include DiscoGAN [18], DualGAN [19], CoGAN, and a domain transfer network [25].Source: D. Unsupervised Domain Mapping · p. 3

III. METHOD

VIGAN initializes cross-domain relations with a CycleGAN and between-view correspondences with a denoising autoencoder, then jointly optimizes both components. The notation identifies the mappings, discriminators, autoencoder, and projections used for paired views.

III. METHOD

  • VIGAN initializes a CycleGAN on unpaired data and a denoising autoencoder on paired data, then jointly optimizes them to denoise GAN outputs and reconstruct missing views.The autoencoder learns shared and private latent spaces for each view.Source: §III · p. 3

A. Notations

  • G1 and G2 map between X and Y, DX and DY discriminate for the respective generators, and A maps paired inputs from X × Y to X × Y.PX and PY project a pair onto its X or Y component, respectively.Source: A. Notations · p. 3

III. METHOD

  • The architecture contains CycleGAN generators G1 and G2, discriminators DX and DY, and a multi-modal denoising autoencoder.The CycleGAN learns cross-domain relations from unpaired data, while the autoencoder learns between-view correspondences from paired data.Source: §III · p. 3, Figure 2 · p. 4

B. The Proposed Formulation

VIGAN combines paired-data correspondences learned by a denoising autoencoder with unpaired-data domain mappings learned by a cycle-consistent GAN. Its joint objective uses the GAN outputs as noisy inputs for reconstructing paired views, while cycle consistency constrains domain mappings.

B. The Proposed Formulation

  • The multi-modal DAE extracts view-specific features, embeds them in a shared representation, and reconstructs each view to capture multi-view structure.Its reconstruction functions and shared representation are jointly optimized.Source: B. The Proposed Formulation · p. 4
  • The DAE reconstructs the original pair (x, y) from either (x, G1(x)) or (G2(y), y), treating the GAN-generated side as corruption.The input pairs are described as corrupted versions of the original pair.Source: B. The Proposed Formulation · p. 4, Figure 3 · p. 4
  • The cycle-consistency loss penalizes the L1 difference between an input and its round-trip mapping, reducing possible mappings and encouraging corresponding cross-view outputs.The rationale is that passing an input through G1 and G2 maps it back to itself; the text says this likely maps x to its corresponding y.Source: B. The Proposed Formulation · p. 5
  • In the final stage, VIGAN combines the DAE, CycleGAN, and adversarial losses so paired correspondences and unpaired domain mappings jointly inform training.The adversarial loss affects both the autoencoder and GAN; the formulation uses adversarial optimization of A with G1 and DY, and A with G2 and DX.Source: B. The Proposed Formulation · pp. 4–5

AEGAN(A, G2, DX) + LY

The overall VIGAN objective is optimized as a minimax problem over the autoencoder, generators, and discriminators. It can use paired and unpaired data, with the paired-data autoencoder loss disappearing when all data are unpaired.

AEGAN(A, G2, DX) + LY

  • The overall loss combines paired and unpaired data and is optimized by a minimax problem over the autoencoder, generators, and discriminators.The loss weights balance its terms.Source: AEGAN(A, G2, DX) + LY · p. 5
  • When all data are unpaired, the autoencoder loss L_AE degenerates to 0, leaving VIGAN as an enhanced CycleGAN whose generators interact with a denoising autoencoder.The DAE denoises generator outputs to improve missing-view estimates.Source: AEGAN(A, G2, DX) + LY · p. 5
  • Even when all data are paired, the cycle and AEGAN losses use self-mappings or randomly sampled views rather than pair correspondence.The text says Eq. (6) can still learn a GAN from unpaired samples drawn from x or y examples.Source: AEGAN(A, G2, DX) + LY · p. 5

1) Training procedure:

VIGAN pretrains a multimodal autoencoder on paired views and a CycleGAN on unpaired views, then jointly re-optimizes them to denoise GAN outputs. The network architecture varies with data type, while the multimodal DAE connects views through hidden layers for reconstruction.

1) Training procedure:

  • Training proceeds from paired-data DAE pretraining to unpaired-data CycleGAN training, followed by combining the networks for continued training.The combined model can continue with paired data, unpaired data, or all data.Source: 1) Training procedure · p. 5, Figure 4 · p. 6
  • The final training stage jointly re-optimizes the pretrained DAE and GAN using paired and unpaired data to denoise GAN outputs and refine missing-view estimates.The DAE receives (x, G1(x)) or (G2(y), y) and targets the true pair (x, y), with one side corrupted by the GAN estimate.Source: 1) Training procedure · p. 5

2) Network architecture:

  • The architecture depends on data type: numeric vectors use fully connected layers, while image imputation uses convolutional layers.The image architecture adapts CycleGAN generators and PatchGAN discriminators, with a vectorized-image multimodal DAE.Source: 2) Network architecture · pp. 5–6
  • The multimodal DAE concatenates the two views as input and connects them in hidden layers to minimize reconstruction error for both views.It functions as a regular multimodal autoencoder during its initial training on true paired samples.Source: 1) Training procedure · p. 5, 2) Network architecture · p. 6

IV. EXPERIMENTS

The experiments evaluate VIGAN on MNIST and two substance-use datasets, comparing it with several imputation approaches. The supplied figure captions identify imputation examples and report bidirectional imputation regardless of which view is missing.

IV. EXPERIMENTS

  • The evaluation uses MNIST, Cocaine-Opioid, and Alcohol-Cannabis datasets and compares VIGAN with matrix completion, a multimodal autoencoder, pix2pix, and CycleGAN.CycleGAN is trained with paired data and with unpaired data.Source: §IV · p. 6
  • Figure 5 presents examples of imputed views.Source: Figure 5 · p. 7
  • Figure 6 reports that VIGAN imputed bidirectionally regardless of which view was missing.Source: Figure 6 · p. 7

A. Image benchmark data

The image benchmark tests VIGAN when either the original digit or its edge-image view is missing. Its combined domain mapping and autoencoder refinement improves reconstruction over domain mapping alone and performs best among the compared methods.

A. Image benchmark data

  • MNIST's original digit images and derived edge images formed two views, with either view treated as completely missing.The 60,000 examples were split into disjoint unpaired sets, one containing original images and the other edge images.Source: A. Image benchmark data · p. 7
  • For a missing view, the GAN translation supplied an initial estimate and the autoencoder's denoised output was the final imputation.Figure 5 illustrates imputation in both directions.Source: A. Image benchmark data · p. 7
  • Using paired and unpaired data together reduced reconstruction RMSE compared with domain mapping alone.The authors report that the multi-stage model using all data further enhanced domain-mapping results.Source: A. Image benchmark data · p. 7, Table I · p. 7
  • VIGAN achieved the best imputation performance among the compared methods, while CycleGAN and pix2pix were comparable to it.The comparison covered X →Y and Y →X after the same number of training epochs; RMSE values are reported in Table I but are not supplied here.Source: A. Image benchmark data · p. 7, Figure 7 · p. 7, Table I · p. 7

B. Healthcare numerical data

In two substance-use-disorder datasets, VIGAN imputed missing symptom views from symptoms for another substance. The authors report higher reconstruction accuracy than competing methods in both directions, using Hamming distance to assess binary symptoms.

B. Healthcare numerical data

  • The application used 12,158 subjects aggregated across studies of cocaine, opioid, alcohol, and cannabis use disorders.The cocaine–opioid and alcohol–cannabis datasets were used to infer missing symptoms from the other substance's known symptoms.Source: B. Healthcare numerical data · p. 8
  • Symptoms were assessed with the SSADDA interview, and diagnosis reliability varied by substance.Reported κ ranges were 0.47–0.60 for cocaine, 0.56–0.90 for opioid, 0.53–0.70 for alcohol, and 0.30–0.55 for cannabis.Source: B. Healthcare numerical data · p. 8
  • For validation and testing, the study reserved 200 subjects for model selection and 300 for reporting results, with both substances exposed.During validation or testing, one view was set missing and imputed from the other view.Source: B. Healthcare numerical data · p. 8
  • VIGAN's reconstruction accuracy was consistently higher than other methods in both imputation directions across the cocaine–opioid and alcohol–cannabis datasets.Reconstruction was assessed with Hamming distance for binary symptoms; Tables III and IV report the comparisons.Source: B. Healthcare numerical data · p. 8, Table III · p. 8, Table IV · p. 9
  • Using unpaired data alongside paired data improved view imputation compared with training from paired data alone.The comparisons distinguish methods using paired datasets only from those using unpaired data during training.Source: B. Healthcare numerical data · p. 9
  • CycleGAN struggled with numeric-data imputation, and the authors suggest the multi-modal DAE may have contributed to improved accuracy.The reported contrast is with CycleGAN's performance on image-to-image domain transfer.Source: B. Healthcare numerical data · p. 9
  • Although tested only with two views, VIGAN is described as extendable to three or more; the authors also report matrix-completion memory failures and VIGAN training within hours on a Tesla K40 GPU.The proposed extension would require constructing and pre-training multi-modal autoencoders.Source: B. Healthcare numerical data · p. 9

V. CONCLUSION

VIGAN combines a cycle-consistent GAN and multi-modal autoencoder in a multi-stage neural network. The authors report effective, efficient missing-view recovery across MNIST and two healthcare datasets.

V. CONCLUSION

  • VIGAN combines a cycle-consistent GAN with a multi-modal autoencoder in a multi-stage neural network.Source: §V · p. 9
  • VIGAN showed empirical effectiveness and efficiency on MNIST and two healthcare datasets, supporting missing-view recovery through domain mappings and view correspondences.Source: §V · p. 9
  • Future work includes extending VIGAN beyond two views, evaluating it on additional large datasets, and identifying which view most affects imputation.Source: §V · p. 9
Loading 1708.06724v5…