Guided Variational Autoencoder for Disentanglement Learning

1. Introduction

• VAEs combine representation and statistical learning, while reconstruction and synthesis quality and representation transparency, interpretability, and controllability remain areas for improvement.The introduction contrasts this representation-learning role with GANs, which focus on generation and lack an encoder in their vanilla version.Source: §1 · p. 1
• Guided-VAE introduces latent-variable guidance in unsupervised and supervised forms while keeping the VAE backbone intact.The method is trained end-to-end and aims to make latent representations more controllable and transparent.Source: §1 · pp. 1–2
• Unsupervised Guided-VAE uses a lightweight deformable-PCA subtask to make latent variables interpretable and controllable.Its secondary decoder learns geometric deformation and a linear subspace alongside the standard VAE backbone.Source: §1 · pp. 1–2
• Supervised Guided-VAE uses adversarial excitation and inhibition to encourage latent-variable disentanglement, informativeness, and controllability.The approach guides latent variables through supervision without replacing the VAE backbone.Source: §1 · pp. 1–2
• The authors report improved controllability over vanilla VAE while retaining its attractive properties and applicability to generative modeling and representation learning.The paper presents guidance as a multi-task objective intended to make learning a good VAE and controllable latents companions rather than conflicts.Source: §1 · pp. 1–2

2. Related Work

• GANs focus on synthesis, whereas vanilla GANs lack an encoder and are not directly suited to representation learning; this work instead makes VAE more controllable and transparent.The comparison is framed against GAN and VAE.Source: §2 · p. 2
• Many latent-variable constraints and regularizers alter the VAE architecture, while β-VAE controls capacity and the independence prior and β-TCVAE adds a total-correlation term.The paper distinguishes its secondary-decoder guidance from those approaches.Source: §2 · p. 2
• PCA offers a representation-learning perspective but has limited modeling capability under its linear-subspace assumption; Guided-VAE combines PCA transparency with VAE modeling power.The paper describes deformable PCA as end-to-end guidance for VAE training.Source: §2 · p. 2
• Supervised Guided-VAE targets generic generative modeling with adversarial excitation and inhibition, unlike conditional image-translation supervision and related formulations such as DANN and DIVA.The paper also notes connections to DSN, latent-variable constraints, and fairness disentanglement, while distinguishing their formulations and objectives.Source: §2 · p. 2

3.1. VAE

• The VAE maximizes an ELBO for log-likelihood estimation, combining reconstruction fit with a KL-divergence term.Under Gaussian output parameterization, the first term is the negative reconstruction loss between input and reconstruction.Source: §3.1 · p. 3

3.2. Unsupervised Guided-VAE

• The unsupervised variant adds Decsub, which reconstructs through a transformation of content using a PCA basis.Its loss penalizes nonorthogonal basis vectors; unit-vector normalization is optional, and the basis is fixed for simplicity.Source: §3.2 · p. 3

3.3. Supervised Guided-VAE

• For each labeled attribute, excitation makes z_t predictive while adversarial inhibition makes the remaining latent variables as uninformative as possible.The inhibition term is subtracted during maximization, discouraging attribute information in the remaining variables.Source: §3.3 · pp. 3–4

4. Experiments

• On MNIST and 2D shapes, unsupervised Guided-VAE disentangles latent embeddings more favorably than prior methods and improves classification performance.The experiments also report improved synthesis and sampling, better disentanglement for classification, and reduced classification errors in meta-learning.Source: §4 · p. 4
• On CelebA, the experiments extend guidance to a supervised adversarial excitation-and-inhibition approach with discriminative objectives for image properties.The section presents this as an extension of the unsupervised guidance approach.Source: §4 · p. 4
• The method is also evaluated on CIFAR 10 image interpolation and Omniglot few-shot classification as architecture-agnostic applications.Source: §4 · p. 4

4.1.1 Qualitative Evaluation

• On MNIST, Guided-VAE assigns rotation and stroke thickness to separate latent variables from content, unlike β-VAE, which does not produce meaningful disentangled representations in this comparison.JointVAE separates class type from continuous factors in the reported comparison.Source: §4.1.1 · p. 4
• On 2D Shapes, the first four latent variables capture x-position, y-position, scale, and rotation, with fewer non-target factors changing during traversals than in the compared models.The dataset contains four ground-truth independent geometry factors.Source: §4.1.1 · pp. 4–5, Figure 3 · p. 5

4.1.2 Quantitative Evaluation

• Guided-β-TCVAE further improves Z-Diff, SAP, and Factor scores, while the broader evaluation reports significant gains over existing methods on disentanglement and representation transfer.Table 1 covers the three disentanglement metrics on 2D Shapes; Table 2 reports MNIST classification error.Source: §4.1.2 · p. 5, Table 1 · p. 5, Table 2 · p. 5
• At dz = 32, Guided-β-TCVAE reduces MNIST classification error to 1.1%, 1.95% lower than baseline VAE.The comparison evaluates classification from latent embeddings across models and latent dimensions.Source: §4.1.2 · pp. 5–6, Table 2 · p. 5
• Classification errors are lower on content variables than deformation variables, and increasing latent dimensions raises deformation-feature errors while lowering content-feature errors.The authors interpret this pattern as evidence of better disentanglement between deformation and content at higher dimensions.Source: §4.1.2 · p. 6, Table 3 · p. 6

4.2.1 Qualitative Evaluation

• On CelebA, Guided-VAE traversals change the intended smile or gender factor without changing other factors such as hair color or baldness, unlike β-VAE traversals that change multiple attributes.The reported setup assigns the first two latent variables to attributes and the remaining variables to content.Source: §4.2.1 · pp. 6–7

4.2.2 Quantitative Evaluation

• For CelebA gender and smile, the corresponding latent variable enables the attribute for zt < −1 and disables it for zt > 1 in tests of 10,000 generated images.The test varies one controlled latent variable while randomly sampling the others.Source: §4.2.2 · p. 7, Figure 6 · p. 7

4.2.3 Image Interpolation

• CIFAR10 interpolation assigns object-type changes to z1:3 and complementary color and pose changes to zrest; the reported model traverses more consistently than ALI-VAE and AC-GAN and produces higher-quality interpolated images by FID.The interpolations use uniformly spaced linear combinations of latent vectors from images sampled from different classes.Source: §4.2.3 · p. 7, Table 4 · p. 7

• The experiment applies supervised guidance to Neural Statistician by adding a guidance network after its statistic network.The supervised guidance uses input labels and Mixup.Source: §4.3 · pp. 7–8
• The Guided Neural Statistician cannot outperform Matching Nets but reaches performance comparable to Bruno (discriminative) on Omniglot few-shot classification.Table 5 covers few-shot classification accuracy on Omniglot.Source: §4.3 · p. 8, Table 5 · p. 8
• The authors could not reproduce Neural Statistician's reported results, as also indicated in.Source: §4.3 · p. 8

5.1. Deformable PCA

• Figure 8 compares sampling results from PCA with results from learned deformable PCA.Source: §5.1 · p. 8, Figure 8 · p. 8

5.2. Guided Autoencoder

• The lightweight decoder improves MNIST representations in an autoencoder, while probabilistic latent modeling remains important for novel-data synthesis.The paper notes that a VAE-like structure is not needed solely for reconstruction and representation learning.Source: §5.2 · p. 8, Table 6 · p. 8

5.3. Geometric Transformations

• Without geometric guidance, geometry is hardly encoded in the first two latent variables, designated for rotation and scaling.The comparison uses a bottleneck of 10, with z1 and z2 assigned to rotation and scaling, respectively.Source: §5.3 · p. 8

5.4. Adversarial Excitation and Inhibition

• Without inhibition, traversing the gender-controlled latent variable changes smiling and sunglasses; the excitation-inhibition mechanism alleviates this issue.Source: Figure 9 · p. 8, §5.4 · p. 8

• Guided-VAE is a representation-learning method for disentanglement, with both unsupervised and supervised versions using lightweight latent-variable guidance.Source: §6 · p. 8
• The authors report improvements over competing methods in disentanglement, image traversal, and meta-learning.Source: §6 · p. 8
• Guided-VAE maintains the VAE backbone and can be applied to other generative-modeling applications.Source: §6 · p. 8