Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting

1 Introduction

• Importance weighting is motivated by the difficulty of learning unbiased distributions and the unavailable likelihood ratios needed for exact correction.In practice, model likelihoods may be intractable or ill-defined, including for variational autoencoders [7] and generative adversarial networks.Source: §1 · p. 1
• A calibrated probabilistic classifier distinguishes data and model samples to estimate density ratios without requiring likelihood evaluation.The method requires samples from both distributions; classifier outputs can be used to recover density ratios.Source: §1 · p. 1
• The model and classifier-based weighting function induce an unnormalized distribution, with conditions on weight quality guaranteeing improved fit to the data distribution.Exact density estimation and sampling from this distribution are intractable, so the paper derives a particle-based approximation for efficient resampling.Source: §1 · p. 2
• Importance weighting reduced sample-quality bias by 23.35% and 13.48% on PixelCNN++ and SNGAN, raised Omniglot classification accuracy from 66.03% to 68.18%, and reduced MBOPE RMSE by 50.25% on 3 MuJoCo environments.The sample-quality figures are averages across commonly used metrics on CIFAR-10; data augmentation was evaluated on Omniglot.Source: §1 · p. 2

2 Preliminaries

• Generative modeling learns a distribution pθ to approximate the unknown data distribution pdata from finite i.i.d. training data.The paper notes maximum likelihood and adversarial training as alternative learning approaches.Source: §2 · p. 2
• For expectation evaluation, generated samples can supplement samples from p and may improve estimate accuracy when pθ is close to p.The model provides virtually unlimited generated samples, ignoring computational constraints.Source: §2 · p. 2
• Data augmentation adds generated labeled examples to training data and optimizes classification loss under a mixture of empirical and generated distributions.The mixture weight m is chosen in [0, 1], and the target expectation in this use case is under pdata.Source: §2 · pp. 2–3

3 Likelihood-Free Importance Weighting

• Classical importance sampling reweights model samples by p(x)/pθ(x), subject to a support condition ensuring the proposal covers relevant target mass.The stated condition is that pθ(x)=0 implies f(x)p(x)=0.Source: §3 · p. 3
• With sample access to p and pθ, a probabilistic classifier estimates the class conditional and yields importance weights from its output.The default weight for a model sample is γ cφ(x)/(1−cφ(x)).Source: §3 · pp. 3–4
• Because practical classifiers are not Bayes optimal, LFIW weights are inexact and can reduce bias without guaranteeing its elimination.The classifier-based estimate is the paper’s default LFIW estimator.Source: §3 · p. 4
• Artifact-sensitive classifiers can assign tiny weights to generated images and increase relative weight variance; normalization, flattening, and clipping are proposed variants.Flattening trades off bias reduction against undesirable variance, while the variants can be combined.Source: §3 · p. 4
• The paper proposes combining empirical and parametric bootstraps to derive confidence intervals for estimated importance weights.The procedure accounts for finite real data and a parametric generative model.Source: §3 · p. 4
• As the number n of real and generated training examples grows, the classifier approaches optimality and its uncertainty shrinks.Figure 1 compares estimated class probabilities with optimal probabilities and shows confidence intervals.Source: Figure 1 · p. 4, §3 · p. 5

4 Importance Resampled Generative Modeling

• Exact density estimation requires the base-model density and partition function, which are typically intractable; Monte Carlo estimates of the partition function can use parallel samples.The estimates may require many samples to reduce variance.Source: §4 · p. 5
• Sampling-importance-resampling provides a tractable particle-based approximation to the induced distribution, recovered in the limit as the particle count T tends to infinity.Expectations under the approximation can be evaluated with the self-normalized LFIW estimator.Source: §4 · p. 5
• The KL-fit analysis assumes pdata is absolutely continuous with respect to both pθ and the resampled density, and derives conditions for improved fit.The paper describes one condition as necessary and sufficient and a follow-up condition as necessary but insufficient.Source: §4 · pp. 5–6
• Table 1 evaluates PixelCNN++ and SNGAN goodness-of-fit on CIFAR-10, with standard errors over 10 runs; higher IS is better, while lower FID and KID scores are better.Source: Table 1 · p. 6

5.1 Goodness-of-fit testing

• Self-normalized LFIW consistently debiases three sample-quality metrics for PixelCNN++ and SNGAN on CIFAR-10, suggesting the resampled distribution better fits the data.The experiments evaluate Inception Score, FID, and KID using 10,000 samples per model.Source: §5.1 · p. 6

5.2 Data Augmentation for Multi-Class Classification

• For Omniglot augmentation, weighting generated points improves classification, whereas generated data alone is not competitive with real training data.Naively combining real and generated data is also overshadowed by the bias it introduces for evaluation and optimization.Source: §5.2 · p. 7
• In Omniglot examples, higher-ranked generated samples often look more plausible, but some reasonable samples are ranked poorly.The authors note rankings cannot be formally validated and may favor high-data-density samples that are unlikely under the generator.Source: §5.2 · p. 7, Figure 2 · p. 7

5.3 Model-based Off-policy Policy Evaluation

• For policy evaluation, the learned dynamics model generates trajectories that can deviate from true trajectories through compounding errors; LFIW weights whole trajectories using transition classifiers.The classifier distinguishes true and generated state-action-next-state triples.Source: §5.3 · p. 8
• On MuJoCo policy evaluation, importance weighting makes estimates more accurate, with accuracy improving as more timesteps are weighted until around 80–100.The multiplicative state-action weights can raise return variance; the dynamics-model estimates also differ substantially from ground truth.Source: §5.3 · pp. 8–9

5 Application Use Cases

• Across experiments, importance weighting yields uniformly better results for sample-quality evaluation and downstream-task utility.The authors expect the black-box wrapper to benefit diverse future tasks.Source: §5.3 · p. 9

5.3 Model-based Off-policy Policy Evaluation

• Importance weighting reduces bias but does not eliminate it: weighted-score confidence intervals still exclude reference scores.The authors suggest stronger classifiers or more data when the generator fails the full-support assumption.Source: §5.3 · p. 9

6 Related Work & Discussion

• Unlike classifier-based generative-model methods aimed at learning model parameters [19] [35] [37] [38] [39] [40] [41], this work uses the classifier to correct bias in a fixed model.The paper distinguishes its goal from prior work that reweights training and test data to address dataset bias.Source: §6 · p. 9
• MCMC and rejection-sampling alternatives explicitly transform or reject generated samples, adding computation that the paper says is unnecessary for many model-based Monte Carlo evaluations [46] [47].The cited use cases include data augmentation and model-based off-policy policy evaluation.Source: §6 · p. 9
• When samples or density estimates from the induced distribution are needed, a particle-based approximation offers a tunable trade-off between statistical accuracy and computational efficiency.The number of particles is the tunable parameter.Source: §6 · p. 9

7 Conclusion

• The proposed framework corrects generative-model bias with importance weights learned likelihood-free via a binary classifier.The conclusion identifies bias relative to a target data distribution as a challenge for using deep generative models as Monte Carlo proposals.Source: §7 · p. 9
• The authors report the framework is useful across goodness-of-fit sample-quality tests, data augmentation, and model-based off-policy policy evaluation.Source: §7 · p. 9
• Characterizing deep generative-model bias is presented as a step toward using these models to guide decisions in high-stakes applications, including healthcare and robust anomaly detection.Source: §7 · p. 9

Appendices

• For a finite dataset, the empirical bootstrap resamples data points with replacement to estimate confidence intervals.Source: A Confidence Intervals via Bootstrap · p. 14
• For a parametric model, the parametric bootstrap draws a fresh dataset from the model before estimating confidence intervals.Source: A Confidence Intervals via Bootstrap · p. 14
• Classifier confidence intervals are estimated by retraining on fresh model samples and a with-replacement resampling of the training data, then taking a suitable quantile across runs.Source: A Confidence Intervals via Bootstrap · p. 14

B Bias-Variance of Different LFIW estimators

• Self-normalization is best or second-best for bias and MSE in all reported cases.The analysis averages over 10 draws of 10,000 samples and 2,048 activation statistics; Table 4 reports the PixelCNN++ and SNGAN results.Source: B Bias-Variance of Different LFIW… · p. 14, Table 4 · p. 15
• Lower α values among the flattening estimators appear to give the best bias-variance trade-off, while clipped estimators perform poorly in this setting.Among clipped estimators, lower β values are slightly preferable to larger ones.Source: B Bias-Variance of Different LFIW… · p. 15
• In the repeated experiment with T = 5,000 samples, variance increases as expected by almost an order of magnitude, while estimator bias remains roughly the same.Table 5 reports the bias-variance analysis for PixelCNN++ and SNGAN at T = 5,000.Source: B Bias-Variance of Different LFIW… · p. 15, Table 5 · p. 15

C.1 Calibration

• Binary classifiers were highly calibrated by default on held-out real and generated samples, so the experiments did not require recalibration.Calibration statistics used 10 bins; the authors attribute the behavior largely to shallow binary classifiers and contrast them with softmax-based multiclass classifiers [62].Source: §C.1 · p. 16, Figure 4 · p. 16

C.2 Synthetic experiment

• The synthetic experiment used a single-hidden-layer, 100-unit MLP trained with cross-entropy on equal numbers of generated and real samples.Source: §C.2 · p. 16

C.3 Goodness-of-fit testing

• Goodness-of-fit evaluation used Inception features and Inception score, FID, and KID; the binary classifier was a 1000-unit MLP on 2048-dimensional Inception features.The classifier used Adam at learning rate 0.001 and batch size 64, converged in fewer than 20 epochs, and was selected by validation accuracy; FID compares real and generated feature statistics, while KID compares feature-space statistics using kernels.Source: §C.3 · pp. 16–17

• The DAGAN generates examples conditioned on training datapoints, while its critic distinguishes real-real pairs from real-generated pairs and encourages same-class generation.Generated examples inherit the conditioning datapoint’s class label.Source: §C.4 · p. 17
• The setup trains both a binary importance-weight classifier and a multiclass classifier using an architecture based on prior Omniglot meta-learning work [66].Training uses 1200 classes, with 15/5/5 training, validation, and test examples per class; the network has four convolutional blocks and is trained for 100 epochs.Source: §C.4 · p. 17

C.5 Model-based Off-policy Policy Evaluation

• The experiments cover Swimmer, HalfCheetah, and HumanoidStandup, using PPO-trained evaluation and behavior policies and a learned dynamics model.The evaluation policy was trained for 1M timesteps and the behavior policy for 500k; the dynamics model was trained for 100k iterations and the classifier for 10k iterations.Source: §C.5 · pp. 17–18, Figure 5 · p. 17

C.5.1 Stepwise LFIW

• Stepwise LFIW reduces OPE bias versus no LFIW, but has higher variance and weaker performance at H = 20, 40; at H = 100 it outperforms trajectory-based estimators on HalfCheetah and HumanoidStandup.Source: Table 7 · p. 18, Figure 6 · p. 18, §C.5.1 · p. 18
• Stepwise LFIW assigns each transition triplet its own importance weight rather than multiplying transition weights across a whole trajectory, then self-normalizes triplet weights.Source: §C.5.1 · p. 18