Self-Guided Generation of Minority Samples Using Diffusion Models

1 Introduction

• The approach aims to generate rare, distinctive samples that can support classification, anomaly detection, medical diagnosis, fairness, and creative AI applications.The introduction motivates these uses as settings where additional instances of rare attributes may be valuable.Source: §1 · p. 1
• Prior minority-generation methods can require substantial resources, annotations, or specialized training, limiting access to pretrained models and practical use.A cited classifier-training example for ImageNet-64 required 40 V100-days.Source: §1 · p. 2
• The sampler obtains low-density guidance solely from a pretrained diffusion model, avoiding separately trained external components such as classifiers.This self-guidance is presented as the defining distinction from prior minority samplers.Source: Figure 1 · p. 2, §1 · p. 3
• A reconstruction loss relative to the posterior mean estimates feature uniqueness in an intermediate latent instance, with theoretical and empirical evidence connecting the metric to existing likelihood measures.The metric is inspired by the low-likelihood measure in [53].Source: §1 · p. 3
• The sampler optimizes its metric during generation, can apply guidance intermittently, and uses timestep schedules to manage guidance strength and sample quality.The introduction gives guidance every five sampling steps as an example of intermittent use.Source: §1 · p. 3
• Experiments across real benchmarks show improved high-quality minority generation, with stronger low-density metrics and better quality metrics than existing techniques.The reported metrics include Average k-Nearest Neighbor and Fréchet Inception Distance.Source: §1 · p. 3
• For text-to-image generation of rare concepts, a related method requires high-quality images describing the target concepts, unlike the pretrained-only approach described here.The cited method is therefore not directly comparable.Source: §1 · p. 4
• The paper distinguishes its low-density objective from methods that seek diverse samples adhering to the ground-truth distribution or improve minority coverage near that distribution.The related-work discussion identifies these as different goals from generating low-density samples.Source: §1 · pp. 3–4

2.1 Diffusion-based generative models

• The forward diffusion process gradually adds Gaussian noise, and the reverse process uses a learned transition to generate samples from noise toward the data distribution.The forward process permits one-shot sampling at a selected timestep; ancestral sampling iterates the reverse process.Source: §2.1 · pp. 4–5

2.2 Guided sampling with diffusion models

• Guided sampling adds a weighted, potentially time-scheduled energy-based guidance term to each reverse step to steer generation toward a desired direction.Classifier guidance is a special case, while minority-generation methods use classifier-based guidance to target low-density regions.Source: §2.2 · p. 5
• Existing minority guidance relies on external components such as classifiers, motivating a self-contained guidance function implemented solely with a pretrained diffusion model.This design is intended to improve the accessibility and practicality of minority generation.Source: §2.2 · pp. 5–6

3.1 Towards an inference-time minority metric

• Direct ODE-based likelihood estimation is computationally expensive, and guiding directly with perturbed-sample likelihood may drive generation off the data manifold.The authors therefore take a different approach to inference-time guidance.Source: §3.1 · p. 6
• The proposed metric measures feature uniqueness in a noisy intermediate sample by reconstructing its posterior-mean clean estimate and comparing it with a second posterior-mean estimate.It extends the minority score to noisy instances, making it usable during inference.Source: §3.1 · pp. 6–7
• The metric assigns higher values to samples with uncommon benchmark features, including hats and eyeglasses in CelebA and complex attributes in LSUN and ImageNet.Fig. 2 displays samples with the smallest, moderate, and highest metric values.Source: §3.1 · p. 7, Figure 2 · p. 7
• With squared-error distance, the proposed metric integrated over timesteps using weights α_s/(1 − α_s) is equivalent to the negative ELBO for the estimated clean sample.The paper presents the negative ELBO as a proxy for log-likelihood.Source: §3.1 · p. 8

3.2 Self-guidance for low-density regions

• The guidance requires no external elements; although its gradient initially requires two backward passes, stop-gradient on ˆˆx0 reduces this to one, often preserving low-likelihood guidance’s performance benefit.Using guidance every 2 sampling steps (n = 2) reduces inference time by 37.7% with marginal impact on performance; n = 5 yields satisfactory performance across diverse benchmarks.Source: §3.2 · pp. 8–9

3.3 Time-scheduling for improved sample quality

• Constant guidance can degrade sample quality at high strength; the proposed schedules decrease guidance over time, and the variance-based schedule generally performs better than fixed scales.The schedules include stopping guidance after a predefined timestep or scaling it by reverse-process variance.Source: §3.3 · pp. 9–10

4.1 Setup

• The experiments cover CelebA and LSUN-Bedrooms unconditional benchmarks, ImageNet-64 and ImageNet-256 class-conditional benchmarks, and additional T2I-generation and medical-imaging scenarios.Baselines and quality, diversity, and low-density metrics vary across the benchmark settings.Source: §4.1 · pp. 10–11

4.2 Results

• Across focused benchmarks, the sampler achieves better or comparable quality and diversity than baselines and remains close to real minority samples, including with intermittent guidance.The comparison uses the most unique real samples, selected by highest AvgkNN, as the real-data baseline.Source: Table 1 · p. 12, §4.2 · pp. 12–13
• On LSUN-Bedrooms, the method and [53] improve all three low-density measures over ADM ancestral sampling.The measures are AvgkNN, LOF, and Rarity Score; higher values indicate less likely samples.Source: §4.2 · p. 13, Figure 5 · p. 13
• Ablations show that stronger guidance improves proximity to real minorities, stop-gradient on the posterior estimate reduces inference time with little performance compromise, and variance scheduling improves quality while retaining respectable cFID.The variance schedule is selected as the final scheduling strategy.Source: §4.2 · pp. 13–14
• On CelebA, classifier training with 50K samples from the method improves every reported metric, while 50K ADM samples do not improve over unaugmented training.The comparison covers 40 attributes and uses an off-the-shelf classifier to label generated samples.Source: §4.2 · p. 14

5 Conclusion

• The framework optimizes diffusion generation with a self-guided minority metric to produce low-likelihood minority features using a pretrained model alone.The conclusion distinguishes this approach from frameworks that require additional components.Source: §5 · p. 14
• Experiments across real-data benchmarks show significant improvements over existing minority samplers.Source: §5 · p. 14
• The sampler adds inference costs and could be misused with negative w to suppress minority-featured samples in favor of high-likelihood majority features.The paper calls for responsible use to uphold fairness and inclusivity.Source: §5 · p. 14

A.1 Proof of Proposition 1

• Summing the noise-prediction errors over timesteps with reciprocal weights yields the negative ELBO up to a constant factor.The proposition states this equivalence for the squared-error minority score.Source: §A.1 · p. 19
• The minority score at timestep t is the expected reconstruction loss between x_0 and its posterior-mean estimate from a perturbed sample.The expectation is over the forward perturbation distribution q_{α_t}(x_t|x_0).Source: §A.1 · p. 19
• With squared-error distance, the reconstruction loss becomes a noise-prediction error weighted by (1 − α_t)/α_t.The derivation uses Tweedie’s formula and the noise-predicting expression for the score network.Source: §A.1 · p. 19

A.2 Proof of Corollary 1

• Weighting the metric across timesteps by α_s/(1 − α_s) makes it equivalent to the negative ELBO with respect to the predicted clean sample.The equivalence follows from Proposition 1 and holds up to a constant factor.Source: §A.2 · pp. 19–20
• The timestep-weighted equivalence follows by applying Proposition 1 to the minority score of the posterior-mean estimate.Source: §A.2 · pp. 19–20
• The proposed metric on the predicted clean sample is the minority score of that sample at timestep s.The proof identifies the metric with the minority score evaluated at the posterior-mean estimate.Source: §A.2 · p. 20

B Additional Details on Experimental Setup

• The experiments use pretrained ADM checkpoints for LSUN-Bedrooms and ImageNet, with the CelebA model constructed to match [53] settings.The [53] sampler uses the same ADM backbones, with U-Net minority classifiers trained on all data except 10% of LSUN-Bedrooms training data.Source: B Additional Details on Experimental… · p. 20
• The evaluation uses AvgkNN and LOF with 5 and 20 neighbors, respectively, and compares generated samples with least-probable real samples selected by AvgkNN.Quality and diversity metrics are computed on 30K generated samples; the real-data baseline uses 10K samples for CelebA and 50K for LSUN-Bedrooms and ImageNet.Source: B Additional Details on Experimental… · p. 21
• The main benchmarks use LPIPS, perturbation strength s of 0.8T or 0.5T depending on the noise schedule, and a single Monte Carlo sample.Guidance uses unit l∞-norm normalization; intermittent rates are 5 for CelebA and LSUN-Bedrooms and 2 for ImageNet.Source: B Additional Details on Experimental… · p. 22

C.1 Problem formulation: a mathematical version

• The formal target is an on-manifold sample x whose density q(x) falls below threshold τ_th, assuming continuity of the data distribution on its support.Source: §C.1 · p. 22

C.2 Effectiveness of the proposed metric

• The authors report theoretical and empirical evidence for the proposed metric, including positive correlations with existing low-density metrics, offering additional empirical validation of its applicability as a minority metric during inference time.Source: §C.2 · p. 23

C.3 Manifold-preserving aspect of the proposed guidance

• Under the stated low-dimensional, locally linear manifold assumption, naive low-density guidance points perpendicular to the clean manifold and may lead samples off the noisy manifold.The reverse diffusion process is interpreted as transitions between adjacent noisy manifolds.Source: §C.3 · pp. 23–24
• LPIPS performs best among the tested distance metrics, moderate perturbation strength such as s = 0.8T is favorable, and one Monte Carlo draw works reasonably well.Fig. 8 also reports that the number of Monte Carlo samples is not critical to the framework.Source: Figure 8 · p. 24
• The proposed guidance is described as tangent to the preceding noisy manifold and as keeping instances on it when the step size is properly chosen.The guidance is presented as an instance of a manifold-constrained gradient.Source: §C.3 · p. 24

C.4 Further ablation studies

• LPIPS perceptual distance is more beneficial than pixel-level distance, although both improve over baseline ancestral sampling.Source: §C.4 · p. 24
• Moderate perturbation strength, such as s = 0.8T, supports good performance; overly strong or weak perturbation degrades the metric’s ability to distinguish low-likelihood features.The text attributes the degradation to reconstruction losses becoming less discriminative at either extreme.Source: §C.4 · p. 25
• A single Monte Carlo sample suffices for the reported main results, and early stopping improves sample quality with only marginal diversity degradation in the cited comparison.Source: §C.4 · p. 25
• The variance-based schedule yields favorable quality–diversity trade-offs versus unprincipled schedules, while all tested schedules improve sample quality over fixed scales.The figure describes Var as using the pretrained model’s noise statistics.Source: §C.4 · p. 25, Figure 9 · p. 26
• Intermittent guidance maintains performance benefits through n = 5 while improving resource efficiency and enabling competitive computational costs.Table 4 reports the intermittent-rate impact; the text points to Table 5 for the complexity comparison.Source: §C.4 · pp. 25–26, Table 4 · p. 25

C.5 Controllability of the proposed approach

• An external semantic classifier can be combined with the sampler to generate low-likelihood samples toward desired semantics, such as gender.The paper states that this retains controllability offered by classifier-based methods.Source: §C.5 · p. 26

C.6 Computational complexities

• The sampler achieves competitive inference times while avoiding external-classifier preparation overhead, which can exceed 40 V100-days for ImageNet-64 baselines.The comparison is reported for CelebA; the text also describes dozens of hours of additional baseline resources.Source: §C.6 · p. 27, Table 5 · p. 27

D.1 Text-to-image generation

• For Stable Diffusion text-to-image generation, samples show more unique minority attributes and higher AvgkNN and LOF values than the standard sampler.The examples use two prompts; the figure compares neighborhood density with real MS-COCO validation data and Stable Diffusion.Source: §D.1 · pp. 27–28, Figure 12 · p. 28, Figure 11 · p. 28

D.2 Medical imaging

• On a 13,640-image brain MRI dataset, the sampler produces more pronounced atrophy and improves minority-generating capability over the baselines without external elements.The comparison includes ancestral-sampling ADM and StyleGAN2-ADA.Source: Table 6 · p. 29, §D.2 · pp. 29–30, Figure 13 · p. 29

D.3 Image editing

• Integrated into SDEdit, the guidance introduces novel visual attributes into reference images compared with baseline editing.Source: §D.3 · p. 30, Figure 14 · p. 30

• On CelebA, both minority samplers produce more unique dataset features than standard sampling, while this method’s gain comes solely from the pretrained model.Source: E Additional Experimental Results · p. 31, Figure 20 · p. 33
• Across CelebA, ImageNet-64, and ImageNet-256, the self-guided sampler consistently matches or outperforms baselines on all three neighborhood-density metrics.The passage identifies AvgkNN, LOF, and Rarity Score as the measures.Source: E Additional Experimental Results · p. 31, Figure 16 · p. 32, Figure 18 · p. 32
• Additional generated-sample comparisons are provided for CelebA, LSUN-Bedrooms, ImageNet-64, and ImageNet-256.Source: E Additional Experimental Results · p. 31, Figure 20 · p. 33, Figure 21 · p. 34