Telescoping Density-Ratio Estimation

1 Introduction

• Density-ratio estimation uses samples to estimate p/q without separately estimating either density and supports applications including energy modeling, MI estimation, and representation learning.Source: §1 · p. 1
• When DKL(p ∥ q) exceeds ∼20 nats, classifiers may achieve near-perfect accuracy while estimating p/q poorly; high-dimensional gaps can reach hundreds of nats.Source: §1 · p. 1
• TRE transports samples through intermediate datasets, estimates ratios between consecutive datasets, then multiplies those chained ratios to estimate the original p/q.Source: §1 · pp. 1–2
• TRE significantly outperforms existing single-ratio methods on high-dimensional problems, and for EBMs extends noise-contrastive estimation with more efficient scaling.Source: §1 · p. 2
• TRE accurately estimates MI values of 30+ nats, a problem recognized as outstanding in the literature.Source: §1 · p. 2
• TRE requires no MCMC during learning, uses a non-adversarial objective, and applies to discrete data, unlike other recent EBM methods described here.Source: §1 · p. 3

2 Discriminative ratio estimation and the density-chasm problem

• Logistic-loss classification estimates p/q without normalization constraints when the ratio model is correctly specified.Other classification losses may yield only an estimate proportional to the true ratio.Source: §2 · p. 3

The density-chasm problem

• An exponential increase in sample size yields only a linear decrease in estimation error in the plotted regime.This sample inefficiency makes increasing sample size a highly inefficient solution and not always possible in practice.Source: The density-chasm problem · p. 3
• The density-chasm problem is reported to occur with other commonly used discriminative losses, not only logistic loss.The authors say the related issues also occur with other losses in appendix experiments.Source: The density-chasm problem · p. 3

3 Telescoping density-ratio estimation

• TRE decomposes a difficult p/q classification task into neighboring waymark-ratio tasks and multiplies the bridge estimates to recover the original ratio.The intermediate distributions are implicitly defined through samples, unlike AIS, which typically defines them explicitly and obtains samples via MCMC.Source: §3 · p. 4

3.1 Waymark creation

• Linear combinations and dimension-wise mixing are the two deterministic waymark-creation mechanisms considered.For linear combinations, coefficients are chosen to preserve variance in the experiments; dimension-wise mixing concatenates subsets of dimensions from the endpoint samples.Source: §3.1 · pp. 4–5

3.2 Bridge-building

• Bridge ratios are learned as logistic-loss classification tasks, using a shared neural-network body and bridge-specific heads.Training uses different samples for each bridge, while all ratios are evaluated at the same input after learning; the authors report no observed mismatch problem but call for deeper analysis.Source: §3.2 · p. 5

3.3 TRE applied to mutual information estimation

• For mutual-information estimation, TRE waymarks hold u fixed while changing v between joint-distribution and product-of-marginals samples.The product-of-marginals samples are obtained by shuffling v vectors across the dataset.Source: §3.3 · p. 6

3.4 TRE applied to energy-based modelling

• TRE-based energy models represent the target density ratio against a known, sampleable density q, yielding a product-of-experts model.The paper identifies Gaussian distributions and normalizing flows as examples of q.Source: §3.4 · p. 6

4.1 1d peaked ratio

• TRE is significantly more accurate than single-ratio estimation across sample sizes, achieving with 100 samples what the baseline achieves with 100,000.The paper describes this as a three-orders-of-magnitude improvement.Source: §4.1 · p. 6

4.2 High-dimensional ratio with large MI

• TRE accurately estimates MI up to 80 nats for 320-dimensional variables, whereas single-ratio estimation becomes severely inaccurate above 20 nats.The experiment uses quadratic bridges; the supplied Figure 3 caption identifies estimated MI as a function of dimensionality.Source: Figure 3 · p. 6, §4.2 · p. 7

4.3 MI estimation & representation learning on SpatialMultiOmniglot

• TRE maintains greater than 97% representation accuracy as Omniglot grid size and MI increase, while single-ratio baselines degrade; TRE also estimates MI near 35 nats.The baselines’ accuracy drops by 20–60% from 4 to 9 characters, while TRE drops by about 3%.Source: §4.3 · p. 7, Figure 4 · p. 7

4.4 Energy-based modelling on MNIST

• For MNIST EBMs, the experiments use Gaussian, Gaussian-copula, and RQ-NSF noise distributions, with quadratic heads and positive-definite W matrices; log-likelihood is estimated using Direct, RAISE, and AIS.The evaluation also uses NUTS for annealed MCMC sampling.Source: RQ-NSF · p. 8, Table 1 · p. 8

RQ-NSF

• TRE with Gaussian noise improves MNIST EBM log-likelihood by around 0.66 bpd (roughly 360 nats) over single-ratio estimation, with more coherent samples despite worse log-likelihood than RQ-NSF.The evaluations use approximate log-likelihood and random samples; log-likelihood and sample quality are only loosely connected.Source: RQ-NSF · p. 8, Table 1 · p. 8, Figure 5 · p. 8
• For copula noise, varying waymarks from 5 to 30 changes approximate log-likelihood by at most 0.03 bpd; omitting z-space waymarks raises RAISE negative log-likelihood to 1.33 bpd.The latter result remains significantly better than single-ratio estimation, while indicating sensitivity to the waymark mechanism.Source: RQ-NSF · p. 9

• TRE is presented as a framework for accurately estimating density ratios between extremely different densities in high dimensions, unlike existing discriminative methods.Source: §5 · p. 9
• The authors identify deeper theory for TRE’s sample efficiency and further research on waymark mechanisms as open directions.They suggest exploring TRE’s structural similarity to annealed importance sampling and using domain- or task-specific assumptions to design waymarks.Source: §5 · p. 9
• The paper focuses on logistic loss and leaves investigation of other discriminative losses for future work.Source: §5 · p. 9

Broader Impact

• TRE could conceivably enhance GAN-generated deepfakes used for fake news or identity fraud, although GAN training is outside this paper’s scope.The authors describe GANs as often trained via density-ratio estimation and able to generate photo-realistic image or video content.Source: Broader Impact · p. 9
• Density-ratio methods can correct dataset bias, but may also introduce new biases while correcting existing ones.The authors identify skewed demographic factors such as race and gender as examples, and call for future work to monitor this possibility.Source: Broader Impact · p. 9

A ResNet architectures with parameter sharing

• Convolution operations are shared across bridges and independent of bridge index; Figure 6 gives the f_k architectures used for SpatialMultiOmniglot and MNIST.ConditionalScaleShift applies bridge-specific elementwise scaling and bias, analogous to class-conditional Batch Normalisation without normalisation; preliminary Instance Normalisation experiments harmed performance.Source: A ResNet architectures with parameter… · p. 14, Figure 6 · p. 14
• MNIST uses average pooling, while SpatialMultiOmniglot searches average versus max pooling; both use LeakyRelu activations with slope 0.3.The MNIST attention block did not improve estimated log-likelihood, and preliminary experiments suggested Spectral Normalisation was not beneficial; the authors caution that negative results are preliminary.Source: A ResNet architectures with parameter… · p. 14

B Waymark number and spacing

• The number of waymarks and, for linear combinations, their spacing are treated as hyperparameters; the authors say observations make finding good values simpler.Both mechanisms yield linearly spaced αk when p = 1. For the first mechanism, p > 1 makes gaps increase with k, while p < 1 makes them decrease.Source: B Waymark number and spacing · pp. 14–15
• Past a certain number of waymarks, MNIST performance plateaus rather than decreases, suggesting little risk of overshooting with extra waymarks.TRE logistic losses saturating close to 0 indicate a density-chasm problem for that bridge and can signal run termination.Source: B Waymark number and spacing · pp. 14–15

C Minibatching

• Coupled minibatches pair samples from p0 and pm to construct intermediate waymark samples, sharing randomness in gradient estimation as a possible variance-reduction mechanism.All experiments use coupling based on preliminary investigations, but forming a full B × M minibatch can force B to be small; the authors speculate this may make coupling inferior to non-coupled minibatches.Source: C Minibatching · p. 15

D 1d peaked ratio toy experiment

• The experiment estimates a ratio between Gaussians with σ0 = 10^-6 and σm = 1, using linear-combination waymarks that remain Gaussian.The bridge model is well-specified because it contains the ground-truth solution, and summing bridge estimates yields an estimate of the original log-ratio.Source: D 1d peaked ratio toy experiment · pp. 15–16

D.1 The density chasm problem for non-logistic loss functions

• The appendix applies NWJ/MINE-f and least-squares losses to both single-ratio estimation and each TRE bridge to test the density-chasm phenomenon beyond logistic loss.The sigmoid in the least-squares expression denotes the sigmoid function.Source: §D.1 · p. 16
• With 10,000 samples, single-ratio NWJ and LSQ objectives saturate across many parameter settings, and their minimisers are far from the true minimisers.For TRE using either loss, each bridge has a quadratic-bowl objective whose finite-sample minimisers almost perfectly overlap the true minimisers.Source: §D.1 · p. 16
• TRE with NWJ or LSQ losses achieves significantly better sample efficiency than single-ratio estimation, with orders of magnitude fewer samples.Single-ratio estimation performs poorly, at best gaining linearly for exponential increases in sample size; the findings match those reported for logistic loss.Source: §D.1 · p. 16

E High-dimensional ratio with large MI toy experiment

• The high-dimensional toy experiment uses a Gaussian p0 with block-diagonal 2 × 2 covariance blocks and compares it with p_m = N(x; 0, I).The task can be viewed either as energy-based modelling, with p_m as a noise distribution, or as mutual-information estimation by splitting x into odd- and even-indexed components.Source: E High-dimensional ratio with large MI… · pp. 16, 18

F.1 Dataset construction

• SpatialMultiOmniglot uses the d largest Omniglot alphabets, with each alphabet represented by character templates and 20 image versions per character.The dataset construction resizes Omniglot images to 28 × 28.Source: §F.1 · p. 18
• For each example, the dataset samples an independent uniform character index from each of d alphabets, then randomizes character versions to form paired views u and v.The views use consecutive characters within their alphabets and are arranged in a raster-ordered grid; the index variables are defined over template images rather than pixel values.Source: §F.1 · pp. 18–19

F.2 Derivation of ground truth MI

• The derivation assumes u and v are conditionally independent given the character-index vector j and that each view identifies its characters without ambiguity.Under these assumptions, the appendix obtains the ground-truth mutual information analytically.Source: §F.2 · p. 19
• The ground-truth mutual information between u and v equals the sum of the log alphabet sizes.The appendix derives an analytical formula for this quantity.Source: §F.2 · p. 19

F.3 Experimental settings

• SpatialMultiOmniglot experiments use d=1, 4, and 9 with 50,000 training and 10,000 validation and test points; representation evaluations fit linear classifiers on g(u).The classifier task predicts each character’s alphabetic position.Source: §F.3 · pp. 19–20
• Training batches total about 512 samples across waymark trajectories, so increasing waymarks decreases the number of trajectories per batch.Models use Adam with cosine annealing for 200,000 iterations on one P100 GPU; average pooling was less prone to overfitting and often performed better, but initially made little progress.Source: §F.3 · p. 20

G Energy-based modelling on MNIST

• MNIST energy-based modeling uses Adam with an initial learning rate of 10^-4 and cosine annealing on a single P100 GPU.The section covers noise parameterization, sampling from learned EBMs, and TRE experimental settings.Source: G Energy-based modelling on MNIST · p. 20

G.1 Noise distributions

• The experiments compare pre-trained multivariate Gaussian, Gaussian copula, and rational-quadratic neural spline flow noise distributions.The copula uses spline-parameterized marginals; the RQ-NSF uses a two-level, eight-step-per-level convolutional architecture.Source: §G.1 · p. 20

G.2 Annealed MCMC Sampling

• Annealed MCMC initializes chains from p_m and samples through intermediate distributions toward p_0; exponentially averaged intermediates provide a smoother interpolation.The sampling sequence can also be run in reverse from data toward the noise distribution.Source: §G.2 · pp. 20–21
• NUTS sampling uses 1,000 intermediate distributions and 100 parallel chains, with sampling performed in z-space before mapping results back to x-space.AIS and RAISE estimate log-likelihoods in z-space, and the change-of-variables formula yields estimates in x-space.Source: §G.2 · p. 21

G.3 Experimental settings

• MNIST is dequantized with uniform noise, rescaled to the unit interval, and mapped to the real line by a logit transformation.The setup uses 50,000 training points and 10,000 validation and test points.Source: §G.3 · p. 21
• The first ratio estimator between p_0 and slightly perturbed p_1 was extremely prone to overfitting, mitigated by setting α_0 to 0.01.The authors conjecture that the issue may reflect MNIST’s discreteness and insufficient dequantizing perturbation.Source: §G.3 · p. 21
• Mini-batches sample 25 points each from p_0 and p_m, making the total batch size 25 × (m + 1).Training runs for 200,000 iterations and takes about a day.Source: §G.3 · p. 21

G.4 Additional results

• X-space waymarks yield lower RAISE performance than z-space and require around 15 waymarks to avoid logistic-loss saturation; Figure 10 plots bpd against waymark number.For x-space, losses saturated near zero with 5 and 10 waymarks.Source: §G.4 · p. 21, Figure 10 · p. 22
• Extended MNIST samples in Figure 11 are ordered from lowest log-density at top left to highest at bottom right.Source: §G.4 · p. 21, Figure 11 · p. 23