Finite-Rank Logistic Gaussian Processes with Exact Likelihood for Conditional Density Estimation

1 Introduction

• For ADNI fractional anisotropy along the corpus callosum, the motivating goal is covariate-adjusted percentile bands that quantify uncertainty and show how covariates and location alter the distribution.The data show variation across individuals and measurement locations, and the clinically relevant question includes where values depart from reference bands and how extensive those departures are.Source: §1 · p. 2
• Existing logistic Gaussian process methods either approximate the normalizing integral or alter or replace the likelihood, and fitted-density moments and quantiles then require numerical integration.ExFR-LGP is presented as avoiding response binning, numerical quadrature, and surrogate loss while targeting the exact posterior under its prior.Source: §1 · pp. 4–5
• ExFR-LGP models the log density with two bivariate functions and achieves a closed-form normalizing constant, mean, and quantiles under its finite-rank prior.The functions capture covariate effects through a location-varying linear index and additional variation across locations; piecewise-linear interpolation on a regular grid defines the prior.Source: §1 · p. 5
• The grid values have Gaussian priors and are sampled by elliptical slice sampling, which requires no tuning parameters.The sampler targets the exact posterior under the finite-rank prior.Source: §1 · p. 5
• When the true log density is a sum of two bivariate functions, the posterior contracts at n^-α/(2α+2) up to a log factor, the minimax rate for an α-smooth bivariate density.The rate applies regardless of covariate dimension because each grid remains two-dimensional.Source: §1 · p. 5

• The paper concludes in Section 7, following its discussion of percentile curves along the corpus callosum with uncertainty.Source: §1 · p. 6
• The supplement provides closed-form calculations, proofs, and additional numerical details.Source: §1 · p. 6

2.1 Model specification

• The log density combines g1(y, w_x,s), capturing covariate effects through a location-varying index, with g2(y, s), capturing variation independent of covariates.The model assigns finite-rank Gaussian-process priors to the two components, represented by piecewise-linear interpolation on a regular grid.Source: §2.1 · pp. 6–7
• Separate components let the model state smoothness directly through their Gaussian-process priors and represent multimodal functions through the piecewise-linear structure.The range of the index is clamped to [0, 1] so g1 is defined on its domain.Source: §2.1 · pp. 7–8
• Piecewise-linear interpolation makes the likelihood, normalizing constant, mean, and quantiles available in closed form without further numerical approximation.The exponent is piecewise linear in the response.Source: §2.1 · p. 8
• The model parameter θ is not identifiable, so covariate effects are summarized through the conditional density fθ.The components share basis functions, allowing parameter changes that leave the density unchanged.Source: §2.1 · p. 8

2.2 Related and alternative models

• Location-varying LGP alternatives use numerical integration or score matching, while spLGP uses a covariate projection and has no location component.Orthogonal-series finite representations do not have a closed-form integral; ExFR-LGP is also compared with a full-grid LGP and logit stick-breaking mixtures.Source: §2.2 · pp. 8–9

3 Theoretical Properties

• The result assumes fixed p and K and a true log density represented by the model’s two bivariate functions and a bounded true index.The observations use independently sampled covariates and responses drawn conditionally from the true density.Source: §3 · p. 11
• n^-α/(2α+2) is the adaptive posterior contraction rate, matching the minimax rate for an α-smooth bivariate function up to a logarithmic factor.The rate is for Hellinger distance and does not require the prior grid size or κ distribution to depend on unknown smoothness α.Source: §3 · pp. 10–11
• The proof reduces contraction to the bivariate functions g1 and g2 and applies Gaussian-process contraction results.The paper also states that the same rate holds when p grows with n under a stated growth condition.Source: §3 · p. 12

4.2 Gibbs sampler

• The Gibbs sampler updates the Gaussian components with elliptical slice sampling, jointly updating the two density components because they are correlated a posteriori.The sampler also updates β by elliptical slice sampling; kernel scales are drawn after the component update.Source: §4 · p. 12, §4.2 · p. 14

4.1 Extended Exact Likelihood Finite-Rank LGP

• Ex2FR-LGP adds δ(y, x, s) to model departures from both the single-index and additive forms of ExFR-LGP.Setting δ to zero recovers ExFR-LGP, and the extended model retains closed-form integration, quantiles, and moments.Source: §4.1 · pp. 12–13

4.2 Gibbs sampler

• The sampler normalizes each β(s) to unit L2 norm, keeping the index within [0, 1] when covariate vectors have norm at most one.This addresses the non-identifiability between the scales of β(s) and g1.Source: §4.2 · p. 14
• Because covariates enter through the index and β rather than a grid over covariates, iteration cost grows linearly in p rather than exponentially.The model’s grids for g1 and g2 do not depend on p.Source: §4.2 · p. 14

Illustrative density estimates

• In Case 1, ExFR-LGP captures the true density and its 95% credible bands cover it; in misspecified Case 2, it recovers unimodal and bimodal shapes.In panel (f), competing methods depart from truth while the ExFR-LGP band covers it.Source: §5 · p. 17

Results

• ExFR-LGP error drops as n increases in both cases; in Case 1 it also drops as K increases.The authors relate this pattern to Theorem 3.1 and to more responses informing functions shared across locations.Source: §5 · p. 17

Uncertainty

• ExFR-LGP’s 95% credible band covers the truth more often than the other methods’ bands, but correlated tract responses can increase error and make bands too narrow.With correlated responses, predictive intervals retain nominal coverage while error grows and credible bands become narrower than actual uncertainty.Source: §5 · p. 17

Diagnosis and FA distribution

• The posterior median difference in conditional density medians between AD and CN individuals is negative at every location, consistent with the reported FA decline associated with Alzheimer’s disease.The comparison holds the other covariates at their overall averages.Source: §6 · p. 21

Covariate-adjusted reference bands

• Against a matched CN reference band, the AD individual’s observed FA falls below the band at locations 21–25 and remains in its lower part through location 50.The reference uses the AD individual’s sex, age, handedness, and education, with AD and MCI set to 0 and MMSE set to the CN group mean.Source: §6 · pp. 21–22, Figure 4 · p. 21
• The reference band provides a covariate-adjusted comparison for a new individual’s FA profile along the tract.The displayed 10th and 90th percentile estimates have pointwise 95% credible bands.Source: §6 · p. 21, Figure 4 · p. 21

Limitations

• Conditional independence across locations may understate posterior uncertainty when an individual’s responses are strongly correlated along the tract.Using the bands as a diagnostic tool would require validation against clinical outcomes.Source: §6 · p. 22

7 Conclusions

• ExFR-LGP represents the log density as two bivariate functions and uses grid-based linear interpolation to obtain a closed-form likelihood, mean, and quantiles.One function uses the response and a location-varying covariate index; the other uses the response and location.Source: §7 · p. 22
• Under the specified additive bivariate representation of the true log density, the posterior contracts at the minimax rate for an α-smooth bivariate function up to a logarithmic factor.Source: §7 · p. 22
• In simulations, ExFR-LGP was more accurate than full-grid LGP and spLGP at the same number of MCMC iterations, and comparable to or more accurate than LSBP.Source: §7 · p. 22
• In the ADNI application, ExFR-LGP produced covariate-adjusted fractional-anisotropy percentile bands along the corpus callosum.Source: §7 · p. 22
• Conditional independence across locations may understate posterior uncertainty when within-individual responses are strongly correlated, and the single-index structure limits each location's covariate effects to one linear combination.Source: §7 · p. 22
• The authors identify multiple-index extensions and Gaussian copula models as future directions, while clinical utility of the summaries requires validation against clinical outcomes.The copula direction is intended to capture within-individual dependence separately from covariate-adjusted FA distributions.Source: §7 · pp. 22–23

Supplementary Materials

• Supplement A gives the closed-form calculations, and Supplement B derives the detailed proof of Theorem 3.1.Source: Supplementary Materials · p. 29
• Supplement C presents the benchmark methods and evaluates credible-band coverage and dependence across locations.Source: Supplementary Materials · p. 29
• Supplement C also reports the posterior contrast in the ADNI data.Source: Supplementary Materials · p. 29

Supplement A: Closed form calculations

• For fixed covariates and location, the supplement starts by representing the response log-density exponent through grid knots and its values at those knots.On each grid interval, the exponent is linear, with a slope and intercept determined by adjacent knot values.Source: Supplement A: Closed form calculations · p. 29
• When the exponent contains an additional piecewise-linear term, the calculation uses the union of its knots and the original grid knots.If Nδ divides N, the union reduces to the original grid.Source: Supplement A: Closed form calculations · p. 29

A.1 Calculation of the normalizing constant

• The normalizing constant is calculated by integrating the exponential of the linear exponent over each grid cell and summing the cell contributions.The zero-slope term is evaluated by its limit.Source: §A.1 · pp. 29–30
• Subtracting the largest knot value before exponentiation and restoring it after taking the logarithm avoids overflow in the normalizing-constant calculation.Source: §A.1 · p. 30
• At most four coefficients contribute to each knot value, so evaluating the location-dependent term costs O(N) operations.The bound follows from at most two nonzero basis functions for each of the two terms.Source: §A.1 · p. 30

A.2 Calculation of Mean

• The conditional mean calculation uses integration by parts.Source: §A.2 · p. 30

A.3 Calculation of quantiles

• To find a quantile, the calculation locates the interval containing the target cumulative probability using partial sums of cell integrals.The selected interval is indexed by the largest partial sum not exceeding the target probability times the normalizing constant.Source: §A.3 · p. 30
• Within the selected interval, the quantile is obtained by solving the integral equation, with separate formulas for nonzero and zero slopes.Source: §A.3 · p. 30

Supplement B: Proof of Theorem 3.1

• The proof verifies the three conditions of Theorem 2.1 in Ghosal, Ghosh, and van der Vaart (2000) to establish the contraction result.Source: Supplement B: Proof of Theorem 3.1 · p. 31
• The proof bounds density distances using log-density error, decomposed into Gaussian-process approximation, grid interpolation, and covariate-index errors.The interpolation error must be small relative to the Gaussian process small-ball probability, which determines how quickly the grid size must grow.Source: Supplement B: Proof of Theorem 3.1 · p. 31
• For the proof's approximation construction, the covariates have norm at most one and the index coefficients are rescaled to have norm at most one.Source: Supplement B: Proof of Theorem 3.1 · p. 31

B.1 Reduction to three conditions

• The reduction treats the sampled observations as i.i.d. and expresses the log density using two interpolated Gaussian-process functions.Source: §B.1 · pp. 31, 33
• The sieve, prior-mass, and entropy conditions yield contraction for the joint density at rate Lεn, and hence contraction for the conditional density.The cited proof states Lεn tends to zero and applies the theorem's contraction result.Source: §B.1 · p. 32

B.2 Auxiliary lemmas

• For g(0) in C^α[0,1]^2, the prior small-ball negative log probability is at most A1κ^2{log(κ/ε)}^3, both near zero and near g(0) under the stated conditions.The bound holds for κ ≥ κ0 and ε ≤ ε0; the g(0)-centered bound additionally requires ε ≥ 2Cακ^-α.Source: §B.2 · p. 34
• The interpolation argument bounds ∥g1−G1∥∞ through increments of G1 between points at most 1/N apart, yielding the stated interpolation-error result (S10).The proof controls the supremum of these increments and applies Borell’s inequality.Source: §B.2 · pp. 36, 38

B.3 Choice of εn, κn and N

• The tuning uses N ≍ n and restricts κ to [κn, 2κn] for the KL condition, while the sieve takes 1 ≤ κ ≤ crκn.The passage states that the remaining proof uses three properties holding for all large n.Source: §B.3 · p. 38

B.4 Sieve condition

• For G_l, the prior probability of falling outside B_n is at most 2e^(-C n ε_n^2) for large n; taking C+2 in place of C gives (S5).The bound splits into Π(κ_l > r_n) plus sup over 1≤κ≤r_n of Π(G_κ ∉ B_n).Source: §B.4 · p. 40

B.5 Metric entropy condition

• The metric-entropy argument constructs nets for the Gaussian-process and coefficient components, transfers their approximation to Hellinger distance, and derives an entropy bound for Fn.The coefficient net uses Euclidean distance, while the process nets use the sup norm.Source: §B.5 · pp. 40–41

B.6 KL condition

• The KL-condition argument concludes that (S6) holds for all sufficiently large n.The supplied passage gives the conclusion but not the preceding KL bound.Source: §B.6 · p. 41
• The proof completes Theorem 3.1 by verifying (S7) from a lower bound on prior probability for the relevant event.Source: §B.6 · pp. 41–42

B.7 A growing number of covariates

• With bounded coordinatewise covariates, unit-ℓ1-norm true index coefficients, and fixed K, Corollary B.6 retains the same ε_n rate as p=p_n grows; for α=2, p_n=o{n^(1/3)(log n)^(5/3)}.Source: §B.7 · p. 43

C.1 Benchmark methods

• The benchmark full-grid LGP uses tensor-grid interpolation and an exactly computed normalizing constant; a single-index comparator omits g2 and uses the ExFR-LGP sampler budget.Source: §C.1 · p. 45

C.2 Coverage of the credible band

• Coverage averaged over settings was 0.92/0.87 for ExFR-LGP (Cases 1/2), versus 0.84/0.82 for LSBP, 0.58/0.55 for spLGP, and 0.22/0.30 for full-grid LGP.Coverage is the probability that a band contains the true density at a response drawn from that density, averaged over test individuals and locations.Source: §C.2 · p. 46

C.3 Dependence across locations

• The experiment fits ExFR-LGP to 50 datasets at each of ρ = 0, 0.5, and 0.9, with n = 250 and K = 20.Responses are generated with correlation ρ^|k−k′| across locations while retaining the target density at each location.Source: §C.3 · p. 46
• Prediction intervals retain nominal coverage at every ρ, but credible bands become narrower than actual uncertainty as dependence rises.The model treats one individual's location-specific responses as independent.Source: §C.3 · pp. 46–47
• As ρ increases, prediction error grows because correlated responses carry less information than independent responses.Source: §C.3 · p. 46

C.4 Posterior contrast in the ADNI data

• −0.086 to −0.007: the posterior median AD-minus-CN contrast is negative at all 83 tract locations.MMSE and other covariates are held at their overall averages.Source: §C.4 · p. 47
• The five training-set fits follow the full-data fit; only one changes sign, at three locations near location 70.Each training set leaves out one fifth of the 213 individuals.Source: §C.4 · p. 47, Figure C.1 · p. 47
• The negative contrast accords with the reported decline of fractional anisotropy associated with Alzheimer’s disease.Source: §C.4 · p. 47