PaperlayerView exact revision on arXiv

Source-linked AI summary

Nonparametric quantile inference using Dirichlet processes

Nils Lid Hjort, Sonia Petrone

arXiv:2608.08355v1math.ST

TL;DR

The chapter addresses nonparametric Bayesian inference for quantiles and related quantities under limited distributional assumptions. It uses Dirichlet-process priors to characterize posterior quantile processes and derive explicit estimators, with extensions to density, inequality, comparison, and regression quantities. The resulting quantile estimator is smooth, and the framework also provides an automatic density estimator and asymptotic posterior theory.

  • Problem

    The chapter studies nonparametric Bayesian inference for quantiles and related quantile-based quantities when the underlying distribution is unknown.

  • Method

    It places a Dirichlet process prior on the unknown distribution and derives posterior quantile distributions and estimators, extending them to density, inequality, comparison, and regression functions.

  • Results

    The posterior quantile distribution, mean, variance, and covariance are characterized explicitly, including a smooth quantile estimator and an automatic density estimator supported on the data range.

  • Takeaways & Limitations

    The framework supplies directly computable Bayesian procedures for quantile inference and several related quantities across one-sample, two-sample, inequality, and covariate settings.

  • Takeaways & Limitations

    When the Dirichlet prior strength grows proportionally with n, frequentist and Bayesian schemes do not agree asymptotically.

Abstract

from arXiv · show

This chapter deals with nonparametric inference for quantiles from a Bayesian perspective, using the Dirichlet process. The posterior distribution for quantiles is characterised, enabling also explicit formulae for posterior mean and variance. Unlike the Bayes estimator for the distribution function, our Bayes estimator for the quantile function is a smooth curve. A Bernstein--von Mises type theorem is given, exhibiting the limiting posterior distribution of the quantile process. Links to kernel-smoothed quantile estimators are provided. As a side product we develop an automatic nonparametric density estimator, free of smoothing parameters, with support exactly matching that of the data range. Nonparametric Bayes estimators are also provided for other quantile-related quantities, including the Lorenz curve and the Gini index, for Doksum's shift curve and for Parzen's comparison distribution in two-sample situations, and finally for the quantile regression function in situations with covariates.

1. Introduction and summary

The chapter develops Bayesian nonparametric inference for quantiles using a Dirichlet process, extending the framework to related quantile-based quantities and covariate settings. It derives explicit posterior calculations, smooth quantile estimators, an automatic density estimator, and Bayesian procedures for comparison and inequality measures.

  • The chapter develops Bayesian inference for quantiles and related quantities from a Dirichlet process prior for the unknown distribution.
  • Explicit posterior distributions, means, variances, and covariances are derived for one or more quantiles.
  • The posterior mean quantile function is a smooth curve, unlike the jump-valued traditional Bayes estimator of the distribution function.
  • A non-informative prior limit yields a Bernstein-type smoothed quantile estimator, while density estimation is obtained by operations on the posterior quantile estimator.
  • The framework supplies nonparametric Bayes estimators for the Lorenz curve, Gini index, Doksum’s shift curve, Parzen’s comparison distribution, and quantile regression.

2. The quantile process of a Dirichlet

This section characterizes prior and posterior distributions for Dirichlet-process random quantiles, including their joint behavior. As the prior strength tends to zero, posterior quantiles concentrate on observed data points with explicit discrete probabilities.

  • A Dirichlet process with parameter aF0 induces random quantiles whose marginal distribution is expressed through Beta distribution functions.
  • The prior quantile distribution satisfies H0,a(x) = Ja(F0(x)), transferring the uniform-quantile distribution through F0.
  • Joint distributions of multiple quantiles are determined by Dirichlet probabilities over intervals between their evaluation points.
  • After observing data, the posterior remains Dirichlet with updated parameter aF0 + nFn, yielding explicit continuous densities between order statistics and point masses at data points.
  • When a tends to zero, posterior quantiles have no mass between observations and their joint selection follows binomial, trinomial, or multinomial structures.
  • For fixed positive a, the posterior mass at observed data points converges to one as n increases.

3. Bayesian quantile inference

The chapter develops directly computable Bayesian estimators for quantile functions and related posterior uncertainty. The non-informative limit produces a smooth order-statistic estimator connected asymptotically to kernel quantile estimation, while posterior bands admit frequentist interpretations.

  • The posterior mean quantile function is the Bayes estimator under quadratic loss, with explicit finiteness conditions and formulas.
  • The posterior mean estimator is increasingly governed by data as n grows, even when the prior strength a is moderate or large.
  • Letting a tend to zero yields a smooth polynomial quantile estimator that climbs from x(1) to x(n).
  • Posterior variances and covariances can be computed explicitly, and bQ0(y) ± 1.96 bV0(y)^1/2 gives asymptotic pointwise 95% confidence and credibility bands.
  • The non-informative estimator is asymptotically equivalent to a kernel quantile estimator using the standard normal kernel and bandwidth {y(1 −y)/n}^1/2.

4. Quantile density and probability density estimators

The chapter derives Bayesian estimators for quantile and probability densities from the Dirichlet-process quantile estimator. In the non-informative case, the resulting density estimator is smooth, automatic, supported on the exact data range, and requires no smoothing parameter.

  • Bayes estimators: The quantile-density and probability-density estimators are obtained by differentiating and inverting the Bayesian quantile estimator.The density estimator bfa is formed from bQa by solving bQa(y) = x and differentiating the resulting bFa.
  • Automatic density estimation: The non-informative estimator bf0 has no smoothing parameter, with inherent smoothing of about {y(1 −y)/n}^1/2.Its smoothing arises from the limiting Dirichlet-process prior rather than a user-selected bandwidth.
  • Bayes estimators: The estimator bfa forms a continuous bridge from the prior density f0 at large a to a genuinely nonparametric, prior-independent estimator at a = 0.The tuning parameter a controls the transition between prior information and the data-driven limit.
  • Automatic density estimation: bf0 is strictly positive on and supported by the exact data range [x(1), x(n)], with unit integral.The construction is obtained by numerically solving bQ0(y) = x for y and differentiating bF0(x).
  • Comparison with histograms: The resulting density estimator can provide a smoother data descriptor than a histogram while avoiding smoothing-parameter selection.The chapter illustrates this behavior with a standard-normal sample of n = 100 data points.
  • Comparison with kernel estimators: Compared with traditional kernel methods, bf0 smooths less, using a locally varying bandwidth of order O(n^-1/2) rather than O(n^-1/5).The traditional comparison assumes two derivatives of the underlying density, whereas bf0 is constructed directly from the data without further smoothness assumptions.

5. The Lorenz curve and the Gini index

The chapter develops nonparametric Bayesian inference for the Lorenz curve and Gini index through the quantile function. Bayes estimators and posterior simulation-based credibility intervals are provided for these inequality summaries.

  • Motivation: The Lorenz curve and Gini index are treated as quantile-related summaries in nonparametric Bayesian inference.The application is motivated by econometric studies of income distributions, where these quantities are used to quantify and compare distributions.
  • Lorenz curve: For a distribution supported on the positive halfline, the Lorenz curve summarizes cumulative income share across population quantiles.It is convex and equals the diagonal only when the underlying distribution is concentrated at a single point.
  • Lorenz curve: Bayesian inference for the Lorenz curve can be performed by simulating quantile curves from the posterior distribution.A natural Bayes estimator is constructed after transforming the quantile-scale solution to the Lorenz scale.
  • Gini index: With a Dirichlet prior and weighted integrated squared-error loss for the quantile function, the chapter obtains Bayes estimators of the Gini index.The non-informative limiting version is singled out as particularly interesting.
  • Gini index: Credibility intervals for the Gini index can be obtained through posterior simulation of Lorenz curves.The Gini index measures the Lorenz curve’s closeness to the diagonal, representing the egalitarian case.

6. Doksum’s shift and Parzen’s comparison

The chapter extends Dirichlet-process Bayesian quantile inference to two-sample comparisons through Doksum’s shift function and Parzen’s comparison distribution. The resulting estimators have interpretable credibility bands and asymptotic coverage properties.

  • Doksum’s shift function: Doksum’s shift function satisfies X + D(X) having the same distribution as X′ and supports comparisons between control and treatment groups.A constant shift corresponds to a location translation, while a linear shift corresponds to a location-and-scale translation.
  • Doksum’s shift function: Bayesian inference for the shift function begins with independent Dirichlet-process priors for the two distributions and yields a posterior distribution evaluable by numerical integration.The posterior at fixed x is expressed through the event F(x) ≤ G(x + t), with Beta distributions used for computation.
  • Uncertainty quantification: The shift-function credibility band has pointwise coverage converging to 90% as the sample sizes grow.This follows from the chapter’s asymptotic theory and applies to the displayed band based on the posterior mean and variance.
  • Doksum’s shift function: For the guinea-pig data, the Bayes shift estimate is quite close to Doksum’s direct estimate, with an approximate pointwise 90% credibility band.The analysis uses 65 control-group and 60 treatment-group animals.
  • Doksum’s shift function: The shift-function band nearly contains a linear curve for the Doksum–Bjerkedal data, indicating a location-and-scale translation.The chapter notes that prior models can link the two distributions for such scenarios instead of assuming prior independence.
  • Parzen’s comparison distribution: The Bayesian estimator of Parzen’s comparison distribution is smoother than the direct nonparametric estimator and is asymptotically equivalent to it.Its simple credibility band reaches 95% coverage in both frequentist and Bayesian settings as sample sizes grow.

7. Large-sample analysis

The chapter establishes large-sample frequentist and posterior limits for Dirichlet-process distribution and quantile estimators. Under regularity conditions, the posterior quantile process has a Brownian-bridge limit, while prior strength determines when Bayesian and frequentist procedures agree.

  • Distribution-process limits: The empirical distribution, posterior mean, and posterior process converge to Brownian-bridge limits under the Dirichlet-process model.Proposition 3 gives the distribution-process results in Skorokhod topology.
  • Prior-strength boundary: The conclusions remain valid when a/√n →0, but for a = cn the posterior mean converges to a prior-data mixture and Bayesian and frequentist schemes no longer agree asymptotically.The mixture is F∞= (c/(c+1))F0+ (1/(c + 1))Ftr.
  • Quantile-process limit: For a true distribution with positive continuous density, the posterior quantile process converges almost surely to qtr(y)W 0(y) on D[ε, 1 −ε].Here qtr(y) is the true quantile-density function and ε excludes the extreme quantiles.
  • Estimator equivalence: The direct empirical quantiles and the non-informative Bayesian quantile estimator become asymptotically equivalent at the √n scale.The result is stated for bQ0 and F_n^-1, and extends to Bayes estimators based on different Dirichlet-process priors.
  • Posterior uncertainty: The posterior variance converges to the variance of qtrW 0, and for a = 0, n bV0(y) converges to qtr(y)^2y(1 −y).These limits support confidence bands with correct limiting pointwise coverage.
  • Two-sample extensions: The two-sample shift and comparison procedures also have Brownian-bridge limits that support pointwise and simultaneous confidence bands.The results cover Doksum’s shift function and Parzen’s comparison distribution, including the equal-distribution case.

8. Quantile regression

The chapter extends Dirichlet-process quantile inference to regression with covariates by combining a parametric location-scale model with a Dirichlet-process residual distribution. Posterior averaging yields estimators for conditional quantiles at new covariate values.

  • Model: Conditional quantile regression assumes Yi = βtxi + σεi, with unknown regression parameters and a scaled residual distribution F.For covariate x, the conditional distribution is F((t −βtx)/σ).
  • Prior and posterior: The prior places an independent density prior on (β, σ) and a Dirichlet-process prior Dir(aF0) on the residual distribution.The posterior is derived jointly for β, σ, and F.
  • Residual quantiles: Given β and σ, the residual quantile process has a Dirichlet posterior updated by standardized residual point masses.The posterior mean bQa(u | β, σ) follows from the earlier quantile-process formulae.
  • Conditional quantile estimator: For each covariate value x0, posterior averaging over β produces an estimator of the conditional quantile function Q(u | x0).The construction uses the posterior mean of β and, in the non-informative limit, a corresponding bQ0 estimator.
  • Computation: A simple implementation samples β from its posterior, sorts the corresponding residual-adjusted responses, and averages the resulting order statistics.This produces posterior means for the order statistics and then for bQ0(u | x0).

9. Concluding remarks

The concluding remarks describe extensions, invariance properties, and practical refinements of the Dirichlet-process quantile framework. They emphasize explicit characterisations, broader prior classes, and extensions to two-sample and other quantile-derived quantities.

  • Other priors: The chapter’s explicit formulae and characterisations partly reduce the simulation required by general quantile-pyramid approaches.The Dirichlet process is presented as a special case of quantile-pyramid constructions.
  • Invariance property: The limiting Bayes rule for quantiles remains unchanged for wider prior classes because extra location-scale parameters cancel from the posterior mean.The argument is stated for monotone transformations Xi = aθ(Zi) with a prior on (θ, G).
  • Invariance property: For scale models Xi = θZi, the scale information is irrelevant to Q(y) = θQG(y) when a is small, supporting the Lorenz and Gini estimators.This connects the invariance argument to the estimators developed for Lorenz-curve and Gini-index inference.
  • Boundary behaviour: The chapter notes that posterior quantile-process results are less direct near y = 0 and y = 1, where alternative methods can provide more informative boundary results.The stated convergence works on D[ε, 1 −ε] for fixed small ε.
  • Simultaneous bands: Simultaneous Bayesian bands for the Doksum shift function can be constructed by simulating posterior shift-function curves and calibrating limiting-process quantiles.The chapter contrasts these with established frequentist simultaneous bands.
  • Further extensions: The chapter also identifies total-time-on-test statistics and structured two-sample priors as further directions for quantile-based inference.Location and location-scale translations motivate informative priors for two-sample comparison problems.

Appendix: various proofs

The appendix supplies proof strategies and finiteness conditions underlying the Dirichlet-process quantile results. It uses Dirichlet decompositions, partial integration, and tail-integrability arguments to establish approximation and existence claims.

  • Beta identities: Partial integration and the Fubini theorem relate Beta cumulative probabilities and densities used in the appendix’s posterior-moment calculations.These identities justify the equivalent integral representations used earlier in the chapter.
  • Proof decompositions: A Dirichlet decomposition separates posterior point masses at observations from probability between observations, whose total mass becomes small as n increases.With high probability, the between-data mass is at most a/√n, making the random quantile close to the empirical quantile.
  • Existence conditions: The posterior mean of a quantile exists only when the relevant extreme-tail integrals are finite.The appendix identifies the first and last integrals outside the observed data range as the critical terms.
  • Existence conditions: For a Dirichlet-process prior, posterior quantile-mean finiteness is characterized through integrability conditions involving the prior distribution’s tails.The conditions apply for any sample size, including the no-sample prior case.
Loading 2608.08355v1…