Source-linked AI summary
Simulation of the matrix Bingham-von Mises-Fisher distribution, with applications to multivariate and relational data
Peter Hoff
TL;DR
The paper addresses the need for computational tools for distributions on orthonormal matrices arising in multivariate and relational models. It develops rejection and Gibbs sampling algorithms for matrix von Mises-Fisher and Bingham-von Mises-Fisher distributions, then applies them to protein-interaction network data. The distributions also arise as conditional posterior distributions in Gaussian and probit models for multivariate data.
Problem
Higher-dimensional manifolds require statistical and computational tools for analysis and matrix decomposition, including models for multivariate and relational data.
Method
The paper develops rejection sampling for matrix von Mises-Fisher distributions and Gibbs sampling schemes for matrix Bingham-von Mises-Fisher distributions.
Results
Members of the Bingham-von Mises-Fisher family arise as conditional posterior distributions in Gaussian and probit models for multivariate data.
Takeaways & Limitations
The sampling schemes provide a tool for studying these multivariate probability distributions and support Bayesian inference and estimation.
Takeaways & Limitations
The Gibbs chain is reducible when m = R, whereas irreducibility and convergence hold for m > R.
Abstract
from arXiv · showhide
Orthonormal matrices play an important role in reduced-rank matrix approximations and the analysis of matrix-valued data. A matrix Bingham-von Mises-Fisher distribution is a probability distribution on the set of orthonormal matrices that includes linear and quadratic terms, and arises as a posterior distribution in latent factor models for multivariate and relational data. This article describes rejection and Gibbs sampling algorithms for sampling from this family of distributions, and illustrates their use in the analysis of a protein-protein interaction network.
1 Introduction
The paper motivates distributions on orthonormal matrices for matrix decomposition and uncertainty modeling, especially in multivariate and relational data. It develops rejection and Gibbs samplers for matrix von Mises-Fisher and Bingham-von Mises-Fisher distributions, with a network-data application.
- Higher-dimensional manifolds require statistical and computational tools for analysis and matrix decomposition.
- Probability models for low-rank orthonormal eigenvector matrices describe variability and uncertainty in matrix-variate data.
- Factor analysis: In factor analysis, orthonormal matrices U and V support latent-factor representations and have matrix von Mises-Fisher conditional posteriors sampled iteratively for joint inference.
- Network data: For network data, a latent eigenvalue model with orthonormal U yields a Bingham distribution under a uniform prior, with parameters A = Z/2 and B = Λ.
- The article presents rejection sampling for matrix von Mises-Fisher distributions and Gibbs sampling for cases where rejection sampling is infeasible.
- A Gibbs algorithm constructs a Markov chain on VR,m whose samples converge in distribution to the matrix Bingham-von Mises-Fisher distribution, and the methods are applied to a 270-protein interaction network.
2 Sampling from the von Mises-Fisher distribution
The section develops rejection and Gibbs samplers for matrix von Mises-Fisher distributions on orthonormal matrices. The rejection method is broadly feasible but can become inefficient under strong concentration or unfavorable dimensional ratios, while Gibbs sampling provides a convergent alternative with boundary cases requiring modification.
- 2.1 A rejection sampling scheme: The rejection sampler constructs an envelope from sequential vector von Mises-Fisher densities while preserving orthogonality among matrix columns.Each column after the first is sampled in the null space of the preceding columns.
- 2.1 A rejection sampling scheme: For non-orthogonal concentration matrix C, the method uses C = UDV^T, samples Y with orthogonal concentration H = UD, and sets X = YV^T.This reparameterization avoids the poor proposal alignment caused by non-orthogonal concentration directions.
- 2.1 A rejection sampling scheme: The simulation study covers m ∈ {10, 20, 200}, R ∈ {2, 4, 6}, and singular values m/2, m, or 2m; results indicate broad feasibility.One hundred samples were generated for each matrix configuration, with average rejected samples recorded in Table 1.
- 2.1 A rejection sampling scheme: Average rejected samples generally increase with the magnitude of D and the ratio R/m.Large D or R can make rejection sampling prohibitively slow because the proposal ratio is rarely near its bound.
- 2.2 A Gibbs sampling scheme: The Gibbs sampler updates each column from its full conditional vector von Mises-Fisher distribution within the null space of the remaining columns.These updates generate a reversible aperiodic Markov chain.
- 2.2 A Gibbs sampling scheme: When m > R, the Gibbs chain is irreducible and converges in distribution to MF(C), but when m = R it is reducible.For m = R, single-column updates leave samples fixed up to column-wise multiplication by ±1; sampling multiple columns remedies this.
3 Sampling from the Bingham-von Mises-Fisher distribution
This section develops Markov chain Monte Carlo methods for the Bingham-von Mises-Fisher distribution by iteratively sampling constrained vector or matrix components. The resulting chains converge to the target distribution under stated dimensional conditions, with multiple-column updates addressing reducibility when m = R.
- The method iteratively resamples each column of X from its full conditional distribution to generate a Markov chain targeting pBMF.
- The vector Gibbs sampler updates constrained coordinates through θ and q, with sign variables handled separately for the BMF density.
- For a column update, the remaining columns define a null space whose orthonormal basis transforms the conditional problem into a vector BMF distribution.
- Iteration produces a reversible, aperiodic chain that is irreducible for m > R and converges in distribution to BMF(A, B, C).
- When m = R, single-column updates are reducible because the relevant null space is one dimensional, so samples remain fixed up to column-wise sign changes.
- The remedy for m = R is to sample multiple columns jointly, reducing the update to a two-dimensional orthogonal problem parameterized by an angle and a sign.
4 Example: Eigenmodel estimation for network data
The paper applies its Bingham-von Mises-Fisher sampling scheme to a latent eigenmodel for a binary protein-interaction network. Two Gibbs chains agree, and the fitted model captures both hub structure and bipartite-like subnetworks.
- The protein-interaction data comprise pairwise measurements among m = 270 essential E. Coli proteins, with yi,j = 1 indicating binding.
- The latent eigenmodel represents protein effects as the rows of an m × R orthonormal matrix U, avoiding scale confounding from normal latent factors.
- Posterior inference jointly Gibbs-samples U, Λ, Z, and c from their full conditional distributions given the observed network Y.
- Two independent length-110,000 Gibbs chains converged to the same parameter-space region after a few thousand iterations.
- One negative and two positive eigenvalues were indicated by the sampled eigenvalue sequences.
- The first three eigenvectors explained 99.95 percent of the posterior mean matrix's sum of squares.
- The model identifies opposite-sign third-coordinate groups that preferentially interact across groups, representing subnetworks resembling bipartite graphs.
5 Discussion
The discussion presents the matrix Bingham-von Mises-Fisher family as important for distributions on the Stiefel manifold and develops sampling tools for studying it. It also shows that BMF distributions arise as conditional posteriors in multivariate Gaussian and probit models.
- The matrix Bingham-von Mises-Fisher family generalizes vector- and matrix-valued von Mises-Fisher and Bingham distributions.
- The article develops rejection sampling for the matrix von Mises-Fisher distribution and Gibbs sampling for the matrix Bingham-von Mises-Fisher distribution.
- These sampling schemes provide a tool for studying complicated multivariate probability distributions and support parameter estimation in otherwise non-standard situations.
- BMF family members arise as conditional posterior distributions in Gaussian and probit models for multivariate data.
- R-functions implementing the article’s sampling schemes are available.