Source-linked AI summary
Fisher information distance: a geometrical reading
Sueli I. R. Costa, Sandra A. Santos, João E. Strapasson
TL;DR
The paper asks how Fisher distance can be interpreted and applied as a measure of distributional dissimilarity and a basis for data averages. It studies normal-distribution models geometrically, using hyperbolic geometry to derive closed forms and relate Fisher distance to Kullback–Leibler divergence. The analysis yields explicit geometric results for several normal-distribution cases while leaving general Gaussian distance without a closed form.
Problem
Applications need measures of distributional dissimilarity and proper averages or centroids, motivating broader interpretations of Fisher distance and its geometry.
Method
The paper analyzes Fisher information geometry for univariate and multivariate normal distributions through classical hyperbolic geometry and derives special-case distance expressions.
Results
The geometric treatment produces closed Fisher-distance expressions for several normal-distribution cases and establishes connections with Kullback–Leibler divergence.
Takeaways & Limitations
Fisher distance admits useful geometric interpretations for normal-distribution models, supporting applications that require distributional comparison or representative averages.
Takeaways & Limitations
For general multivariate normal distributions with arbitrary symmetric positive definite covariance matrices, the authors could not derive a general closed-form Fisher distance and numerical approaches are required.
Abstract
from arXiv · showhide
This paper is a strongly geometrical approach to the Fisher distance, which is a measure of dissimilarity between two probability distribution functions. The Fisher distance, as well as other divergence measures, are also used in many applications to establish a proper data average. The main purpose is to widen the range of possible interpretations and relations of the Fisher distance and its associated geometry for the prospective applications. It focuses on statistical models of the normal probability distribution functions and takes advantage of the connection with the classical hyperbolic geometry to derive closed forms for the Fisher distance in several cases. Connections with the well-known Kullback-Leibler divergence measure are also devised.
1 Introduction
The paper situates Fisher distance within information geometry and applications requiring distributional dissimilarity or representative averages. It presents a geometric treatment of normal-distribution models, using hyperbolic geometry to derive closed forms and connect Fisher distance with Kullback–Leibler divergence.
- Information geometry equips probability-distribution manifolds with a Riemannian metric defined by the Fisher information matrix.The framework supports analysis across statistical inference, information theory, mathematical programming, and neurocomputing.
- Fisher-matrix geometry also connects information-theoretic quantities with geometric properties of typical sets, linking entropy and surface-area arguments to geometric inequalities.
- Applications use Fisher distance and related divergences to measure distributional dissimilarity and construct data averages or centroids.These representatives can serve as inputs to distance-geometry problems, while related work applies Fisher–Rao geometry to clustering, image processing, dimensionality reduction, and sampling.
- The paper develops a geometric reading of Fisher information for univariate and multivariate normal distributions to broaden interpretations for prospective applications.Its approach uses classical hyperbolic geometry to derive closed Fisher-distance expressions in special multivariate-normal cases and deduces connections with other dissimilarity measures.
- The paper proceeds from univariate Gaussian geometry to multivariate normal models, deriving closed forms in common parameters and relationships with Kullback–Leibler divergence.The univariate results are associated with equations (9)–(14) and Figures 6–7.
2 Univariate normal distributions: a geometrical view
The paper models univariate normal distributions in a Fisher-metric half-plane and uses its correspondence with hyperbolic geometry to derive distances, geodesics, averages, and relations to Kullback–Leibler divergence.
- 2.1 The hyperbolic model of the mean × standard deviation half-plane: Fisher circles are ellipses with the same eccentricity whose centers lie below their Euclidean centers, and equidistant Gaussian pairs can occur at different parameter heights.Figure 2 illustrates equal-distance geodesic endpoints, while Figure 4 gives an equal-distance value of 2.37687.
- 2.1 The hyperbolic model of the mean × standard deviation half-plane: Closed expressions for Fisher distance are obtained from hyperbolic distance formulas, including separate cases for vertically aligned and nonaligned points.The construction uses the logarithmic cross-ratio expression for Poincaré distance.
- 2.1 The hyperbolic model of the mean × standard deviation half-plane: The Fisher metric makes the mean × standard deviation half-plane hyperbolic, with geodesics represented by vertical lines or half-ellipses.This geometry follows from the correspondence between the Fisher and Poincaré half-planes.
- 2.1 The hyperbolic model of the mean × standard deviation half-plane: The Fisher metric defines an average distribution as the midpoint of a geodesic segment, equidistant from the two endpoint distributions.The construction is illustrated for A = (1.5, .75) and Q = (1.0610, 0.1646), yielding M = (1.1400, 0.3711).
- 2.2 Univariate normal distributions described in other usual parameters: Changing normal-distribution coordinates changes the plotted geodesic shape: half-ellipses occur in classic parameters, parabolas in source and expectation parameters, and half-hyperbolas in natural parameters.Figure 6 compares classic, source, natural, and expectation parameterizations.
- 2.3 The Kullback-Leibler divergence and the Fisher distance: For univariate normals, symmetrized Kullback–Leibler divergence approaches Fisher distance when the distributions become close in parameter space.The paper states that this local agreement also holds for multivariate normal distributions as one parameter approaches the other.
3 Fisher information geometry of multivariate normal distributions
The paper develops Fisher information geometry for multivariate normal distributions, using hyperbolic geometry to obtain closed-form distances for round and diagonal covariance cases. General covariance matrices are substantially more complex, and a general closed form is unavailable.
- 3.1 Round Gaussian distributions: Closed-form Fisher distances are derived for round Gaussian distributions, whose parameter space is identified with a hyperbolic half-space.The corresponding geodesics are lines or half ellipses orthogonal to the boundary hyperplane.
- 3.2 Diagonal Gaussian distributions: The diagonal-covariance model has a product metric on IH2p, yielding a closed-form Fisher distance for independent multivariate normal distributions.Its metric has constant negative mean curvature equal to −1/[2(2p−1)].
- 3 Fisher information geometry of multivariate normal distributions: The round and diagonal models induce hyperbolic metrics with constant negative mean curvature equal to −1/[p(p+1)] and −1/[2(2p−1)], respectively.These expressions and related geometric properties follow from Poincaré models and products of Riemannian manifolds.
- 3.3 General Gaussian distributions: For general multivariate normal distributions with arbitrary positive definite covariance matrices, the parameter space has nonconstant sectional curvatures and remains far from fully developed.The bivariate case can be represented using eigenvalues and an eigenvector turning angle, producing rotated elliptical level sets.
- 3.3 General Gaussian distributions: A general closed form for the Fisher distance is unavailable, so numerical estimates, symmetrized Kullback-Leibler approximations, or bounds from isometric embeddings are used.For fixed means, the distance can be expressed through eigenvalues; restricting to diagonal covariances recovers the corresponding restricted metric.
4 Final remarks
The paper presents a geometrical view of Fisher distance for normal distributions, using hyperbolic geometry to obtain closed forms and connect the measure with Kullback–Leibler divergence.
- Closed forms for Fisher distance were derived for special instances of multivariate normal distributions.
- For univariate Gaussian models, hyperbolic geometry yields closed Fisher-distance forms in commonly used parameters.
- The analysis also derives a relationship between Fisher distance and the Kullback–Leibler divergence measure.