Source-linked AI summary
Hyperbolic Image Embeddings
Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan Oseledets, Victor Lempitsky
TL;DR
Computer vision commonly uses Euclidean or spherical embeddings, while many visual tasks contain hierarchical relationships that those geometries may not represent naturally. The paper adapts hyperbolic neural-network layers to vision models and finds that hyperbolic embeddings often improve performance, especially in few-shot classification, while noting task-, dataset-, and numerical-precision dependence.
Problem
Computer vision tasks use Euclidean or spherical embeddings despite hierarchical relationships arising in retrieval, recognition, and other visual data.
Method
The paper adapts hyperbolic neural-network layers to standard vision architectures and evaluates dataset hyperbolicity using Gromov δ-hyperbolicity.
Results
Across a number of tasks, especially few-shot image classification, hyperbolic embeddings can produce a substantial accuracy boost; hyperbolic ProtoNet outperforms its Euclidean counterpart by more than 10% in both settings.
Takeaways & Limitations
Hyperbolic embeddings may better conform to the intrinsic geometry of some image manifolds with hierarchical structure.
Takeaways & Limitations
Benefits vary substantially across tasks and datasets, and fixed-precision arithmetic can create numerical-precision effects between hyperbolic geometry models.
Abstract
from arXiv · showhide
Computer vision tasks such as image classification, image retrieval and few-shot learning are currently dominated by Euclidean and spherical embeddings, so that the final decisions about class belongings or the degree of similarity are made using linear hyperplanes, Euclidean distances, or spherical geodesic distances (cosine similarity). In this work, we demonstrate that in many practical scenarios hyperbolic embeddings provide a better alternative.
1. Introduction
The paper argues that hierarchical relationships are common in visual data and motivates hyperbolic embeddings as a geometry suited to represent them. It applies hyperbolic neural-network machinery to vision tasks and reports benefits across several settings.
- Motivation: Visual hierarchies arise in retrieval from whole images to detail fragments and in recognition when degraded images become ambiguous across identities or classes.Composite images can similarly relate to images containing class representatives in isolation.
- Motivation: Hyperbolic spaces can represent hierarchical visual relationships because their volume expands exponentially, unlike the polynomial expansion of Euclidean spaces.The paper presents hyperbolic spaces as continuous analogues of trees and therefore plausible models for hierarchical visual data.
- Approach: The authors adapt hyperbolic neural-network layers to standard image-classification and few-shot-learning architectures.The paper also studies image classification, one-shot and few-shot learning, and person re-identification.
- Contributions: Experiments on few-shot learning and person re-identification models and datasets demonstrate that hyperbolic embeddings are beneficial for visual data.The paper frames these experiments as its first main contribution.
- Contributions: The paper proposes evaluating dataset hyperbolicity with Gromov δ-hyperbolicity and using it to estimate a suitable Poincaré-disk radius.This metric is intended as a practical tool for practitioners.
2. Related work
The paper connects hyperbolic embeddings to hierarchical visual data and adapts hyperbolic neural-network components for computer-vision tasks. It also introduces δ-hyperbolicity to assess datasets and estimate suitable Poincaré-disk radii.
- Hyperbolic architectures: Hyperbolic neural-network layers allow standard image-classification and few-shot architectures to operate on hyperbolic embeddings.The paper modifies existing architectures by adding hyperbolic layers near the network output.
- Embedding geometry: In hybrid models, distance from the Poincaré-ball origin tends to place generic or ambiguous objects nearer the center and specific objects nearer the boundary.The authors use distance to the origin as an uncertainty estimate and center distance as a confidence measure.
- Hyperbolicity estimation: The paper estimates dataset hyperbolicity with the scale-invariant metric δrel(X)=2δ(X)/diam(X), using batched samples and repeated runs.The metric lies in [0,1], with values closer to 0 indicating stronger hyperbolicity.
- Empirical findings: Image datasets show δrel values significantly closer to 0 than to 1, suggesting that visual tasks can benefit from hyperbolic representations.For previously studied datasets, the radius formula estimates c∼0.33, although c=0.05 sometimes performs better.
4. Hyperbolic operations
Hyperbolic networks replace ordinary vector operations with geometry-aware mappings and aggregation in the Poincaré ball. The curvature parameter c controls the transition between hyperbolic and Euclidean behavior, while numerical clipping stabilizes training.
- 4. Hyperbolic operations: Hyperbolic neural networks use Möbius gyrovector operations because hyperbolic spaces do not support standard vector-space arithmetic.These operations provide generalized addition and other building blocks for hyperbolic networks.
- 4. Hyperbolic operations: The curvature parameter c balances hyperbolic and Euclidean geometries, with c → 0 recovering the usual Euclidean formulas.The same curvature parameter defines the ball constraint c∥x∥^2 < 1.
- 4. Hyperbolic operations: Exponential and logarithmic maps provide a bijection between Euclidean vectors in R^n and points in the hyperbolic ball.The maps support transitions between Euclidean and Poincaré-ball representations.
- 4. Hyperbolic operations: Hyperbolic averaging maps Poincaré-ball points to Klein coordinates, computes an Einstein midpoint, and maps the result back.This generalizes feature-vector averaging used in image processing and prototypical networks.
- 4. Hyperbolic operations: After the exponential map, norm clipping constrains representations to 1/√c(1 − 10^-3) for numerical stability.The clipping is an implementation measure used to stabilize training.
5. Experiments
The experiments evaluate hyperbolic embeddings for uncertainty estimation and few-shot classification across MNIST, Omniglot, MiniImageNet, and CUB. Hyperbolic ProtoNet improves substantially over Euclidean ProtoNet on MiniImageNet and CUB.
- Distance to the origin as the measure of uncertainty: Distances to the Poincaré-ball origin distinguish MNIST from Omniglot embeddings in three of four tested dimensions better than Euclidean maximum class probability.The MNIST-trained hyperbolic classifier is evaluated on both datasets, and the distributions are compared using Kolmogorov-Smirnov distances.
- Distance to the origin as the measure of uncertainty: Unclear MNIST images cluster near the Poincaré-ball center, whereas easily classified images lie closer to the boundary.This supports using distance to the origin as a confidence-related signal.
- Few-shot classification: Hyperbolic ProtoNet is evaluated on 1-shot and 5-shot few-shot classification tasks using MiniImageNet and CUB.The experiments compare hyperbolic and standard prototypical networks across these datasets and task settings.
- Few-shot classification: Hyperbolic ProtoNet significantly improves accuracy over standard ProtoNet on MiniImageNet, especially in the one-shot setting.In several cases, its accuracy exceeds results from more advanced methods, including larger-capacity architectures.
- Few-shot classification: More than 10%: hyperbolic ProtoNet outperforms its Euclidean counterpart in both CUB few-shot settings.It also outperforms many other algorithms on this fine-grained bird-classification dataset.
6. Discussion and conclusion
The paper finds that hyperbolic embeddings can substantially improve accuracy across several vision tasks, particularly few-shot classification, while remaining compatible with Euclidean backbones. The authors qualify this benefit as task- and dataset-dependent and note numerical-precision concerns.
- Discussion and conclusion: Hyperbolic models use Euclidean operations in most layers and apply an exponential map to move embeddings into hyperbolic space at the network end.This makes the approach compatible with existing Euclidean backbone networks.
- Discussion and conclusion: Across several tasks, hyperbolic embeddings produce a substantial accuracy boost, particularly for few-shot image classification.The authors speculate that negative curvature better conforms to the intrinsic geometry of some hierarchically structured image manifolds.
- Discussion and conclusion: The benefit of hyperbolic embeddings varies substantially across tasks and datasets, so when hyperbolic geometry is warranted remains insufficiently understood.The authors identify this as an area for future work.
- Discussion and conclusion: Fixed-precision arithmetic breaks the continuous equivalence among hyperbolic models, making numerical precision an important practical consideration.The authors observed that alternative hyperbolic models may offer more favorable floating-point performance.
A. Hyperbolic Neural Networks
The appendix adapts common neural-network operations to hyperbolic space, including linear layers, concatenation, and multiclass logistic regression. These constructions let standard vision architectures operate on Poincaré-ball embeddings.
- Linear layer: A hyperbolic linear layer replaces Euclidean matrix multiplication and bias addition with Möbius matrix-vector multiplication followed by Möbius addition.The Möbius matrix-vector product is defined separately, including a zero-output case when the product vanishes.
- Concatenation of input vectors: Hyperbolic concatenation maps vectors from two Poincaré balls into a target ball rather than directly concatenating their Euclidean coordinates.The construction uses trainable matrices M1 and M2 with dimensions n3 × n1 and n3 × n2.
- Multiclass logistic regression: Hyperbolic multiclass logistic regression represents class logits through distances to hyperbolic analogues of Euclidean hyperplanes.The class parameters include learnable points in the Poincaré ball.
B. Experiment details
The experiments instantiate hyperbolic models for Omniglot, MiniImageNet, CUB, and person re-identification using Poincaré-ball mappings and hyperbolic classification or distance operations. Dataset-specific backbones, dimensions, and training settings are varied.
- Omniglot: Omniglot uses a 64-dimensional ProtoNet baseline, while the hyperbolic model maps the final convolutional output into the Poincaré ball.The hyperbolic model otherwise builds on the convolutional ProtoNet architecture.
- MiniImageNet: MiniImageNet experiments use 4-Conv and ResNet18 backbones with embedding dimensions 1024 and 512, respectively.The models are trained for one-shot and five-shot evaluation under specified learning-rate schedules and curvature values.
- Caltech-UCSD Birds: CUB experiments use a 4-Conv architecture with 512-dimensional embeddings and separate one-shot and five-shot training settings.The learning rates, curvature values, step sizes, and decay rates differ between the two settings.
- Person re-identification: Person re-identification uses ResNet50 with 32-, 64-, and 128-dimensional embeddings; the hyperbolic version maps descriptors to the Poincaré ball and applies multiclass logistic regression.The Euclidean baseline instead adds a classification linear layer followed by cross-entropy loss.
C. Visualizations
The few-shot embeddings are visualized with hyperbolic UMAP projections for 5-shot tasks on CUB and MiniImageNet. Different classes appear near the circle boundary and well separated.
- Hyperbolic UMAP projects high-dimensional embeddings from trained few-shot models into two dimensions.The projections are used for visual inspection of the learned embeddings.
- Different classes are positioned near the circle boundary and appear well separated.
- The visualization covers 5-shot tasks on CUB and MiniImageNet.Figure 6 presents CUB on the left and MiniImageNet on the right.