Source-linked AI summary
Exploring Dowker Homology for Sentence Similarity
Marius Huber, Juri Opitz
TL;DR
The paper asks whether Dowker homology can capture sentence similarity from token embeddings represented as paired point clouds. It evaluates persistence-diagram features and derived single-number summaries across fine-tuned and non-fine-tuned transformer models, finding that DH captures similarity information and visual model differences, but does not consistently outperform established pooling methods.
Problem
The paper investigates whether Dowker homology captures sentence similarity and can provide readily applicable similarity measures.
Method
The authors treat token embeddings from sentence pairs as point clouds, compute Dowker persistence diagrams, vectorize them into persistence images, and evaluate regressions and single-number summaries.
Results
Dowker homology captures sentence similarity information and supports visual inspection, while its derived summaries work reasonably well without consistently outperforming established pooling methods.
Takeaways & Limitations
DH provides quantitative and visual insight into sentence-similarity data and transformer representations, including effects associated with fine-tuning.
Takeaways & Limitations
The experiments use only the STS-B dataset and relatively modest sample sizes because of computational constraints, and focus on zero-dimensional DH.
Abstract
from arXiv · showhide
Dowker homology is a topological tool that may be used to analyze the relative position of two point clouds living in a common space. We investigate whether Dowker homology captures sentence similarity information by treating the embeddings of the tokens that constitute a sentence pair as a pair of point clouds in the latent space of a transformer model, using both models that have and have not been fine-tuned for sentence similarity. We find that Dowker homology captures sentence similarity information, as measured by regressing Dowker homology features onto ground-truth similarity scores, and that it can be used for visual inspection of similarity data and models. In an attempt to make Dowker homology readily applicable, we derive from it single-number summaries that we expect to capture sentence similarity directly. These turn out to work reasonably well, but without outperforming standard sentence similarity measures based on established pooling methods.
1 Introduction
The paper introduces Dowker homology as a new NLP application for analyzing sentence similarity and asks whether it can distinguish similar from dissimilar sentences and extract similarity scores. The authors find that DH provides quantitative and visual insight, while its single-number summaries perform reasonably well but do not exceed established methods.
- Dowker homology analyzes the relative position of two point clouds in a shared metric space.The paper applies this topological data-analysis tool to sentence representations.
- The study applies DH to Siamese sentence transformer models as a novel NLP application.It investigates models both with and without sentence-similarity fine-tuning.
- The paper asks whether DH can distinguish similar from dissimilar sentences and extract sentence similarity.
- DH elucidates sentence-similarity data and models quantitatively and visually.
- Single-number summaries extracted from DH capture ground-truth similarity reasonably well but do not outperform established methods.
2 Background on Dowker Homology
Dowker homology constructs scale-dependent graphs from two point clouds and records component births and deaths in persistence diagrams. Spatial separation shifts birth values to larger scales, while the construction remains unchanged when the point-cloud roles are swapped.
- The review focuses on zero-dimensional DH, the version used in the experiments.
- At scale r, DH builds a graph on points in X1, with X2 determining which vertices and edges are connected through r-close relationships.
- Connected-component births and deaths are recorded as pairs and represented as points in a persistence diagram.
- Greater spatial separation between point clouds produces larger birth values in the corresponding Dowker persistence diagrams.When the clouds nearly coincide, features appear at small scales; identical clouds have birth values of zero.
- Swapping X1 and X2 does not change the resulting persistence diagram, making DH an undirected quantity.
3 Dowker Homology for Similarity
The paper applies Dowker homology to token-embedding point clouds from sentence pairs to assess sentence similarity, using visual, regression-based, and single-number analyses. Dowker features capture similarity information, while derived Dowker similarities perform reasonably well without surpassing established pooling methods.
- Applying Dowker homology: Token embeddings from each sentence form two point clouds in a transformer’s latent space, whose Dowker persistence diagram reflects their spatial separation.Similar sentences are expected to produce colocalized clouds and predominantly small birth values; dissimilar sentences should produce larger birth values.
- Visual findings: Fine-tuning substantially changes Dowker diagrams for dissimilar pairs but leaves diagrams for similar pairs comparatively unaffected.The reported example uses non-fine-tuned MiniLM-L6 and fine-tuned all-MiniLM-L6; the pattern generalizes across similar and dissimilar STS-B pairs.
- Evaluation: The study evaluates Dowker information by regressing ground-truth sentence similarities on persistence-diagram and token-level features across transformer layers.Experiments use STS-B sentence pairs annotated with human similarity scores from 0 to 5, alongside BOS-, EOS-, max-, and mean-pooled cosine-similarity baselines.
- Dowker similarity: Four Dowker similarity variants aggregate minimum, maximum, mean, or lifetime-weighted mean birth values into single-number similarities between 0 and 1.Identical sentences yield similarity 1 because all birth values vanish when their token point clouds coincide.
4 Experiments and Results
The experiments compare Dowker-homology features with pooled sentence representations using regression and single-number similarity summaries across transformer layers. Dowker methods perform competitively but generally do not surpass established pooling methods, while results vary by model, layer, and evaluation design.
- Experimental setup: 512 training and 256 test sentence pairs from STS-B were used, with experiments run across hidden layers of fine-tuned and non-fine-tuned transformer models.The dataset contains human similarity scores, and computational constraints limited the dataset and sample sizes.
- Layer regression experiment: Dowker regression never outperformed the best baseline, but it always ranked at least third and consistently beat BOS- and EOS-pooling.Max-pooling produced the best features for 9/10 models, despite sentence-transformer fine-tuning with mean-pooling.
- Layer regression experiment: Fine-tuned models generally achieved higher Spearman correlations in later layers, whereas Dowker features ranked second in one of the two MiniLM-L6 cases.The comparison covered all methods at each layer of fine-tuned and non-fine-tuned MiniLM-L6 models.
- Layer correlation experiment: Single-number Dowker summaries ranked behind max- and mean-pooling for 5/10 models, while winning first or second place for 2/10 models each.Max-pooling exceeded mean-pooling on average, although mean-pooling reached the highest maximal correlation for sentence-transformer models at the final layer.
- Layer correlation experiment: smean_Dow outperformed the other Dowker-summary versions for 9/10 models and outperformed all methods in two cases.Those two cases were MiniLM-L6 and MiniLM-L12.
5 Discussion
Dowker homology captures sentence similarity quantitatively and visually, but its single-number summaries do not consistently outperform established methods.
- Dowker persistence diagrams reflect sentence similarity and provide visual insights into how fine-tuning affects representations of similar and dissimilar sentence pairs.
- All versions of Dowker summaries capture sentence similarity to some extent, but the version averaging birth values performs best.It correlates with ground-truth similarities essentially as well as a trained linear regression.
- Single-number Dowker summaries capture ground-truth sentence similarity reasonably well, but do not consistently outperform established methods.
- The findings support further research on connections between topology and sentence similarity, including additional single-number summaries.
A.1 Dowker Homology
Dowker homology records how the topology of two related point sets changes across distance scales, encoding component births and deaths in a persistence diagram.
- Dowker homology captures the topology of two sets X and Y together with a relation R ⊆ X × Y between them.The experiments use Dowker–Rips homology, an implementation provided by Huber and Schnider (2025).
- For metric-space point sets, the relation R_r connects x ∈ X and y ∈ Y whenever d(x, y) ≤ r.
- The relation induces a graph whose vertices are points in X, with an edge when two vertices share a point in Y within distance r.
- As r increases, vertices and edges appear and connected components can be born or merge, producing a sequence of component births and deaths.
- A persistence diagram places each component’s birth and death scales at coordinates (b_i, d_i), while vertical distance from the diagonal represents its lifetime d_i − b_i.
A.2 Vectorizing Persistence Diagrams
Persistence images convert variable-sized persistence diagrams into fixed-length vector representations suitable for machine-learning pipelines.
- Persistence images begin by shearing the diagram so its diagonal maps to a horizontal line, then placing a Gaussian at each transformed point.The Gaussian standard deviation is called the bandwidth, and the resulting heat map is a persistence surface.
- The persistence surface becomes a fixed-length persistence image by discretization onto a pixel grid with chosen resolution.Bandwidth and resolution are treated as application hyperparameters.
A.3 Models used
The section points readers to Table A.1 for the models used in the experiments.
- Table A.1 lists the models used in the experiments.
A.4 Hyperparameter Selection
The layer regression experiment used ridge regressions with SVD solving and selected α by grid search over 10^-3 through 10^2.
- Ridge regressions used scikit-learn’s SVD solver, with α tuned by grid search over 10^-3, 10^-2, …, 10^2.Other scikit-learn parameters remained at their defaults.
A.5 Results
The appendix provides plots for the paper’s regression and correlation experiments, alongside persistence-image schematics and results across model pairs.
- A.5 Results: Appendix plots report results from the two experiments conducted in the paper.These plots are presented analogously to the main-text plots.
- A.5 Results: Figure A.1 illustrates shearing persistence diagrams from (birth, death) to (birth, death-birth) coordinates and the resulting persistence surface.The transformation is shown from left to right through the resulting persistence surface.
- A.5 Results: Figure A.2 presents mean persistence images for the model pairs listed in Table A.1.The persistence images are described as capturing the same quantities explained for Figure 3 in the main text.
- A.5 Results: Figures A.3–A.6 show layer regression results for four pairs of sentence-transformer models and their base models.The pairs include MiniLM, distilroberta, mpnet, and xlm-roberta variants.
- A.5 Results: Figures A.7–A.10 show layer correlation results for the same four model pairs, with legends ordered by decreasing average Spearman correlation across layers.The figures cover MiniLM, distilroberta, mpnet, and xlm-roberta variants.