How smoothing the affinity matrix affects neighborhood preservation in t-SNE

Shirin Mohebi; Guillaume Bied; Jefrey Lijffijt

Source-linked AI summary

How smoothing the affinity matrix affects neighborhood preservation in t-SNE

Shirin Mohebi, Guillaume Bied, Jefrey Lijffijt

arXiv:2608.17190v1Aug 17, 2026cs.LGcs.CV

TL;DR

t-SNE’s local neighborhood preservation may be improved by modifying its affinity matrix, an underexplored question relevant to visualizing high-dimensional data. The paper introduces a row-wise power transform and finds that sharpening favors very nearest neighbors, whereas smoothing favors broader local neighborhoods.

Problem
The study asks whether modifying t-SNE’s affinity scores can improve local neighborhood preservation beyond standard t-SNE.
Method
A row-wise power transform smooths or sharpens t-SNE affinity probabilities, equivalently rescaling Gaussian bandwidths while producing point-dependent effective perplexities.
Results
Sharpening preserves very nearest neighbors better, while smoothing improves broader local neighborhoods and can outperform global-perplexity increases and multiscale alternatives.
Takeaways & Limitations
Gamma offers a lightweight way to shift t-SNE’s preservation focus toward nearest-neighbor fidelity or broader local and global structure.
Takeaways & Limitations
The study focuses mainly on NO@k/QNX(k), and its improvements are consistent but modest in absolute terms.

Abstract

from arXiv · show

Dimensionality reduction methods are instrumental to visualize high-dimensional data, and t-SNE stands as one of the most widely used methods due to its emphasis on local neighborhood preservation. A central component of t-SNE is the affinity matrix, which expresses pairwise similarities in the form of symmetrized probabilities, over which the optimization problem of t-SNE is defined. We study how the sharpness of this probability distribution affects neighborhood preservation at different scales. We introduce a row-wise power transform controlled by a parameter gamma that can smooth or sharpen each row of the affinity matrix while preserving sparsity and rank order. We show that this transform is equivalent to rescaling the Gaussian bandwidth and thus to changing the perplexity. However, as the sharpness of the probability distribution varies per point, a fixed gamma leads to point-dependent effective perplexities, making it distinct from changing the global perplexity. Empirically, we find that sharpening improves preservation of the very nearest neighbors, while smoothing improves preservation of broader local neighborhoods, outperforming alternative affinity constructions including multiscale methods in the mid-local range.

1 Introduction

The study examines how smoothing or sharpening t-SNE’s affinity matrix changes neighborhood preservation across scales. It introduces a gamma-controlled transform that alters effective perplexities point-dependently rather than globally.

Motivation: t-SNE is widely adopted because it focuses on preserving local neighborhood structure when projecting high-dimensional data into two dimensions.Points close in the high-dimensional space are encouraged to remain close in the embedding.
Motivation: Although t-SNE often outperforms other methods on local-preservation benchmarks, its absolute neighborhood-preservation scores can remain low and errors may be unevenly distributed.Other methods can outperform t-SNE on global structure preservation.
Methodology: The study varies the smoothness of t-SNE’s affinity matrix because the matrix determines which neighborhood relations are emphasized and how strongly they influence the embedding.After the affinity matrix P is constructed, optimization is driven only by those affinities.
Methodology: Standard t-SNE uses point-specific Gaussian bandwidths to give every point the same global perplexity, interpreted as its effective number of neighbors.Perplexity is defined as 2H(P_i), where H(P_i) is the Shannon entropy of a conditional probability row.
Theoretical and empirical results: Sharpening improves preservation of the very nearest neighbors, whereas smoothing improves preservation of broader local neighborhoods and global structure.The transform changes effective Gaussian bandwidth and perplexity while producing point-dependent effective perplexities.

2 Related work

Prior work establishes t-SNE as a strong method for local-structure preservation and has improved its scalability and practical use. Related affinity-construction studies motivate this work’s narrower analysis of how row-wise probability weighting affects neighborhood preservation.

Local-structure preservation: t-SNE is particularly effective at revealing local structure and shows strong local-neighborhood preservation against UMAP, TriMap, and PaCMAP.A separate benchmark reports the strongest local structure preservation among the evaluated dimensionality-reduction methods.
Practical improvements: Barnes-Hut t-SNE, FIt-SNE, and openTSNE improve computational efficiency or accessibility through approximations, interpolation, fast Fourier transforms, and modular implementation.Other work also studies optimization choices including initialization, learning rate, and early exaggeration.
Affinity construction: Standard t-SNE uses target perplexity to control neighborhood scale, motivating multiscale affinity constructions that question a single-scale representation.Multiscale Stochastic Neighbor Embedding averages softmax-based similarities over multiple bandwidths.
Scope of this work: This work instead changes only the row-wise probability distribution after standard affinities are constructed, using a power transform that preserves neighbor support and rank order while changing sharpness.The design isolates whether neighborhood preservation depends on neighbor presence alone or also on the strength of neighbor weighting.

3 Methodology

The method applies a row-wise power transform to t-SNE’s conditional affinities, smoothing or sharpening probability mass without changing neighbor order or sparsity. The transform is equivalent to changing each Gaussian bandwidth, but induces point-dependent effective perplexities rather than a single global target.

t-SNE affinity construction: t-SNE converts high-dimensional distances into conditional probabilities, symmetrizes them into affinity matrix P, and minimizes KL divergence against low-dimensional similarities.Larger affinity values emphasize stronger attractive neighborhood relations during optimization.
Motivation and implementation: At perplexity 30, affinity rows can be highly skewed, with a few neighbors dominating; smoothing redistributes mass more evenly across the neighborhood.For γ = 0.5, all neighbors receive meaningful probability mass while the usual 3·ρ-neighbor support remains unchanged.
Power transform: The row-wise power transform uses γ to redistribute conditional probability mass: γ < 1 smooths rows, γ > 1 sharpens them, and γ = 1 recovers t-SNE.The transform preserves row-wise rank order and leaves the existing neighbor support unchanged.
Bandwidth interpretation: The power transform rescales each Gaussian precision as ˜βi = γβi, so smoothing widens the effective bandwidth and sharpening narrows it.Thus, the transform changes the concentration of each conditional distribution through an equivalent bandwidth adjustment.
Effective perplexity: A constant γ ≠ 1 can produce different smoothed entropies and effective perplexities across rows, unlike standard t-SNE with one new target perplexity.The result is a point-adaptive, heterogeneous spread of effective neighborhood sizes rather than a uniform shift.

4 Experiments

Experiments across three datasets evaluate how row-wise affinity smoothing and sharpening affect neighborhood preservation at different scales, whether smoothing differs from increasing global perplexity, and how it compares with alternative affinity constructions. Smoothing improves broader local and global structure, while sharpening favors the very nearest neighbors.

Experimental setup: Experiments use MNIST, UCI Adult, and mouse cortex data with matched openTSNE initialization and optimization settings across affinity variants.The only difference between standard and smoothed t-SNE is the row-wise affinity transformation, and reported metrics average five runs.
Evaluation: NO@k measures the fraction of high-dimensional k-nearest neighbors retained among the embedding’s k-nearest neighbors.The metric is also known as QNX(k) in dimensionality-reduction evaluation.
Neighborhood scale: At perplexity ρ = 30 on MNIST, sharpening performs better at small k, whereas smoothing performs better at larger k, with its advantage increasing as k grows.The crossover toward smoothing begins around k ≈ 2/3ρ.
Global perplexity comparison: For k > 25, smoothed t-SNE with ρ = 30 and γ = 0.7 outperforms standard t-SNE at both ρ = 30 and matched ρ = 53, showing that smoothing is not equivalent to increasing global perplexity.The matched perplexity equals the median effective perplexity of the smoothed variant.
Sensitivity and global structure: Near-local preservation improves with sharpening and lower perplexity, while global structure preservation generally improves with smoothing and higher perplexity.The best near-local AUC is 0.450 at ρ = 30, γ = 2; the best global result is at ρ = 200, γ = 0.75.
Alternative affinity constructions: Compared with alternative affinity constructions, γ = 1.5 gives the highest small-k NO@k at approximately 0.5 versus 0.42 for standard t-SNE, while γ = 0.7 leads at k ≈25–90.In that mid-local range, γ = 0.7 outperforms MultiscaleMixture and Multiscale.

5 Conclusion · Appendix: How smoothing the affinity matrix

The study identifies affinity sharpness as a scale-dependent control of t-SNE neighborhood preservation: sharpening favors very nearest neighbors, whereas smoothing favors broader local structure. A row-wise power transform provides this control through point-dependent effective perplexities, while the appendix supplies preprocessing and full multi-dataset results.

5 Conclusion: The row-wise power transform γ smooths or sharpens affinity probabilities, equivalently rescales Gaussian bandwidths while producing point-dependent effective perplexities.This distinguishes γ from changing a single global perplexity.
5 Conclusion: Sharpening improves preservation of very nearest neighbors, whereas smoothing improves preservation of broader local neighborhoods.Smoothing redistributes probability mass toward mid-ranked neighbors, counteracting standard t-SNE’s concentration on the first few neighbors.
5 Conclusion: Affinity sharpness is a meaningful control axis, with sharper affinities suited to nearest-neighbor fidelity and smoother affinities suited to broader local and global structure.The parameter γ offers a lightweight way to shift preservation focus according to visualization goals.
5 Conclusion: The study’s improvements are consistent but modest in absolute terms, and their practical significance remains difficult to establish.The analysis focuses primarily on NO@k/QNX(k), while future work could optimize ρ and γ for specific NO@k objectives.
5 Conclusion: The work acknowledges funding from Flemish, FWO, EU/ERC, and Ghent University programs, with views stated as those of the authors.The funding disclosure includes the ERC project VIGILIA, 101142229.
5 Conclusion: The authors declare no competing interests relevant to the article.This disclosure concerns the content of the study.
Appendix: How smoothing the affinity matrix: The appendix reports preprocessing details and full experimental results for MNIST, mouse cortex, and UCI Adult, beyond the main paper’s MNIST-only results.It also includes effective perplexity analyses and detailed results for each main-paper experiment.

A Dataset preprocessing details

The study preprocesses three datasets with dataset-specific normalization and encoding, reducing MNIST and mouse cortex data to 50 principal components before constructing t-SNE affinities.

MNIST: MNIST images are scaled to [0, 1] and reduced to 50 principal components before constructing t-SNE affinities.The dataset contains 70,000 images with 784 pixels, following Kobak and Berens’ recommended preprocessing.
UCI Adult: UCI Adult numerical variables are standardized and categorical variables are one-hot encoded before constructing t-SNE affinities.The dataset contains 48,842 records and 14 variables.
Mouse cortex: Mouse cortex data use sequencing-depth normalization, feature selection, log-transformation, and reduction to 50 principal components before constructing t-SNE affinities.The single-cell RNA sequencing data contain 23,822 observations grouped into 133 hierarchically organized clusters.

B Effective perplexity analysis

The row-wise power transform changes effective perplexity according to γ: sharpening lowers it, smoothing raises it, and the effect varies across points based on their affinity distributions. Sharper affinity rows receive larger increases under smoothing, while bandwidth-based relationships are weaker and less consistent.

Effective perplexity across γ and ρ: At γ = 1.0, effective perplexity matches ρ; sharpening lowers it, smoothing raises it, and γ = 0 yields 3 · ρ.At γ = 2.0 and ρ = 30, effective perplexity falls to 5.840 on MNIST.
Point-level variation: Smoothing affects effective perplexity pointwise according to each row’s probability distribution.The analysis examines smoothing with γ = 0.7 and ρ = 30 across MNIST, mouse cortex, and Adult.
Predictors of point-level change: Bandwidth σi shows a weaker and less consistent relationship with smoothing-induced effective-perplexity changes.For MNIST and Adult, sparser points tend to be more affected.
Predictors of point-level change: Points with sharper affinity rows receive larger effective-perplexity increases after smoothing, with Spearman correlations of 0.667, 0.797, and 0.789 for MNIST, mouse cortex, and Adult.Sharpness is measured by top-5 conditional mass Mi(5).

C Effect of smoothing on local neighborhood preservation

Smoothing and sharpening shift t-SNE’s neighborhood-preservation strengths across scales: sharpening favors very nearest neighbors, whereas smoothing favors broader local neighborhoods. Embeddings also show dataset-dependent visual effects, including greater separation where standard t-SNE clusters overlap.

Neighborhood preservation by scale: At perplexity ρ = 30, smoothed variants (γ < 1) perform better at higher k, while sharpened variants (γ > 1) perform better at small k.The performance gap between smoothed variants grows as k increases.
Neighborhood preservation by scale: Sharpening improves very-nearest-neighbor preservation because it concentrates probability mass on nearest neighbors, increasing their attractive force during optimization.Sharpened variants outperform standard t-SNE for small k.
Embedding structure: Smoothing produces more separated clusters for MNIST and slightly more separated clusters for mouse cortex, while Adult embeddings remain similar.For mouse cortex, both embeddings preserve the overall hierarchical structure; Adult lacks strong cluster structure.

D Smoothing versus increasing perplexity

Smoothing widens each conditional row’s effective bandwidth and raises effective perplexity, but its neighborhood-preservation behavior is not reproduced by simply increasing standard t-SNE’s perplexity. With γ = 0.7, smoothed t-SNE peaks at lower perplexities while achieving higher peak NO@30 across the evaluated datasets.

Effective perplexity: For γ < 1, smoothing widens the effective bandwidth to σ̃_i = σ_i/√γ and raises every conditional row’s effective perplexity.The transform therefore changes perplexity in a point-dependent way through each row’s bandwidth.
Matched perplexity: Matched-perplexity standard t-SNE does not reproduce smoothing’s neighborhood-preservation behavior on mouse cortex and Adult, with MNIST reported in the main paper.Figure 5 compares standard t-SNE, smoothed t-SNE with γ = 0.7, and standard t-SNE matched to the median effective perplexity.
Perplexity sweep: With γ = 0.7, standard t-SNE performs best around perplexity 40–50, whereas smoothed t-SNE peaks around 15–25 across the datasets.The same pattern is reported for mouse cortex and Adult, with MNIST shown in the main paper.
Perplexity sweep: Smoothed t-SNE attains a higher peak NO@30 than standard t-SNE across all three datasets.This result is shown while varying perplexity, with γ = 0.7 held constant rather than tuned.

E Sensitivity to γ and ρ · F Effect of smoothing on global structure preservation

Across datasets, sharpening improves near-local neighborhood preservation but harms mid-local preservation and global structure, whereas smoothing helps broader neighborhoods and generally improves global structure, especially at higher perplexities. These effects vary with perplexity and dataset, with Adult showing weaker mid-local shifts but the clearest global-structure effect.

E Sensitivity to γ and ρ: Sharpening improves near-local preservation, while smoothing hurts it across datasets and perplexities.This pattern holds for mouse cortex, Adult, and MNIST.
E Sensitivity to γ and ρ: Moderate smoothing improves mid-local preservation for MNIST and mouse cortex, especially at lower perplexities.Adult shows the same shift, but more weakly.
E Sensitivity to γ and ρ: At ρ = 30 and 50, γ = 0.7 gives the best mid-local result for Adult.At higher perplexities, the preferred γ shifts toward 1.0–1.2.
E Sensitivity to γ and ρ: At ρ = 100 and 200, γ = 1.0–1.2 is best for Adult, while γ = 0.7 falls below standard t-SNE.Thus, smoothing is not uniformly beneficial across perplexities.
F Effect of smoothing on global structure preservation: Sharpening consistently degrades global structure preservation, whereas smoothing generally improves it across all three datasets.The improvement is most clear at higher perplexities.
F Effect of smoothing on global structure preservation: Adult shows the strongest absolute global-structure values and clearest effect, while MNIST shows the weakest effect.The weaker MNIST effect is consistent with its strong local cluster structure dominating optimization.

G Comparison with alternative affinity constructions

Across mouse cortex and Adult, the affinity variants reproduce MNIST’s pattern: γ = 1.5 performs best at small k, while γ = 0.7 leads in the mid-local range. Dataset-specific differences emerge at larger scales, with multiscale methods more competitive on mouse cortex and γ = 0.7 retaining a broader advantage on Adult.

Cross-dataset comparison: Across mouse cortex and Adult, γ = 1.5 leads at small k, while γ = 0.7 outperforms standard t-SNE in the mid-local range.This overall pattern from MNIST is reproduced in the NO@k comparison for k = 1, . . . , 200.
Mouse cortex: On mouse cortex, multiscale methods track closely with γ = 0.7 from the mid-local to large range, while γ = 0.0, Uniform, and FixedSigmaNN dominate at large k.The multiscale constructions are more competitive on mouse cortex than on MNIST at larger neighborhood sizes.
Adult: On Adult, γ = 0.7 maintains its advantage across a wider range than on mouse cortex.The figure describes this broader advantage relative to the alternative affinity constructions.