PaperlayerView exact revision on arXiv

Source-linked AI summary

What Converges in the Platonic Representation Hypothesis? Structure over Geometry

Junwon You, Mihyun Jang, Sangwoo Mo, Jae-Hun Jung

arXiv:2609.27252v1cs.LGcs.CVmath.AT

TL;DR

Prior work leaves open whether representational convergence is local or global, because comparisons vary both scale and whether they assess relations or quantitative geometry. This paper matches those comparisons in a 2×2 framework, finding robust relational convergence at both scales but weaker convergence under stricter metric agreement. The same pattern appears with a Riemannian approximation and in video-text representations.

  • Problem

    Prior local-neighborhood findings leave unclear whether convergence depends on structural scale or on comparing relation identities rather than quantitative metric geometry.

    Source: §1 · p. 1

  • Method

    A controlled 2×2 framework compares relational structure and metric geometry at local and global scales, using H0 skeleton overlap as a global counterpart to mKNN.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Relational alignment remains robust at both scales after calibration, while increasingly stringent distance agreement weakens alignment and progressively flattens capacity-dependent trends.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Relational convergence extends beyond local neighborhoods to global spanning structure, whereas metric geometry exhibits substantially weaker convergence.

    Source: §1 · pp. 2–3

    The analysis is limited to H0 spanning structure, a fixed-relation formulation of metric agreement, and one Riemannian metric approximation.

    Source: §5 · p. 10

Abstract

from arXiv · show

The Platonic Representation Hypothesis suggests that increasingly capable models converge toward shared representations. Recent work narrows this claim to shared local neighborhood relationships, finding that capacity-dependent trends in several global similarity measures largely disappear after calibration. We challenge this interpretation by showing that prior local-global comparisons confound structural scale (local versus global) with what is compared: relational structure, defined by which samples are related, versus metric geometry, characterized by quantitative relations such as distances, similarities, or correlations. To disentangle these factors, we construct a controlled $2\times2$ framework that evaluates both relational structure and metric geometry at local and global scales. We introduce $H_0$ skeleton overlap as a global counterpart to mutual $k$-nearest neighbors, together with matched distance-aware variants. Across vision-language models, relational structure exhibits robust representational convergence at both scales after calibration, whereas increasingly stringent distance agreement substantially weakens alignment and progressively flattens the capacity-dependent trend. We further extend the analysis beyond ambient Euclidean geometry by evaluating distance agreement under a Riemannian metric approximation and recover the same structure-geometry pattern. The pattern is also reproduced in video-text representations. Together, these results show that relational convergence extends beyond local neighborhoods to global spanning structure, whereas metric geometry exhibits substantially weaker convergence.

1 INTRODUCTION

The paper argues that prior comparisons of representational convergence confound local-versus-global scale with relational structure versus metric geometry. Its controlled framework finds relational convergence at both scales, while stricter distance agreement weakens capacity-dependent alignment.

1 INTRODUCTION

  • Prior local-global comparisons change both structural scale and what is compared, leaving unclear whether differences reflect scale or relational structure versus metric geometry.Relational structure concerns which samples are related; metric geometry concerns quantitative relations such as distances, similarities, or correlations.Source: §1 · pp. 1–2
  • The 2×2 framework evaluates relational structure and metric geometry independently at local and global scales, pairing mKNN with global H0 skeleton overlap.H0 death edges coincide with minimum-spanning-tree edges, letting the authors compare global spanning relations without requiring distance agreement.Source: §1 · p. 2
  • Across vision-language models, relational alignment remains robust after calibration and generally strengthens with capacity at both scales, while stricter distance agreement weakens and flattens that trend.The weakening also appears with a Riemannian metric approximation and in video-text representations.Source: §1 · pp. 2–3
  • The findings challenge a local-neighborhood-only interpretation: relational convergence extends to global spanning structure, whereas stringent metric agreement substantially weakens convergence.The paper frames the key question as which aspects of representation structure are shared across models.Source: §1 · pp. 2–3

2 RELATED WORK

Related work measures representational similarity through different numerical or relational information, and recent calibration results sharpen questions about what converges. This paper addresses the remaining ambiguity by matching local and global comparisons of relational structure and metric geometry.

2.1 REPRESENTATIONAL SIMILARITY MEASURES

  • CCA-based methods, CKA, and RSA compare numerical relations such as correlations, similarities, or dissimilarities, with different invariances and constructions.An increase in one similarity score does not by itself establish convergence in every aspect of representation geometry.Source: §2.1 · p. 3

2.2 THE PLATONIC REPRESENTATION HYPOTHESIS AND ITS RECENT CRITIQUES

  • The Platonic Representation Hypothesis proposes convergence toward a shared statistical representation, while recent calibration work finds scaling trends weaken in several global measures but persist in neighborhood measures.The prior work also reports increasing agreement on local neighbor identities without matching alignment in pairwise distances.Source: §2.2 · p. 3
  • The prior contrast leaves open whether differences reflect scale or what is compared; this work pairs local mKNN with global H0 skeleton overlap and matched distance-aware variants.The framework also extends distance evaluation beyond ambient Euclidean geometry using a Riemannian metric approximation.Source: §2.2 · p. 4

2.3 TOPOLOGICAL COMPARISON AND ALIGNMENT OF REPRESENTATION SPACES

  • H0 skeleton overlap compares identities of labeled spanning edges, discarding filtration values to isolate global relational structure from edge distances.H0 persistence and minimum spanning trees are connected, and related work has used topology for representation comparison and alignment.Source: §2.3 · p. 4
  • Unlike topology-based optimization methods, this paper uses H0 skeleton overlap diagnostically and adds matched distance-aware variants at both scales.These comparisons probe metric agreement while preserving the underlying relations.Source: §2.3 · p. 4

3 A CONTROLLED FRAMEWORK FOR REPRESENTATIONAL CONVERGENCE

The framework separates structural scale from the information compared: it measures relational structure using selected relation identities, then probes metric geometry by adding distance agreement without changing those relations. It applies this comparison locally and globally, calibrates alignment scores, and tests an alternative geometric model.

3 A CONTROLLED FRAMEWORK FOR REPRESENTATIONAL CONVERGENCE

  • The controlled framework matches local and global comparisons while varying whether alignment uses relation identities alone or also their distances.This separates structural scale from what is compared.Source: §3 · p. 4, §3.1 · p. 4

3.1 PROBLEM SETUP AND COMPARISON FRAMEWORK

  • At each scale, paired representations of the same samples yield selected relation identities for relational structure and distances on those same relations for metric geometry.This produces a 2×2 comparison framework.Source: §3.1 · p. 4

3.2 RELATIONAL STRUCTURE ALIGNMENT

  • mKNN compares local neighbor identities, while H0 skeleton overlap compares global spanning-edge identities and discards edge lengths.H0 death edges coincide with minimum-spanning-tree edges, so their overlap measures agreement in the global spanning structure.Source: §3.2 · pp. 4–5

3.3 DISTANCE-AWARE ALIGNMENT

  • Distance-aware variants weight shared neighbors or spanning edges by agreement in their normalized distances; smaller τ requires closer agreement.As τ approaches infinity, the distance-aware measures recover their corresponding relational-structure measures.Source: §3.3 · pp. 5–6

3.4 AGGREGATION-AWARE PERMUTATION CALIBRATION

  • Permutation calibration uses the maximum alignment across layer pairs and a null distribution from shared sample-correspondence permutations, mapping scores below the calibration threshold to zero.The same aggregation and calibration procedure is applied to all framework measures.Source: §3.4 · p. 6

3.5 BEYOND EUCLIDEAN GEOMETRY

  • The Riemannian extension keeps relation identities fixed and changes only the distances used to evaluate agreement, treating the approximation as one geometric model rather than uniquely correct intrinsic geometry.It is constructed from local covariance information.Source: §3.5 · p. 6

4 EXPERIMENTS

Across vision-language experiments, relational convergence survives calibration at local and global scales, whereas stricter distance agreement weakens alignment and flattens capacity trends. The pattern persists under a Riemannian approximation and in video-text representations.

4.1 EXPERIMENTAL SETUP

  • The evaluation uses 1,024 paired WIT image-text samples, 12 language models, and 17 vision transformers, yielding 204 model pairs.Alignment values are aggregation-aware calibrated scores, with false-discovery-rate control across model pairs.Source: §4.1 · p. 7

4.2 RELATIONAL STRUCTURE CONVERGES AT BOTH LOCAL AND GLOBAL SCALES

  • All 204 vision-language model pairs remain significant after correction for both calibrated mKNN and H0 skeleton overlap, whose alignment generally increases with model capacity.Across pairs, the measures are strongly correlated (Spearman ρ = 0.968).Source: §4.2 · p. 7, Figure 2 · p. 7

4.3 STRICTER DISTANCE AGREEMENT WEAKENS CAPACITY-DEPENDENT CONVERGENCE

  • At τ = 1, local and global alignment retain 85.6% and 83.0% of relational baselines; at τ = 10−2, they retain 5.8% and 5.5%, respectively.Capacity-dependent trends weaken as distance agreement becomes more stringent; at τ = 10−4, 89.7% of local versus 40.2% of global alignments remain significant.Source: §4.3 · p. 8, Figure 3 · p. 8

4.4 EXTENDING BEYOND THE AMBIENT EUCLIDEAN METRIC

  • Under the Riemannian approximation, at τ = 10−2 only 3.2% and 3.0% of local and global relational baselines remain, and capacity trends progressively flatten with stricter agreement.This reproduces the structure-geometry pattern beyond ambient Euclidean geometry.Source: §4.4 · p. 9

4.5 EXTENSION TO VIDEO-TEXT REPRESENTATIONS

  • All 143 video-text model pairs show significant calibrated relational alignment for mKNN and H0 skeleton overlap, while stricter distance agreement weakens alignment and flattens capacity trends.At τ = 10−4, 86.0% of local versus 46.9% of global alignments remain significant.Source: §4.5 · p. 9, Figure 4 · p. 9

5 CONCLUSION

The conclusion argues that representational convergence is not confined to local neighborhoods: relational structure converges at both local and global scales, while stricter metric agreement weakens convergence. The experiments evaluate vision-language models across capacities using local and global relational measures and distance-aware variants.

5 CONCLUSION

  • Relational structure converges at both local and global scales, whereas increasingly stringent metric agreement weakens convergence.The conclusion also reports that stricter distance agreement progressively flattens capacity-dependent trends.Source: §5 · p. 9
  • The analysis is limited to H0 spanning structure, fixed-relation metric agreement, and one Riemannian metric approximation.Other global structures, metric-agreement formulations, and approximations of intrinsic geometry may show different patterns.Source: §5 · p. 10

A.1 VISION-LANGUAGE MODELS AND REPRESENTATIONS

  • The study evaluates language and vision models at multiple scales on 1,024 paired image-text samples from WIT.It uses 12 language models and 17 vision models, yielding 204 vision-language model pairs.Source: §A.1 · p. 15
  • Model-pair alignment is the maximum score across every pair of vision-model and language-model layers.The same aggregation procedure is applied across all alignment measures.Source: §A.1 · p. 15
  • Local relational structure is measured with mKNN at k = 10, while global relational structure is measured with H0 skeleton overlap.Distance-aware variants use τ values from 1 to 0.0001 to impose increasingly stringent distance agreement.Source: §A.1 · p. 15
  • Distance agreement is evaluated on fixed structural supports using ambient Euclidean distances and a locally estimated Riemannian metric.The Riemannian analysis keeps the underlying supports fixed to test whether the pattern depends on metric choice.Source: §A.1 · p. 16

A.2 VIDEO-TEXT EXPERIMENTAL SETUP

The appendices describe the video-text setup, shared permutation calibration and significance procedures, and the construction and interpretation of the H0 skeleton. The skeleton records global spanning relations, coincides with an MST under the same edge ordering, and represents connectivity across the sample set.

A.2 VIDEO-TEXT EXPERIMENTAL SETUP

  • Video-text experiments use 1,024 PVD test videos and evaluate 143 video-text model pairs across video-native, frame-level, and language representations.The video-native scale series uses VideoMAE Base and Large pretrained checkpoints and a Huge checkpoint fine-tuned on Kinetics.Source: §A.2 · p. 16

B.1 AGGREGATION-AWARE PERMUTATION CALIBRATION

  • Permutation calibration applies the same layer-wise aggregation to observed and permuted alignment statistics, using the permutation critical value as the reference threshold.Scores at or below the threshold map to zero, and the calibrated score remains bounded above by one.Source: §B.1 · pp. 16–17
  • The null permutes sample correspondences across all layers and reapplies maximum aggregation, accounting for inflation from selecting the maximum layer-pair score.A common permutation preserves within-model correspondence across layers while disrupting cross-model correspondence.Source: §B.1 · pp. 16–17

B.2 PERMUTATION SIGNIFICANCE TESTS AND MULTIPLE-TESTING CORRECTION

  • Significance is assessed with one-sided empirical permutation p-values and Benjamini-Hochberg false discovery rate correction across model-pair tests.The correction is applied separately for each alignment measure and τ setting at q = 0.05.Source: §B.2 · p. 17

C.1 FORMAL DEFINITION OF THE H0 SKELETON

  • The H0 skeleton consists of labeled edges that merge previously disconnected components as edges are processed in nondecreasing weight order.The edge weights come from the representation’s pairwise distance matrix.Source: §C.1 · p. 18
  • H0 records the sample pairs that merge disconnected components, retaining spanning-relation identities while discarding filtration values.These labeled edge identities are compared across representations to measure global relational alignment.Source: §C.1 · p. 18

C.2 H0 DEATH EDGES AND MINIMUM SPANNING TREES

  • Under the same deterministic edge ordering, the labeled H0 death-edge set coincides with the minimum spanning tree edge set.For a connected complete weighted graph, the resulting tree contains n − 1 edges.Source: §C.2 · p. 18

C.3 TIE HANDLING AND INTERPRETATION

  • When tied weights allow multiple MSTs, deterministic Prim tie-breaking selects a labeled tree consistent with a deterministic refinement of edge ordering.When all pairwise weights are distinct, the MST is unique and no tie refinement is required.Source: §C.3 · p. 19

C.4 GLOBAL NATURE OF THE H0 SKELETON

A fixed-k neighborhood graph can remain disconnected even when the H0 skeleton connects the full sample set. The H0 skeleton provides a global counterpart to mKNN because its edge choices depend on connectivity across the point cloud.

C.4 GLOBAL NATURE OF THE H0 SKELETON

  • An MST connects every sample into one structure, whereas fixed-k neighborhoods can remain confined to disconnected regions.In the two-cluster example, the MST must include an edge crossing the gap even when the fixed-k graph has none.Source: §C.4 · p. 19
  • For every fixed k, a finite point set can have a symmetrized k-nearest-neighbor graph with two components while its H0 skeleton is connected.The H0 skeleton contains an edge joining the components that is absent from the k-nearest-neighbor graph.Source: §C.4 · p. 19
  • The H0 skeleton selects edges according to connectivity induced by previously processed lower-weight edges, so its support is determined over the full point cloud.By contrast, mKNN compares relations defined within sample-centered local neighborhoods.Source: §C.4 · p. 20
  • Here, “global” means connectivity across the entire sample set; individual MST edges need not be geometrically long-range.The H0 skeleton is used for global relational structure, while mKNN measures local relational structure.Source: §C.4 · p. 20

D ADDITIONAL RELATIONAL STRUCTURE RESULTS

The section tests whether capacity-dependent relational alignment is shared across vision-model families rather than driven by family averaging. Family-wise results broadly reproduce the aggregate local and global pattern after calibration.

D ADDITIONAL RELATIONAL STRUCTURE RESULTS

  • The analysis disaggregates the aggregate results into individual vision-model variants within five families to test whether convergence is family-specific.The families are ImageNet-21K supervised, MAE, DINOv2, CLIP, and ImageNet-12K fine-tuned CLIP.Source: D ADDITIONAL RELATIONAL STRUCTURE… · p. 20, §D.1 · p. 20

D.1 FAMILY-WISE RESULTS

  • Across families, local mKNN and global H0 alignment generally increase with language-model capacity, while larger vision variants tend to align more strongly.Aggregation-aware calibration lowers absolute scores but largely preserves these relationships, which are broadly reproduced within constituent families.Source: §D.1 · p. 20

E ADDITIONAL DISTANCE-AWARE RESULTS

The additional analyses examine distance-agreement sweeps and how their capacity-dependent pattern appears across model families. The family-wise display contrasts local and global relational structure by vision-model family.

E ADDITIONAL DISTANCE-AWARE RESULTS

  • The additional analyses examine how the capacity-dependent convergence signature changes as distance agreement tightens and whether the pattern is consistent across families.Figure 5 separately displays local mKNN and global H0 relational alignment across five vision-model families.Source: E ADDITIONAL DISTANCE-AWARE RESULTS · pp. 20–21, Figure 5 · p. 21

E.1 FULL DISTANCE-AGREEMENT SWEEP

The distance-agreement sweep shows that stricter agreement weakens calibrated alignment and its capacity ordering at both structural scales. Family-wise tests and rank-correlation analyses support this pattern beyond score-magnitude changes.

E.1 FULL DISTANCE-AGREEMENT SWEEP

  • Under tolerant agreement, larger vision models generally align more strongly, but calibrated alignment and its capacity separation weaken as τ decreases at both scales.The trend is clear at τ = 1 and, to a lesser extent, τ = 10−1; the clearest capacity ordering disappears under the most stringent settings.Source: §E.1 · pp. 21–22, Figure 6 · p. 21
  • More than sixfold, relative calibrated alignment decreases between τ = 10−1 and 10−2 at both scales; at τ = 10−4, 89.7% of local pairs and 40.2% of global pairs remain significant.The local-global significance difference emerges mainly under stringent agreement.Source: §E.1 · p. 22, Table 1 · p. 22

E.2 FAMILY-WISE DISTANCE-AGREEMENT SWEEPS

  • Across families, increasingly stringent distance agreement weakens calibrated local and global alignment and makes within-family capacity ordering less pronounced or stable.The transition varies by family, but the weakest, least structured trajectories occur under the most stringent settings across all families.Source: §E.2 · p. 22

E.3 STATISTICAL VALIDATION OF THE CAPACITY-DEPENDENT TREND

  • For language capacity, mean local correlation falls from 0.735 at τ = 1 to 0.293 at τ = 10−4, with significant endpoint and full-sweep decreases.The intermediate trajectory is not strictly decreasing at every τ.Source: §E.3 · p. 25, Table 2 · p. 25
  • For vision capacity, mean global correlation falls from 0.850 at τ = 1 to −0.174 at τ = 10−4, the largest endpoint decrease among the four comparisons.The passage cautions that the negative strict-setting mean is not evidence of a general inverse scaling relationship; the earlier positive ordering is no longer preserved.Source: §E.3 · pp. 25–26
  • Spearman rank association weakens with stricter agreement at both scales and along both capacity axes, beyond the mechanically expected decline in score magnitude.A uniform rescaling would leave rank association unchanged; the observed weakening therefore reflects changes in capacity ordering.Source: §E.3 · p. 26

F.1 SENSITIVITY TO DISTANCE NORMALIZATION

Alternative distance-normalization quantiles preserve the main distance-agreement pattern, and a cosine-similarity construction reproduces the weakening. A locally estimated Riemannian metric is evaluated while the relational supports are held fixed.

F.1 SENSITIVITY TO DISTANCE NORMALIZATION

  • Changing the distance-normalization quantile leaves local and global alignment trajectories highly consistent as agreement tightens.The main setting uses q = 0.90, with q ∈ {0.50, 0.75, 0.95} as alternatives; alignment approaches zero under the most stringent agreement across choices.Source: §F.1 · p. 26, Figure 8 · p. 26

F.2 SIMILARITY-AWARE ALIGNMENT

  • The similarity-aware construction applies cosine-similarity agreement only to shared relations, retaining the same local and global structural supports.Smaller τ requires closer similarity agreement; as τ → ∞, the relational-structure measures are recovered.Source: §F.2 · p. 27
  • The similarity-aware sweep reproduces the distance-aware transition: alignment weakens at both scales as cosine-similarity agreement becomes stricter.At τ = 10−2, similarity-aware alignment retains 4.73% locally and 5.57% globally of relational-structure baselines.Source: §F.2 · p. 27, Figure 9 · p. 28

G ROBUSTNESS UNDER A LOCALLY ESTIMATED RIEMANNIAN METRIC

  • The Riemannian test changes the metric used to evaluate agreement while keeping local kNN and global MST supports fixed.Reconstructing those supports under the alternative metric would change relational structure and metric information simultaneously.Source: G ROBUSTNESS UNDER A LOCALLY ESTIMATED… · p. 28

G.1 RIEMANNIAN DISTANCE CONSTRUCTION

  • For local distance-aware mKNN, the original k = 10 neighbor identities are retained while their distances are replaced by Riemannian lengths.The lengths are normalized by the 90th percentile of positive Riemannian graph-edge lengths.Source: §G.1 · p. 29
  • For global distance-aware H0 overlap, ambient-Euclidean MST edge identities are retained and evaluated using Riemannian shortest-path distances on the weighted neighbor graph.The positive distances are normalized by their 90th percentile before evaluation on the fixed MST support.Source: §G.1 · p. 29

G.2 FULL RIEMANNIAN RESULTS

Under a locally estimated Riemannian metric, stricter distance agreement weakens calibrated alignment and flattens capacity trends at both scales. The video-text extension shows the same broad structure-geometry pattern.

G.2 FULL RIEMANNIAN RESULTS

  • As Riemannian distance agreement becomes more stringent, calibrated alignment weakens and capacity-dependent patterns become less pronounced at both structural scales.This parallels the pattern under the ambient Euclidean metric.Source: Figure 10 · p. 30, §G.2 · p. 30
  • At τ = 10−2, Riemannian variants retain 3.24% of the local and 2.97% of the global relational-structure baselines.At τ = 1, the corresponding fractions are 79.0% and 72.4%; they fall below 0.5% at τ = 10−3 and approach zero at τ = 10−4.Source: Table 5 · p. 30, §G.2 · p. 30
  • All 204 model pairs remain significant through τ = 10−1; at τ = 10−4, 175 local pairs but only 13 global pairs remain significant.At τ = 10−3, 204 local and 160 global pairs remain significant.Source: §G.2 · pp. 30–31
  • Replacing ambient Euclidean distances with a locally covariance-adapted Riemannian metric does not restore robust capacity-dependent convergence.The same overall structure-geometry pattern persists under this alternative metric.Source: §G.2 · p. 31

H ADDITIONAL VIDEO-TEXT RESULTS

  • In video-text representations, relational alignment remains robust at both scales, while stricter distance agreement weakens alignment and capacity trends.This reproduces the qualitative structure-geometry pattern from the main vision-language setting.Source: H ADDITIONAL VIDEO-TEXT RESULTS · p. 31

H.1 FAMILY-WISE RELATIONAL STRUCTURE RESULTS

Across video-text model families, local and global relational alignment show broadly similar capacity trends. In the distance-agreement sweep, stricter requirements weaken alignment at both scales, with clearer scale differences in the stringent regime.

H.1 FAMILY-WISE RELATIONAL STRUCTURE RESULTS

  • Across VideoMAE, DINOv2, and CLIP, relational alignment generally increases with language-model capacity, and larger visual-model variants tend to align more strongly.The qualitative local and global patterns remain consistent across visual-model families despite differing absolute score ranges.Source: §H.1 · pp. 31–32, Figure 11 · p. 31

H.2 FULL DISTANCE-AGREEMENT SWEEP

  • Stricter video-text distance agreement sharply decreases calibrated alignment and progressively flattens capacity separation at both local and global scales.The transition occurs in parallel for the two measures; differences become more apparent under stringent agreement.Source: §H.2 · p. 32, Figure 12 · p. 32
  • At τ = 10−2, video-text alignment retains 3.65% of the local and 4.22% of the global relational-structure baselines.At τ = 1, the fractions are 81.1% and 79.4%; at τ = 10−1, they are 29.3% and 29.1%.Source: Table 6 · p. 32, §H.2 · p. 33
  • At τ = 10−4, 123/143 local pairs (86.0%) remain significant, compared with 67/143 global pairs (46.9%).All 143 local pairs remain significant through τ = 10−3, while significant global pairs decrease to 131/143 at that setting.Source: Table 6 · p. 32, §H.2 · p. 33

Related papers

Loading 2609.27252v1…