PaperlayerView exact revision on arXiv

Source-linked AI summary

Gromov-Wasserstein Distillation for Inductive Multi-View Embedding

Rafael Pereira Eufrazio, Eduardo Fernandes Montesuma, Charles Casimiro Cavalcante

arXiv:2609.40047v1cs.LGstat.ML

TL;DR

GW-MDS embeds observed samples but does not provide an explicit mapping for unseen data. The paper distills its transport-plan correspondence into sample-aligned targets for neural students, including multi-view variants. Across synthetic and real-world evaluations, distilled models consistently outperform direct neural GW training in sample-indexed relational preservation.

  • Problem

    GW-MDS is transductive and supplies no explicit mapping for unseen samples, while direct GW objectives can agree structurally under a non-identity coupling without recovering sample-indexed positions.

    Source: §1 · pp. 1–2

  • Method

    A GW-MDS teacher’s transport plan projects its latent support into sample-aligned barycentric targets, which a neural student learns to map from observations.

    Source: §1 · p. 2

  • Results

    Distilled models consistently improve over direct neural GW training in sample-indexed relational preservation across synthetic and real-world evaluations.

    Source: §1 · p. 2, §4.4 · p. 8

  • Takeaways & Limitations

    Barycentric projection provides a bridge from transductive GW embeddings to inductive neural mappings for single-view and multi-view data.

    Source: §1 · p. 2, §5 · p. 9

    Future work includes scalable teacher optimization and broader evaluations with larger datasets, repeated splits, and additional multi-view settings.

    Source: §5 · p. 10

Abstract

from arXiv · show

Gromov-Wasserstein multidimensional scaling (GW-MDS) learns low-dimensional representations from relational data but remains transductive, providing no explicit mapping for unseen samples. We introduce an inductive framework based on barycentric distillation. A GW-MDS teacher learns a latent support and an optimal transport plan from the training data, and barycentric projection converts the resulting coupling into sample-aligned targets. A neural student then learns an explicit out-of-sample mapping, avoiding additional relational-matrix construction and GW optimization at inference. We formulate the approach for single-view data and extend it to Mean-GWMDS and Multi-GWMDS teachers through consensus and selected-projection targets learned by a multi-view student with view-specific encoders. We also investigate a direct neural baseline trained solely with a GW objective. Experiments on synthetic and real-world data using Euclidean, geodesic, and cosine relations show that the distilled models preserve the teacher geometry on unseen samples and consistently outperform direct neural GW training in sample-indexed relational preservation. These results establish barycentric projection as an effective bridge between transductive GW embeddings and inductive neural mappings.

1 Introduction

GW-MDS can compare relational structures across domains without shared coordinates, but it only embeds observed samples. This paper uses barycentric distillation to create sample-aligned targets for an inductive neural mapping.

1 Introduction

  • GW-MDS optimizes coordinates for observed samples but does not learn a mapping for unseen data.Its transport plan may also assign latent support points to samples non-identically, so direct neural GW training can miss sample-indexed positions.Source: §1 · pp. 1–2
  • Barycentric distillation projects the teacher’s transport plan and latent support into sample-aligned targets that a neural student learns to predict.After training, unseen samples are embedded by a forward pass without constructing new relational matrices or solving another GW problem.Source: §1 · p. 2
  • The framework extends from single-view GW-MDS to Mean-GWMDS and Multi-GWMDS teachers for multi-view data.The paper evaluates on synthetic and real-world data across multiple relational geometries, reporting consistent gains over direct neural GW and competitive or superior performance to PCA-based inductive baselines.Source: §1 · p. 2

2 Background

Gromov–Wasserstein methods compare internal relational structures rather than coordinates, enabling embeddings across domains with different ambient spaces. GW-MDS learns a latent support, while its transport plan leaves sample alignment unresolved and the resulting embedding remains transductive.

2.1 Gromov–Wasserstein Discrepancy

  • GW discrepancy compares within-domain relational structures without requiring shared ambient spaces, and its optimal plan encodes structural correspondence.The review situates GW discrepancy as a basis for relational dimensionality reduction.Source: §2.1 · p. 2, §2 · p. 2

2.2 Gromov–Wasserstein Multidimensional Scaling

  • GW-MDS learns a low-dimensional latent support by matching its induced geometry to an input dissimilarity matrix through an optimized transport plan.The input matrix can encode Euclidean, cosine, geodesic, or other application-dependent dissimilarities.Source: §2.2 · pp. 2–3
  • Barycentric projection converts the transport plan and latent support into a representation aligned with the original samples.GW-MDS itself remains transductive and provides no explicit mapping for unseen observations.Source: §2.2 · p. 3

2.3 Multi-View Relational Data

  • Multi-view relational data describe the same observations in the same row order through views with potentially different feature dimensions and ambient spaces.The views can encode complementary relational structures, motivating shared low-dimensional representations that do not require coordinate-wise comparability.Source: §2.3 · p. 3

3 Proposed Methods

The method turns transductive GW embeddings into inductive mappings by using a teacher’s transport plan to construct sample-aligned barycentric targets for neural students. It extends this strategy to multi-view teachers and compares distillation with direct neural GW optimization.

3.1 Single-View Inductive GW-MDS

  • The GW-MDS teacher’s transport plan projects its latent support into targets aligned row by row with the source samples.Directly pairing each input sample with the same-index latent point would impose an arbitrary correspondence.Source: §3.1 · p. 3
  • A fixed transport plan and latent support yield barycentric targets that uniquely minimize the plan-induced quadratic reconstruction cost.The uniqueness is conditional on the selected plan and coordinate realization; isometrically equivalent supports produce projections differing only by a global Euclidean isometry.Source: §3.1 · pp. 3–4

3.1.1 Neural student

  • The neural student regresses to standardized barycentric targets and maps unseen samples through a forward pass.Under uniform empirical weights, the pointwise objective also controls an upper bound on average squared pairwise-distance discrepancy relative to the targets.Source: §3.1.1 · p. 4

3.2 Multi-View Barycentric Distillation

  • Mean-GWMDS and Multi-GWMDS serve as complementary teachers for multi-view barycentric distillation.The method uses normalized view relations and weights their contributions in a shared objective.Source: §3.2 · p. 4

3.2.1 Inductive Mean-GWMDS

  • Inductive Mean-GWMDS uses a consensus barycentric target derived from its teacher support and corresponding optimal plan.The passage identifies the mean target and plan as the components used in this extension.Source: §3.2.1 · p. 4

3.2.2 Inductive Multi-GWMDS

  • Multi-GWMDS retains view-specific projections, then selects the one with the highest aggregated agreement with training views when one representation is required.Agreement is measured using Pearson correlations between induced and training-view distance matrices, with selection based on training data only.Source: §3.2.2 · p. 4

3.2.3 Multi-view student

  • The multi-view student encodes each view separately, concatenates learned features, and maps them through a shared output head to teacher coordinates.This preserves sample-aligned teacher supervision while fusing learned view-specific features.Source: §3.2.3 · p. 5

3.3 Direct Neural GW Baseline

  • The direct neural GW baseline recomputes transport plans from current outputs at each iteration and holds them fixed during backpropagation.The single-view case sets V = 1, and embeddings are centered and scaled using training-set statistics.Source: §3.3 · p. 5

4 Experiments and Discussion

Across synthetic and real-world tests, barycentrically distilled students preserve sample-indexed relational structure better than direct neural GW, while results vary by dataset, view, and relational geometry. Mean-GWMDS is strongest across the reported real-world comparisons, and geodesic relations are particularly effective for ERA5.

4.3 Experimental Protocol

  • The experiments compare Euclidean, cosine, and geodesic relations, average real-world metrics across views, and use held-out samples for out-of-sample evaluation.The S-curve uses 1,000 training and 250 test samples; real-world splits are 80% training and 20% test, with 15% of training data reserved for validation.Source: §4.1 · p. 5, §4.3 · p. 7

4.1 Single-View Evaluation on a Synthetic Manifold

  • On the S-curve, geodesic Inductive GW-MDS reaches Pearson 0.9980, Spearman 0.9974, trustworthiness 0.9981, and stress 0.0337.Compared with direct neural GW, geodesic Pearson rises from 0.7142 to 0.9980 and stress falls from 0.3905 to 0.0337; the distilled student more closely reproduces teacher sample organization, while rotations and reflections are not errors.Source: Table 1 · p. 5, §4.1 · pp. 5–6, Figure 1 · p. 6
  • On the S-curve, cosine relations also yield strong correlation and neighborhood preservation, while Euclidean Inductive GW-MDS is comparable to PCA and achieves the highest trustworthiness.The reported metrics are Pearson and Spearman correlations, trustworthiness, and scale-adjusted stress.Source: §4.1 · p. 5

4.2 Real-World Datasets

  • The real-world evaluation uses ERA5’s four co-registered atmospheric-variable views over 483 locations and rMD17–Aspirin’s two rotation-invariant molecular views from 1,000 selected conformations.The ERA5 views are hourly time series for temperature, dew-point temperature, surface pressure, and precipitation; the molecular views encode interatomic distances and force-derived structure.Source: §4.2 · p. 6

4.4 ERA5 Results

  • On ERA5, Inductive Mean-GWMDS achieves the best value for all four metrics under every geometry; geodesic results are r = 0.8507, ρ = 0.8439, trustworthiness 0.9373, and stress 0.2603.Direct multi-view GW reaches almost the same final objective as its teacher but obtains r = 0.3409, versus 0.7855 for Inductive Multi-GWMDS and 0.8507 for Inductive Mean-GWMDS; the distilled embeddings also show clearer latitude organization.Source: Table 2 · p. 7, §4.4 · pp. 7–8

4.5 Results on rMD17–Aspirin

  • On rMD17–Aspirin, Inductive Mean-GWMDS outperforms all competing methods across every geometry and metric; for cosine, its Pearson correlation is 0.5242 versus 0.4222 for concatenated PCA.Cosine favors global distance agreement, while Euclidean and geodesic relations provide slightly better local-neighborhood and metric preservation; Figure 3 shows distinct held-out organizations across methods.Source: Table 2 · p. 7, §4.5 · pp. 8–9, Figure 3 · p. 9

4.4 ERA5 Results

  • ERA5 view-wise results are uneven: geodesic Mean-GWMDS raises precipitation correlation from 0.4203 under Euclidean relations to 0.7021, while cosine surface-pressure Pearson is 0.2667 and Spearman is 0.6706.Multi-GWMDS remains competitive with PCA under geodesic geometry, but Mean-GWMDS improves the other three views while sacrificing some temperature-view fidelity.Source: §4.4 · pp. 7–8

4.5 Results on rMD17–Aspirin

  • For rMD17–Aspirin, the selected Multi-GWMDS projection favors interatomic-distance structure, whereas Mean-GWMDS improves force-derived-view preservation and yields a more balanced consensus.Under cosine, Multi-GWMDS correlations are 0.8204 and 0.0669 for the interatomic-distance and force-derived structures, compared with 0.7740 and 0.2743 for Mean-GWMDS; the two structures are only partially compatible in a shared two-dimensional representation.Source: §4.5 · pp. 8–9

5 Conclusion

The paper presents barycentric distillation as an inductive extension of GW-MDS and reports that its students preserve unseen-sample relational structure while consistently outperforming direct neural GW training. It identifies larger-scale optimization and broader evaluations as future work.

5 Conclusion

  • Distilled students preserve relational structure on unseen samples and consistently outperform direct neural GW training across the evaluated settings.Inductive Mean-GWMDS has the strongest view-averaged preservation, while Inductive Multi-GWMDS favors the structure associated with its training-selected projection.Source: §5 · p. 9
  • Barycentric projection turns the teacher’s transport coupling into sample-aligned targets that a neural student learns to predict for single-view and multi-view data.The multi-view targets use the Mean-GWMDS consensus or the Multi-GWMDS selected projection.Source: §5 · p. 9
  • Future work includes scalable teacher optimization, a clustering-oriented mini-batch variant, and broader evaluations with larger datasets, repeated splits, and additional multi-view settings.Source: §5 · p. 10

A Supplementary Methodological and Experimental Analysis

The appendix adds theoretical and methodological details and extends experimental evidence for the framework, including analyses on ERA5 and rMD17-Aspirin.

A Supplementary Methodological and Experimental Analysis

A.1 Theoretical Properties of Barycentric Distillation

The appendix characterizes barycentric distillation through optimality, isometry, and distortion-control properties. For a fixed transport plan, the projection is the unique sample-indexed representation minimizing the reconstruction cost.

A.1 Theoretical Properties of Barycentric Distillation

  • The barycentric projection is equivariant under global rotations, reflections, and translations of the latent support, preserving pairwise relational geometry.Source: §A.1 · pp. 12–13
  • The appendix also establishes that pointwise distillation error controls an upper bound on the discrepancy between student-predicted and barycentric-target pairwise Euclidean distances.Source: §A.1 · p. 12
  • For a fixed transport plan and latent support, the barycentric projection uniquely minimizes the corresponding quadratic reconstruction cost.In Mean-GWMDS this applies to the consensus plan; in Multi-GWMDS it applies separately to each view-dependent plan and yields candidate targets for projection selection.Source: §A.1 · p. 13

Then its barycentric projection satisfies

The barycentric targets transform with global isometries of the latent support, leaving their pairwise Euclidean geometry unchanged. The appendix also states that distillation error controls relational distortion.

Then its barycentric projection satisfies

Then the induced pairwise Euclidean distances satisfy

The proof establishes that pointwise student error controls the average discrepancy between pairwise distances induced by student predictions and barycentric targets.

Then the induced pairwise Euclidean distances satisfy

  • For each sample pair, the proof bounds the distance difference using the reverse triangle inequality and then the triangle inequality.It weights the squared pairwise bound by a_i a_j and sums over all pairs.Source: Then the induced pairwise Euclidean… · p. 14
  • Summing the weighted pairwise bounds simplifies because each sample error term is independent of the other pair index, yielding the stated average-error control.The proof uses nonnegative weights and the definition of the weighted distillation loss.Source: Then the induced pairwise Euclidean… · pp. 14–15
  • Mean-squared pointwise distillation error upper-bounds the average discrepancy between the student-prediction and barycentric-target pairwise Euclidean distance matrices.The conclusion specializes Proposition A.3 to uniform empirical measures, where a_i = 1/n.Source: Then the induced pairwise Euclidean… · p. 15

A.2 Algorithmic Summary

The multi-view algorithm builds a neural mapping from view-specific teacher targets, selecting a projection by training-view agreement when a single Multi-GWMDS representation is needed.

A.2 Algorithmic Summary

  • For Multi-GWMDS, candidate projections are scored by aggregated Pearson correlations between induced distances and training-view relational structures, and the highest-scoring projection is selected.The aggregation may use the arithmetic mean, median, or minimum; the experiments use the arithmetic mean.Source: §A.2 · pp. 15–16
  • The algorithm fits preprocessing on training data, constructs and normalizes each view’s relational matrix, and obtains teacher targets before student training.It splits training indices into fitting and validation subsets and returns the checkpoint with the lowest validation loss.Source: §A.2 · p. 16
  • The student uses view-specific encoders, concatenates their features, and maps them to the Mean-GWMDS consensus target or selected Multi-GWMDS projection.For one view, Algorithm 1 reduces to the single-view formulation.Source: §A.2 · p. 16
  • At inference, the trained student embeds unseen observations with a forward pass, without test relational matrices, test-set couplings, or additional GW optimization.Training-fitted preprocessing is applied before the forward pass.Source: §A.2 · p. 16

A.3 Additional Experimental Analysis on ERA5

On ERA5, distilled students closely reproduce teacher relational behavior, while consensus and selected-projection targets trade balanced multi-view preservation against stronger fidelity to particular views. Direct neural GW can reach a similar objective yet produce weaker sample-indexed test preservation.

A.3 Additional Experimental Analysis on ERA5

  • Across geometries and metrics, the selected-projection student’s largest difference from its Multi-GWMDS teacher is 0.0081; Mean-GWMDS student Pearson differences are at most 0.0002 for Euclidean and geodesic relations.Under cosine dissimilarity, Mean-GWMDS Pearson correlation decreases from 0.5175 to 0.4960, while the other reported metrics remain close to teacher values.Source: §A.3 · p. 16
  • Held-out ERA5 results remain comparable to training values, with consistency indicating transfer of teacher relational organization rather than memorization of barycentric targets.Training and test metrics use different pairwise sets: 386 training locations and 97 held-out locations.Source: §A.3 · p. 16
  • For total precipitation, Inductive Mean-GWMDS Pearson correlation rises from 0.4203 with Euclidean relations to 0.7021 with geodesic relations.Under cosine dissimilarity, surface pressure has Pearson/Spearman correlations of 0.2667/0.6706, indicating stronger ordering than magnitude preservation.Source: §A.3 · p. 17
  • Under geodesic relations, the selected Multi-GWMDS projection raises temperature correlation from 0.9194 to 0.9393 versus Mean-GWMDS, but lowers correlations for the other three views.Mean-GWMDS provides a more balanced consensus across temperature, dewpoint temperature, surface pressure, and precipitation.Source: §A.3 · p. 17
  • The direct neural GW objective differs from the Multi-GWMDS teacher by less than 1% under every geometry, yet its geodesic model has substantially smaller test correlation.This comparison illustrates that optimized-coupling GW agreement does not ensure sample-indexed correspondence.Source: §A.3 · p. 17

A.3.1 Computational Cost

The additional analyses report low distillation cost and coherent ERA5 out-of-sample embeddings, while rMD17-Aspirin exposes view trade-offs and greater validation difficulty. The teacher stage still requires dense relational structures and transport plans.

A.3.2 Qualitative Results and Optimization Behavior

  • On ERA5, student distillation takes less than 3% of teacher runtime, and the complete Inductive Mean-GWMDS pipeline is approximately 2.3–2.9 times faster than direct multi-view GW.Mean-GWMDS students preserve a consensus organization; selected Multi-GWMDS projections favor their corresponding view, while global orientation differences are not errors because GW objectives are invariant to rotations and reflections.Source: §A.3.1 · p. 18, §A.3.2 · p. 19
  • The ERA5 students produce coherent held-out embeddings without a test transport plan; Mean-GWMDS is more balanced across four views, while Multi-GWMDS favors temperature.Direct neural GW yields less informative sample-indexed organization despite a final objective close to the teacher’s.Source: §A.3.2 · pp. 19–20

A.4 Additional Experimental Analysis on rMD17-Aspirin

  • For rMD17-Aspirin, training losses approach zero but best validation losses range approximately from 0.105 to 0.214, making validation-based early stopping particularly important.The passage identifies students trained exclusively from the force-Gram projection as a particular case for early stopping.Source: §A.4 · p. 21
  • On rMD17-Aspirin, Mean-GWMDS trades a small reduction in interatomic-distance preservation for markedly stronger Force-Gram agreement and the best view-averaged results under every geometry.The selected Multi-GWMDS projection is more strongly associated with interatomic distances, with lower Force-Gram correlations.Source: §A.4 · p. 22
  • For cosine relations on rMD17-Aspirin, Inductive Multi-GWMDS test Pearson correlation is 0.4437 versus 0.2942 for direct multi-view GW, a difference of 0.1495 despite nearly identical GW objectives.The teacher and direct-GW objectives differ by approximately 0.09% under cosine relations.Source: §A.4 · pp. 22–23

A.3.1 Computational Cost

  • The complete Inductive Mean-GWMDS pipeline takes approximately 325.22 seconds on rMD17-Aspirin, versus 546.63 seconds for direct neural multi-view GW.The complete Inductive Multi-GWMDS pipeline takes approximately 608.92 seconds, about 11.4% slower than direct multi-view GW during initial training.Source: §A.4 · p. 23

A.5 Limitations

  • The teacher remains transductive and requires dense pairwise relational matrices and transport plans, so training-stage memory grows quadratically with observation count.Repeated GW updates also remain expensive for large datasets.Source: §A.5 · p. 23

Related papers

Loading 2609.40047v1…