PaperlayerView exact revision on arXiv

Source-linked AI summary

Similarity of Neural Network Models: A Survey of Functional and Representational Measures

Max Klabunde, Tobias Schumacher, Markus Strohmaier, Florian Lemmerich

arXiv:2305.06329v4cs.LG

TL;DR

Understanding neural-network similarity is difficult because models can appear similar in outputs or intermediate representations without matching in the other perspective. This survey unifies measures for both perspectives, analyzes their properties and relationships, and identifies open research challenges. Its central conclusion is that the measures provide complementary information whose scores require contextual interpretation.

  • Problem

    Neural-network similarity has multiple perspectives, while existing measure literatures are often disconnected and provide only partial views of model similarity.

  • Method

    The survey formally defines and systematically reviews representational and functional similarity measures, including their properties, relationships, and practical applicability.

  • Results

    The survey concludes that functional and representational measures are complementary, generally need not correlate, and should be interpreted in context.

  • Takeaways & Limitations

    Combining both perspectives provides a more comprehensive view of neural-network similarity across layers and can help validate representational measures using functional outputs.

  • Takeaways & Limitations

    The survey focuses on representational and functional measures and differs from methods used to assess or optimize similarity during model training.

Abstract

from arXiv · show

Measuring similarity of neural networks to understand and improve their behavior has become an issue of great importance and research interest. In this survey, we provide a comprehensive overview of two complementary perspectives of measuring neural network similarity: (i) representational similarity, which considers how activations of intermediate layers differ, and (ii) functional similarity, which considers how models differ in their outputs. In addition to providing detailed descriptions of existing measures, we summarize and discuss results on the properties of and relationships between these measures, and point to open research problems. We hope our work lays a foundation for more systematic research on the properties and applicability of similarity measures for neural network models.

1 Introduction

Neural-network similarity has multiple perspectives, so this survey unifies representational and functional measures into a comprehensive overview of their definitions, properties, relationships, and open challenges.

  • Similarity measures are used to study learning dynamics, architecture, supervision, robustness, updates, distillation, ensembles, language representations, and generalizability.
  • Representational measures compare intermediate activations, whereas functional measures compare task-related outputs; together they provide complementary views across network layers.
  • The survey provides a unified overview of representational and functional similarity measures, addressing research lines that are often disconnected.
  • It formally defines both similarity problems, with functional similarity treated in the context of classification, and systematically categorizes existing measures.
  • The survey analyzes practical properties such as robustness to noise and confounding, examines connections between measures, and identifies open research challenges.
  • The survey focuses on representational and functional measures rather than methods typically used to assess or optimize similarity during model training.

2 Similarity of Neural Network Models

The paper frames neural-network similarity through representations and outputs, formalizes how representations are compared under transformations, and explains why the two perspectives require contextual interpretation.

  • Neural-network similarity is quantified by measures of intermediate activations or output behavior for a given classification task.
  • Representational Similarity: Representational measures compare activations from corresponding layers using the same fixed inputs, assigning a score to representation matrices from two models.
  • Representational Similarity: Representational equivalence depends on transformations such as permutations, orthogonal transformations, scaling, translations, invertible linear maps, and affine maps.
  • Representational Similarity: A similarity measure should be invariant to exactly the transformations under which the compared representations are considered equivalent.
  • Representational Similarity: Permutation and orthogonal transformations preserve Euclidean distances and angles, whereas invertible linear and affine transformations generally preserve neither.
  • Relationship Between Similarity Perspectives: Functional and representational scores need contextualization: functional dissimilarity implies representational dissimilarity at some layer under an explicit classification-layer assumption, but the reverse does not generally hold.

3 Representational Similarity Measures

Representational similarity measures compare neural activations through complementary strategies, including canonical correlations, alignment, topology, and descriptive statistics. Their interpretations depend on invariances, dimensionality requirements, preprocessing, and metric properties.

  • Measure categories: Representational similarity measures are organized by comparing projected angles, aligned representations, pairwise-similarity matrices, neighborhoods, manifolds, or representation statistics.These categories provide different operational views of similarity between representations.
  • Canonical Correlation Analysis-Based Measures: CCA aggregates canonical correlations, with mean and mean-squared correlation among standard choices for quantifying similarity.CCA finds maximally correlated linear combinations of two representations and can produce multiple canonical correlations.
  • Canonical Correlation Analysis-Based Measures: CCA is invariant to affine transformations, and equivalent representations have mCCA(R, R′) = 1 and mCCA2(R, R′) = 1.The equivalence condition applies when all canonical correlations equal one.
  • Canonical Correlation Analysis-Based Measures: SVCCA first denoises representations with PCA, retaining enough components to explain a fixed variance amount, usually 99 percent.For CNNs with many features, a discrete Fourier transform can make CCA computation more efficient.
  • Canonical Correlation Analysis-Based Measures: PWCCA weights canonical correlations by their importance to the raw representation, but the resulting measure is asymmetric and bounded in [0, 1].The weights are computed from one representation, and a score of one indicates equivalent representations.
  • Alignment-Based Measures: Alignment-based measures optimize a transformation before comparing representations, while transformation choice determines the corresponding invariances and usually requires equal dimensionality.Shape metrics can be bounded distances, whereas GULP uses ridge-regularized covariance comparisons and is unbounded.
  • Topology-Based Measures: The topology-based mGS measure compares mean relative living times, is bounded in [0, kmax], and equals zero for equivalent representations.It is invariant to orthogonal transformations, isotropic scaling, and translations.
  • Descriptive Statistics: Graph-based modularity measures assess connectivity among same-label neurons, with high modularity indicating dense within-label and sparse between-label connections.Both variants are invariant to orthogonal transformations, and one is additionally invariant to isotropic scaling.

4 Functional Similarity Measures

Functional similarity measures compare neural-network outputs for the same inputs and task. They are organized by the granularity of output information required, with the full set summarized in an overview table.

  • Output-based comparison: Functional similarity measures compare output matrices O and O′, whose entries represent class probabilities or scores for each input.Predictions are obtained by selecting the class with the highest output value.
  • Measure organization: The measures are mainly categorized by the granularity of model outputs they require.This categorization is illustrated in Figure 4.
  • Measure organization: An overview of the functional similarity measures is provided in Table 2.

C Soft Prediction-Based Measures

Functional similarity measures compare model outputs at different granularities, from performance and hard predictions to soft predictions, gradients, adversarial examples, and model stitching. These measures differ in nuance, assumptions, and computational or implementation costs.

  • Performance-Based Measures: Performance-based measures define similarity as the absolute difference between models’ quality scores, such as accuracy or F1 score.This scalar comparison is easy to implement but can obscure more nuanced functional differences.
  • Hard Prediction-Based Measures: Hard prediction-based measures compare predicted classes, reporting high similarity when predictions agree across inputs regardless of correctness or confidence.Disagreement is bounded in [0, 1], with zero indicating perfect agreement; its attainable range depends on model quality.
  • Hard Prediction-Based Measures: Error-corrected disagreement normalizes observed disagreement using model error rates, while min-max normalization provides a symmetric comparison against attainable bounds.The normalized measures retain zero as perfect agreement, but their upper limits depend on model error rates.
  • Soft Prediction-Based Measures: Soft prediction-based measures compare class probabilities or scores, capturing confidence differences that hard predictions discard.Jensen-Shannon divergence is a common symmetric choice for comparing probability distributions across instances.
  • Model Stitching-Based Measures: Model stitching can require additional layer training and may be unstable because the stitching layer is trained nondeterministically.The original model parameters are typically frozen while only the stitching layer is optimized.

5 Properties and Application of Similarity Measures

Similarity measures support both understanding and improving deep learning systems, but their evaluation and selection remain uneven across representational and functional approaches. The survey organizes practical guidance around measure properties, applications, computational trade-offs, and the choice of data, layers, and preprocessing.

  • Applications in the Literature: Similarity measures have been used to study model architecture, objectives, training variation, universality, input features, fine-tuning, and cross-system representations.Applications span comparisons of transformers and CNNs, visual representations against biological systems, and effects of training seeds or data.
  • Properties and Evaluation: Existing tests of representational measures usually evaluate limited subsets, and no measure generally stands out across the ranked comparisons.The ReSi benchmark is a notable broader evaluation resource, implementing 24 representational similarity measures across graph, language, and vision settings.
  • Applications in the Literature: Similarity measures also support ensemble design and knowledge distillation, although system-improvement applications are comparatively rarer than analyses aimed at understanding deep learning.Studies encourage representational diversity, penalize similar soft predictions, or transfer knowledge from larger teachers to smaller students.
  • Similarity Measure Selection: Functional measure selection is application-dependent, and combining multiple measures can provide more nuanced insights while prediction- and performance-based measures are usually linear-time.Robust analyses should control for confounding factors such as random agreement and error rate.
  • Similarity Measure Selection: Representational measure selection is difficult because neural representations are opaque and their equivalences and sensitivities are often unclear.The survey recommends filtering measures by known transformation invariances, relevant prior analyses, and model objective functions.
  • Similarity Measure Selection: Alignment-based measures are less flexible in their invariances than RSM- and neighborhood-based measures, while topology-based measures face substantial computational challenges.These trade-offs can make topology-based approaches unsuitable for large-scale applications.
  • Additional Practical Considerations: Input data influences similarity estimates, so diverse data improves generalizability while increasing computational cost; layer choice creates a similar trade-off.Representation preprocessing may also be necessary, such as standardizing skewed language-model dimensions before cosine-based comparison.

6 Discussion and Open Research Challenges

The survey identifies major open challenges in applying similarity measures, especially determining applicability, interpreting scores, and ensuring robustness. These challenges limit informed measure selection and reliable conclusions about neural network similarity.

  • Applicability of Representational Similarity Measures: Different representational similarity measures can produce materially different results for the same pair of models.Choosing a measure therefore requires understanding which representational differences matter for the application.
  • Applicability of Representational Similarity Measures: Broad research on the applicability and geometry of representational similarity measures remains limited.The ReSi benchmark is described as an initial effort toward more systematic analysis.
  • Interpretability: Similarity scores are not always directly interpretable because nonlinearities and input context can make their numerical values misleading.For example, cosine similarity of 0.95 can correspond to an 18° angle, while disagreement of 0.05 can have different meanings across classification settings.
  • Interpretability: More research is needed to improve score interpretability through expected values or boundaries linked to input similarity and dimensionality.The survey emphasizes that contextual factors can strongly affect obtained scores.
  • Robustness of Representational Similarity Measures: Perturbing single-instance representations can strongly affect CKA scores, threatening reliability in applications that require robust similarity estimates.The survey highlights model-reuse identification in intellectual-property protection as one such application.

7 Conclusion

The survey presents representational and functional similarity as complementary perspectives and unifies their measures through formal definitions, categorization, and practical guidance. It also identifies limited analysis of measure properties and applicability as an important research gap.

  • Conclusion: Representational and functional similarity provide complementary perspectives for analyzing and comparing neural networks.Representational measures concern intermediate activations, whereas functional measures concern model outputs.
  • Conclusion: The survey formally defines 53 similarity measures and categorizes them into different types with pedagogical illustrations.This provides a systematic overview of measures spanning both perspectives.
  • Conclusion: The survey reviews salient measure properties and provides guidance for practical application.It connects this synthesis to the use of similarity measures in deep learning-related problems.
  • Conclusion: Limited unified research on representational-measure properties and applicability constrains the quality of practical recommendations.The survey argues that additional research is needed for more informed application and understanding of neural network similarity.
  • Conclusion: The survey establishes notation and reviews matrix norms used by some representational similarity measures.These include Frobenius and nuclear norms, with the Frobenius norm noted as invariant to orthogonal transformations.

A.3 Similarity Functions for RSMs

This section describes instance-wise similarity functions, transformation invariances, and preprocessing choices used in representational similarity measures. It emphasizes that normalization, dimensionality matching, and flattening affect what comparisons are made invariant to and how representations enter the measures.

  • Similarity Functions: Instance-wise similarity functions strongly influence which transformations RSM-based measures are invariant to.The survey presents common functions and summarizes their induced invariances.
  • Transformation Invariances: For ReLU, intertwiner-group invariance includes permutation invariance and can constrain measures such as Orthogonal Procrustes under normalized-column assumptions.The relevant transformations combine permutation matrices with positive diagonal matrices.
  • Preprocessing: Higher-dimensional CNN representations must be flattened before measures expecting matrix inputs can be applied.Flattening must account for model-specific representation structure.
  • Preprocessing: Normalization changes which geometric properties are preserved and therefore changes the assumptions behind similarity comparisons.Mean-centering imposes translation invariance, while different rescalings preserve or alter angles and Euclidean distances in different ways.
  • Preprocessing: Representations with unequal dimensionality require preprocessing such as zero-padding or dimensionality reduction.The survey specifically mentions PCA as a dimensionality-reduction method.

C.1 Correlation between Functional and Representational Measures

Studies comparing functional and representational similarity report heterogeneous relationships across measures, models, and evaluation settings. Some measures correlate more strongly with functional behavior in particular experiments, but no measure consistently outperforms others across broad tests, and CKA can be altered without changing functional properties.

  • Comparative Findings: Only limited work has examined whether functional and representational similarity track one another.Ding et al. studied this relationship using BERT and ResNet models under induced functional changes.
  • Comparative Findings: Orthogonal Procrustes and CKA correlated more with functional similarity than CCA measures after CNN neuron deletion.This finding came from ImageNet-trained CNNs.
  • Comparative Findings: CKA correlated less with model disagreement than RTD, while GULP correlated better with regression-output differences than CCA-based measures and CKA.These comparisons used different functional proxies and model settings.
  • Comparative Findings: No representational similarity measure consistently outperformed others across accuracy, disagreement, and Jensen-Shannon Divergence tests.The ReSi benchmark evaluated over 20 measures across vision, text, and graph models and multiple datasets.
  • Sensitivity to Transformations: CKA can approach zero for identical representations after translating one instance, without changing class-based separability.This demonstrates sensitivity to representational manipulations that need not affect functional similarity.

C.2 Discriminative Abilities of Representational Similarity Measures

Representational similarity measures differ substantially in what discriminative tests they pass, with performance depending on the task and experimental setting. No measure consistently dominates across layer matching, dimensionality, clustering, and other evaluations.

  • PWCCA was most robust to representations containing noise, whereas mean CCA was least robust when half the dimensions were noisy.The test assumed similar representations should share an identical subset of columns despite added random noise.
  • No measure consistently outperformed others for matching corresponding layers across models differing only in training seed.The desired match was the corresponding layer in the other model.
  • At higher sampled dimensionalities, CKA, Riemannian distance, RSA, and RSM norm difference all assigned high similarity to samples and their baseline representation.The experiments sampled between 10 and 50 dimensions from a neuroscience representation.
  • CKA, Orthogonal Procrustes, and GULP generally produced good architecture clusters, while CCA-based measures tended to perform worse; optimized GULP performed best overall.The comparison used ImageNet-trained models and pairwise representational similarity.
  • Results were inconclusive for whether models trained on similar datasets receive higher similarity scores than models trained on dissimilar datasets.Tang et al. compared CKA and NBS on this desideratum.
  • ContraSim outperformed CKA when comparing multilingual text representations and matching images with their true captions rather than captions from other images.ContraSim was specifically trained for each respective task.
  • The ReSi benchmark evaluates discrimination of behavioral differences from random labels, shortcut features, and augmentation, plus similarity's relationship with layer depth.The tests span vision, language, and other model settings.

C.3 Influence of Inputs

Similarity scores can be strongly influenced by the chosen inputs, including their similarities, specific features, number, and distribution. These effects can make models appear more similar or alter score stability independently of the intended comparison.

  • Input-instance similarity can make random networks score higher than same-task networks under RSA and CKA.Early-layer low-level features may not distinguish similar instances, and regression-based de-confounding was proposed.
  • Specific input features can inflate representational similarity when models exploit different features or shortcuts.Class-leaking pixels affected RSA, while feature co-occurrence overestimated CKA; eliminating co-occurrences sharply reduced similarity.
  • Angular Shape Metric similarity generally decreased as the input-to-dimension ratio N/D increased, then stabilized with additional inputs.This result was observed for two CNNs trained on CIFAR-10.
  • Inputs from different data distributions significantly affected CKA and Orthogonal Procrustes similarity scores.In-distribution and out-of-distribution model similarities were significantly correlated, but the correlation depended heavily on the dataset.

D Details on Evaluation of Representational Similarity Measures

The survey aggregates results from diverse representational-similarity tests into comparative rankings, while emphasizing that averaged ranks conceal substantial variation across test variants and are not significance tests. The evaluation therefore requires consulting the originating studies for application-specific interpretation.

  • Table 3 rankings average ranks across test variants before assigning an overall rank to each measure.Variants can differ in datasets and models.
  • Aggregated rankings can vary widely across model and data variants and do not indicate statistically significant differences.K-NN Jaccard similarity ranked 1 to 18 across variants in the JSD correlation test.
  • The survey recommends consulting the originating study for nuanced judgments about a measure's applicability to a specific scenario.The recommendation follows from variance across test variants and the interpretation limits of aggregated ranks.
  • Correlation-based tests: The evaluation ranks measures using correlations with disagreement, Jensen-Shannon divergence, squared error, or output differences, depending on the test.These rankings draw on reported Spearman correlations and, for disagreement, reported results from the evaluated benchmarks.
  • Benchmark-specific criteria: Other tests rank measures by areas under curves, matching accuracy, Kendall's τ, standard-deviation ratios, or AUPRC, according to each benchmark's reported criterion.The criteria cover noise addition, layer matching, dimension subsampling, cluster count, architecture clustering, shortcut affinity, augmentation, label randomization, and layer monotonicity.
  • Accuracy-based tests: Accuracy-based rankings compare measures using results across multilingual and image-caption test variants.ContraSim had higher accuracy than CKA in every considered variant for both benchmarks.

E Neural Network Similarity Beyond This Survey

Beyond the survey's representational and classification-focused functional measures, neural-network similarity can be studied for other tasks, through alternative comparison objects, and in training-oriented settings. These extensions introduce task-specific constraints and do not all generalize to post-training analysis.

  • Functional similarity beyond classification: Many functional similarity measures transfer beyond classification when suitable performance measures or white-box access are available.Soft and hard prediction measures remain limited to discrete-label outputs, while structured outputs make equivalence harder to define.
  • Functional similarity beyond classification: Structured outputs such as generated text or images complicate functional similarity because outputs may differ while remaining semantically equivalent.The passage specifically notes visually imperceptible differences between generated images.
  • Alternative similarity notions: Alternative similarity notions compare decision regions, neuron activations, weight matrices, or the influence of training data on predictions.Some approaches are specialized to particular network types, while weight- and influence-based methods are more universal.
  • Dimensionality effects: The survey's Orthogonal Procrustes illustration shows similarity increasing with dimensionality until N = D, then remaining flat.For N > D, the dimensionality relationship follows a power law; the figure uses N = 1000 rows.
  • Training-oriented similarity: Training-oriented similarity methods prioritize differentiability and computational efficiency and often assume a shared representation space.They evaluate instance-level or fixed-layer representations and can map different-model representations into a joint space during training.
  • Training-oriented similarity: Training-oriented approaches do not generalize to post-training analysis of neural networks.The survey distinguishes their optimization assumptions and objectives from its post-training similarity measures.

F Orthogonal Procrustes and Dimensionality

The experiment examines how Orthogonal Procrustes similarity scores vary with representation dimensionality using synthetic matrices and shuffled-row baselines.

  • Orthogonal Procrustes scores are plotted over varying representation dimensions to assess dimensionality effects.
  • The synthetic matrix pairs use standard-normal entries, random Haar-distributed orthogonal transformations, and added normal noise.
  • For each D in {10, 50, 100, 200, 300, 1000, 2000, 3000}, the process is repeated ten times and mean distances are reported.
  • A shuffled-row baseline is computed by permuting the original matrix rows ten times for each representation pair and reporting the mean score.
  • The broader test overview distinguishes synthetic-data tests from tests using models, datasets, or raw neuroimage data.

G Transformations for Figure 2

Figure 2 uses named transformations to illustrate invariances, including rotation, reflection, and axis permutation operations.

  • The Figure 2 illustrations include AT, ILT, and TR invariances.
  • The illustrations also use OT, PT, and IS invariances with specified transformation parameters.
  • Transformation Q rotates the representation by 120 degrees and reflects it across the 15 degree axis.
  • Permutation P swaps the axes in the coordinate system.
Loading 2305.06329v4…