PaperlayerView exact revision on arXiv

Source-linked AI summary

Similarity of Neural Network Representations Revisited

Simon Kornblith, Mohammad Norouzi, Honglak Lee, Geoffrey Hinton

arXiv:1905.00414v4cs.LGq-bio.NCstat.ML

TL;DR

The paper asks how neural-network representations can be compared when conventional CCA-based indexes are invariant to transformations that erase meaningful structure. It introduces CKA, which compares representational similarity matrices, and shows that it avoids the high-dimensional limitation while reliably identifying correspondences across differently trained networks.

  • Problem

    Similarity indexes invariant to invertible linear transformations cannot measure meaningful differences between representations whose width is at least the number of data points.

  • Method

    The paper introduces centered kernel alignment (CKA), which compares representational similarity structures and connects this approach to CCA and related indexes.

  • Results

    CKA reliably identifies layer correspondences across networks trained from different initializations and with different widths, and across different architectures.

  • Takeaways & Limitations

    CKA provides a similarity index for comparing neural-network representations across layers and networks in settings where previously proposed indexes fail.

  • Takeaways & Limitations

    The paper notes that invertible-linear-invariant similarity indexes give the same result when representation width reaches the dataset size and can discard meaningful scale information.

Abstract

from arXiv · show

Recent work has sought to understand the behavior of neural networks by comparing representations between layers and between different trained models. We examine methods for comparing neural network representations based on canonical correlation analysis (CCA). We show that CCA belongs to a family of statistics for measuring multivariate similarity, but that neither CCA nor any other statistic that is invariant to invertible linear transformation can measure meaningful similarities between representations of higher dimension than the number of data points. We introduce a similarity index that measures the relationship between representational similarity matrices and does not suffer from this limitation. This similarity index is equivalent to centered kernel alignment (CKA) and is also closely connected to CCA. Unlike CCA, CKA can reliably identify correspondences between representations in networks trained from different initializations.

1. Introduction

The paper studies how to measure similarities between neural network representations and introduces CKA to compare their similarity structures. CKA identifies correspondences across layers, initializations, widths, architectures, and datasets more broadly than prior indexes.

  • Motivation: The paper frames representational similarity as a way to study what networks learn across architectures, initializations, datasets, and layers.These comparisons can provide insight into learned representations and their relationship to training and data.
  • Contributions: The authors analyze similarity-index invariance properties and their implications for comparing neural network representations.
  • Contributions: They introduce centered kernel alignment (CKA) and relate it to linear regression, canonical correlation analysis, and related methods.
  • Results: CKA determines correspondences between hidden layers of networks trained from different initializations and with different widths, where earlier indexes fail.
  • Results: Wider networks learn more similar representations, with early-layer similarity saturating at fewer channels than later-layer similarity.
  • Results: Early layers, but not later layers, learn similar representations when networks use different datasets.

2. What Should Similarity Be Invariant To?

The paper argues that useful similarity indexes should ignore orthogonal transformations and isotropic scaling but retain meaningful information lost under arbitrary invertible linear transformations. Such invariance becomes uninformative when representation width reaches the number of examples.

  • Desired invariances: The authors favor invariance to orthogonal transformation and isotropic scaling, but not to arbitrary invertible linear transformation.
  • Invertible-linear invariance: Invertible-linear invariance makes every representation of width p2 ≥ n receive the same similarity result.The limitation follows from the fact that full-rank representations can become indistinguishable to any such invariant index.
  • Invertible-linear invariance: This limitation is practical because some neural-network layers contain more neurons than examples in the training dataset.
  • Training dynamics: Neural-network training is not invariant to arbitrary invertible transformations, since optimization can depend on input scale and covariance structure.
  • Training dynamics: Representations from different initializations share similar large principal components, so invertible-linear-invariant indexes discard potentially meaningful scale information.The associated Euclidean distances between examples are similar across networks and support perceptual applications.
  • Desired invariances: Orthogonal transformations preserve scalar products and Euclidean distances and remain well-defined when p2 > n.

3. Comparing Similarity Structures

The paper compares representations by comparing their pairwise similarity structures rather than directly matching multivariate features. This leads to CKA, a normalized kernel-based index connected to dot products, HSIC, and established multivariate similarity measures.

  • Similarity structures: The central idea is to compare the similarity between every pair of examples in each representation, then compare the resulting similarity structures.
  • Dot-product similarity: For centered representations, dot products between examples are encoded in XX^T and YY^T, whose relationship measures similarity between inter-example structures.The same quantity can also be expressed through squared dot products between feature pairs.
  • Kernel similarity: HSIC generalizes the dot-product construction to kernels by comparing kernel matrices through the Hilbert-Schmidt norm of cross-covariance.
  • Centered kernel alignment: Centered kernel alignment normalizes HSIC to make it invariant to isotropic scaling.
  • Connections: With a linear kernel, CKA is equivalent to the RV coefficient and Tucker’s congruence coefficient.
  • Kernel selection: RBF and linear kernels produce similar results across most experiments, so the paper uses linear CKA unless otherwise specified.

4. Related Similarity Indexes

The paper reviews linear regression, CCA, SVCCA, PWCCA, neuron-alignment procedures, and mutual information as approaches to comparing neural network representations.

  • Linear Regression: Linear regression relates representations by fitting each feature in X as a linear combination of features from Y.Its summary statistic is the total fraction of variance explained by the fit.
  • Canonical Correlation Analysis: CCA finds bases for two representations whose projected variables maximize corresponding correlation coefficients.Canonical weights transform the original data into canonical variables, with constraints enforcing orthogonality.
  • CCA Variants: SVCCA improves CCA robustness by applying it to truncated singular value decompositions, retaining components that explain a fixed variance proportion.Its invariance to invertible linear transformation holds only when the retained subspace remains unchanged.
  • CCA Variants: PWCCA reduces CCA’s perturbation sensitivity through projection weighting and is closely related to linear regression.The paper connects this relationship explicitly in Appendix C.3.
  • Neuron Alignment Procedures: Neuron-alignment methods match individual neurons or seek mutually representable neuron subsets rather than comparing representation subspaces.Maximum matching subsets were found to be very small for intermediate layers.
  • Mutual Information: Mutual information is invariant to arbitrary invertible transformations, but the paper argues it is not useful for comparing deterministic representations of the same input.The supplied passage introduces this concern but truncates its supporting explanation.

5. Linear CKA versus CCA and Regression

The paper relates linear CKA to CCA and linear regression by weighting shared directions according to explained variance. This weighting is symmetric and avoids matrix decomposition.

  • Relationship to Existing Indexes: Linear CKA is closely related to CCA and linear regression for centered representations.The relationship follows from expressing the representations through orthonormal bases and their scalar products.
  • Variance Weighting: Linear CKA weights corresponding eigenvectors by the variance each explains in the representations.This distinguishes it from unweighted canonical-correlation summaries.
  • Empirical Comparison: Figure 2 compares similarity-index correspondence patterns across layers of CNNs trained from different random initializations.The figure reports that CKA reveals consistent relationships whereas CCA, linear regression, and SVCCA do not.
  • Computational Form: Linear CKA incorporates this weighting symmetrically and can be computed without a matrix decomposition.The paper contrasts this construction with related CCA-based alternatives.
  • Alternative Weightings: The paper identifies canonical ridge regularization as an alternative weighting that interpolates between CCA, linear regression, and linear CKA.The interpolation is described for an appropriately normalized similarity index.

6. Results

Across experiments, CKA passes the layer-correspondence sanity check and exposes structural relationships across depth, architecture, width, and training datasets. It also reveals when deeper networks become pathological.

  • 6.1. A Sanity Check for Similarity Indexes: CKA passes the layer-correspondence sanity check on architecturally identical CNNs, whereas other tested indexes perform substantially worse.The comparison includes CCA, SVCCA, PWCCA, and linear regression.
  • 6.1. A Sanity Check for Similarity Indexes: All similarity indexes achieve non-trivial correspondence accuracy on Transformer encoder sublayers, with RBF CKA and CCA slightly outperforming other methods.The representations differ in feature scale between feed-forward and self-attention sublayers, which CCA does not capture because it is invariant to non-isotropic scaling.
  • 6.2. Using CKA to Understand Network Architectures: At 8x depth, more than half of a CNN’s later layers are very similar to its last layer, while classification accuracy plateaus less than halfway through the network.Doubling depth improved accuracy, but greater depth multipliers hurt accuracy.
  • 6.2. Using CKA to Understand Network Architectures: In ResNet-62, same-block layers are more similar than layers from different feature-map scales, producing a grid pattern originating from the architecture.Post-residual activations resemble other post-residual activations, whereas within-block activations do not.
  • 6.2. Using CKA to Understand Network Architectures: Across architectures, CKA indicates that increasing depth effectively inserts new layers between the original layers.Other similarity indexes fail to reveal meaningful relationships between the architectures shown.
  • 6.2. Using CKA to Understand Network Architectures: As width increases, CKA approaches 1 and earlier-layer similarity saturates faster than later-layer similarity.Networks are generally more similar to other networks of the same width than to the widest trained network.
  • 6.2. Using CKA to Understand Network Architectures: Models trained on CIFAR-10 and CIFAR-100 develop similar early-layer representations, whereas similarity with untrained networks is much lower.The cross-dataset comparison uses CKA on test-set representations.
  • 6.4. Analysis of the Shared Subspace: For large eigenvectors, two representational similarity matrices have similar actions, but their shared subspace can be much lower-rank than activation dimensionality.In the penultimate CIFAR-10 layer, the shared-subspace dimensionality is approximately 10, matching the number of classes.

7. Conclusion and Future Work

The paper frames representation similarity as ill-defined and proposes CKA, which compares representational similarity structures to identify correspondences across networks.

  • Similarity between neural-network representations is ill-defined because it is unclear which aspects a similarity index should measure.
  • The shared subspace of two independently initialized CIFAR-10 networks is primarily spanned by eigenvectors with the largest eigenvalues.
  • CKA compares pairwise example-similarity structures rather than directly comparing multivariate feature vectors.
  • CKA consistently identifies layer correspondences across networks trained from different initializations and across different architectures, unlike earlier methods.
  • The paper leaves open whether kernels beyond linear and RBF kernels, or alternative Equation 14 weightings, would better analyze neural-network representations.

A. Proof of Theorem 1

The proof establishes that full-row-rank representations become indistinguishable under any similarity index invariant to invertible linear transformations, and connects regression-based measures with CCA.

  • If X and Y both have rank n, any index invariant to invertible linear transformations gives s(X,Z)=s(Y,Z).
  • The proof constructs an invertible transformation A by extending X and Y with bases for their row-null spaces.
  • Orthogonalizing representation columns makes any orthogonally invariant similarity index invariant to invertible linear transformations.
  • Linear regression uses an orthonormal basis from a thin QR decomposition, with fitted values obtained by projecting one representation onto the other.
  • CCA computes canonical correlations as singular values of QX^TQY, equivalently as square roots of the eigenvalues of A^TA.
  • Projection-weighted canonical correlation connects to linear regression when dot products and canonical correlations are squared.

D.1. Canonical Ridge

Canonical ridge regularizes CCA-related quantities, but normalization reveals connections to regression and CKA while its hyperparameters limit exploratory usefulness.

  • Canonical ridge introduces regularization into CCA-related objectives, producing partially orthogonalized bases from the singular decompositions of X and Y.
  • Regularized singular values no longer measure canonical-variable correlations and shrink as either regularization parameter increases.
  • With κX=κY=0, the ridge objective reduces to the sum of squared canonical correlations, whose normalized form recovers R2.
  • When κY=0 and κX approaches infinity, normalization by the specified bound recovers R2.
  • Normalizing by the multiplicatively separable bound yields CKA, and equal regularization followed by κ→∞ yields CKA(XXT,YYT).
  • Canonical ridge is less useful for exploratory analysis because selecting its hyperparameters by cross-validation is computationally expensive and sample-size biased.

E. Architecture Details

The experiments use compact convolutional and Transformer architectures, with similarity evaluated across layers, sublayer locations, and corresponding Transformer encoders.

  • Convolutional architectures: The convolutional experiments use All-CNN-C-based models, including Tiny-10 and Plain-(8n + 2) architectures, with some modern architectural modifications.Plain models place the final linear layer after average pooling and use batch normalization; the models train in minutes on modern hardware.
  • Transformer architecture: The Transformer encoder alternates between self-attention and feed-forward network sublayers across 12 sublayers.Representations are sampled at four possible locations within each sublayer for similarity analysis.
  • Similarity evaluation: Similarity methods are evaluated by identifying corresponding sublayers from maximum similarity across architecturally identical 12-sublayer Transformer encoders.The evaluation covers four representation locations within each sublayer.
  • Transformer experiments: Transformer representations are computed for 75,804 tokens from the WMT18 English-to-German translation development set.The models use tensor2tensor and the newstest2013 development set, excluding end-of-sentence tokens.
  • Similarity evaluation: CKA reveals a checkerboard similarity pattern aligned with alternating attention and FFN sublayers, including structure before the attention/FFN operation that CCA misses.CCA gives identical similarities before and after channel-wise scaling because it is invariant to non-isotropic scaling.
  • Similarity evaluation: SVCCA was tested across truncation thresholds beyond 0.99, but no threshold revealed the network structure.The threshold experiments extend the threshold suggested by Raghu et al. (2017).

F.4. Additional CKA Results

Additional experiments examine CKA across normalization settings, initialization, classes, and architectures, finding consistent or architecture-specific representation correspondences.

  • Normalization: CKA finds similar representations in batch-normalized and non-batch-normalized networks trained from different random initializations.The largest difference occurs at the last convolutional layer; average accuracies are 93.9% with batch normalization and 91.5% without it.
  • Initialization: Layer similarity at initialization is evaluated for deeper networks, extending the analysis beyond the primary network comparisons.The supplied figure passage identifies the analysis but does not report a specific outcome.
  • Class structure: CKA measures similar within-class structure to the structure obtained from all examples.Within-class CKA is computed separately for each CIFAR-10 class using Plain-10 representations and then averaged.
  • Different architectures: Only CKA reveals meaningful correspondence between layers of Tiny-10 and ResNet-14 architectures.The comparison uses the CIFAR-10 training set because CCA-based methods are sensitive to the number of examples.
Loading 1905.00414v4…