PaperlayerView exact revision on arXiv

Source-linked AI summary

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability

Maithra Raghu, Justin Gilmer, Jason Yosinski, Jascha Sohl-Dickstein

arXiv:1706.05806v2stat.MLcs.LG

TL;DR

Understanding learned representations is central to interpreting deep networks, but comparing them across layers and architectures is challenging. The paper introduces SVCCA, combining SVD and canonical correlation analysis, and uses it to examine dimensionality, learning dynamics, and class sensitivity. It reports bottom-up convergence to final representations and proposes Freeze Training as a way to reduce computation and potentially improve generalization.

  • Problem

    Understanding learned representations is central to interpreting deep networks, and prior work compared network representations by finding maximally correlated neuron permutations.

    Source: §1 · p. 1

  • Method

    SVCCA treats neuron activations over a finite input set as vectors and combines SVD with canonical correlation analysis to compare representations.

    Source: §1 · p. 1, §2 · p. 2

  • Results

    Networks broadly converge to final representations from the bottom up, while SVCCA also reveals layer dimensionality and class sensitivity and supports faster convolutional comparisons using discrete Fourier transforms.

    Source: §1 · pp. 1–2, §3.1 · p. 5

  • Takeaways & Limitations

    The bottom-up convergence finding motivates Freeze Training, which sequentially freezes lower layers to reduce training computation and may improve generalization.

    Source: §1 · p. 2, §5 · p. 8

    Representations are defined by neuron responses over a finite training or validation input set, rather than all possible inputs.

    Source: §2 · p. 2

Abstract

from arXiv · show

We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less. Code: https://github.com/google/svcca/

1 Introduction

Understanding learned representations is central to interpreting deep networks, while prior work had compared network representations by matching correlated neurons. The paper introduces SVCCA and uses it to study layer dimensionality, learning dynamics, class sensitivity, and efficient training.

1 Introduction

  • SVCCA combines SVD and canonical correlation analysis to compare layers as subspaces of neuron activation vectors.In the toy example, individual neuron traces differ across networks while their top SVCCA directions are nearly identical up to sign.Source: §1 · p. 1, Figure 1 · p. 2
  • Networks broadly converge to final representations from the bottom up, motivating Freeze Training, which sequentially freezes lower layers to reduce training computation.The paper also reports that this technique may improve generalization.Source: §1 · pp. 1–2
  • Trained networks can perform equally well using carefully selected directions numbering only a fraction of the layer’s neurons, revealing potential over-parameterization.The paper explores implications for model compression.Source: §1 · p. 1
  • SVCCA captures class semantics through class sensitivity: similar classes have similar sensitivities, and dissimilar classes have dissimilar sensitivities.Source: §1 · p. 2
  • A discrete-Fourier-transform method greatly speeds SVCCA for convolutional neural networks.Source: §1 · p. 2

2 Measuring Representations in Neural Networks

The paper defines representations through neuron activation vectors over a finite dataset, treating layers as their spanned subspaces. SVCCA combines SVD and CCA to compare such subspaces and reveal compact, distributed directions.

2 Measuring Representations in Neural Networks

  • A neuron is represented by its responses across a finite dataset, and a layer is the subspace spanned by its neuron vectors.The paper uses training or validation examples because full input-output tables are infeasible and its interest is in features of a specific dataset.Source: §2 · pp. 2–3
  • SVCCA first uses SVD to retain directions explaining typically 99% of variance, then uses CCA to align the reduced subspaces and report correlated direction pairs.CCA's affine-transform invariance supports comparisons without natural alignment, while SVD identifies important directions in the original spaces.Source: §2 · p. 3, §2.1 · p. 4

2.1 Distributed Representations

  • The method is a subspace analysis rather than a neuron-axis-aligned comparison, and a toy regression demonstration shows it can find hidden similarities between representations.The paper also reports that projecting onto top SVCCA directions can preserve the network's task performance.Source: §2.1 · pp. 3–4, §2 · p. 3
  • 25 SVCCA directions provide nearly the same accuracy as all 512 dimensions of a fully connected layer, while random and max-activation neuron baselines need more directions.The accuracy rises rapidly with the first few directions and plateaus quickly.Source: §2.1 · p. 4, Figure 2 · p. 4
  • Some useful SVCCA directions are distributed across neurons: 100 fc1 neurons cannot match the accuracy of the first 20 SVCCA directions.This comparison suggests those 20 directions are spread across five or more neurons each, on average.Source: §2.1 · pp. 3–4

3 Scaling SVCCA for Convolutional Layers

For convolutional layers, SVCCA compares corresponding layers by matching spatial views and compares different layers by flattening their activations. A per-channel Fourier transform makes exact computation more tractable under stated translation and boundary assumptions.

3 Scaling SVCCA for Convolutional Layers

  • For same-layer comparisons, concatenating pixel coordinates makes each channel a vector across spatial positions and datapoints, with corresponding views aligned across versions of the layer.For different layers, the method instead flattens each convolution into h×w×c neurons because their image patches have no natural correspondence.Source: §3 · p. 5

3.1 Scaling SVCCA with Discrete Fourier Transforms

  • Convolutional layers make SVCCA costly because they contain many neurons, especially early in the network.The large matrices involved make direct computation prohibitively expensive.Source: §3.1 · p. 5
  • Applying a discrete Fourier transform to each channel makes covariance matrices block-diagonal, allowing exact SVCCA through blockwise operations on the diagonal blocks.The theorem states that the blocks have size c×c for translation-invariant image data.Source: §3.1 · p. 5
  • The exact Fourier-based result assumes circular boundary conditions in preceding convolution or pooling layers and a dataset containing all translations of each image.The paper notes that the activation symmetries used in the proof typically exist in natural images even without translation invariance.Source: §3.1 · p. 5

4 Applications of SVCCA

SVCCA is used to examine learning dynamics, class sensitivity, and similarities across architectures, and to motivate Freeze Training and model compression. The applications show bottom-up representational convergence, semantic patterns in class sensitivity, and reduced training or inference costs in the studied settings.

4.1 Learning Dynamics with SVCCA

  • SVCCA comparisons across layers and training times show broad bottom-up convergence, with the very top layers an exception.The plots compare each layer at each timestep with layers at the final timestep.Source: §4.1 · p. 6, Figure 4 · p. 7

4.2 Freeze Training

  • Freeze Training saves computation and appears to improve generalization on CIFAR-10 for both convolutional and residual networks.The convolutional network freezes individual layers at evenly spaced times; the residual network freezes whole blocks.Source: §4.2 · p. 6, Figure 5 · p. 7

4.3 Interpreting Representations: when are classes learned?

  • Class-logit similarity reflects semantic relationships: firetruck separates from the two dog-breed pairs, whose members have similar curves.Firetruck sensitivity also appears earlier; the dog breeds were selected as visually similar pairs.Source: §4.3 · p. 6, Figure 6 · p. 8

4.4 Other Applications: Cross Model Comparison and compression

  • Across a CIFAR-10 ResNet and ConvNet, lower layers are more similar than higher layers, and the ResNet’s highest layers are least similar to the ConvNet’s lower layers.Source: §4.4 · p. 7, Figure 7 · p. 8
  • Projecting layer activations onto top SVCCA directions creates a bottleneck that reduces parameter count and inference cost; one case reduced each top layer to 0.35 of its original size with little performance effect.Source: §4.4 · pp. 7–8

5 Conclusion

The paper concludes that SVCCA enables comparisons of learned representations across layers and architectures, yielding insights into dynamics and interpretability and motivating training and compression techniques. Its mathematical details describe CCA’s search for aligned directions and SVCCA’s SVD-then-CCA procedure.

5 Conclusion

  • SVCCA compares learned distributed representations across neural-network layers and architectures, and its analyses motivate Freeze Training and model compression.The conclusion also notes class sensitivity that reflects semantic properties and bottom-up learning in recurrent networks.Source: §5 · p. 8

A Mathematical details of CCA and SVCCA

SVCCA Summary

  • The SVCCA summary procedure takes X and Y, performs SVD on each, then applies CCA to the resulting representations to output aligned directions and their correlations.Source: SVCCA Summary · p. 11

B Additional Proofs and Figures from Section 2.1

The proof shows that orthonormal transformations of the input sets preserve the CCA correlation coefficients.

B Additional Proofs and Figures from Section 2.1

B.0.1 The importance of SVD: how many directions matter?

CCA coefficients alone can make subspaces with very different useful content look identical. Applying SVD first to retain directions explaining 99% of variance distinguishes these cases.

B.0.1 The importance of SVD: how many directions matter?

  • CCA recovers an original activation signal after the compared directions have been permuted, rotated, and scaled.The example compares a penultimate-layer neuron’s activations across ordered CIFAR-10 image ids with distorted directions.Source: §B.0.1 · p. 12
  • The subspaces B and C have identical canonical-correlation coefficients with A, although C contains 150 additional useful directions.B’s 150 unaligned directions are noise, while C’s are useful but different from A’s.Source: §B.0.1 · p. 12
  • SVD selects the directions needed to explain 99% of observed variance, allowing SVCCA to distinguish pathological cases that coefficient aggregation misses.This distinction matters when canonical correlations are aggregated to measure similarity in network learning dynamics.Source: §B.0.1 · p. 12

C Proof of Theorem 1

The proof uses translation structure and the DFT to simplify convolutional-layer covariances. DFT preprocessing preserves CCA vectors up to rotation.

C Proof of Theorem 1

  • The proof section establishes the DFT covariance structure using an n by n channel representation and column-stacked vectorization.The supplied proof introduces the channel shape and vectorization before deriving the covariance result.Source: C Proof of Theorem 1 · pp. 12–13
  • For a translation-invariant dataset, covariance in the DFT basis is exactly diagonal; without translation augmentation it is approximately diagonal, while pixel-basis covariance remains dense.The figure contrasts pixel and DFT bases with and without translation augmentation.Source: C Proof of Theorem 1 · p. 13
  • The DFT-transformed neurons have the same CCA vectors as the original neurons, up to rotation.The proof rewrites vectorized DFT as multiplication by a unitary Kronecker product, which preserves CCA up to rotation.Source: C Proof of Theorem 1 · p. 13

C.1 Proof of Lemma 1

The proofs connect translation invariance to circulant covariance structure, which the DFT simplifies into diagonal or block-diagonal forms. That structure also yields a lower-cost, parallelizable computation.

C.1 Proof of Lemma 1

  • Translation-invariant inputs produce translation-invariant channel outputs, giving covariance a block-circulant structure.The proof relates shifts in neuron coordinates to corresponding shifts in the input and covariance indices.Source: §C.1 · pp. 13–14

C.2 Proof of Lemma 2

  • The proof invokes the DFT’s diagonalization of circulant matrices to establish the relevant matrix result.It notes that the DFT matrix’s eigenvectors are orthogonal.Source: §C.2 · p. 14

C.3 Proof of Theorem 2

  • Using the circulant and block-circulant covariance structure, the proof derives diagonal covariance after DFT transformation.The argument covers the cross-covariance case as well as the self-covariance case.Source: §C.3 · p. 15

C.4 Proof of Theorem 1

  • Theorem 1’s proof arranges the transformed neuron directions so nonzero covariance terms occupy diagonal blocks.The proof identifies n^2 blocks of size k by k.Source: §C.4 · p. 15

C.5 Computational Gains

  • The covariance computation costs roughly kn log n + n^2k^2.5, versus (kn^2)^2.5 naively, and the DFT blocks can be computed in parallel.The algorithm takes the DFT of each channel and computes covariance operations within the nonzero blocks.Source: §C.5 · p. 15

E Additional Figure from Section 4.4

  • Different initializations of the same convolutional architecture show similar representations in the initial and final layers, but differing similarity in middle layers.The comparison is of converged CIFAR-10 representations.Source: E Additional Figure from Section 4.4 · p. 16

G Learning Dynamics for an LSTM

The compression figure compares SVCCA representations and reports compression of the final five fully connected layers. A Penn Tree Bank LSTM shows a similar bottom-up convergence pattern, with lower layers converging faster.

G Learning Dynamics for an LSTM

  • pool1, fc1, bn3, fc2, and bn4 are each compressed to 0.35 of their original size.The figure reports compression of fully connected layers in a CIFAR-10 convnet.Source: G Learning Dynamics for an LSTM · p. 17
  • Using layer activations and logits as the SVCCA pair gives better compression results than comparing two differently initialized and trained convnets.The two alternatives are the figure’s sources for the SVCCA representations.Source: G Learning Dynamics for an LSTM · p. 17
  • In a Penn Tree Bank language-modeling LSTM, lower layers converge faster than upper layers.The figure reports a learning-dynamics pattern similar to convolutional networks trained on image data.Source: G Learning Dynamics for an LSTM · p. 17
Loading 1706.05806v2…