Opening the Black Box of Deep Neural Networks via Information

Ravid Shwartz-Ziv; Naftali Tishby

Source-linked AI summary

Opening the Black Box of Deep Neural Networks via Information

Ravid Shwartz-Ziv, Naftali Tishby

arXiv:1703.00810v3Mar 2, 2017cs.LG

TL;DR

The paper addresses the lack of a comprehensive understanding of DNN optimization and internal organization by visualizing layer representations in the Information Plane. It finds distinct fitting and compression phases, near-IB representations, and a computational benefit from hidden layers, while noting limitations in generality and real-world information estimation.

Problem
DNNs have achieved major success, but their optimization process and internal organization remain insufficiently understood.
Method
The paper analyzes DNN layers in the Information Plane using mutual-information measurements and Information Bottleneck concepts, alongside SGD experiments.
Results
The analysis identifies fitting and compression phases, finds converged layers near the IB bound, and reports that hidden layers reduce stochastic relaxation times.
Takeaways & Limitations
DNNs appear to learn efficient representations through SGD-driven compression, with hidden layers offering a computational advantage.
Takeaways & Limitations
The authors leave open whether the findings generalize to other decision rules and network architectures, and note that real-world information estimation is beyond the study’s scope.

Abstract

from arXiv · show

Despite their great success, there is still no comprehensive theoretical understanding of learning with Deep Neural Networks (DNNs) or their inner organization. Previous work proposed to analyze DNNs in the \textit{Information Plane}; i.e., the plane of the Mutual Information values that each layer preserves on the input and output variables. They suggested that the goal of the network is to optimize the Information Bottleneck (IB) tradeoff between compression and prediction, successively, for each layer. In this work we follow up on this idea and demonstrate the effectiveness of the Information-Plane visualization of DNNs. Our main results are: (i) most of the training epochs in standard DL are spent on {\emph compression} of the input to efficient representation and not on fitting the training labels. (ii) The representation compression phase begins when the training errors becomes small and the Stochastic Gradient Decent (SGD) epochs change from a fast drift to smaller training error into a stochastic relaxation, or random diffusion, constrained by the training error value. (iii) The converged layers lie on or very close to the Information Bottleneck (IB) theoretical bound, and the maps from the input to any hidden layer and from this hidden layer to the output satisfy the IB self-consistent equations. This generalization through noise mechanism is unique to Deep Neural Networks and absent in one layer networks. (iv) The training time is dramatically reduced when adding more hidden layers. Thus the main advantage of the hidden layers is computational. This can be explained by the reduced relaxation time, as this it scales super-linearly (exponentially for simple diffusion) with the information compression from the previous layer.

1. Introduction

The paper uses Information-Plane visualization to investigate DNN training dynamics and internal representations, identifying distinct fitting and compression phases and links to the Information Bottleneck.

Information-Plane visualization is used to study DNN training dynamics, learning processes, and internal representations.
SGD has distinct empirical-error-minimization and representation-compression phases with different gradient signal-to-noise ratios.The compression phase is characterized by gradient fluctuations larger than their means and diffusion-like weight changes.
Most optimization epochs are spent compressing input information in internal representations under the training-error constraint.The paper reports that this occurs without explicit regularization or sparsity.
Converged layers lie on or near the Information Bottleneck bound and satisfy self-consistent encoder-decoder relationships.The optimized hidden layers move upward in the information plane as the training sample size increases.
The paper reports that hidden layers provide a computational benefit by reducing stochastic relaxation times.

2. Information Theory of Deep Learning

This section frames DNN representations through mutual information, Markov-chain structure, and the Information Bottleneck tradeoff between compression and prediction. It also states the scope and limitations of applying this analysis.

A DNN produces a Markov chain of hidden-layer representations optimized by SGD to predict labels from input patterns.
Each layer is treated as one random variable characterized by encoder and decoder distributions and information-plane coordinates I(X; T) and I(T; Y).
The information plane represents layers as connected points whose label information is constrained by the Data Processing Inequality.The ordering is I(X; Y) ≥ I(T1; Y) ≥ I(T2; Y) ≥ ... ≥ I(Tk; Y) ≥ I(Ŷ; Y).
The Information Bottleneck framework seeks approximate minimal sufficient statistics by optimizing the tradeoff between input compression and label prediction.
The IB Lagrange multiplier β determines the relevant information retained by a representation, with solutions specified by self-consistent equations along the information curve.
For deterministic functions, mutual information cannot reflect function complexity, while stochastic label rules can encode decision-boundary information and learning complexity.
For large real-world problems, the required distributions and mutual information values must be estimated from samples or modeling assumptions, which this work does not address.

3. Numerical Experiments and Results

The numerical studies use fully connected feed-forward DNNs trained with standard SGD and cross-entropy settings on binary, rotation-invariant classification rules over 12 binary inputs.

The experiments use fully connected feed-forward networks with up to 7 hidden layers and no explicit regularization.Neurons use shifted hyperbolic-tangent activations, with a sigmoidal final layer, SGD, and cross-entropy loss.
The tasks are binary decision rules invariant under O(3) rotations of a sphere, using 12 binary inputs representing uniformly distributed sphere points.
The 4096 input patterns are partitioned into 64 disjoint rotation-group orbits forming a minimal sufficient partition for the symmetric rules.
Labels are generated by thresholding a spherically symmetric function and then softening the rule with a sigmoid.The sigmoid gain is chosen so that I(X; Y) is approximately 0.99 bits.

3.2 Estimating the Mutual Information of the Layers

The analysis treats each hidden layer as a multivariate variable and estimates its mutual information with inputs and labels from discretized neuron activations and known input-output distributions.

Each network layer Ti is treated as a single variable, and mutual information with inputs and labels is calculated for every hidden layer.
Neuron arctan activations are binned into 30 equal intervals between -1 and 1 to estimate joint distributions.
The encoder and decoder mutual informations I(X; Ti) and I(Ti; Y) are computed using the Markov chain Y → X → Ti.
The calculations are repeated across 50 randomized weight initializations and random training-sample selections.

3.3 The dynamics of the training by Stochastic-Gradient-Decent

The study tracks 50 randomized networks in the Information Plane by plotting each layer’s input and label mutual information during SGD.

The analysis plots IX = I(X; Ti) and IY = I(Ti; Y) for every layer across 50 randomized initializations.

3.4 The two optimization phases in the Information Plane

The Information-Plane trajectories reveal a short label-fitting phase followed by a much longer representation-compression phase, with sample size affecting the latter’s label information.

During optimization, layers first increase IY and later decrease IX, compressing their representations.
The trajectories of differently randomized networks follow similar paths and converge to nearby Information-Plane points.
The fast ERM phase lasts a few hundred epochs, whereas the later compression phase is much longer and removes irrelevant input information.
Without explicit regularization or sparsification, SGD still produces representation compression, which the authors identify as their main result.
In the 5% sample condition, compression significantly reduces label information, while with 85% samples it mostly increases label information.

3.5 The drift and diffusion phases of SGD optimization

SGD transitions from high-signal drift to low-signal diffusion, linking gradient behavior to representation compression and leaving unresolved why layers converge differently.

The gradient dynamics transition from a high-SNR drift phase to a low-SNR diffusion phase as empirical error saturates.
During drift, large gradient means rapidly increase IY; during diffusion, small means and large fluctuations add noise while preserving the training-error constraint.
Diffusion can produce more compressed representations by minimizing IX for each layer under an empirical-error or IY constraint.
The reason different hidden layers converge to different Information-Plane points remains unclear, although differing gradient noise may lead to different maximum-entropy distributions.
The final weights are randomized without evidence of vanishing connections or norm decreases, and weak neuron correlations imply many networks can achieve essentially optimal performance.

3.6 The computational benefit of the hidden layers

Adding hidden layers substantially accelerates useful learning by shortening representation compression, while extra width does not provide the same benefit.

Computational benefit: Adding hidden layers dramatically reduces the epochs needed for good generalization.With one hidden layer, good IY values were not reached after 10^4 epochs, whereas six hidden layers reached full relevant output information within 400 epochs.
Computational benefit: Each layer’s compression phase is shorter when it starts from a previously compressed representation.Networks with five or six hidden layers reached their endpoints in roughly half the epochs required with four layers.
Compression dynamics: Deeper, narrower layers compress faster and pull lower layers along during the diffusion phase.Additional intermediate representations accelerate compression, reversing the drift-phase ordering in which lower layers move first.
Width versus depth: Even wide hidden layers eventually compress their inputs, so adding width does not improve learning in these simulations.A width-12 first hidden layer remained near the upper-right information-plane corner until sufficient SGD epochs had elapsed.
Figure 5: Figure 5 compares information-plane paths across architectures with different hidden-layer counts, using line colors to indicate training epochs.The architectures use widths starting at 12 neurons, with each added layer two neurons narrower; the final two-neuron layer is common to all panels.

3.7 The computational benefits of layered diffusion

Layered diffusion reduces the relaxation time required for information compression because compression is divided into smaller steps across successive hidden layers.

Diffusion mechanism: Simple diffusion requires exponentially more steps as entropy, and therefore compression, increases.Entropy grows logarithmically with time under Gaussian convolution, while a constraining potential leads asymptotically to a maximum-entropy distribution.
Diffusion mechanism: A diffusion-based compression of ΔI_X is expected to require approximately exp(ΔI_X/D) optimization epochs.Here D is the diffusion constant, and the estimate applies to the SGD diffusion phase.
Layered diffusion: With K hidden layers, total compression is approximately divided into K smaller compression steps.Each layer compresses from the representation produced by the preceding compressed layer.
Layered diffusion: The number of epochs decreases exponentially with the number of layers when the per-layer compression amounts are similar.Operations still grow only linearly with layer count, so the epoch reduction can remain significant when relaxation time is super-linear in compressed entropy.

3.8 Convergence to the layers to the Information Bottleneck bound

The paper tests whether converged DNN layers satisfy Information Bottleneck encoder-decoder relations and finds that empirical layers lie close to the theoretical IB limit.

IB optimality test: The analysis compares each layer’s empirical encoder and decoder distributions with IB-consistent distributions across values of β.The empirical distributions are estimated from quantized neuron values, and an optimal β_i is selected by minimizing averaged KL divergence.
Figure 6: Figure 6 plots the layers’ information values against the IB information curve.The figure evaluates convergence using five empirical layers and layer-specific optimal β values.
Convergence result: The five SGD-trained empirical layers lie remarkably close to the theoretical IB information curve.The curve slope β^-1 matches each layer’s estimated optimal β⋆.
Convergence result: The layer encoder-decoder distributions satisfy the IB self-consistent equations within numerical precision.The fitted β decreases for deeper layers, and near the information curve ΔI_Y is proportional to β^-1ΔI_X.

3.9 Evolution of the layers with training sample size

Training sample size shifts converged hidden-layer representations toward higher relevant information and, in deeper layers, better compression of irrelevant input information.

Sample-size effects: Increasing training size pushes the layers’ true label information I_Y upward toward the theoretical IB bound.The experiment uses six-hidden-layer networks trained with sample sizes ranging from 3% to 85% of the patterns.
Sample-size effects: Converged layers for different training sizes lie on smooth, regular lines corresponding to finite-sample information curves.These curves can be calculated from the IB self-consistent equations using the empirical decoder distribution.
Layer-specific effects: Training size has little effect on information in lower layers because random weights already preserve much of the mutual information on X and Y.The observed stability applies to both input and label mutual information in the lower layers.
Figure 6: Figure 6 reports standard-error bars from N=50 and plots five points per line for different layers, with β optimized separately for each point.The caption identifies β as the optimal value found for the corresponding layer.
Layer-specific effects: Deeper layers preserve more information about Y and better compress irrelevant information about X as training samples increase.Larger samples also shift middle-layer representations toward higher I_X because more input details become relevant to Y.

4. Discussion

The discussion interprets SGD noise as a mechanism for efficient representation compression and connects the observed dynamics to computational benefits from hidden layers. It also tests robustness across architectures while identifying open questions about generality, scale, and practical implications.

Optimization dynamics: SGD stochasticity generates efficient internal representations through compression by diffusion, rather than serving only to escape local minima.The compression phase occurs without explicit regularization or sparsity and occupies most optimization epochs.
Open questions: The authors identify unresolved questions about whether the findings generalize across rules and architectures, scale to larger real-world problems, and yield practical algorithms.These questions are posed as directions for further discussion rather than settled conclusions.
Robustness across architectures: A non-symmetric committee machine exhibits information-plane paths and SGD phases similar to the symmetric rule, including diffusion, compression, and gradient-SNR equilibration.The compression phase is faster in this case, while the overall training dynamics remains similar.
Scope: The proposed generalization-through-noise mechanism depends on multilayer structure and may extend to other deep models, unlike the single-layer perceptron case described.Bayesian networks and random forests are mentioned as possible settings for related behavior.
Algorithmic implications: The authors propose that simpler stochastic-relaxation methods could replace SGD during the diffusion phase, which consumes most training epochs.They specifically mention Monte-Carlo relaxations as a potentially more efficient alternative.
Algorithmic implications: If layers converge to IB bounds, analytic encoder-decoder relationships could be combined with stochastic relaxation to accelerate DNN training.The proposed benefit is conditional on the observed IB optimality of the layers.