A mathematical theory of semantic development in deep neural networks

Andrew M. Saxe; James L. McClelland; Surya Ganguli

Source-linked AI summary

A mathematical theory of semantic development in deep neural networks

Andrew M. Saxe, James L. McClelland, Surya Ganguli

arXiv:1810.10531v1Oct 23, 2018cs.LGcs.AIq-bio.NCstat.ML

TL;DR

The paper asks how neural circuits can extract abstract semantic knowledge from many item-specific experiences despite the absence of an analytic theory covering diverse developmental phenomena. It mathematically analyzes learning in deep linear networks and finds that exact solutions qualitatively reproduce hierarchical differentiation, semantic illusions, typicality, category coherence, developmental changes in induction, and cross-species similarity structure. The results connect these phenomena to interactions between environmental statistics and nonlinear deep-learning dynamics.

Problem
The central gap is the lack of an analytic mathematical theory explaining how neural circuits integrate individual experiences into abstract semantic knowledge across development.
Method
The paper derives exact learning dynamics for deep linear networks trained to map item inputs onto associated properties and analyzes their representations and knowledge over time.
Results
The deep linear model qualitatively captures diverse semantic-development phenomena, including hierarchical differentiation, semantic illusions, typicality, category coherence, changing induction, and conserved neural similarity structure.
Takeaways & Limitations
The analysis provides conceptual insight into how environmental statistical structure can interact with deep-learning dynamics to produce regularities in semantic cognition.

Abstract

from arXiv · show

An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, thereby raising a fundamental conceptual question: what are the theoretical principles governing the ability of neural networks to acquire, organize, and deploy abstract knowledge by integrating across many individual experiences? We address this question by mathematically analyzing the nonlinear dynamics of learning in deep linear networks. We find exact solutions to this learning dynamics that yield a conceptual explanation for the prevalence of many disparate phenomena in semantic cognition, including the hierarchical differentiation of concepts through rapid developmental transitions, the ubiquity of semantic illusions between such transitions, the emergence of item typicality and category coherence as factors controlling the speed of semantic processing, changing patterns of inductive projection over development, and the conservation of semantic similarity in neural representations across species. Thus, surprisingly, our simple neural model qualitatively recapitulates many diverse regularities underlying semantic development, while providing analytic insight into how the statistical structure of an environment can interact with nonlinear deep learning dynamics to give rise to these regularities.

A Deep Linear Neural Network Model

The paper analyzes how a deep linear network integrates item-feature experiences over time, using exact learning dynamics to study internal representation development. Despite linear components, its analytically derived trajectories reproduce hierarchical differentiation seen in a deep nonlinear network.

A Deep Linear Neural Network Model: The model learns item-associated properties by propagating inputs through two weight matrices and adjusting them to reduce prediction error.Its composite map is ˆy = W2W1x, with W1 connecting input to hidden activity and W2 connecting hidden activity to output.
A Deep Linear Neural Network Model: The analysis contrasts this three-layer deep linear network with a shallow network using one direct input-to-output matrix.The shallow model computes ˆy = Wsx, whereas the deep model separates the transformation into two learned stages.
A Deep Linear Neural Network Model: Analytic learning trajectories for the deep linear network are compared with simulated trajectories from a multilayer nonlinear network using MDS visualizations.Panel A shows simulated nonlinear-network dynamics, while Panel B shows analytically derived deep-linear-network dynamics in a hierarchically structured domain.
A Deep Linear Neural Network Model: Both analyses exhibit progressive hierarchical differentiation, separating animals from plants before birds from fish, trees from flowers, and individual items.The deep linear analysis derives this phenomenon mathematically from the network’s learning dynamics and structured input domain.

Acquiring Knowledge

The model decomposes item–property correlations into semantic modes and learns these modes through nonlinear deep-network dynamics. Stronger modes are learned faster, producing hierarchical transitions and, through competing mode contributions, non-monotonic property predictions.

Learning from experience: Small learning rates allow incremental item-wise updates to extract abstract domain structure from the statistical structure of many examples.Averaging updates across epochs yields dynamics driven by input and input-output correlation matrices.
Semantic modes: SVD decomposes item–property correlations into object-analyzer and feature-synthesizer vectors linked by singular values.The analyzers locate items along semantic dimensions, while synthesizers encode associated feature distinctions.
Deep learning dynamics: The deep network’s effective singular values follow sigmoidal trajectories toward the data singular values, whereas the shallow network approaches them exponentially.These dynamics assume small random initial weights and are confirmed by simulation in the tabula-rasa regime.
Learning timescales: Modes with stronger explanatory power are learned faster, and small initial weights can make deep-network transitions arbitrarily sharp.The learning time scales as O(1/sα) up to a logarithmic factor, while shallow-network mode timescales depend only weakly on singular value.
Hierarchical differentiation: Hierarchically structured data produce SVD distinctions that mirror the taxonomy, so deep learning generates progressive waves of internal-representation differentiation.Broader distinctions have stronger statistical structure and are learned before finer distinctions; shallow networks do not produce the same depth-dependent dynamics.
Illusory correlations: Competing mode contributions can make individual property predictions non-monotonic, including inverted-U trajectories and illusory correlations.Deep networks can transiently increase errors on specific item–property pairs while reducing global prediction error; shallow networks monotonically approach correct values.

Organizing and Encoding Knowledge

The network’s learned representations provide mathematical accounts of item typicality and category prototypes, category coherence, and the encoding of diverse environmental structures. These quantities emerge from the statistical structure of the environment and govern category learning and representation.

Item typicality: The network’s typicality definition is fixed by environmental statistics through the SVD and provably correlates with task performance because more typical items produce larger response magnitudes.This replaces heuristic feature weighting with a mathematically determined weighting derived from the entire environment.
Item typicality: Item typicality is defined by an object’s coordinate along a learned semantic distinction axis, with extremal coordinates marking the most typical category members.For the bird-fish distinction, positive and negative extremes correspond to typical birds and fish, while intermediate objects lie near the origin.
Prototypes: Category prototypes and item typicality form a duality: typicality is projection onto a prototype, while the prototype averages object features weighted by typicality.The feature synthesizer provides the prototype, and its components indicate the importance of features for the distinction.
Category coherence: Category coherence is a singular-value-based quantity that determines whether disjoint categories are learned and how accurately they are recovered.Performance is not uniform across coherence values: categories with C ≤ 1 are not learned, whereas learning improves as coherence increases above one.
Category coherence: In hierarchical environments, category coherence depends on within-cluster similarity relative to between-cluster similarity across lower levels, so different hierarchy levels can gain a coherence advantage.Basic-level or subordinate distinctions become advantaged when categories at the corresponding level are separated by negative similarity.
Structural representations: Probabilistic graphical models with clusters, trees, rings, grids, orderings, hierarchies, or cross-cutting structure produce corresponding object-analyzer vectors in the network’s internal representations.Examples include cluster-membership vectors, tree-structured wavelets, Fourier modes, ordering encodings, and vectors spanning hierarchical distinctions.

Deploying Knowledge: Inductive Projection

The network deploys learned knowledge through similarity-based inductive projection in a shared hidden representational space. As hierarchical representations progressively differentiate during development, projections narrow from broad to more specific items.

Developmental change: Children’s inductive generalizations generally become more specific with age, and the model analytically reproduces this developmental narrowing with hierarchically structured data.Young children may project novel properties broadly, whereas older children restrict projection to more closely related items.
Similarity-based projection: A novel property assigned to a familiar item is projected more readily to other items whose hidden representations are close to that item.The hidden-layer Euclidean similarity between item representations determines the strength of inductive projection.
Similarity-based projection: A novel item assigned a familiar feature receives other familiar features more readily when their hidden representations are similar to the assigned feature’s representation.The same similarity principle operates in the feature-to-novel-item direction.
Shared semantic space: The hidden layer provides a common semantic space in which both items and features are represented, enabling parallel forms of inductive generalization.Novel features are placed near familiar items, and novel items are placed near familiar features.
Developmental change: Progressive differentiation of hidden representations over development combines with similarity-based projection to explain the shift from broad to specific inductive generalization.The developmental change follows from increasingly differentiated internal geometry rather than from a separate projection rule.

Linking Behavior and Neural Representations

The theory links neural and behavioral similarity structures under optimal learning, while predicting that behavioral distinctions are stronger than their neural counterparts. This correspondence depends on smallest-norm solutions and remains an empirical prediction in some respects.

Neural similarity: The theory gives the model a neural interpretation through representational similarity analysis, a method that compares similarity structures across neural population responses.RSA has identified semantic organization in high-level visual cortices, including animate–inanimate differentiation.
Optimal learning: Optimal learning produces identical representational similarity matrices across networks despite differences in individual neural tuning patterns.The condition is that networks solve the task with the smallest-norm weights; large random initializations can instead yield different similarity matrices.
Behavioral alignment: Behavioral and neural similarity share the same singular vectors, so behavioral patterns preserve the neural categorical structure while expressing distinctions more strongly.The stronger behavioral expression scales with the square of each singular value because output weights contribute to behavior but not hidden-layer similarity.
Open test: The precise mathematical relation between behavioral and neural similarity remains to be tested in detail, although some evidence suggests greater category separation in behavior.The theory therefore offers a qualified account of observed cross-system similarity rather than a completed empirical explanation.
Behavioral alignment: The neural–behavioral match appears for networks trained from small random weights but not for networks trained from large random weights.The latter networks learn the same composite input–output map with suboptimal large-norm weights.

Discussion

The discussion presents deep linear networks as a mathematically tractable model that qualitatively captures diverse semantic-development phenomena. It also defines semantic properties through learned weights, predicts neural–behavioral alignment under optimal learning, and states important limits of linearity.

Discussion: Deep linear networks qualitatively capture a diverse array of phenomena in semantic development and cognition through exact analytical solutions of nonlinear learning dynamics.The account includes hierarchical differentiation, semantic illusions, typicality, category coherence, inductive projection, and neural similarity.
Mechanisms: Singular values of input–output correlations determine transition timing, while global prediction optimality can produce semantic illusions between rapid developmental transitions.Hierarchically structured data contains a hierarchy of singular values that supports differentiated conceptual learning.
Semantic representations: The learned weights define object analyzers for category membership and feature synthesizers for category-specific feature importance.The extracted categories are determined by environmental statistics, and item typicality is defined by strength in an object analyzer.
Neural representations: Across optimally trained networks, internal similarity structures are identical and closely related to behavioral similarity even when single-neuron correspondences are absent.Neither alignment is obligatory for suboptimal networks, and the proposed explanation for human–monkey IT alignment is explicitly suggestive rather than proven.
Limitations: Because the model is linear, it captures only a limited portion of semantic cognition and does not address phenomena requiring complex nonlinear processing or memory.Examples include context dependence, damaged-network dementia, theory of mind, causal deduction, and binding items to event roles.

Supplementary Material

The supplementary material identifies the paper’s authors and their institutional affiliations.

Authors: The paper is authored by Andrew M. Saxe, James L. McClelland, and Surya Ganguli.The listed affiliations are the University of Oxford, Stanford University, and Google Brain.
Affiliations: The authors are affiliated with the University of Oxford, Stanford University, and Google Brain.The affiliations listed are Oxford, UK; Stanford, California; and Mountain View, California.

Acquiring Knowledge

The paper derives learning dynamics for a three-layer deep linear network by averaging stochastic-gradient updates and solving mode-specific equations in an SVD basis. The explicit solution applies primarily to gradual, balanced, weakly coupled learning from small random initial weights.

Learning rule: Training uses stochastic gradient descent on squared error, with updates equivalent to standard backpropagation.The update for W2 is ΔW2 = λe_ih_i^T, where h_i is the hidden activity and e_i is the output error.
Learning dynamics: A small learning rate permits averaging updates over examples and taking a continuum limit in which input and input–output correlation matrices drive coupled nonlinear weight dynamics.The resulting equations can contain cubic interactions in the weights, with time measured in learning epochs.
SVD solution: The analysis decomposes input–output correlations with an SVD and tracks each mode through scalar strength dynamics in a decoupled basis.When initial weights are approximately diagonal in the SVD basis, off-diagonal couplings decay and the modes can be analyzed independently.
Explicit solution: The overall input–output map preserves the SVD’s object analyzers and feature synthesizers while replacing singular values with time-dependent effective strengths A(t).In the tabula rasa regime, each effective strength starts near zero and approaches the corresponding singular value as learning proceeds.
Assumptions: The explicit solution is restricted to gradual learning, roughly balanced initial mode strengths, and weak intermode coupling, conditions approximated by small random initial weights.It does not describe learning when substantial prior knowledge is already embedded in the network.
Learning-rate regime: Stronger SVD structure or more training samples requires a smaller learning rate, expressed by the condition λ ≪ 1/(s1P).Here s1 is the largest singular value and P is the number of training examples.
Error dynamics: The squared error begins proportional to output variance and ends at the variance unexplained by the best linear model.Early learning has A(t) ≈ 0, whereas late learning has A(t) ≈ S.

Shallow network

Deep linear networks learn input-output modes on timescales inversely proportional to singular-value strength, producing sigmoidal, quasi-stage-like transitions unlike shallow networks.

Learning dynamics: Deep networks exhibit sigmoidal learning trajectories, whereas shallow networks approach learned values exponentially.The deep trajectory spends much of training progressing slowly before a brief rapid transition.
Hierarchical data: Hierarchical data are generated by independently diffusing binary feature values down a branching tree, linking statistical structure to learning dynamics.The analysis computes singular values and vectors from the resulting hierarchical dataset.
Learning dynamics: Learning time for mode α scales as O(τ/sα), so stronger correlation modes are learned earlier.The mode projects inputs through right singular vector vα and contributes to outputs through left singular vector uα.

Hierarchical feature vectors from a branching diffusion process

A branching diffusion process generates hierarchically correlated feature vectors, whose correlation eigenstructure determines the order and timescale of learned distinctions.

Data generation: Each feature begins with a random ±1 root value and independently flips sign with small probability along tree links.The values at the leaves define that feature across items.
Data generation: Pairs of examples sharing a more recent common ancestor have stronger feature overlap than pairs whose common ancestor is more distant.The overlap parameters qk decrease as the last common ancestor moves upward in the tree.
Eigenstructure: The correlation matrix has hierarchical block structure, giving eigenvectors organized by tree level.These eigenvectors are the input modes that project examples into internal representations.
Developmental ordering: Because eigenvalues decrease with level, coarse distinctions are learned before finer-grained distinctions.Structure below level l cannot appear internally until structure at level l−1 has been learned.
Developmental ordering: For constant branching factor B, the learning timescale grows exponentially with tree depth.The timescale is proportional to the square root of the number of ancestors at the relevant level.
Semantic illusions: Deep networks can produce U-shaped predictions for individual features, but total error across features and items never increases.Shallow networks remain monotonic and therefore do not produce illusory correlations.

Organizing and Encoding Knowledge

The SVD organizes semantic structure through category coherence, typicality, and global relations among items and features, while deep-network dynamics determine which distinctions emerge first.

Category coherence: A single coherence quantity determines category-recovery performance in the analyzed large-system regime.Recovery occurs above a threshold expressed through the coherence and item-feature ratio.
Category coherence: Category coherence can be defined as the singular value associated with a categorical distinction in the input-output correlation SVD.This definition can produce a basic-level advantage depending on category similarity structure.
Category coherence: Strong anticorrelation between sibling categories can make an intermediate category level more coherent and faster to learn than a superordinate level.Between-category anticorrelation boosts coherence at the intermediate level while reducing superordinate coherence.
Global organization: The first k SVD modes minimize total prediction error among linear predictors restricted to k relations.Early stopping therefore yields a globally optimal rank-k approximation under the Eckart–Young–Mirsky theorem.
Global organization: Adding one feature can substantially reorganize categorical structure even when an item’s existing features remain unchanged.This follows because the SVD depends on global relations across the dataset.
Learning speed: Shared category structure is typically learned faster than idiosyncratic item-specific information because its singular value is larger.The result is established for the stated graph construction and parameter regime.

Linking Behavior and Neural Representations

The paper links invariant hidden-layer similarity structure to optimal minimum-norm implementations, while showing that equivalent input-output mappings can otherwise produce different internal representations.

Identical hidden-layer similarity structure across networks trained on the same statistics arises only for optimal minimum-norm implementations.Networks can implement the same input-output task with widely different internal similarity structures when this optimality condition is absent.
For white probe inputs, the hidden activity matrix H provides the neural similarity structure through H^T H.The probe inputs are assumed to satisfy X^T X = I, matching the learning statistics.
General invertible transformations do not preserve hidden similarity structure, because their ambiguity matrix need not satisfy Q^T Q = I.Thus, task-equivalent networks may have different internal similarity structures outside the orthogonal case.
Minimum-norm solutions constrain the hidden-layer ambiguity to orthogonal transformations, preserving similarity structure.The learned factors satisfy P = R^T and Q = R for an orthogonal matrix R, with R^T R = I.
Behavioral similarity and neural similarity share categorical structure, but behavioral distinctions are amplified according to the square of each singular value.The stronger behavioral distinction reflects semantic relations encoded in the output weights, which do not influence neural similarity in the same way.