PaperlayerView exact revision on arXiv

Source-linked AI summary

Statistical field theory for neural networks

Moritz Helias, David Dahmen

arXiv:1901.10416v1cond-mat.dis-nncond-mat.stat-mech

TL;DR

Formal field-theory methods have only sparsely entered neuroscience, and the origins of influential mean-field results are often difficult to follow. These notes introduce those methods and apply them to neural networks, including derivations of self-consistent dynamics and the TAP approximation.

  • Problem

    Formal field-theory methods have sparsely entered neuroscience, and the derivations underlying influential mean-field results have often been omitted or bypassed.

    Source: §I · p. 5

  • Method

    The notes introduce statistical field theory for neural networks, including systematic fluctuation calculations and applications to random-connectivity systems and the pairwise maximum-entropy model.

    Source: §I · pp. 4–5, §XII · p. 76, §XIV · p. 85

  • Results

    The derivations reduce a disordered network to non-interacting units in a self-consistent fluctuating field, describe curvature-dependent fluctuation corrections, and develop a diagrammatic derivation of TAP theory.

    Source: C. Average over the quenched disorder · p. 51, A. Motivating example for the… · p. 64, §XII · p. 76

  • Takeaways & Limitations

    The formal developments aim to equip readers to understand prior work and extend statistical field theory toward new theoretical-neuroscience questions.

    Source: §I · p. 5

Abstract

from arXiv · show

These notes attempt a self-contained introduction into statistical field theory applied to neural networks of rate units and binary spins. The presentation consists of three parts: First, the introduction of fundamental notions of probabilities, moments, cumulants, and their relation by the linked cluster theorem, of which Wick's theorem is the most important special case; followed by the diagrammatic formulation of perturbation theory, reviewed in the statistical setting. Second, dynamics described by stochastic differential equations in the Ito-formulation, treated in the Martin-Siggia-Rose-De Dominicis-Janssen path integral formalism. With concepts from disordered systems, we then study networks with random connectivity and derive their self-consistent dynamic mean-field theory, explaining the statistics of fluctuations and the emergence of different phases with regular and chaotic dynamics. Third, we introduce the effective action, vertex functions, and the loopwise expansion. These tools are illustrated by systematic derivations of self-consistency equations, going beyond the mean-field approximation. These methods are applied to the pairwise maximum entropy (Ising spin) model, including the recently-found diagrammatic derivation of the Thouless-Anderson-Palmer mean field theory.

I. INTRODUCTION

Neural recordings capture only restricted views of multiscale activity, while phenomenological mean-field descriptions often lack a systematic derivation and treatment of fluctuations. These notes introduce statistical field theory as a systematic framework for studying neural networks, including random-connectivity dynamics.

I. INTRODUCTION

  • Neural networks exhibit activity correlated across spatial and temporal scales, but recordings either undersample individual neurons or provide coarser collective signals.Single-neuron recordings have high temporal resolution but serious subsampling, whereas local field potentials combine activity from many neurons and have moderate temporal resolution.Source: §I · p. 4
  • Models of interacting neurons can bridge microscopic dynamics and population-level measurements, while statistically homogeneous connectivity can support population equations.The many synaptic inputs received by each cell can permit application of the law of large numbers.Source: §I · p. 4
  • Functional and path-integral methods offer systematic approximation schemes for neural activity, addressing limitations of intuition-led mean-field descriptions and supporting study of fluctuations.The notes introduce perturbation theory and loop-wise expansion as examples of these schemes.Source: §I · p. 4
  • Neuronal interactions are directed, delayed, and pulse-mediated, unlike the reciprocal, instantaneous Coulomb interactions used to motivate many solid-state systems.The notes present field theory as a way to investigate this distinctive physical system.Source: §I · p. 5
  • The notes distinguish Curie-Weiss mean field, which neglects order-parameter fluctuations at tree level, from spin-glass mean field, which retains fluctuations through a Gaussian variable.They also trace dynamic mean-field theory for random networks to a single unit in a Gaussian fluctuating field with self-consistently determined statistics.Source: §I · p. 5
  • The notes develop statistical field theory to help readers understand prior work and extend it to new questions in theoretical neuroscience.They begin with finite-dimensional random variables to separate conceptual challenges from infinite-dimensional mathematics, then address time-dependent systems and random-connectivity networks.Source: §I · pp. 5–6

A. Probabilities, observables, and moments

The section represents a system by its state and observables, then relates repeated-measurement averages to moments and the moment-generating function.

A. Probabilities, observables, and moments

  • A system state x ∈ R^N can represent activities across N neurons or one neuron's activity across N time points, and observables are functions of that state.The density p(x) describes the system's state variables.Source: A. Probabilities, observables, and… · p. 6
  • The expectation of an observable is its average over repeated measurements, stochastic realizations, or time.The notation ⟨·⟩ denotes this average.Source: A. Probabilities, observables, and… · p. 6
  • The characteristic function contains the same information as the probability density when its Fourier transform exists.It is the Fourier transform of p and is also called the moment-generating function.Source: A. Probabilities, observables, and… · p. 7
  • Moments are obtained by differentiating the moment-generating function, and they are its Taylor coefficients.The moments also determine expectations of observables that possess a Taylor expansion.Source: A. Probabilities, observables, and… · p. 7

B. Transformation of random variables

For a transformed random variable y=f(x), the section derives its density and moment-generating function from those of x.

B. Transformation of random variables

C. Cumulants

The section defines cumulants through the logarithm of the moment-generating function, which separates dependencies not already accounted for at lower orders.

C. Cumulants

  • For independent variables, the joint density and moment-generating function factorize, so joint moments decompose into products of first moments.Source: C. Cumulants · p. 8
  • Taking the logarithm of the moment-generating function turns the independent-variable product into a sum, and its Taylor coefficients define cumulants.Source: C. Cumulants · p. 8
  • Mixed cumulants of independent variables vanish, while changing the normalization of the generating function does not affect cumulants.The logarithm of the partition function corresponds to free energy in statistical physics.Source: C. Cumulants · p. 8

D. Connection between moments and cumulants

Moments and cumulants are linked through their generating functions: the moment-generating function comes from the probability distribution, and its logarithm gives the cumulant-generating function. The k-th moment can then be expressed as a sum over partitions into cumulants of orders up to k.

D. Connection between moments and cumulants

  • The moment-generating function Z follows from the probability p, and taking its logarithm gives the cumulant-generating function.Moments and cumulants are obtained as Taylor coefficients of their respective generating functions.Source: D. Connection between moments and… · p. 9
  • Repeated variables are handled by deriving the result for distinct indices first, then counting each distinct assignment of identical variables separately.A repeated power corresponds to repeated occurrences of the same index.Source: D. Connection between moments and… · pp. 9–10
  • Differentiating Z(j) = exp(W(j)) builds the k-th moment through prefactors whose recursion follows from the product rule.At zero sources, the prefactor is the k-th moment.Source: D. Connection between moments and… · p. 9
  • The k-th moment is the sum over all partitions of k points into subgroups, with each subgroup contributing a cumulant of its size.The recipe also counts distinct assignments of variables to subgroup positions.Source: D. Connection between moments and… · p. 10

III. GAUSSIAN DISTRIBUTION AND WICK’S THEOREM

The Gaussian distribution is central to the subsequent development because field theory can be viewed as a reorganization of Gaussian integrals. This section derives fundamental properties of the distribution.

III. GAUSSIAN DISTRIBUTION AND WICK’S THEOREM

  • The Gaussian distribution is essential to the further development of the field-theory treatment.The section introduces its fundamental properties.Source: §III · p. 11
  • Field theory is framed as a clever reorganization of Gaussian integrals.Source: §III · p. 11
  • The section derives fundamental properties of the Gaussian distribution.Source: §III · p. 11

A. Gaussian distribution

The section defines a centered N-variable Gaussian using a positive-definite symmetric matrix A. Positive definiteness is equivalent to all eigenvalues of A being positive.

A. Gaussian distribution

  • A centered Gaussian distribution of N variables is defined using a positive-definite symmetric matrix A.Source: A. Gaussian distribution · p. 11
  • A is positive definite when x^T A x > 0 for every nonzero x, equivalently when all its eigenvalues are positive.A is also described as self-adjoint with respect to the Euclidean scalar product.Source: A. Gaussian distribution · p. 11
  • The distribution is properly normalized, with the normalization factor derived in a later section.Source: A. Gaussian distribution · p. 11

B. Moment and cumulant generating function of a Gaussian

For the Gaussian distribution, shifting the integration variable by A^-1j evaluates the moment-generating function. Its cumulant-generating function is quadratic, so only second-order cumulants remain, yielding the covariance matrix.

B. Moment and cumulant generating function of a Gaussian

C. Wick’s theorem

Wick’s theorem expresses Gaussian moments through products of second cumulants. Its pairing rule makes odd moments vanish and gives a combinatorial factor when variables are identical.

C. Wick’s theorem

  • Gaussian moments are sums of products of the distribution’s second cumulants, a relation known as Wick’s theorem [23, 50].Source: C. Wick’s theorem · p. 12
  • Odd moments vanish, while each even moment is formed by pairing the variables in all distinct ways and replacing each pair with its second cumulant.Source: C. Wick’s theorem · p. 12
  • When all four variables are identical, the same pairing term occurs three times, producing a combinatorial factor of three.Source: C. Wick’s theorem · p. 12

D. Graphical representation: Feynman diagrams

This section develops Feynman diagrams to represent Gaussian contractions and organize perturbative calculations, including rules for external sources and normalized generating functions. It shows how diagrams isolate connected contributions and relate moments to n-point correlations, illustrating the methods with a φ³ + φ⁴ example.

D. Graphical representation: Feynman diagrams

  • A fourth Gaussian moment has three distinct pairings, represented by connecting the four legs in pairs; the initial leg choice does not add a multiplicity.For identical variables this reproduces the factor three in the one-dimensional Gaussian fourth moment.Source: D. Graphical representation: Feynman… · pp. 12–13

E. Appendix: Self-adjoint operators

  • For a self-adjoint operator, the eigenvectors form orthogonal columns of its diagonalizing matrix, whose transformation preserves the scalar product.With normalized eigenvectors, U^-1 = U^T.Source: E. Appendix: Self-adjoint operators · p. 13

F. Appendix: Normalization of a Gaussian

  • Diagonalizing a positive matrix by an orthogonal transform preserves the integration measure, and the Gaussian normalization uses the product of the matrix’s eigenvalues.The orthogonal transform has determinant magnitude 1.Source: F. Appendix: Normalization of a Gaussian · pp. 13–14

A. General case

  • For a perturbation around a solvable theory, the generating function reduces to calculations of moments under that theory, which can be expanded in the perturbing potential.For a Gaussian solvable part, Wick’s theorem evaluates the required moments; the potential need not be restricted to Gaussian perturbations.Source: A. General case · pp. 14–15, B. Special case of a Gaussian solvable… · p. 15

B. Special case of a Gaussian solvable theory

  • Gaussian contractions become connecting lines, while Taylor coefficients of the potential become interaction vertices whose legs are paired in all possible ways.Each connection contributes the inverse covariance, and order k contributes a factor ϵ^k; summing over indices and distinct vertex combinations supplies the remaining factors.Source: B. Special case of a Gaussian solvable… · p. 16

C. Example: Example: “φ3 + φ4” theory

  • In the one-dimensional φ3 + φ4 example, odd powers vanish for the centered Gaussian, while quartic contractions produce a factor of 3.The quartic factor follows either from the three distinct pairings of four legs or from the fourth moment of a unit-variance Gaussian.Source: C. Example: Example: “φ3 + φ4” theory · pp. 17–18

D. External sources

  • External source factors act as monopole vertices with labeled external legs, and diagrams with l external lines contribute to the l-th moment.The rules include a factor 1/l! for l external lines and treat the associated source factors as distinct legs.Source: D. External sources · p. 18

E. Cancellation of vacuum diagrams

  • The expansion of Z(j) includes the same vacuum diagrams as Z(0), including disconnected vacuum components unconnected to the vertices attached to external lines.Z(0) itself consists of diagrams without external lines.Source: E. Cancellation of vacuum diagrams · pp. 18–19
  • Vacuum diagrams cancel against the normalization Z(0), leaving only diagrams connected to external lines.The external-line diagrams carry the same combinatorial factors as the vacuum diagrams in the normalization.Source: E. Cancellation of vacuum diagrams · p. 20

F. Equivalence of graphical rules for n-point correlation and n-th moment

  • The graphical rules for ⟨x^n⟩ match those for an n-point correlation of distinct variables, because assigning n sources to external legs gives n! equivalent choices.The n sources formally couple to the same variable x, so every assignment yields the same result.Source: F. Equivalence of graphical rules for… · p. 20

G. Example: “φ3 + φ4” theory

  • In the φ3 + φ4 example, the first-order three-point vertex gives a tadpole contribution to the mean, while the four-point vertex vanishes there by Wick’s theorem.The four-point contribution is proportional to ⟨x^5⟩0 = 0; direct Wick evaluation also gives ⟨x^4⟩0 = 3(K−1)2.Source: G. Example: “φ3 + φ4” theory · pp. 20–21
  • At second order, a mixed three-point/four-point contribution counts 3 × 4 × 3 contractions, with no extra factor of two for choosing either three-point leg first.The external source has three attachment choices, followed by four and then three choices for connecting the remaining legs.Source: G. Example: “φ3 + φ4” theory · p. 21
  • Some second-moment diagrams factor into products of first-moment contributions, and these redundant terms are subtracted when calculating the variance, or second cumulant.The example identifies one such second-order contribution as one-half the square of a first-moment contribution.Source: G. Example: “φ3 + φ4” theory · p. 21

V. LINKED CLUSTER THEOREM

This section develops diagrammatic links between the action, cumulant-generating function, and cumulants. Taking the logarithm isolates linked clusters and avoids repeating diagrams already obtained at lower orders.

V. LINKED CLUSTER THEOREM

  • The section derives direct links from the action to the cumulant-generating function and cumulants, following the cancellation of vacuum graphs without external lines by normalization.Source: §V · p. 22
  • Keeping only connected diagrams avoids recalculating components that already appeared at lower perturbative orders.Source: §V · p. 22
  • Expanding ln Z turns products into sums, so mixed source terms occur only when their variables belong to the same connected component, or linked cluster.Source: §V · p. 22

A. General proof of the linked cluster theorem

This section develops functional and diagrammatic methods for perturbation theory, stochastic dynamics, and path-integral representations. It applies these methods to nonlinear stochastic differential equations and random-network dynamics, including their correlation and response functions.

A. General proof of the linked cluster theorem

  • The general proof shows that perturbative contributions to W(j) are connected and carry the same factors as the direct perturbation expansion.Each interaction vertex attaches to existing components, and the induction excludes disconnected diagrams.Source: A. General proof of the linked cluster… · pp. 23–25

B. Dependence on j - external sources - two complimentary views

  • Making the external-source dependence explicit lets derivatives construct cumulants directly from connected diagrams with matching numbers of external legs.The two equivalent views either retain W_l(j) as a function of j or expand its source dependence on external legs.Source: B. Dependence on j - external sources… · pp. 25–26

C. Example: Connected diagrams of the “φ3 + φ4” theory

  • In the φ3 + φ4 example, successive orders combine bare interaction vertices with diagrams already generated, producing repeated diagrammatic elements.The example works through cumulant corrections up to second order and then notes that the construction generates recurring elements.Source: C. Example: Connected diagrams of the… · pp. 26–28

VI. FUNCTIONAL PRELIMINARIES

  • The section introduces functional calculus as a set of rules needed in subsequent sections.It treats functionals mapping smooth functions to real numbers.Source: §VI · p. 28

A. Functional derivative; 1. Product rule

  • Functional derivatives measure sensitivity to the field at a particular time, and ordinary differentiation rules such as the product rule carry over.The definition is traced to ordinary derivatives; the product rule is introduced as a consequence.Source: A. Functional derivative · p. 28, §1 · p. 28

2. Chain rule

  • The functional chain rule is derived by discretizing time into N bins, applying the finite-dimensional rule, and taking the infinitesimal-discretization limit.Source: §2 · p. 28

3. Special case of the chain rule: Fourier transform

  • Under Fourier transformation, functional derivatives acquire inverse-transform properties, including the opposite sign of ω and a factor of 1/2π.The resulting invariance of a frequently encountered expression is used when evaluating Feynman diagrams in the Fourier domain.Source: §3 · p. 29

B. Functional Taylor series

  • A functional Taylor series represents f[x] through symmetric coefficients, which are identified from functional derivatives with a k! factor.After taking the k-th functional derivative and setting x = 0, only the term of order k remains.Source: B. Functional Taylor series · p. 30
  • Functional Taylor expansion generalizes to expansion around x0 by replacing x with x − x0, and to multiple fields by expanding each field dependence.The expansion is presented in a form reminiscent of the usual n-dimensional Taylor expansion.Source: B. Functional Taylor series · p. 30

VII. FUNCTIONAL FORMULATION OF STOCHASTIC DIFFERENTIAL EQUATIONS

  • The Ito convention defines the discrete-time SDE interpretation, while Ito and Stratonovich yield the same continuous-time limit for additive noise but different path-integral representations.The section treats additive noise and retains a general discretization parameter α.Source: §VII · pp. 30–31

B. Martin-Siggia-Rose-De Dominicis-Janssen (MSRDJ) path integral

  • The MSRDJ path integral introduces a response field to avoid the square in the Onsager–Machlup action, supporting perturbation calculations and treatment of disordered systems.The Onsager–Machlup action’s square can complicate analytical calculations, including quenched averages.Source: §VII · p. 30, B. Martin-Siggia-Rose-De… · p. 32

A. Onsager-Machlup path integral*

  • For Gaussian noise, the Onsager–Machlup path integral has a squared term in its action, which can complicate analytical treatment.The square is one motivation for introducing the response-field formulation.Source: B. Martin-Siggia-Rose-De… · p. 32

C. Moment generating functional

D. Response function in the MSRDJ formalism

  • A perturbation source couples to the response field, so functional differentiation with respect to that source yields the system’s response to perturbations.The resulting response function can serve as a kernel for the mean response to an external perturbation of arbitrary temporal shape.Source: D. Response function in the MSRDJ… · pp. 34–35

A. Definition

  • The Ornstein–Uhlenbeck process is introduced as a linear SDE and as the free Gaussian part of the dynamical theory.The formulation extends to coupled SDEs with Wiener increments having a covariance matrix.Source: A. Definition · p. 36

B. Propagators in time domain

C. Propagators in Fourier domain

  • Fourier transformation diagonalizes the Gaussian operator by frequency, allowing its two-by-two matrix to be inverted separately at each frequency.For stable one-dimensional dynamics, the covariance is positive and the response function is causal.Source: C. Propagators in Fourier domain · pp. 38–39

IX. PERTURBATION THEORY FOR STOCHASTIC DIFFERENTIAL EQUATIONS

  • The Ornstein-Uhlenbeck process provides the solvable Gaussian baseline for calculating perturbative corrections from nonlinearities in stochastic differential equations.The nonlinear terms form the non-Gaussian part of the action.Source: §IX · p. 39

A. Vanishing moments of response fields

  • All moments of the response field vanish, following from normalization of the generating function even with a nonzero response source.The text states this holds irrespective of the value of the response source ˜j.Source: A. Vanishing moments of response fields · p. 40

B. Vanishing response loops

  • Causal response functions remain causal at every perturbative order, and same-vertex response loops cancel in both Ito and Stratonovich conventions.The inductive argument extends causality to arbitrary orders; in Stratonovich, the closed-loop contributions cancel the functional-determinant term.Source: B. Vanishing response loops · pp. 40–41

C. Feynman rules for SDEs in time domain and frequency domain

  • Diagram rules encode frequency conservation at vertices and propagators, with the remaining independent frequency integrals corresponding to loops.Time-translation-invariant problems can be expressed in the frequency domain, where each vertex enforces that incoming frequencies sum to zero.Source: C. Feynman rules for SDEs in time… · pp. 42–43
  • At first order in α, the nonlinear term shifts the stationary expectation by αD/4.The correction affects the stationary expectation at ω = 0; the text relates αD/4 to the unperturbed variance D/2.Source: C. Feynman rules for SDEs in time… · p. 43

D. Diagrams with more than a single external leg

  • Repeated external legs in diagrams are accounted for by the corresponding repeated factor in the expansion.The passage introduces this rule for diagrams with more than one external leg but does not specify the factor in the supplied excerpt.Source: D. Diagrams with more than a single… · p. 43
  • The covariance correction is symmetric after differentiating the source-dependent contribution, and its frequency-space form has a loop integral with sources attached by propagators.The inner loop integral is identified as an effective 1PI vertex; individual terms need not be symmetric because the response propagators are nonsymmetric.Source: D. Diagrams with more than a single… · p. 44

E. Appendix: Unitary Fourier transform

  • The Fourier transform changes the path-integral measure only by a constant determinant, while real-valued time-domain fields require X(−ω) = X(ω)*.The constant has no consequence for derivatives of generating functionals.Source: E. Appendix: Unitary Fourier transform · p. 45

X. DYNAMIC MEAN-FIELD THEORY FOR RANDOM NETWORKS

  • The chapter presents formal calculations reproducing key results of Sompolinsky et al. and extends the model to additive uncorrelated Gaussian white noise.Setting noise amplitude D = 0 recovers the original results.Source: §X · p. 46

A. Definition of the model and generating functional

  • The random-network model uses Gaussian random couplings, a nonlinear gain function, and pairwise uncorrelated Wiener noise, with a generating functional for activity moments and response functions.The action separates single-unit properties from the explicit coupling term.Source: A. Definition of the model and… · p. 46

B. Property of self-averaging

  • To leading order in network size N, the two-functional average factorizes, making observable variance vanish and supporting a typical-network mean-field description.Self-averaging means observables vary little across realizations of frozen connectivity; averaging the generating functional can therefore approximate quantities for most networks.Source: B. Property of self-averaging · pp. 47–48

C. Average over the quenched disorder

Averaging over random connectivity and introducing auxiliary fields reduces the interacting network to a single-unit problem in a self-consistent field. At the saddle point, that field’s statistics are determined by the units’ activity correlations.

C. Average over the quenched disorder

  • Introducing Q1 decouples four-field terms, while the large-N action motivates a saddle-point approximation that neglects fluctuations at tree level.The central limit theorem motivates treating Q1 as approaching a Gaussian distribution.Source: C. Average over the quenched disorder · pp. 49–50
  • The saddle point selects auxiliary-field values that maximize their contribution to the overall probability mass.Figure 2 identifies the maximizing value Q* and marks b[Q*].Source: C. Average over the quenched disorder · p. 50, Figure 2 · p. 50
  • The saddle-point equations determine the auxiliary fields from expectations under the resulting single-unit statistics, with Q2* vanishing.The fields’ values affect the statistics used to compute those expectations, so the equations must be solved self-consistently.Source: C. Average over the quenched disorder · p. 51
  • The saddle-point action decomposes the network into non-interacting units exposed to a common field, reducing the problem to a single unit.The field statistics are determined self-consistently, and the particular realization of the activity is not needed.Source: C. Average over the quenched disorder · pp. 50–51
  • Each unit’s effective input is Gaussian noise whose covariance is the summed activity autocorrelation weighted by the connectivity variance.Only the second cumulant of this effective noise is nonzero.Source: C. Average over the quenched disorder · pp. 51–52

D. Stationary statistics: Self-consistent autocorrelation of as motion of a particle in a potential

For stationary statistics, the self-consistent autocorrelation equation can be represented as motion in a potential, whose shape constrains the possible covariance solutions. The section also derives the largest Lyapunov exponent to assess the transition to chaos and leading-order self-averaging.

D. Stationary statistics: Self-consistent autocorrelation of as motion of a particle in a potential

  • For stationary statistics, c(τ) obeys a self-consistent differential equation that can be rewritten as motion in a potential.The covariance is symmetric, and the equation includes the noise term Dδ(τ).Source: D. Stationary statistics:… · pp. 52–53
  • For D = 0 and g < 1, the only bounded solution is the vanishing covariance c(τ) ≡ 0.In this regime the parabolic part of the potential dominates for all c0.Source: D. Stationary statistics:… · p. 53
  • With D > 0, the bounded covariance solution starts at c0 and reaches zero at long times under the self-consistent energy condition.The initial kinetic energy is set by the noise, and the resulting covariance decays from c0 to zero.Source: D. Stationary statistics:… · p. 54
  • For g > 1, the potential curvature can change sign as c0 increases, producing an intermediate double-well regime with multiple solutions.Only the solution that decays to zero as τ → ∞ is stable, including in the noiseless case.Source: D. Stationary statistics:… · p. 54
  • The self-consistent autocorrelation solution agrees with direct simulation, with connectivity variance compensating for the suppressive effect of the saturating gain.The input covariance and autocorrelation have strong similarities.Source: D. Stationary statistics:… · p. 54

E. Transition to chaos

  • The transition-to-chaos analysis derives the largest Lyapunov exponent and uses the replicated moment-generating function’s leading contribution to conclude leading-order self-averaging.The exponent assesses conditions for a transition into the chaotic regime.Source: E. Transition to chaos · p. 54

F. Assessing chaos by a pair of identical systems

The section analyzes chaos by comparing two copies of a network with the same connectivity and noise, then studies the stability of their identical-trajectory solution. It recasts the perturbation growth problem as a Schrödinger equation whose fastest mode determines the distance’s growth rate.

F. Assessing chaos by a pair of identical systems

  • The copies share both the coupling matrix and, when D > 0, the same Gaussian-noise realization, which correlates their dynamics through distinct noise and disorder terms.The common-noise term represents shared temporal fluctuations; the shared-connectivity term is static.Source: F. Assessing chaos by a pair of… · pp. 54, 57
  • Across perturbed trials, the squared distance diverges with a stereotyped form that seems dominated by one largest Lyapunov exponent.The section derives the rate of this divergence from the fastest-growing mode.Source: F. Assessing chaos by a pair of… · p. 55, Figure 6 · p. 56
  • Symmetry among unit indices reduces the two-copy system from 2N units to two representative units, and the resulting activity statistics are Gaussian.Only products of pairs of fields appear in the action.Source: F. Assessing chaos by a pair of… · p. 58
  • The two-copy dynamics has an identical-trajectory fixed point with d(t) = 0, and its instability can indicate chaotic dynamics.Each subsystem retains the single-system solution; stability is tested by perturbing the copies’ cross-correlation.Source: F. Assessing chaos by a pair of… · pp. 58–59

G. Schrödinger equation for the maximum Lyapunov exponent

  • A change to sum-and-difference time coordinates turns the linearized cross-correlation equation into a time-independent Schrödinger equation, whose ground state gives the fastest distance-growth mode.The distance is squared, so its growth exponent is twice the deflection’s exponent.Source: G. Schrödinger equation for the… · pp. 59–60

H. Condition for transition to chaos

This section identifies the noisy-network transition to chaos through a zero-curvature condition, then develops effective-action and diagrammatic loopwise methods for organizing fluctuation corrections in self-consistent equations and stochastic dynamics. It connects cumulants, vertex functions, and one-particle-irreducible diagrams, with applications including the TAP approximation and perturbative resummations.

H. Condition for transition to chaos

  • With noise, the transition to chaos occurs when the ground-state eigenvalue reaches zero, requiring vanishing autocorrelation curvature at τ=0 as external and intrinsic decorrelation balance.The curvature changes sign as the dominant source of decorrelation shifts from external input to chaotic dynamics.Source: H. Condition for transition to chaos · pp. 60–62
  • A necessary transition condition is that the Jacobian spectral radius exceed one, marking the point where linear stability is lost.The condition follows from the ground-state energy being no smaller than the potential.Source: H. Condition for transition to chaos · pp. 61–62
  • In the chaotic regime, the replica-coupling field has only a physically admissible vanishing solution, so the system is self-averaging to leading order in N.The text relates this result to the noiseless transition criterion and extends it to nonzero noise.Source: H. Condition for transition to chaos · p. 62

XI. VERTEX GENERATING FUNCTION

  • When interactions are strong and activity peaks far from zero, fluctuations can be expanded systematically around the true mean rather than around zero.This motivates the effective-action approach developed in the following sections.Source: §XI · p. 63

A. Motivating example for the expansion around a non-vanishing mean value

  • In the homogeneous-network example, Gaussian fluctuations around x* diverge at J0φ'(x*)=1, while the next-order mean correction has the sign of φ''.The example's ad hoc treatment is limited to Gaussian fluctuations around a fixed point and motivates a later diagrammatic treatment of non-Gaussian corrections.Source: A. Motivating example for the… · pp. 63–64

B. Legendre transform and definition of the vertex generating function Γ

  • The effective action Γ is the Legendre transform of W(j), and its equation of state yields the mean x* for a source j, including fluctuation corrections.The construction chooses the reference value x* to equal the mean, leaving fluctuations with zero mean.Source: B. Legendre transform and definition… · pp. 65–66

C. Perturbation expansion of Γ

  • The perturbative scheme decomposes S into S0 and εV, then expands Γ into solvable and interaction corrections to obtain recursive approximations and graphical rules.The correction term ΓV collects interaction contributions to a chosen perturbation order.Source: C. Perturbation expansion of Γ · p. 67
  • The correction to the effective action is obtained by iterating order by order in interaction vertices and taking the iteration limit.At first order, connected diagrams contain one interaction vertex and cumulant connections from the unperturbed theory.Source: C. Perturbation expansion of Γ · pp. 67–68

D. Generalized one-line irreducibility

  • The Legendre-transform subtraction cancels reducible graphs, while connections through higher-order cumulants can survive with a minus sign.The surviving graphs are absent in the Gaussian case and can be enumerated through skeleton diagrams.Source: D. Generalized one-line irreducibility · pp. 70–71

E. Example

  • In the third-order-vertex example, diagrams joined by a single line cancel, but contributions attaching to higher-order cumulants remain uncanceled.Those additional terms arise from the second contribution in the iteration and cannot arise from the first.Source: E. Example · p. 72

F. Vertex functions in the Gaussian case

  • For a Gaussian theory, all connections are second cumulants, so reducible diagrams cancel and no non-standard diagrams are produced.The resulting diagrams are one-line irreducible.Source: F. Vertex functions in the Gaussian case · p. 73
  • For a Gaussian expansion, vertex generation selects connected 1PI diagrams and amputates their external legs, replacing each attached field with x∗−x0.The equation of state is obtained from diagrams with one external amputated leg.Source: F. Vertex functions in the Gaussian case · p. 73

G. Example: Vertex functions of the “φ3 + φ4”-theory

H. Appendix: Explicit cancellation until second order

I. Appendix: Convexity of W

  • A differentiable W is convex because its Hessian is a positive-semidefinite covariance matrix, so its Legendre transform Γ is well defined even when W has a linear segment.The double Legendre transform returns W only when W is convex to begin with.Source: §I · pp. 75–76

J. Appendix: Legendre transform of a Gaussian

XII. APPLICATION: TAP APPROXIMATION

  • The diagrammatic derivation of TAP gives a second-order effective action whose terms are independent-spin entropy, Curie-Weiss mean field, and the Onsager reaction term.The resulting equation of state can be solved numerically, for example by bisection.Source: §XII · pp. 76, 78, 79

Inverse problem

  • The inverse problem uses the effective-action approximation to infer parameters hi and Jij from measured mean activities mi and covariances cij.The covariance-matrix inverse is used in solving this data-analysis problem.Source: Inverse problem · p. 79
  • The inverse procedure determines couplings from off-diagonal correlations and then biases; the positive sign gives Kij = 0 when those correlations vanish.The procedure evaluates the expression for Jij at i ≠ j.Source: Inverse problem · p. 79

XIII. EXPANSION OF CUMULANTS INTO TREE DIAGRAMS OF VERTEX FUNCTIONS

  • Cumulants expressed through vertex functions have tree structure, with fluctuations implicitly contained in the vertex functions and full propagator W (2).This decomposition extracts frequency integrals, which then need to be computed only once.Source: §XIII · pp. 82–83

A. Self-energy or mass operator Σ

  • A vanishing eigenvalue of the effective-action Hessian Γ(2) makes fluctuations diverge in that direction, while the self-energy Σ corrects the bare Hessian in Dyson’s equation.Σ contributions are connected diagrams that cannot be disconnected by cutting a single line.Source: A. Self-energy or mass operator Σ · pp. 84–85

XIV. LOOPWISE EXPANSION OF THE EFFECTIVE ACTION - TREE LEVEL

  • For large l, probability mass concentrates at the stationary mean x*, improving the tree-level, mean-field approximation as fluctuations are neglected.The tree-level effective action is the convex envelope of −S; the background field expansion provides a systematic way to include fluctuations around x*.Source: §XIV · p. 86

A. Counting the number of loops

  • With scale l, each graph’s contribution is damped by l^(k−n_Δ), where k is its number of vertices and n_Δ its number of propagator lines.The propagator scales as l^-1 and each vertex as l, linking the scale of fluctuations to diagram structure.Source: A. Counting the number of loops · p. 86
  • Cumulants of order n_j are suppressed by l^-n_j, while contributions at fixed order diminish as the diagrams’ loop count increases.The external legs supply the cumulant suppression; the amputated diagram contributes the loop-dependent factor.Source: A. Counting the number of loops · p. 87

B. Loopwise expansion of the effective action - Higher numbers of loops

  • A diagram with n_L loops contributes to Γ_fl with factor l^-n_L, and the iterative construction cancels one-particle-reducible terms, leaving only 1PI diagrams.The cancellation is between vacuum diagrams and diagrams assembled from lower-loop subgraphs.Source: B. Loopwise expansion of the effective… · pp. 89, 91

C. Example: φ3 + φ4-theory

  • The tree-level term is O(l), the one-loop term is O(1), and the two-loop corrections are O(l^-1); the resulting approximations of Γ are shown in figure 10.Source: C. Example: φ3 + φ4-theory · p. 92

D. Appendix: Equivalence of loopwise expansion and infinite resummation

  • The tree-level approximation resums infinitely many diagrams from ordinary perturbation theory, and this resummation may have better convergence properties than the original series.The appendix also indicates that this resummation property persists at higher loopwise orders.Source: D. Appendix: Equivalence of loopwise… · p. 93
  • Expanding about nonzero x* resums infinitely many perturbative terms, with each term’s vertices forming a single loop and the resulting series taking a logarithmic form.The passage identifies the resummation with ln(−S^(2)) = ln(lA) + ln(1 − A^−1V^(2)).Source: D. Appendix: Equivalence of loopwise… · p. 94

E. Appendix: Interpretation of Γ as effective action

  • The effective action yields the theory’s fluctuation-corrected solution through its stationary point and consists of one-line-irreducible diagrams.Its vertex functions and effective propagator generate connected contributions through tree-level diagrams.Source: E. Appendix: Interpretation of Γ as… · pp. 94–95

F. Loopwise expansion of self-consistency equation

  • The one-loop correction recovers the earlier self-consistency correction as a systematic next-to-leading fluctuation term, with higher-order corrections and other moments also accessible.Because Σ_xx ≡ 0, diagrams with two external x-legs vanish and need not be calculated.Source: F. Loopwise expansion of… · p. 97

XV. LOOPWISE EXPANSION IN THE MSRDJ FORMALISM

  • The loopwise expansion is a systematic perturbative technique whose number of diagram loops measures fluctuations in the system.The section applies this expansion to stochastic differential equations in the MSRDJ path-integral formalism.Source: §XV · p. 98

A. Intuitive approach

  • For weak noise, the lowest-order loopwise expansion matches the naive mean-drive approximation, while higher orders provide systematic corrections that account for fluctuation-induced changes to the mean.The finite stationary solution has ˜x ≡ 0; for stable dynamics, the fluctuation δx is centered.Source: A. Intuitive approach · pp. 98–99

B. Loopwise corrections to the effective equation of motion

The section uses loopwise expansions and self-energy corrections to derive self-consistency equations for stochastic dynamics, with a stationary Fokker–Planck solution as a check. It also reviews how the nomenclature relates generating functions and propagators to field-theory objects.

B. Loopwise corrections to the effective equation of motion

  • For D = 0, the trivial fixed point is stable for J < 1, while x± are stable for J > 1; α > 0 permits bistability, whereas α < 0 leads to runaway at high excitation.For α < 0 and 0 < J < 1, or α > 0 and J < 1, nontrivial fixed points exist but are unstable.Source: B. Loopwise corrections to the… · p. 100
  • The one-loop calculation sets the mean response field to zero and yields a self-consistency equation for the mean activity, checked against stationary Fokker–Planck fixed points.Figure 15 compares the numerical stationary Fokker–Planck solution, the one-loop approximation, and its first-order expansion in D.Source: Figure 15 · p. 100, B. Loopwise corrections to the… · p. 102

C. Corrections to the self-energy and self-consistency

D. Self-energy correction to the full propagator

E. Self-consistent one-loop

  • Replacing bare propagators with full propagators yields a self-consistent one-loop approximation, identified as Hartree–Fock.The equal-time covariance used in the diagram is the process variance.Source: E. Self-consistent one-loop · pp. 104–105

F. Appendix: Solution by Fokker-Planck equation

XVI. NOMENCLATURE

  • The nomenclature identifies Z as the moment-generating function, W = ln Z as the cumulant-generating function, and Γ as the generator of vertex functions or 1PI diagrams.Source: §XVI · pp. 105–106
Loading 1901.10416v1…