Balanced Excitation and Inhibition are Required for High-Capacity, Noise-Robust Neuronal Selectivity

• Although balanced networks were known to respond quickly and linearly to changing stimuli, their benefits for general information processing remained unclear.Source: Introduction · p. 3
• The paper argues that strong, balanced excitation and inhibition are needed for selective responses robust to output noise and for stable associative-memory states.Source: Introduction · p. 3
• The analysis combines nonnegative inputs, a firing threshold, sign-constrained bounded synapses, and input and output noise, and predicts strong balanced synapses for large input populations.It also predicts an optimal ratio of excitatory to inhibitory inputs for maximizing performance.Source: Introduction · p. 3

• The neuron is a binary threshold unit that classifies plus inputs as active and minus inputs as quiescent, using nonnegative afferent activities and sign-constrained synaptic weights.The exemplar activities are drawn from distributions with nonnegative means and specified covariance assumptions.Source: Results · p. 4
• Unbalanced solutions use weights of order 1/N, while balanced solutions use stronger weights of order 1/√N whose excitatory and inhibitory contributions approximately cancel.The balanced case keeps mean membrane potential near threshold; the model bounds the weight norm by Γ.Source: Results · p. 4
• Input noise reflects stimulus and sensory fluctuations and scales with synaptic weights, whereas output noise arises from unrelated afferents or internal processes and is weight-independent.Both sources produce membrane-potential fluctuations that can change the neuron's output state.Source: Results · p. 5
• Balanced solutions produce an O(1) signal between plus and minus patterns, comparable to both noise sources; unbalanced signals are overwhelmed by O(1) output noise.The authors conclude that balanced solutions offer the only hope for selectivity robust to both input and output noise.Source: Results · p. 5

Balanced and unbalanced solutions

• Only balanced solutions remain robust to substantial output noise: both solution types can tolerate input noise, but output-noise fluctuations are independent of weight magnitude.Input noise scales with synaptic weight magnitude, whereas output noise does not; balanced solutions spread membrane potentials farther from threshold.Source: Balanced and unbalanced solutions · p. 6, Figure 1 · p. 6
• For the constrained perceptron, capacity vanishes at fexc = 0 and rises to αc = 1 once fexc reaches f⋆exc, which matches the relative tuning of excitatory and inhibitory inputs.The critical excitatory fraction depends on input statistics; beyond it, capacity is half the unconstrained perceptron’s αc = 2.Source: Balanced and unbalanced solutions · p. 7
• Balanced solutions exist up to αb = αc when fexc ≤ f⋆exc; above f⋆exc, αb falls below αc and reaches zero at fexc = 1.For fexc > f⋆exc, typical solutions switch from balanced to unbalanced as load crosses αb, with weight norm collapsing from Γ to order 1/N.Source: Figure 2 · p. 8, Balanced and unbalanced solutions · p. 9
• Balanced solutions have near-zero input imbalance of order 1/N, whereas unbalanced solutions have imbalance of order 1; imbalance rises sharply above αb.Below αb, typical solutions are balanced and their weight norm is saturated at Γ.Source: Balanced and unbalanced solutions · p. 9

Noise robustness of balanced and unbalanced solutions

• Maximum input robustness remains order 1 up to αc, but substantial maximum output robustness occurs only below αb, where maximum-output-robustness solutions are balanced.These balanced solutions also achieve input robustness close to the maximum and the greatest possible among balanced solutions; solutions robust to both noise types exist below αb.Source: Noise robustness of balanced and… · p. 10

Balanced and unbalanced solutions for spiking neurons

• In the leaky integrate-and-fire model, balanced maximal-κout weights retain performance under increased output noise, while unbalanced maximal-κin weights deteriorate.Both solutions classify well without output noise, including with Poisson input fluctuations.Source: Balanced and unbalanced solutions for… · pp. 10, 12, Figure 4 · p. 13

Balanced and unbalanced synaptic weights in associative memory networks

• In associative-memory networks, large basins of attraction correspond to high input-noise robustness, while retrieval robustness to output noise corresponds to high output-noise robustness.The section considers recurrent weights that maximize input robustness (unbalanced in the chosen regime) and balanced weights that maximize output robustness.Source: Balanced and unbalanced synaptic… · pp. 12, 14
• Memory states in balanced networks tolerate output-noise levels an order of magnitude larger than in unbalanced networks, despite the unbalanced networks’ moderately larger attraction basins.The basins are estimated by the maximum initial-state distortion that still permits convergence to the parent memory with high probability.Source: Balanced and unbalanced synaptic… · p. 14, Figure 5 · p. 15
• As memory load rises, σmax_out declines smoothly to zero at αb, the maximum load supporting balanced solutions; for αb < α < αc, only unbalanced solutions exist.Below capacity, memory states remain stable when output noise stays below σmax_out.Source: Balanced and unbalanced synaptic… · p. 16

The role of inhibition in associative memory networks

• Plasticity in I-to-E connections suppresses spatiotemporal fluctuations in inhibitory feedback, yielding balanced performance with attraction basins comparable to or larger than those of unbalanced fully learned networks.Shuffling the I-to-E weights significantly degrades network performance, showing that inhibition contributes beyond global feedback.Source: The role of inhibition in associative… · p. 16

Learning Robust Solutions

• Learning with output noise is predicted to be sufficient for excitation-inhibition balance and robustness to both input and output noise, while input-noise learning alone can leave solutions output-noise-sensitive.κin can be large while κout remains small, but κout of order 1 implies weights and κin are also of order 1.Source: Learning Robust Solutions · p. 17
• High-output-noise training with a local, online, history-independent perceptron rule produces balanced weights with substantial κout and κin, whereas weak-output-noise training yields small κout.The rule includes sign constraints and weight decay; the figure shows substantial output robustness only with high output noise during learning.Source: Learning Robust Solutions · p. 17, Figure 6 · p. 18
• For αb < α < αc, learning fails to achieve reasonable performance under large output noise; reducing noise to the allowed κmax_N value yields unbalanced solutions with robustness near the regional maximum.This regime contains only unbalanced solutions.Source: Learning Robust Solutions · p. 17

Robustness trade-off

• Maximizing κout yields balanced solutions with minimal output-noise sensitivity and nearly the input robustness of maximal-margin solutions, at a modest capacity trade-off.Maximizing κin often instead yields unbalanced solutions that are extremely sensitive to output noise.Source: Discussion · pp. 18–19

Optimal input ratio

• Balanced-regime capacity is maximal at an excitatory-input fraction f⋆exc determined by the stimulus-tuning coefficients of variation of excitatory and inhibitory inputs.The discussion notes that fexc ≃ 0.8 would be optimal for CVexc/CVinh ≃ 4.Source: Discussion · p. 19

Plasticity assumptions

• The analysis generally assumes inhibitory neurons learn memory representations like excitatory neurons, although the main results also hold when inhibitory neurons do not explicitly code memories.The authors say how their learning rules map to experimentally observed plasticity remains unclear given cell- and synapse-specific rules and neuromodulatory control.Source: Discussion · p. 19

Associative memory

• For the same memory states, maximal-κin unbalanced weights have the largest basins but are sensitive to mild stochasticity; balanced weights trade somewhat smaller basins for greater output-noise robustness.The paper contrasts this with Hebbian associative-memory models, which generally have poorer capacity and smaller basins except at extremely sparse activity.Source: Discussion · p. 20

Balanced-network comparison

• Learned balanced networks support multiple stable fixed points, unlike prior dynamically balanced networks described as typically lacking the population nonlinearity needed for multiple attractors.In both cases, balance amplifies small changes in external or synaptic inputs.Source: Discussion · p. 20

Conclusion

• The paper attributes robust functionality to excitation-inhibition balance under basic biological constraints across single neurons and recurrent networks, and suggests balance may emerge through plasticity.The stated constraints include fixed-sign weights, non-negative afferent activity, a positive threshold, and input and output noise.Source: Discussion · p. 20

Finding perceptron solutions

• Constrained perceptron weight solutions maximizing κout or κin were found by solving conic-programming optimization problems.The algorithms enforce the imposed biological constraints.Source: Finding perceptron solutions · p. 21

Simulations of recurrent networks

• Recurrent-network simulations used randomly selected binary memory patterns as noiseless stable fixed points and initialized states with a controlled distortion while matching mean activity levels.The distortion level is δ, and the initialization procedure preserves the memory state's mean activity in excitatory and inhibitory populations.Source: Simulations of recurrent networks · p. 21

Perceptron learning algorithm

• The perceptron algorithm presents labeled patterns sequentially, adds Gaussian output noise, computes an error signal, and updates excitatory and inhibitory weights with sign-constrained rules and weight decay.Patterns are presented in randomized order each learning cycle.Source: Perceptron learning algorithm · p. 21

Dynamics of Leaky Integrate-and-Fire neuron

• The leaky integrate-and-fire model receives Poisson input spike trains, integrates synaptic and output-noise inputs, resets after threshold crossings, and classifies patterns by output spike count.The spike-count threshold is chosen to maximize classification performance; perceptron-derived balanced and unbalanced weights are compared in this neuron.Source: Dynamics of Leaky Integrate-and-Fire… · p. 22

Figures parameters

• Figure settings specify network sizes, pattern loads, input statistics, noise conditions, averaging procedures, and other simulation parameters across the main and supplementary figures.Reported examples include N = 3000 and 100 samples for Fig. 3, and N = 1000, P = 1000 for Fig. 4.Source: Figures parameters · p. 22

Software

• Computations used the Python scientific-computing stack, CVXOPT for convex conic optimization, and IPyparallel for cluster simulation parallelization.Source: Software · p. 23

Code availability

• Simulation and saddle-point-equation code is available upon request.Source: Code availability · p. 23

4.3 Tuning properties of cortical neurons

• Section 4.3 begins by invoking cortical-neuron tuning properties, but the passage does not state the resulting claim.Source: Supplementary Information: Balanced… · p. 29

Figure S1

• Figure S1 estimates αc at the load where solution-existence probability reaches 1/2 and measures αb from the probability of finding balanced maximal-κout solutions.The panels compare theoretical and measured capacity quantities across excitatory-input fractions and output proportions.Source: Figure S1 · p. 31, §1 · p. 31

Figures S2–S3

• Maximal-κin solutions can be balanced or unbalanced depending on input statistics and κmax, while silent-weight fractions vary by solution type and regime.For unbalanced maximal-κin solutions, half of both excitatory and inhibitory weights are silent; for unbalanced maximal-κout solutions, the excitatory silent fraction exceeds one half and the inhibitory fraction is below one half.Source: §1 · p. 32

Figure S3

• In the balanced regime, maximal-κout solutions saturate the weight-norm bound, making κout maximization equivalent to κin maximization under that norm constraint.The supplementary plot also presents the rescaled weight norm across system sizes.Source: Figure S3 · p. 33, §1 · p. 33

Figure S4

• With zero output noise, both balanced maximal-κout and unbalanced maximal-κin weights allow mean spike count to classify the input patterns.The figure compares plus- and minus-class spike-count histograms for an LIF neuron.Source: Figure S4 · p. 34

Figure S5

• Shuffling learned inhibitory weights while preserving their distribution reduces recurrent-network performance and stability.The comparison uses networks with the same distribution of inhibitory weights per excitatory neuron.Source: Figure S5 · p. 35

Figure S6

• For αb < α < αc, learning under high output noise leaves substantial training error and negative input and output robustness, whereas robustness is positive when output noise scales at most as 1/N.The compared learning conditions differ in output-noise magnitude while other model and learning parameters are held fixed.Source: §1 · p. 36, Figure S6 · p. 36

Conic formulation

• Both robustness objectives can be expressed as convex conic optimization problems, with sign, norm, linear, and quadratic constraints on the effective weights and threshold.The authors identify the use of conic optimization for maximizing κout as a novel contribution; maximizing κin is equivalent to the margin maximization used by Support Vector Machines.Source: §2 · p. 37

Optimization objectives

• Maximizing κin is equivalent to minimizing the effective-weight norm, while maximizing κout is equivalent to minimizing the effective threshold.The effective quantities are defined so that κin = 1/|u| and κout = Vth/b.Source: §2 · p. 37

Handling unrealizable constraints

• A global regularization variable τ permits the optimization to have a solution even when the classification constraints are unrealizable; τ = 0 indicates a zero-error solution exists.When τ is positive, the labeled patterns are not linearly separable and no zero-error solution exists.Source: §2 · pp. 37–38

Capacity dependence

• The capacity αc of an unconstrained perceptron depends on the fraction of plus patterns, pout.The passage introduces the capacity expression but does not supply a complete readable formula.Source: §3 · p. 38

Order parameter

• The capacity calculation uses an order parameter ∆ determined by an equation.The supplied passage gives only the equation's concluding description.Source: §3 · p. 38

Numerical measurement

• Figure S1 compares theoretical and measured αc and αb, with each quantity shown relative to its corresponding capacity scale for two pout values.The text identifies the comparison but supplies no numerical outcomes.Source: §3 · p. 38

4 Effects of excitatory and inhibitory input statistics

• The properties discussed depend on λ and φ, and φ determines the optimal excitatory fraction f⋆exc.λ is the ratio of inhibitory to excitatory input standard deviations; φ is the ratio of their coefficients of variation.Source: §4 · p. 38

4.1 Balanced and unbalanced maximal κin solutions

• Maximal-κin solutions may be balanced or unbalanced depending on fexc, φ, λ, and κmaxin.Source: §4.1 · p. 39

4.2 Fraction of ‘silent’ weights in maximal κin and maximal κout solutions

• For unbalanced maximal-κout solutions, silent excitatory weights exceed half and silent inhibitory weights fall below half; unbalanced maximal-κin solutions have half of each type silent.In the balanced regime, the qualitative silent-weight behavior for maximal-κout solutions depends on λ.Source: §4.2 · p. 39

4.3 Tuning properties of cortical neurons suggest that in cortex f⋆

• Under the stated cortical assumptions, inhibitory neurons have higher firing rates and broader tuning, implying λ and φ exceed 1 and informing f⋆exc.The analysis models afferents with Gaussian tuning curves and assumes a uniformly distributed stimulus and narrow tuning widths.Source: §4.3 · p. 39
• In purely excitatory networks, all solutions are unbalanced; maximal-κout solutions become sparser as load decreases, unlike maximal-κin solutions with a constant silent-weight fraction of one half.The text predicts higher measured connectivity for maximal-κin than maximal-κout solutions.Source: §4.3 · p. 40
• Experiments cannot readily distinguish silent synapses from synapses that were never available as inputs during learning.Source: §4.3 · p. 40

6 Recurrent networks with non-learned inhibition

• The basic associative-memory model specifies both excitatory and inhibitory activity in desired states and learns all connections; the section then replaces learning of E-to-I and I-to-I connections with random weights.The alternative where inhibitory activity is optimized using the full connectivity matrix is not addressed.Source: §6 · p. 40

6.1 Choosing random synapses for inhibitory neurons

• The random inhibitory network is parameterized to yield mean inhibitory activity mI at excitatory memory states, where activity is asynchronous and differs across memory states.The model assumes weak correlations and a Gaussian membrane-potential distribution at large N.Source: §6.1 · pp. 40–41

6.2 Training set definition

• Because inhibitory activity is nonstationary at desired memory states, training samples inhibitory patterns after running the inhibitory dynamics and adds them to the excitatory memory patterns.The inhibitory state is sampled after T = 100N time steps; nsample patterns per memory are used.Source: §6.2 · p. 41

6.3 Learned network stability

• Training convergence alone does not establish dynamical memory stability, so the authors separately test stability and attraction basins after releasing excitatory activity.The stability test runs the network for T = 500N time steps after initialization at the memory state.Source: §6.3 · p. 41

6.4 Learning only E to E connections

• Random I-to-E feedback combined with learned E-to-E weights failed above the tested memory capacity because feedback and inhibitory-activity variability were substantial relative to the signal.The authors attribute the restriction to heterogeneous I-to-E connections and variable inhibitory activity.Source: §6.4 · p. 42

6.5 Learning both E to E and I to E connections

• At out = 0.15, pII = pIE = 1/2, mI = 0.4, and nsample = 40, solutions implemented all desired memory states for the extended training set; excitatory states were stable with very high probability, with no unstable pattern observed.Source: §6.5 · p. 42

7 Replica theory for sign and norm constrained perceptron

• The calculation averages the logarithm of the Gardner volume over input distributions and labels, then determines order parameters from stationary-phase saddle-point equations.The model uses six order parameters: q, Q, θ, ∆, B, and C.Source: §7.1 · p. 43

7.1 The order parameters

• The order parameters q, Q, and θ describe correlations between solutions and properties of a typical solution’s induced potentials, while ∆ has a stated physical interpretation.The supplied passages explicitly define q as the typical correlation between potentials induced by two solutions to the same task.Source: §7.1 · pp. 43–44
• The saddle-point solutions separate into unbalanced cases, where θ is set by its equation, and balanced cases, where θ is clipped to its lower bound.q and Q remain order 1 across weight-norm scalings, whereas θ changes with the scaling.Source: §7.1 · pp. 43–44

7.2 Summary of main results

• Typical solutions are balanced below the balanced-capacity boundary αb and unbalanced above it, up to the total capacity αc.Balanced solutions exist for α < αb, while unbalanced solutions exist for αb < α < αc.Source: §7.2 · p. 44
• At the critical load, q approaching 1 marks the maximal load with solutions, and the saddle-point equations describe maximal-robustness solutions.The calculation relates constrained critical loads to those of the classic unconstrained perceptron.Source: §7.2 · p. 45
• For sparse binary inputs with mean activity s ≪ 1, the authors report that output robustness is given by κ0.The passage links this result to increased input tuning.Source: §7.2 · p. 45
• The balanced-capacity line depends on the excitatory-input fraction, and the optimal fraction f⋆ maximizes balanced capacity for the given input statistics.When the excitatory fraction exceeds f⋆, high loads can preclude balanced solutions.Source: §7.2 · p. 46

7.3.2 Saddle point equations for typical solutions

• At typical loads, solutions are balanced for α < αb and unbalanced for α > αb.The norm and imbalance index are reported as properties of these solutions.Source: §7.3.2 · p. 48

7.3.3 Solutions with significant κout are balanced

• All unbalanced solutions have output robustness of order 1/√N, so solutions with κout of order 1 are balanced.This result is stated as a theorem and derived from the saddle-point equations.Source: §7.3.3 · p. 49

7.3.4 Saddle point equations for critical solutions

• In the critical limit q → 1, the saddle-point equations are used to determine capacity and maximal input or output robustness.The critical equations relate constrained critical loads to the classic unconstrained perceptron’s critical load.Source: §7.3.4 · p. 49

7.3.5 Capacity and balanced capacity

• At capacity, the solution is balanced for excitatory fractions below f⋆; the section derives separate balanced and unbalanced capacity cases.The validity of the unbalanced solution is subject to a lower-bound condition on θ.Source: §7.3.5 · pp. 51–52

7.3.6 Coexistence of balanced and unbalanced solutions below the balanced capacity line

• Below αb, balanced and unbalanced solutions can coexist, but the minimum norm permitting unbalanced solutions diverges as α approaches αb.As the norm limit grows, the capacity approaches the balanced-capacity line.Source: §7.3.6 · p. 53

7.3.7 Saddle point equations for the maximal κin solution

• For maximal input robustness, the transition between balanced and unbalanced solutions depends on φ, λ, and fexc; solutions are balanced for φ > φ⋆.The transition point is defined by B = 0 and θ̃ = 0.Source: §7.3.7 · p. 54

7.3.8 Saddle point equations for the maximal κout solution

• The maximal-output-robustness analysis treats balanced and unbalanced cases separately, and its equations are equivalent to those derived in.For the balanced case, κout Vth/σexcΓ replaces κin in the corresponding equations.Source: §7.3.8 · p. 55

7.3.9 Distribution of synaptic weights

• The authors derive mean synaptic-weight distributions for excitatory and inhibitory synapses in critical solutions and give the corresponding silent-synapse fractions.The fractions are expressed using H(−B) for excitatory and H(Bφ) for inhibitory synapses.Source: §7.3.9 · p. 55