Neural Logic, Invariance, and the Retina---McCulloch and Pitts

Nima Dehghani

Source-linked AI summary

Neural Logic, Invariance, and the Retina---McCulloch and Pitts

Nima Dehghani

arXiv:2609.02183v1Sep 2, 2026q-bio.NCcond-mat.dis-nncs.FLcs.NEphysics.bio-ph

TL;DR

The chapter asks how physical neural systems can realize logic, preserve task-relevant relations across transformations, and identify sensory operations. It reconstructs the McCulloch-Pitts program historically while applying modern formal tools to its dynamics, invariance mechanisms, and retinal evidence. The result is a unified account of neural computation as causal organization with explicit idealization limits.

Problem
The chapter addresses how a material nervous system can realize logical relations and recognize forms despite changes in pitch, size, position, and other physical conditions.
Method
It combines historical reconstruction of neural logic with finite-state analysis, Boolean threshold theory, group averaging, feedback canonicalization, and spike-triggered analysis.
Results
The chapter connects neural logic, heterarchy, invariance, and frog-retina physiology as operations through which physical networks preserve task-relevant relations.
Takeaways & Limitations
McCulloch and Pitts’ program is best understood as a theory of causal organization in matter, extending from logical circuits to invariant sensory operations.
Takeaways & Limitations
The idealization lacks general data-driven learning, explicit noise tolerance, and unbounded memory, while the retinal channels are qualitative models rather than fits to original spike trains.

Abstract

from arXiv · show

This chapter reconstructs the McCulloch-Pitts program as a physics of neural computation rather than the familiar cartoon of a binary neuron. The 1943 logical calculus is developed in both directions: given a net, characterize the propositions realized by its activity; given an admissible logical expression, construct a net that realizes it. We recover the original distinction between thresholded excitatory summation and absolute inhibitory veto-one the weighted-threshold form cannot preserve for arbitrarily large excitatory inputs-and read unit-time delay as the physical realization of logical depth. Recurrence is treated exactly: an autonomous, deterministic network of finitely many binary units has a finite state space, so every trajectory eventually enters a periodic orbit-a fact about finite-state dynamics, not unbounded Turing computation. A single threshold element realizes only linearly separable Boolean functions, whereas finite feedforward networks of them synthesize any Boolean function on a finite domain. We then follows McCulloch and Pitts beyond threshold logic. The 1945 heterarchy paper turns cyclic preference into an obstruction to representation by a scalar utility. The 1947 work on universals asks how a physical network can identify inputs related by nuisance transformations, developed here via group averaging and feedback canonicalization. The 1959 frog-retina study makes the adequate-stimulus question experimental, revealing parallel invariant operations before the brain proper. Spike-triggered analysis shows how a nonlinearly driven neuron can have a vanishing first-order average while second-order statistics recover its hidden selectivity: methodological failure can masquerade as physiological absence. Modern mathematical tools are used without projecting their notation onto the historical papers, and limitations of the idealization are stated explicitly.

CHAPTER ORIENTATION

The chapter reconstructs McCulloch and Pitts as developing a physics of neural computation grounded in causal networks, logical relations, invariance, and experimental sensory operations. It connects the 1943 calculus to later work on heterarchy, universals, and frog-retina physiology while separating historical claims from modern formalization.

Chapter orientation: The chapter treats neural activity as a causal formal system involving threshold, inhibition, delay, recurrence, and logical relations.This frames cognition as organization of matter in time rather than as a binary-neuron cartoon.
Chapter orientation: The chapter follows the program beyond threshold logic through cyclic preference, transformation invariance, and experimentally identified retinal operations.The 1945, 1947, and 1959 works are presented as connected by questions about topology, invariance, and sensory function.
Chapter orientation: A fixed recurrent network of N binary units has at most 2^N states, while a single threshold element computes only linearly separable Boolean functions.Finite networks of threshold elements can nevertheless synthesize every finite Boolean function.
Chapter orientation: Modern tools such as group averaging, feedback canonicalization, and spike-triggered analysis are used without attributing their current notation to the historical papers.The chapter distinguishes exact historical claims from later mathematical formalization and uses controlled replications rather than digitized historical figures.
Chapter orientation: The chapter positions McCulloch and Pitts within existing traditions including mathematical biophysics, network models, switching theory, mathematical logic, relay circuits, and effective computation.Pitts is presented as a central mathematical coauthor rather than a junior assistant.
Chapter orientation: The 1943 synthesis analyzes propositions realized by a net and constructs nets realizing admissible logical expressions.These are the complementary directions of analysis and synthesis.

III. THE 1943 MODEL: A CAUSAL CALCULUS IN DISCRETE TIME

The 1943 model represents neural computation as a causal calculus in discrete time, combining excitatory thresholding with absolute inhibitory veto and interpreting spikes as propositions about prior activity. Its formal semantics captures realized causal relations while remaining an idealization rather than a complete account of neural representation.

Degrees of freedom and update law: Each unit receives excitatory and inhibitory inputs, with thresholded excitatory summation combined with an absolute inhibitory veto.Any active inhibitory input suppresses the output, a distinction that ordinary weighted-threshold notation can obscure.
Degrees of freedom and update law: Equation (2) is a later generalization that matches the original veto only on specified input domains when negative weights are sufficiently large.It is not identical to the original rule for arbitrary input combinations.
Degrees of freedom and update law: The unit time step makes each output a proposition about earlier activity and gives circuit depth a temporal meaning.Because updates use state t to determine state t + 1, directed cycles do not create algebraic loops.
Degrees of freedom and update law: The update rule can be read as a two-stage physical test: inhibitory silence first, followed by sufficient excitation, with output deposited after one delay.This combines logic, causal ordering, and an idealized inhibition mechanism.
Degrees of freedom and update law: The model selects an equivalence class of nets sharing causal input-output behavior rather than identifying each formal element with one literal cell.Alternative assumptions about threshold, delay, and inhibition may yield behaviorally equivalent nets with different units or elapsed times.
Degrees of freedom and update law: Under the formal semantics, a spike is a proposition about causal antecedents sufficient for firing, called an adequate stimulus.Logical describability does not by itself determine the animal’s representational ontology.

A. AND, OR, and NOT

McCulloch–Pitts units provide thresholded logical operations, and functionally complete constructions extend single-unit operations to arbitrary finite Boolean functions. Delays and recurrence make the resulting computation temporal and stateful rather than merely truth-functional.

A. AND, OR, and NOT: Threshold two computes AND, while threshold one computes OR.
A. AND, OR, and NOT: NOT uses tonic excitation together with inhibitory input from the signal being negated.
A. AND, OR, and NOT: AND and NOT are functionally complete, so every Boolean function can be synthesized through disjunctive normal form.
A. AND, OR, and NOT: XOR cannot be computed by one weighted threshold unit because its positive-case inequalities contradict the negative case for (1, 1).
A. AND, OR, and NOT: A two-stage threshold construction computes XOR exactly, with inhibitory veto implementing the final exclusion.
A. AND, OR, and NOT: If a path contains d successive units, its result appears after d time steps, so recombining branches require delay matching.
A. AND, OR, and NOT: With fixed input, recurrence defines an autonomous map on {0, 1}^N; finite deterministic trajectories enter periodic orbits after transients.

B. Memory is state dependence, not necessarily synaptic change

Neural memory can reside in recurrent state rather than synaptic change. Finite recurrent nets therefore implement finite-state dynamics, not unbounded computation, while circuit realizability remains distinct from learning and robustness.

B. Memory is state dependence, not necessarily synaptic change: Reverberating loops retain past events in current internal state after the original input disappears.
B. Memory is state dependence, not necessarily synaptic change: Dynamical memory is carried by x(t), whereas slower structural memory is carried by changing parameters W(t).
B. Memory is state dependence, not necessarily synaptic change: The 1943 calculus specifies fixed connections and does not provide a general plasticity rule for structural memory.
B. Memory is state dependence, not necessarily synaptic change: A recurrent network of N binary units has at most 2^N internal states and can recognize regular temporal patterns with an input stream.
B. Memory is state dependence, not necessarily synaptic change: A fixed finite network does not become Turing complete merely by containing a directed cycle.
B. Memory is state dependence, not necessarily synaptic change: Constructing a circuit for a function does not supply an algorithm that learns that circuit from examples.

A. Robustness is a separate theorem

Logical realizability, robustness, scalar valuation, and invariance are separate questions. The chapter distinguishes noise tolerance from circuit construction and frames cyclic preference and nuisance transformations through topology and symmetry.

A. Robustness is a separate theorem: A positive classification margin preserves outputs under perturbations smaller than that margin on the finite labeled set.
A. Robustness is a separate theorem: Logical realizability alone does not imply noise tolerance, graceful degradation, or biological reliability.
A. Robustness is a separate theorem: The cycle A ≻ B, B ≻ C, and C ≻ A would require the impossible ordering U(A) > U(B) > U(C) > U(A).
A. Robustness is a separate theorem: A directed preference graph admits a strictly increasing scalar potential exactly when it is acyclic.
A. Robustness is a separate theorem: Heterarchical dynamics need not be integrable into a global scalar potential, unlike strict descent of one energy function.
A. Robustness is a separate theorem: Nuisance transformations can be handled by averaging transformed presentations or using feedback to drive inputs toward a standard form.
A. Robustness is a separate theorem: Group averaging makes representations invariant by generating equivalent presentations and forgetting which orbit point was observed, but it can discard task-relevant information.

B. Invariant, equivariant, and selective

Invariant representations can be built by averaging transformed presentations or by feedback-driven canonicalization, while equivariant maps preserve spatial structure before pooling. These mechanisms improve nuisance tolerance but introduce computational and ambiguity-related trade-offs.

Invariant and equivariant representations: Averaging over a transformation orbit produces invariance but can blur distinctions and discard information treated as nuisance.The same transformation orbit may contain task-relevant differences, so invariance must be selective.
Invariant and equivariant representations: Equivariant feature maps move activity with the stimulus, allowing later pooling to create invariance while preserving earlier selectivity.Retinotopic organization is approximately translation equivariant.
Orbit matching: Max matching over 45 translated and scaled templates raises mean normalized match from 0.270 to 0.717 over tested probes.This is a finite modern demonstration rather than a reconstruction of the 1947 circuits.
Feedback canonicalization: Feedback canonicalization estimates a transformation and reverses it, making equivalent transformed inputs share a canonical representation.The approach can use biological orienting movements instead of internally representing every transformed template.
Feedback canonicalization: In a one-dimensional coded example, absolute pose error falls from 0.56 to 0.0038 after twelve discrete corrections.The mismatch functions as a Lyapunov function, linking invariance to dissipative dynamics.
Limits: Canonicalization can fail under symmetry, estimator discontinuities, or delayed and excessively strong feedback.These are physical constraints on invariant computation rather than merely abstract mathematical concerns.

X. FROM SYNTHETIC NETS TO THE FROG’S VISUAL SYSTEM

The frog-retina study treated visual computation as an experimental search for adequate stimuli and found parallel, illumination-robust operations represented as spatial maps. The chapter models these selectivities qualitatively while preserving the study’s species and modeling limits.

The experimental question: Single-fiber recordings tested which stimulus properties were essential by varying candidate features after finding strongly activating stimuli.The study used spots, edges, geometric objects, movement, and behaviorally suggestive targets in intact frogs.
The experimental question: The retina decomposed visual input into four principal parallel operations that were nearly independent of general illumination.A rare fifth group tracked absolute darkness over a wide area but was not treated as a principal channel.
Parallel maps: Each operation was represented across retinal position, with four depth-separated registered retinotopic maps in the tectum.Contrast, convexity, moving-edge, and dimming terminals occupied progressively deeper tectal layers.
Selectivity and interpretation: The memorable “bug perceiver” label denotes a conjunction of size, contrast polarity, boundary geometry, position, and motion history, not context-free semantic recognition.Behavioral relevance motivates the stimulus family without turning one axon into a proposition.
Effective modeling: The chapter’s effective model uses normalized spatial and temporal filters to generate contrast, compact-dark-object, moving-edge, and dimming fields.It is pedagogical rather than a fit or digitization of the 1959 recordings.
Effective modeling: Under a roughly 30-fold illumination ramp followed by global dimming, object-linked channels follow the object while the dimming response peaks at the global step.A representative receptive field responds when the object crosses its center.

A. From filter field to spikes

The chapter connects stimulus filtering to spike inference by showing when first-order spike-triggered averages recover selectivity and when nonlinear responses require second-order statistics. It uses these results to frame adequate-stimulus analysis as an identifiability problem.

From filters to spikes: An LN/LNP description maps stimulus history through a spatiotemporal filter and nonlinear rate function, but one linear filter cannot represent convexity or contrast energy.Those features require multiple subunits or quadratic combinations.
From filters to spikes: Controlled stimulus optimization tests a proposed feature by seeking strong responses and then varying nuisance variables while preserving that feature.A robust hypothesis predicts both excitation and invariance rather than only one maximally effective stimulus.
Spike-triggered averages: For monotonic responses to a linear Gaussian projection, the spike-triggered average recovers the adequate filter up to scale.Rotational symmetry makes the conditional mean parallel to the true filter.
Nonlinear selectivity: Even contrast-energy responses can produce zero spike-triggered average because opposite stimuli drive equal firing and cancel in the first-order statistic.Spike-triggered covariance can instead reveal a direction proportional to kkT.
Nonlinear selectivity: With 120,000 seeded samples, STA correlation is 0.990 for a linear-nonlinear unit and 0.061 for an energy unit, while covariance recovers the latter at 0.989.The comparison shows how analysis failure can masquerade as absent selectivity.
Stimulus design: Stimulus ensembles determine which model dimensions are identifiable, so behaviorally meaningful probes and controlled perturbations can complement white noise.Naturalistic objects can expose nonlinear conjunctions, while perturbations test candidate terms.

XIII. WHAT EXACTLY WAS MCCULLOCH’S SCIENTIFIC ADVANCE?

McCulloch’s scientific advance was to make neural activity a causal formal system spanning logical realization, feedback, topology, invariance, and experiment. The program connected formal circuit synthesis to an experimental account of invariant operations and adequate stimuli.

Neural activity became a causal formal system in which spikes functioned as propositions and nets as temporally indexed logical relations.
The 1943 calculus was bidirectional: it characterized propositions realized by given nets and synthesized nets for admissible expressions.
Feedback made internal activity and temporal patterns computational state variables without requiring recurrent computation to descend an energy function.
Cyclic preference exposed a topological obstruction to representing value with a single scalar hierarchy.
Invariance became a circuit-building problem through transformation averaging and feedback to canonical presentations.
The frog study extended the adequate-stimulus logic experimentally by identifying parallel invariant operations in single fibers before the brain proper.

XIV. LIMITS, BIOLOGICAL INTERPRETATION, AND EFFECTIVE SCALE

The McCulloch–Pitts model is an effective theory of causal organization, not a cellular reconstruction. Its usefulness depends on whether the retained variables close at the task’s observational scale, while the retinal models remain operation-level approximations.

The binary net omits spike timing within bins, firing rate, dendritic location, adaptation, neurotransmitter, stochasticity, and heterogeneous conduction.
If microscopic states sharing the same binary x(t) produce different x(t + 1), the binary state is not a sufficient Markov state.
A binary point-neuron description remains informative when omitted variables only renormalize thresholds, effective delays, or noise over the task timescale.
The frog channel equations are operation-level approximations that earn their role by predicting stimulus-response relations rather than reproducing every anatomical detail.
The numerical figures test mathematical mechanisms under specified conditions but do not establish biological sufficiency, reconstruct historical circuits, replicate original spike trains, or fully analyze frog ganglion cells.

5. Count units and edges for the direct DNF construction

The supplied passages frame finite-network analysis through graph structure, recurrence, invariance, retinal channel identification, and spike-triggered methods rather than giving the requested direct DNF unit-and-edge count. They specify several proof and comparison tasks across these mechanisms.

The exercises ask readers to separate parity-network depth, size, and fan-in when comparing direct constructions with trees of two-input XOR subnetworks.
Finite recurrent networks are examined through fixed points, cycles, transients, and the threshold-circuit cost of an N-bit binary counter.
The invariance exercises compare averaging, equivariance, canonicalization, stabilizers, and the accuracy-cost tradeoff from sampling transformation groups.
Retinal-model exercises test whether stimulus-response matrices identify channels and distinguish curvature detectors from small-dark-object detectors under matched stimulus conditions.
Spike-triggered exercises derive why symmetric stimulation makes the STA zero and how covariance changes recover nonlinear selectivity.

XVII. A PRIMARY-LITERATURE READING SEQUENCE

The reading sequence moves from the 1943 logical calculus through finite-state dynamics, heterarchy, invariance, feedback, and frog physiology, while using modern formalizations to clarify rather than replace the primary sources. It ends by identifying a continuing problem: learning effective variables and invariances while preserving dynamical closure.

1943 logic: Begin with the 1943 paper’s definitions and follow both directions: characterizing nets and constructing nets from admissible expressions.
Finite-state dynamics: Read Shannon, Turing, and Kleene alongside the 1943 work to distinguish switching algebra, effective procedure, neural-net synthesis, finite automata, and regular expressions.
Heterarchy: Use the 1945 heterarchy paper to connect preference cycles with the obstruction to a scalar potential, while treating the modern theorem as a foothold rather than a replacement.
Invariance and feedback: Study universals through averaging and feedback standardization, identifying the transformation group, preserved selectivity, and physical cost of traversing or correcting the orbit.
Frog physiology: Read the 1959 frog paper from behavior to anatomy, documenting stimulus classes, null stimuli, illumination, temporal persistence, field size, conduction class, and tectal depth.
Open problem: The unresolved task is to learn effective variables and invariances while keeping the physical neural system dynamically closed under its own behavior.