Source-linked AI summary
Non-Zero Radius: The Missing Component of Neural Networks
Tiansi Dong, Mateja Jamnik, Pietro Liò
TL;DR
Conventional neural networks still face challenges in logical reasoning despite their broad empirical success. This paper represents concepts as spheres and uses a transition map for syllogistic reasoning without training data. It reports that SphNNs construct a satisfying sphere configuration for any satisfiable set of syllogistic constraints in one epoch.
Problem
Conventional neural networks face challenges in reliable logical reasoning, including simple abstract reasoning errors and uncertainty about whether LLMs truly reason.
Source: §2 · p. 2Method
Results
For any satisfiable set of syllogistic constraints, SphNN constructs a satisfying sphere configuration in one epoch.
Source: §6 · p. 4Takeaways & Limitations
The authors argue that sphere representations restore region-based structure so logical transitions can be encoded directly rather than learned indirectly from examples.
Source: §7 · p. 4The sphere embedding assumes that each concept denotes a set.
Source: §5 · p. 3
Abstract
from arXiv · showhide
Inspired by the analogy between the all-or-none activation of biological neurons and the true-false of logic circuits, McCulloch and Pitts proposed a mathematical model of the biological neuron in 1943 with the aim of simulating logical reasoning. Although Norbert Wiener considered the model overly simplistic at the time, it later became a building block of modern neural networks, including large language models. Despite their remarkable empirical success, however, conventional neural networks remain largely black-box systems and struggle with reliable symbolic reasoning. Viewing vectors as spheres of zero radius, we relax the zero-radius constraint and represent concepts as spheres with non-zero radii. We argue that sphere-based representations can capture additional features of biological cognition, including representing spatial concepts as regions and reasoning through model construction and inspection. Building on sphere embeddings, we introduce Sphere Neural Networks (SphNNs), which use a transition map to achieve symbolic-level Aristotelian syllogistic reasoning without training data, thereby taking an initial step toward the original goal of McCulloch and Pitts. We suggest that the zero-radius constraint may reflect an aspect of the oversimplification identified by Wiener and may limit conventional neural networks' capacity for symbolic-level logical reasoning.
1 Introduction
The introduction connects neural models to logical deduction, then asks whether adding radius could address an oversimplification in the original model. It uses disjunctive syllogism as a concrete example of the reasoning at stake.
1 Introduction
- A proposed AGI test asks whether a model trained only through 1911 could derive general relativity as Einstein did in 1915.Source: §1 · p. 1
- Disjunctive syllogism has the form “A∨B. ¬ A. ∴B” and is described as foundational reasoning used across real-world domains.The passage cites education, business, medical decision-making [67], and legal judgement.Source: §1 · p. 2
- McCulloch and Pitts linked biological neurons’ all-or-none firing to logic circuits in their 1943 mathematical model.Source: §1 · p. 2
- The authors ask whether extending point representations to spheres supplies the radius missing from the oversimplified model.Source: §1 · p. 2
2 The McCulloch–Pitts’ Model Falls Short of Symbolic-Level Syllogistic Reasoning
The section describes persistent limits in conventional neural networks’ logical reasoning despite broad task performance. Prior evaluations report inconsistent results across model sizes, syllogism types, and lexical variations.
2 The McCulloch–Pitts’ Model Falls Short of Symbolic-Level Syllogistic Reasoning
- Conventional neural networks can err on simple abstract reasoning and give correct answers with incorrect explanations, while whether LLMs truly reason remains unclear [18] [11].Source: §2 · p. 2
- PaLM 2-Small outperformed PaLM 2-Large, so syllogistic accuracy did not consistently increase with model size [18].Source: §2 · p. 2
- Average zero-shot accuracy was 70% on generalised modus ponens and 23% on disjunctive syllogism for the evaluated model families [67].The passage also reports sensitivity to surface-level lexical variations in zero- and few-shot settings [67].Source: §2 · p. 2
3 Mental Model Construction with Regions
The section draws on spatial cognition and mental-model accounts to motivate representations of concepts as regions and reasoning as constructing and inspecting models.
3 Mental Model Construction with Regions
- Spatial objects can be represented as regions, with nested regions capturing spatial hierarchies and parent–child relationships.Source: §3 · p. 2
- Mental-model theory describes reasoning as model construction, inspection, and variation, including searches for counter-models that would invalidate a conclusion.Source: §3 · p. 2
- The preferred mental-model account treats initial model construction as deterministic, then generates alternatives through local, minimal-change transformations.The passage describes transitions among models with a neighbourhood graph.Source: §3 · p. 2
4 Vectors are Zero-Radius Spheres
The section defines sphere inputs as a generalization of vectors and describes their use for geometric matching. It also presents figures illustrating set relations.
4 Vectors are Zero-Radius Spheres
- SphNN inputs encode a center vector and radius, with vectors recovered as the zero-radius case.The input sphere contains each input vector within distance r of its center.Source: §4 · p. 3
- The paper extends the diameter-limited perceptron idea by explicitly defining its input sphere, while Figure 5 depicts subset, reverse-subset, disjoint, and partial-overlap cases.The sphere definition is described as consistent with convex regions in mental representation.Source: §4 · p. 3, Figure 5 · p. 3
- The non-zero-radius extension models receptive-field matching by representing stimuli and receptive fields as spheres and applying a non-monotonic activation function.The activation indicates containment when Δ(O1, O2) = 0; for equal radii, larger Δ means greater separation between centers.Source: §4 · p. 3
5 Representing Set-Theoretic Relations and Syllogistic Statements as Sphere Relations
The section represents concepts as spheres and encodes set-theoretic relations as geometric relations between them. It then maps the four Aristotelian statement forms onto those relations.
5 Representing Set-Theoretic Relations and Syllogistic Statements as Sphere Relations
- Set inclusion is represented by one sphere being part of another, disjointness by disconnected spheres, and partial overlap by neither containment nor disconnection.The representations assume that each concept denotes a set.Source: §5 · p. 3
- In the example, “rich” is the shared concept relating Greeks and Philosophers in the premises.Source: §5 · p. 3
- All X are Y maps to containment, No X are Y to disconnection, Some X are Y to non-disconnection, and Some X are not Y to non-containment.Source: §5 · p. 3
6 Symbolic-Level Neural Reasoning through Map-guided Model Construction without Training
SphNNs test syllogistic validity by constructing sphere configurations that satisfy the premises and negated conclusion. A built-in transition map guides the construction, which is guaranteed to find a satisfying configuration within one epoch when one exists.
6 Symbolic-Level Neural Reasoning through Map-guided Model Construction without Training
- An argument is invalid when a sphere configuration satisfies its premises and the negation of its conclusion; otherwise, it is valid.Source: §6 · p. 4
- The transition map integrates neighbourhood transitions, including a partial-overlap step between disconnection and containment; SphNNs use this map rather than training on data.Figure 6 depicts the map in n-dimensional space and on a sphere's surface.Source: §6 · p. 4, Figure 6 · p. 4
- For any satisfiable set of syllogistic constraints, the gradual construction procedure is guaranteed to produce a satisfying sphere configuration in one epoch; completing an epoch without a counterexample implies validity.Source: §6 · p. 4
- In the example, the counterexample search encodes the premises as sphere relations and seeks a configuration satisfying those relations plus containment of Greeks in Philosophers.Source: §6 · p. 4
- Embedding a set as arcs allows its complement to be represented as an arc as well.Source: Figure 7 · p. 4
7 Summary and Outlook
The section argues that non-zero-radius spheres restore region-based structure absent from conventional vector representations. It describes how sphere configurations support syllogistic and additional forms of reasoning, including on a sphere's surface.
7 Summary and Outlook
- Conventional vectors are degenerate sphere representations with zero radii, collapsing regions into points and removing the geometric relations needed for SphNN's transition mechanism.Source: §7 · p. 4
- Sphere configurations encode set relations and let SphNNs represent logical transitions directly, whereas conventional networks typically learn to approximate such reasoning from training data.Source: §7 · p. 4
- On a circular universe, spheres become arcs, and the transition map can update arcs representing sets or their complements for disjunctive syllogism and 16 additional syllogistic-style reasoning types [7].Source: §7 · p. 4
- The paper suggests that the zero-radius restriction may be a neglected barrier to AGI and that removing it brings AI closer to symbolic-level rational reasoning.Source: §7 · p. 4