Source-linked AI summary
Gradland: On Phenomenal Experience, Differentiated Across Many Dimensions
David Balduzzi
TL;DR
The paper asks whether the first-order structure of physical interactions can characterize phenomenal experience. It develops a differential-functional approach using Jacobians and measures of their structure, then applies it to neural-network examples involving duration, vividness, learning, and rich experience. The paper concludes that this framework accounts for several differentiated aspects of experience, while remaining a minimal and open-ended exemplar for more realistic models.
Problem
The paper addresses how phenomenal experience relates to the physical interactions that generate it.
Method
The paper studies Jacobians and their compositions in idealized neural networks, using matrix-based measures of interaction structure across worked examples.
Results
The framework accounts for patterns involving duration, vividness, temporal integration, distinct and confused ideas, learning, and the function of rich experience.
Takeaways & Limitations
The paper presents differential functionalism as a minimal exemplar for studying how physical interaction structure may relate to phenomenal experience.
Takeaways & Limitations
The approach is developed in idealized neural-network models, and how well it applies to more realistic brain models remains an open question.
Abstract
from arXiv · showhide
This paper investigates the hypothesis that the first-order structure of physical interactions, i.e. gradients or Jacobians, characterizes the structure of phenomenal experience. It does so in an idealized world inhabited by neural networks, Gradland, where the physics are known and the functions are (mostly) differentiable. The paper introduces two measures of Jacobian structure: effective rank and cohesion, based on Kirchhoff complexity. Applying the measures to a series of worked examples shows the hypothesis accounts for: (1) the duration of experience, that it can prolong over hundreds of milliseconds; (2) the difference between what is experienced vividly and obscurely; (3) the experience of texture; (4) the blooming buzzing confusion presumably experienced by newborns; (5) the difference between ideas that are held distinctly in mind and ideas that are confused; (6) what learning is like; and finally (7) the paper explains the function of rich, dense experience.
1 Introduction
The paper asks how phenomenal experience relates to physical interactions, proposing gradients and matrices as tools for studying that relationship. It frames experience as transparent and cohesive, while noting that the approach remains exploratory and faces difficult measurement and boundary questions.
- Analytical tools: Applied mathematics supplies three relevant tools: differentiation, aggregation, and diagonalization.These tools are used to analyze how physical interactions compose.
- Core hypothesis: The paper studies whether the first-order structure of physical interactions can characterize phenomenal experience.It focuses on gradients, matrices, and their compositions as tools for relating physical structure to experience.
- Transparency: Experience is transparent because systems can experience the world through their own eyes, neurons, bodies, and other components.The paper connects this compositional structure to gradients and matrices.
- Cohesion: Experience is cohesive because interactions bind activity across time and space rather than localizing it in one component.Cohesion is presented as a measure of interaction density spanning matrix inputs and outputs.
- Scope: The paper studies these ideas in Gradland, an idealized neural-network world, through examples concerning duration, learning, and the organization of experience.The paper presents the approach as a differential-functional account rather than a complete theory of consciousness.
2 Motivations
The motivations combine a behavioral starting point, a response to mechanical pictures of mind, and a methodological choice to study relations through matrices. The paper treats interactions and their transformations as more informative starting points than isolated things.
- Experience and interaction: Experience is treated as what it is like to interact, with behavior potentially extending across neurons, glia, bodies, and the environment.This approach rejects a complete separation between experience and physical functioning.
- Relations and unification: The paper uses convergent interactions as an alternative to treating colors or other phenomena as lifeless sets of isolated elements.The lens metaphor illustrates how relations can unify differentiated elements without reducing them to a single abstract category.
- Why matrices: Matrices are chosen because derivatives represent linear transforms, and matrices represent those transforms in a fixed basis.The paper extends this relational perspective from observable quantities to the study of phenomenal experience.
- Scope of the matrix analogy: Linear approximations are presented as useful tools for representing relations, without claiming that reality consists of matrices at every scale.The paper’s claim is to focus on relations and begin with ubiquitous matrix representations.
- Methodological stance: The paper adopts the methodological principle of studying relations between observable quantities rather than directly observing phenomenal experience.It explicitly works in media res rather than attempting to identify a base layer of reality.
3 Gradland
Gradland models neural networks as mostly differentiable physical systems with known input-output functions, assuming that first-order interactions characterize experience. It analyzes experience using Jacobians, effective rank, and cohesion, connecting transparency and interaction density to phenomenal structure.
- 3 Gradland: Gradland rebrands mathematical models used by AI researchers as an idealized world with mostly differentiable systems, observed inputs and outputs, and known physics.Hypothesis G is assumed true in this setting: experience is identified with first-order interactions.
- 3 Gradland: The paper defines neural-network systems as subgraphs with nonempty, disjoint inputs and outputs, while parameters of input nodes are ignored.The system may be the loss, a layer’s outputs, or outputs of all layers except the first.
- 3 Gradland: The framework represents MLPs as feedforward layer compositions, RNNs as feedforward systems with recurrent updates, and temporal copies of MLPs as disconnected when there is no cross-time interaction.Standard layer computation is z^(l) = f(a^(l)) with a^(l) = W_l z^(l−1).
- 3 Gradland: Jacobians are obtained by differentiating outputs with respect to inputs or parameters, including tensor-valued parameter Jacobians for network layers.For an MLP, Jy←Wl is the Jacobian of selected outputs with respect to parameters in layer l.
- 3 Gradland: Hypothesis G states that Jacobians characterize the structure of experience and vice versa, with transparency measuring effective rank and cohesion measuring how interactions bind inputs and outputs.Effective rank captures vividness and the number of impressions, while cohesion is zero for noninteracting blocks and increases with mixing.
- 3 Gradland: Effective rank and cohesion can diverge: identity-like Jacobians maximize transparency while having zero cohesion, whereas strongly mixing matrices can maximize cohesion.A system with zero cohesive rank is either inert or decomposes into independent subsystems; maximizing cohesive rank is proposed as one way to identify experiencing systems.
4 Living in Gradland
The worked examples argue that Jacobian structure tracks how experience is woven across inputs, time, vividness, texture, learning, and simultaneously held ideas. Cohesion captures interactions that bind experience, while gradient structure distinguishes what remains vivid from what fades into obscurity.
- Weaving the world in me: Input Jacobians can capture overlapping experience while parameter Jacobians preserve independent decisions in separate neural networks viewing the same tree.Shared perturbations produce nonzero cohesion across outputs, whereas each network’s parameter effects decompose into independent sub-Jacobians.
- The experience of duration: RNN Jacobians are lower-triangular rather than diagonal, so past inputs influence present outputs and experience endures through time.The paper attributes duration to temporal mixing, not recurrent connectivity specifically; transformers can prolong experience similarly.
- The experience of duration: Cohesive rank peaks between the near-diagonal and dense-triangle extremes because increasing temporal mixing raises cohesion while reducing transparency.Different decay timescales weave a denser knot than a single shared timescale, and increasing their spread increases cohesive rank.
- Vivid and obscure ideas: Sigmoid saturation breaks the clean gradient-based relation between output magnitude and vividness, while deep randomly initialized ReLU networks can produce incoherent, noise-like experiences.In the ReLU example, gradient correlations decrease exponentially with depth and spatial structure is annihilated by depth ≥20 layers.
- Vivid and obscure ideas: Attention makes attended content vivid while saturated key-query decisions fade into obscurity, because concentrated attention blocks gradients through the selection path.Gradients continue through the value-output path even when the key-query path saturates.
- Texture and distinct ideas: Convolutional structure supports textured experience across spatial scales, while stable task-specific circuits support coherent simultaneous ideas learned through practice.The paper presents texture as neighboring points interwoven by convolutional kernels and links learned circuit separation to abilities such as swimming while thinking.
5 Learning in Gradland
The paper treats learning as a physical interaction whose Jacobians characterize a higher-order experience: neural nets accumulate objective-specific experience into their weights. Rich, cohesive experience broadens responses to varied environments, but present-day neural nets remain fragmented and objective-dependent.
- Learning as experience: Learning modifies the structure of experience and is presented as the paper’s first example of a higher-order experience.
- Learning as experience: In Gradland, stochastic gradient descent repeatedly computes a loss gradient on a random sample and makes a small opposite-direction weight update.
- Learning as experience: The experience of learning is analyzed through Jacobians of weight updates and includes second-order terms from differentiating the objective.
- Learning as experience: Neural nets accumulate particular, objective-dependent facets of experience into their weights, making their knowledge empirical.
- The function of rich experience: Rich experience has an energetic cost, while effective rank measures response capacity and cohesion measures combinatorial coverage across impressions and reactions.
- The function of rich experience: Lower effective rank narrows possible impressions and reactions, while lower cohesion increases blind spots; rich, dense experience supports continual adaptation but is not sufficient without learning.
- Monadology and sociology: Current neural nets can have disjoint forward-pass experiences, separate backward-pass organization, and fragmented experience because of batching and architecture.
- Monadology and sociology: Neural-net harmony depends on the training loss: random-data training can fit perfectly while leaving shared goals unclear, and different prompts can engage different circuits.
6 Discussion
The discussion presents differential functionalism as a Jacobian-based bridge between physical interactions and phenomenal experience. It treats Gradland as a minimal exemplar whose worked examples are suggestive but not yet a physical or consciousness theory.
- Differential functionalism relates phenomenal experience to the composition of physical interactions, yielding a Mach-inspired form of monism.
- The approach treats phenomenal experience and physical interactions as intrinsic and extrinsic views of reality, while noting that substrates supporting interesting experiences are rare.
- Jacobians are compositional transformations whose chain-rule structure produces an endlessly rich, almost fractal organization.
- Differential functionalism differs from computational functionalism by taking first-order structure, rather than Turing-machine equivalence, as its computational basis.
- Gradland’s worked examples connect Jacobians with duration, vividness, confusion, learning, and other aspects of experience.
- The framework remains a minimal exemplar: its adequacy for realistic brain models is open, and calculus and linear algebra alone do not constitute a theory of consciousness.
A.1 Singular value decomposition
Singular value decomposition expresses a matrix as orthogonal rotations and diagonal rescaling. Its singular values and singular vectors provide the components used to characterize the matrix’s structure.
- An n × d matrix A is factorized by singular value decomposition into orthogonal matrices U and V and a diagonal rectangular matrix Σ.
- The diagonal entries σ_i of Σ are the singular values, ordered from largest to smallest, and matrix rank counts the nonzero singular values.
- Geometrically, SVD represents every matrix as a rotation, rescaling, and another rotation.
A.2 Effective rank
Effective rank softens ordinary rank by accounting for how evenly a matrix’s nonzero singular values are distributed. It ranges from one to rank and falls as squared-singular-value unevenness increases.
- Effective rank, or participation ratio, is defined from a matrix’s nonzero singular values and lies between 1 and the matrix rank.
- Related work uses participation ratios to measure neural-representation dimension and reports maxima at synaptic-connectivity degrees observed in specific biological systems.
- Effective rank can also be interpreted through traces, inverse collision probability, and the spread of a normalized squared-singular-value distribution.
- Effective rank equals rank when the nonzero singular values are equal, while unequal squared singular values reduce it.
- Entropic rank is another softened rank measure, maximized when singular values are equal and approximately one when a single singular value dominates.
A.3 Cohesiveness and Kirchhoff complexity
The section defines cohesion through graph structure and Kirchhoff complexity, using spanning trees and Laplacian properties to characterize whether interactions form a connected whole. It also extends the construction to bipartite graphs and identifies a scale-related limitation of cohesion.
- Graph representation: For a weighted graph, edges encode nonnegative pairwise interactions, while the graph Laplacian is symmetric and has at least one zero eigenvalue.The zero eigenvalue follows from L(G) · 1 = 0.
- Kirchhoff complexity: Kirchhoff complexity is normalized from the spanning-tree expression in Kirchhoff’s matrix tree theorem.The theorem relates a Laplacian principal-minor determinant to the sum of products of edge weights over all spanning trees.
- Cohesiveness: Kirchhoff complexity is zero exactly when the graph is disconnected.Disconnection means the nodes split into subsets with zero-weight edges between them.
- Bipartite graphs: A matrix A can be represented as a bipartite graph whose two node blocks have sizes n and d and whose edges run only between blocks.The associated symmetric Laplacian uses diagonal matrices chosen so both row and column sums vanish.
- Bipartite graphs: Bipartite cohesion requires normalization based on n+d nodes, n+d−1 spanning-tree edges, and the spanning-tree count for K_n,d.These constants differ from those for a complete graph because the bipartite graph has a different structure.
- Scale limitation: For large matrices, the root normalization can pull cohesion values toward 1 and weaken detection of approximate bottlenecks.A logarithmic, root-free alternative is proposed as a more robust measure at scale.
A.4 The experience of symbol manipulation
The section uses induction heads to examine the experience of symbol manipulation through gradients. It proposes tracing how training on increasingly abstract patterns may change those gradients and the associated experience.
- Induction heads: Induction heads respond to a second occurrence of A by producing B when AB appeared earlier in the prompt.This can occur even when the AB pattern was absent from training data.
- Gradient-based analysis: The proposed exercise constructs an induction head with a two-layer transformer and computes gradients during AB, A, B prediction.The exercise then considers training on abstract pairs such as capital-country relations rather than symbol pairs.
A.5 Theories of consciousness
Differential functionalism addresses experience at a more fundamental level than theories focused on when processing becomes conscious, while remaining distinct from a complete theory of human consciousness. The section relates it to global workspace, recurrent processing, higher-order, integrated-information, and plasticity approaches.
- Scope: Differential functionalism is presented as relevant to consciousness research but explicitly not as a theory of consciousness.A theory of human consciousness would require understanding human brain function.
- Major theories: Global workspace theory links consciousness to neural states entering a workspace that integrates processing across specialized modules.The theory concerns when neural processing is conscious.
- Major theories: Recurrent processing theory holds that conscious visual perception requires recurrent processing and that feedforward processing is insufficient.Its proposed criterion is recurrent processing rather than feedforward activity alone.
- Major theories: Higher-order theories relate consciousness to first-order states being monitored by higher-order states, while radical plasticity emphasizes learning.Radical plasticity is characterized as a form of higher-order theory.
- Theoretical contrast: Differential functionalism and integrated information theory address experience more fundamentally than theories centered on criteria for conscious cortical processing.They are grouped together as approaches in a distinct theoretical camp.
- Possible connections: The paper suggests that a future theory building on differential functionalism would likely connect to global workspace, recurrent processing, and higher-order theories.It specifically associates dense global-workspace experience with recurrent processing and treats learning as a higher-order experience.
A.6 Integrated information
The section positions differential functionalism as a Jacobian-based alternative motivated partly by dissatisfaction with integrated information’s technical and conceptual features. It introduces effective rank and cohesive rank, relates them to existing concepts, and presents Jacobian geometry as a computationally cleaner framework while acknowledging limited scope.
- Motivation: Differential functionalism grew from dissatisfaction with some technical and conceptual features of integrated information.The section therefore frames the approach partly as a response to limitations in that framework.
- Effective rank: Effective rank is analogous to effective information and measures the range of differences that make a difference.It is described as exponentially easier to compute at scale than effective information.
- Cohesive rank: Cohesive rank is analogous to integrated information and captures whether a system decomposes into non-interacting subsystems.It is described as doubly exponentially easier to compute at scale because integrated information requires computation over all partitions.
- Jacobian geometry: Jacobian geometry replaces qualia-space geometry with a framework based on linear transforms that is easier to compute and work with.The paper emphasizes that Jacobians support analysis across layers, parameters, activations, and mixtures.
- Toy-system results: Cohesive rank reproduces integrated-information results for toy systems, including low values for modular systems with approximately block-diagonal Jacobians.The section also notes analogous low cohesion for systems with saturated activity.
- Conceptual connection: The Jacobian formulation reimagines information integration in the language of gradients and matrices.The paper presents this as a step toward connecting interaction structure with experience.
A.7 Open problems
The framework leaves open how to choose measures, units, scales, boundaries, and derivative orders, while identifying empirical tests and extensions. Its claims remain conditional on determinate specifications and on gradients adequately characterizing experience.
- Measure design: Effective rank and cohesion depend on arbitrary choices, including cohesion normalization; alternative measures may better capture bottlenecks or blind spots.The paper notes that Fiedler eigenvalues or Eq. (8) may be better starting points for those purposes.
- Derivative order: Gradients may not be the whole story: zeroth- and second-order terms, finite differences, symbolic differentiation, or information-theoretic approaches could alter the picture.The vividness results assume gradients alone characterize interactions, and higher-order terms may matter in learning and biological systems.
- Units: Units pose an unresolved problem because singular-value-based measures, luminosity, effective rank, and cohesion do not account for heterogeneous Jacobian units.The paper suggests that units might themselves be relevant to qualia, while requiring any selection of relevant interactions to remain compatible with matrix composition.
- Extensions: The framework should be extended to probabilistic and continuous-time systems, while biological neurons may prevent clean separation of parameters from other changing structural features.The paper also speculates about basis changes and commutativity as possible directions for future work.
- Specification: The relevant spatiotemporal scale, perturbations, and system boundaries remain empirical and difficult to determine, because answers should map to lived experience.The paper specifically leaves open whether nodes should be atoms, cells, or another level, and how systems such as brains should be bounded.
- Falsifiability: A key empirical test is whether cohesive rank drops under anaesthesia and correlates with existing measures across sleep stages and seizures.These states are presented as cases where experience is diminished or abolished, enabling comparison with the perturbational complexity index.