PaperlayerView exact revision on arXiv

Source-linked AI summary

A Unified Framework for the Mechanics of Information in Convolutional Neural Network Image Space

Aryan Shukla, Matthew Toews

arXiv:2608.26363v1cs.CV

TL;DR

The paper asks how information transforms through CNN filtering and develops a framework linking discrete filter symmetry and rectified convolution to relativistic energy–momentum analogies. It reports Lorentz-like image displacement, Gaussian scale-space, and scale-invariant Morse critical points across physical image domains, while noting several scope limitations.

  • Problem

    The paper addresses the need to understand the mechanical rules governing information transformation between neural-network layers despite the black-box nature of neural networks.

  • Method

    The paper models CNN filtering with real-valued linear filters, nonlinear rectification, and scalar image data, decomposing filters into symmetric and antisymmetric components.

  • Results

    The demonstrations show Lorentz-like image displacement and scale-invariant Morse critical points across phenomena ranging from atoms and molecules to brains, galaxies, and the CMB.

  • Takeaways & Limitations

    Repeated filtering supplies Gaussian scale space and a topological description in which critical points identify spherical density concentrations and bubble-like density depletions.

  • Takeaways & Limitations

    The Lorentz-transform correspondence is close but not exact at intermediate β, and the maximum propagation displacement result applies only to discrete 2 × 2 and 3 × 3 filters.

Abstract

from arXiv · show

This paper introduces a unified mathematical framework for modeling information propagation through convolutional neural networks (CNNs), with the aim of connecting descriptions of physical space and information space. A correspondence is presented linking discrete filter symmetry and the relativistic energy--momentum relation under the widely used nonlinear rectified convolution operation. Specifically, symmetric filter components (e.g. the sum $Σ= [1,1]$) operate analogously to rest energy $mc^2$ in preserving the image centre of mass (e.g. isotropic diffusion), whereas antisymmetric components (e.g. the gradient $\nabla = [-1,1]$) operate analogously to the momentum term $pc$ in generally inducing a displacement (e.g. vibration or translation). For typical small discrete filters, this displacement is determined by the ratio of antisymmetric to total filter energy, analogously to how the displacement of a relativistic particle relates to a Lorentz transform with beta parameter $β= \frac{v}{c}=\frac{pc}{E}$ equal to the ratio of momentum $pc$ to total energy $E$. Repeated filtering leads to the Gaussian scale-space and emergent scale-invariant features. These constructions share a Laplacian-driven structure with the classical heat (diffusion) equation and, via standard mathematical correspondences, with the Schrödinger equation and aspects of the Friedmann equations, together with emergent Morse topological structure. Demonstrations in 3D images reveal blob-like, scale-invariant Morse critical points in images spanning a wide range of physical scales, including organic sugar molecules and inorganic silicon crystals, human and primate brains in magnetic resonance images (MRI), galaxies and the cosmic microwave background (CMB).

D. Derivation of the Lorentz Transformation

The section develops rectified convolution in 1D as diffusion, vibration, or translation, then relates mixed gradient paths to bounded, Lorentz-like displacement. It extends the construction toward higher-dimensional filters and emergent phenomena.

  • Generalization: The framework generalizes primary sum and difference operators from 1D to 2D and 3D image space and to larger filters using symmetry and superposition.The paper motivates this extension as a basis for demonstrating emergent phenomena.
  • Diffusion mode: Repeated convolution with the symmetric sum operator produces a binomial pyramid that approaches a Gaussian density with variance σ^2 proportional to iteration count.Because the impulse and sum filter are non-negative, rectification does not alter this pyramid.
  • Propagation modes: Rectified convolution of an impulse with a difference operator shifts information left or right according to the operator orientation.Successive operations form paths of left or right steps.
  • Propagation modes: Equal left and right gradient operations produce zero net displacement, corresponding to a vibration mode around the origin.For equiprobable orientations, the most probable path has equal counts of left and right shifts.
  • Propagation modes: A sustained single gradient orientation produces one-directional drift and a pure translation mode, although such paths are least probable in mixed-orientation networks.The signal shifts rightward when the same difference orientation is repeatedly applied.
  • Propagation modes: Mixed-orientation paths typically form small diffusive random walks, while filter width bounds the maximum displacement per layer.The maximum displacement is described as (Width −1)/2 pixels per layer, with 0.5 pixels for a 2-pixel gradient.

III. ELEMENTARY OPERATORS

This section decomposes image-space operators into symmetric and antisymmetric components and extends symmetry from left–right functions to radial and spherical image structure. These components are treated as orthogonal, with radial symmetry determined by distance from a centre.

  • Symmetry and antisymmetry: In 1D, symmetric and antisymmetric functions correspond respectively to even and odd functions under left–right inversion.The section uses these terms synonymously.
  • Symmetry and antisymmetry: A function can be uniquely decomposed into symmetric and antisymmetric components that are orthogonal under the relevant inner product.The squared norm therefore obeys a Pythagorean identity.
  • Radial symmetry: Radial symmetry in n-dimensional space means that a function’s value depends only on distance from a fixed centre, not direction.Points at the same Euclidean distance have equal function values.
  • Radial symmetry: Subtracting a shell-averaged radial component yields a remainder with zero mean on each radial shell and orthogonality to the radial component.This construction generalizes the odd-function remainder used in one dimension.

B. Radial Symmetry in Images

The paper defines radial symmetry for 3D images through distance-based voxel shells and averaging, then situates these operators within a broader CNN filtering framework using DCT components.

  • Radial symmetry: A 3D scalar image is represented as a cubic volume indexed over discrete voxel coordinates.
  • Radial symmetry: Voxels at the same radial distance are grouped into shells, whose intensities are averaged to define the radially symmetric component.
  • Radial symmetry: In 2D and 3D, radial symmetry produces circular or spherical patterns whose intensities vary with distance but not direction.
  • Radial symmetry: The antisymmetric component is defined as the shell-demeaned remainder, separating directional structure from the radial average.
  • Operators in image space: The framework generalizes sum and gradient operators to larger filters and multidimensional image spaces using symmetry, including DCT-related components.
  • Operators in image space: Under rectified convolution, symmetric filtering preserves image centre of mass through isotropic diffusion, whereas antisymmetric filtering generally induces directional displacement.

1. Lorentz-like Motion from Rectified Convolution

Rectified convolution produces centre-of-mass motion that closely follows a Lorentz-transform curve as filters mix symmetric sum and antisymmetric gradient components. Repeated filtering yields diffusion, vibration, or translation depending on gradient direction and mixing.

  • Lorentz-like displacement: 0 pixels displacement occurs for pure diffusion at β2 = 0, while pure translation reaches 10 pixels after ten iterations at β2 = 1.These are the stationary and maximal centre-of-mass outcomes reported for the rectified impulse experiment.
  • Propagation modes: A non-zero gradient component produces motion, with symmetric sum filtering causing stationary, isotropic diffusion.The gradient component determines whether propagation becomes vibration or translation, whereas pure sum preserves the centre of mass.
  • Propagation modes: Alternating gradient directions produce vibration about µx = 0, while a single gradient direction produces translation.For β2 ≤ 0.5, the pattern vibrates and contracts horizontally; for β2 = 1, a narrow wave front propagates.
  • Propagation modes: A single-pixel wave front travels up to 100 pixels when β2 = 1 under repeated rectified convolution.The pattern bulk disappears, leaving a front that moves from the circle’s right contour.
  • Propagation modes: The maximum per-iteration displacement is set by filter width, reaching one pixel for 3 × 3 filters when β2 = 1.Net displacement depends on the balance between right- and left-directed gradient operations.

Governing PDE in Physics and Images (3D)

The paper formulates 3D image scale-space as diffusion driven by the Laplacian, with image scale σ related to physical diffusion time by t = σ2. Gaussian solutions and differential operators then support scale-invariant keypoints and Morse topology.

  • Diffusion equations: The 3D physical heat equation and image scale-space PDE share a Laplacian-driven diffusion structure.In image space, the evolving image is conditioned by an initial image and uses an isotropic diffusion constant D > 0.
  • Diffusion equations: Image diffusion is parameterized by scale σ rather than physical time, with t = σ2 linking the two descriptions.This transformation makes the correspondence between physical diffusion and image-scale evolution explicit.
  • Gaussian scale-space: The diffusion equation’s fundamental solution is convolution with an isotropic 3D Gaussian kernel whose standard deviation σ sets image scale.The solution is written as I(x, y, z, σ) = G3(x, y, z; σ) ∗ I0(x, y, z).
  • Gaussian scale-space: For small k − 1, Difference-of-Gaussian filtering approximates the Laplacian, and its extrema identify 3D scale-invariant keypoints.Geometric scale sampling uses σt = kσt−1, with k determined by the number of samples per octave.
  • Morse topology: Morse critical points have vanishing LoG gradient and nondegenerate Hessians, with all-positive or all-negative eigenvalues forming bubbles or droplets.Their topological charge is determined by the determinant’s sign, positive for index 0 and negative for index 3.
  • Morse topology: Fine-to-coarse Gaussian blurring merges many small bubbles into fewer larger structures, with neighbouring structures separated by saddle points.The construction is described as an image-heating process with progressively coarser topology.

Bridging Physics, Diffusion, and SIFT in 3D

The paper frames 3D SIFT as Gaussian diffusion whose Laplacian-based extrema yield scale-invariant bubbles and droplets across molecular, biological, galactic, and cosmological images. These constructions are mathematically analogous to heat, Schrödinger, and Friedmann scale evolution, not physically identical.

  • Diffusion and SIFT: 3D Gaussian scale-space, its Laplacian, and SIFT keypoint detectors form one coherent diffusion framework.Gaussian smoothing is the scale-space PDE solution; DoG approximates the Laplacian, while Hessian structure supports blob detection.
  • Physical-scale demonstrations: In primate brain MRI, positive features tend to coincide with white matter and sulci, whereas negative features tend to coincide with ventricles and extracerebral spaces.Certain brain regions exhibit feature signatures that may support more efficient indexing and matching.
  • Physical-scale demonstrations: Galactic density projections show positive features at concentrations and negative features in voids, while CMB maps and molecular images extend the same feature analysis across scales.The reported galactic features do not explain galactic rotation-curve anomalies.
  • Diffusion and SIFT: SIFT extrema across spatial coordinates and scale identify bubble-like and droplet-like density concentrations or depletions at characteristic, resolution-invariant scales.The features emerge by evolving images through Gaussian scale-space and detecting DoG peaks and valleys.
  • Physics analogies: The Schrödinger and Friedmann connections are mathematical or conceptual analogies based on shared Laplacian or scale-factor structure, not claims of physical identity.The scale-space Hubble-like correspondence is illustrative rather than dimensional because its quantities have different dimensions.

V. DISCUSSION

The discussion presents CNN filtering as an information-mechanical analogy linking rectified convolution, relativistic energy–momentum, quantum state evolution, and scale-space topology. It also identifies practical indexing possibilities and explicit scope limitations.

  • Framework: CNN filtering is modeled with real-valued linear filters, nonlinear rectification, and scalar images to study how information transforms between layers.The framework is motivated by the black-box nature of neural networks and demonstrated computationally through Lorentz-like image behavior.
  • Framework: Symmetric filter components preserve image centre of mass through isotropic transformations, while antisymmetric components produce directional momentum-like displacement.The correspondence is presented as an analogy to rest mass and momentum in relativistic energy.
  • Framework: The maximum information displacement per layer is bounded by discrete filter width, analogously to a relativistic speed limit.The stated bound is dx/dt ≤ (Width −1)/2 pixels per layer.
  • Scale-space consequences: Gaussian scale-space and Morse critical points provide a common description of structure from atom-scale electron density through MRI and galaxy-scale images.The paper positions these descriptors as invariant to scale, translation, and rotation.
  • Scale-space consequences: Scale-invariant descriptors may support indexing similar MRI, biological, and molecular structures and potentially inform architectures beyond local CNN correlations.The paper specifically mentions memory-based learning and possible transformer applications.
  • Limitations: The demonstrated Lorentz-like motion is approximate, the analysis focuses on primary sum and gradient modes, and maximum displacement occurs only for discrete 2 × 2 and 3 × 3 filters.The paper also notes that square filters coarsely approximate ideal round or spherical filters.

APPENDIX A: 3 × 3 Image Decomposition into Radially Symmetric and Antisymmetric Components

Appendix A decomposes 3 × 3 filters into radially symmetric and antisymmetric components and verifies their orthogonality. It situates this construction within the paper’s broader symmetry-based image-space framework.

  • Filter decomposition: The 3 × 3 filter is decomposed into a radially symmetric component and an antisymmetric shell-demeaned remainder.The decomposition is centered at (1, 1) and separates radial structure from antisymmetric content.
  • Orthogonality check: The component inner product is evaluated by separating corner, edge, and centre contributions.The appendix shows cancellation of the corner and edge terms before evaluating the centre contribution.
  • Orthogonality check: The corner and edge terms cancel, while the centre contributes zero, confirming orthogonality between the symmetric and antisymmetric components.The appendix states ⟨Is, Ia⟩ = 0.

Spatial Eigenvalue Problems

The appendix derives diffusion solutions through separated spatial eigenvalue problems and connects the resulting Gaussian scale-space to SIFT. On unbounded image domains, Gaussian smoothing corresponds exactly to diffusion at time t = σ².

  • Eigenvalue solution: The rectangular-domain solution is formed by superposing integer spatial modes and matching the initial condition through coefficients A_mn.The corresponding eigenfunctions and coefficients determine the expansion of the initial field.
  • Gaussian scale-space: On R², the heat-equation solution is convolution with the Gaussian heat kernel, written as u(·, t) = G(·; t) ∗ f.This formulation avoids boundary effects for moderate σ when images are much larger than the Gaussian kernel’s effective support.
  • Gaussian scale-space: Gaussian smoothing at scale σ is exactly the diffusion solution at time t = σ², with image-space diffusion constant D > 0.This identifies SIFT’s scale parameter with diffusion time.
  • Keypoint detection: SIFT keypoints are extrema of the Difference-of-Gaussians, which approximates the Laplacian up to a constant factor.The Laplacian link connects keypoint detection to the diffusion operator.

APPENDIX C: Scale–Space Schr¨odinger Analogy

The appendix maps the Schrödinger equation to image diffusion through a Wick rotation, identifying the evolving wavefunction with a scale-space image. The resulting propagator is the Gaussian blur used in scale-space methods.

  • Schrödinger-to-diffusion mapping: A Wick rotation transforms unitary Schrödinger evolution into real diffusion in image scale-space.The substitution maps the wavefunction’s time evolution to an evolving image parameterized by σ.
  • Diffusion equation: The image satisfies a diffusion PDE, ∂I/∂σ = 2σD∇^2I, with D = ℏ^2/(2m).Here σ plays the role of the square root of diffusion time.
  • Spatial modes: The separated spatial eigenproblem represents k_x and k_y as image-space spatial frequencies describing how rapidly pixel values change.These frequencies characterize variation along the image’s x and y dimensions.
  • Gaussian scale-space: The combined solution is exactly the Gaussian-scale-space blur with variance σ^2.The two factors of σ ensure that t = σ^2 recovers the usual heat-kernel variance.
  • Interpretation: Under t = iσ^2, the propagator becomes the Gaussian kernel used in SIFT and other scale-space methods.This summarizes the correspondence between Ψ(x,y,t), the image I(x,y,σ), and Gaussian diffusion.

APPENDIX D: Derivation of the Lorentz Transformation

The appendix derives the Lorentz transformation from linear frame changes, invariant light speed, and the mapping of a moving origin’s worldline. It then introduces four-vector and Minkowski-metric notation for the transformation’s relativistic formulation.

  • Assumptions: The derivation assumes a linear transformation and constant light speed in all inertial frames.These are stated as foundational assumptions for obtaining the standard transformation.
  • Lorentz boost: A boost with velocity v uses β = v/c and maps the moving origin’s worldline to the primed frame’s time axis up to scale factor μ.The worldline condition supplies the geometric constraint on the boost.
  • Metric convention: The formulation uses the mostly-minus Minkowski metric convention (+, −, −, −), while noting the equivalent opposite-signature convention.The tensors g_μν and g^μν are treated as inverse metric representations.
  • Four-vector notation: The four-vector χ^μ = (ct, x, y, z) uses μ = 0,1,2,3, with covariant components and Einstein summation notation.The appendix also relates covariant and contravariant vectors through the metric tensor.

APPENDIX F: Dirac’s Equation

The appendix develops Dirac’s first-order relativistic operator by factoring the energy–momentum relation with anticommuting matrices. The resulting operator acts on a four-component wavefunction.

  • Relativistic operator: Dirac’s construction makes energy and momentum first-order differential operators while preserving the relativistic relation P^μP_μ = (mc)^2.The appendix frames this as taking a linear square root of the quadratic relation.
  • Matrix algebra: The factorization requires matrices whose squares equal one and whose distinct products anticommute.The stated conditions include A^2 = B^2 = C^2 = D^2 = 1 and AB + BA = 0 through AD + DA = 0.
  • Gamma matrices: Dirac’s solution uses four 4 × 4 matrices because the required anticommutation prevents the coefficients from being ordinary real or complex numbers.The matrices provide the linear factorization of the relativistic operator.
  • Spacetime components: The gamma matrices associate one component with the time derivative and three skew-diagonal matrices with spatial derivatives in x, y, and z.The identity matrix associated with rest mass forms the diagonal component.
  • Wavefunction: The constructed Dirac operator acts on ψ, the four-vector form of the wavefunction.This applies the operator after replacing the four-vectors with their differential-operator representations.

APPENDIX G: Experimental code and data

The appendix documents data sources, software, and acquisition details for the experimental demonstrations. These materials cover molecular, crystal, medical, simulated, and visual 3D images used for SIFT bubbles and information-propagation visualizations.

  • Software: 3D SIFT feature extraction and diffusion-bubble experiments are supported by publicly listed source code and compiled binaries.Separate repositories provide SIFT extraction, diffusion-bubble generation, and information-propagation results.
  • Brain MRI: Macaque, chimpanzee, and human brain MRIs were acquired at 0.5 mm, 0.6 mm, and 0.7 mm voxel sizes, respectively.The datasets used 3T Siemens scanners and came from the cited macaque, chimpanzee, and Human Connectome resources.
  • Cosmology and visualization: The appendix identifies a galactic simulation source and 3D Slicer as the software used for 3D bubble visualizations.These resources support the cosmological and visualization components of the experiments.
  • Molecules and crystals: Molecular electron-density images use density-functional-theory calculations, PubChem atomic arrangements, and a supplied doped silicon crystal.The cited tools include GPAW and SDF molecular-structure data.
  • Additional image data: The documented image sources also include COPDGene lung CT scans and a Minecraft 3D scene with a corresponding 2D display.These sources broaden the experimental inputs beyond molecular and medical brain images.
  • Molecular demonstration: Figure 13 depicts 3D SIFT bubbles from an H2 molecule.The figure is presented as part of the molecular image demonstrations.
Loading 2608.26363v1…