PaperlayerView exact revision on arXiv

Source-linked AI summary

Exact ReLU realization of binary affine refinement iterates via reflection folding and cone switching

Boldsaikhan Bolorkhuu, Tsogtgerel Gantumur

arXiv:2609.11962v1math.OCcs.LG

TL;DR

Binary affine refinement iterates require exact ReLU realizations without separating forcing profiles from binary cell seams. The paper uses reflection doubling, fixed cone switching, and residual-memory replay to obtain exact fixed-width, depth-O(n) realizations, with extensions to stage-dependent forcing and reflection-equivariant parity reduction.

  • Problem

    For binary affine refinement, the earlier offset-frame approach requires the forcing profile to be separated from ordinary cell seams, unlike the M-ary setting with M≥3.

  • Method

    Pair each residual profile with its reflection, replace the two branch matrices by one fixed block matrix and swap involution, and use a CPwL cone switch with residual memory for reverse-order replay.

  • Results

    The paper proves exact fixed-width, depth-O(n) ReLU realizations for finite binary vector-valued affine refinement iterates, with weights and biases bounded by C2Λn.

  • Takeaways & Limitations

    The construction removes seam separation, supports stage-dependent forcing from a fixed finite-dimensional family, and reduces the doubled cascade under genuine reflection equivariance.

  • Takeaways & Limitations

    The theorem is exact in real arithmetic and makes no claim about finite-precision stability or bit complexity.

Abstract

from arXiv · show

We study vector-valued binary affine refinement operators with finitely supported matrix masks and compactly supported continuous piecewise linear input and forcing data. We prove that every finite refinement iterate admits an exact ReLU realization of fixed width and depth linear in the number of iterations. No separation of the forcing profile from the binary cell seams is required. The main mechanism is universal reflection doubling. Pairing each residual profile with its reflection replaces the two binary transition matrices by one fixed block matrix together with a fixed swap involution. The cell-seam identity makes the two branch candidates agree at the tent fold, while their swap-odd component is bounded linearly by the distance to the fold. This permits exact branch selection by a fixed continuous piecewise linear cone switch, without multiplication by a variable selector. The resulting primal recursion requires the residual orbit in reverse order. We obtain exact backward replay from the residual memory controller developed previously for affine refinement, interpreted here through the reflection quotient of circle doubling. The construction propagates the full vectorized profiles rather than decomposing the input and forcing into reference atoms. We also treat stage-dependent forcing from a fixed finite-dimensional family and show that genuine reflection equivariance reduces the doubled cascade to a single parity sector.

1 Introduction

The paper establishes exact fixed-width ReLU realizations for finite vector-valued binary affine refinement iterates, removing the forcing seam-separation restriction through reflection doubling and cone switching. A residual memory controller supplies reverse-order evaluation, while the construction also supports stage-dependent forcing and parity reduction under reflection symmetry.

  • 1.1 Background and motivation: The binary seam-separation restriction is removed by doubling each residual profile with its reflection and encoding both branches through one fixed block matrix and a swap involution.The cell-seam identity makes branch candidates agree at the tent fold, while Lipschitz control bounds the swap-odd component there.
  • 1.2 Main results: The architecture propagates complete vectorized input and forcing profiles, accommodates stage-dependent forcing from a fixed finite-dimensional family, and avoids atomic decomposition.A compatible reflection symmetry further reduces the universal doubled cascade to a single parity sector.
  • 1.2 Main results: Exact ReLU realizations use fixed width and depth linear in the refinement iterate count for compactly supported CPwL vector-valued binary affine data.The main theorem assumes a preserved support window and gives constants independent of n; weights and biases are bounded exponentially in n.
  • 1.3 Contribution and relation to earlier constructions: A fixed CPwL cone switch selects the identity or swap exactly on the cascade-generated cone without multiplying the evolving state by a variable selector.The swap-odd component is controlled by distance to the tent fold, enabling continuous exact switching.
  • 1.3 Contribution and relation to earlier constructions: Because the folded recursion is primal and evaluated in reverse residual order, a polygonal-loop residual memory controller provides exact backward replay through a two-sheeted lift of tent dynamics.Reflection doubling resolves branch ambiguity, whereas residual memory supplies the required reverse chronology.

2 Binary vectorization and reflection folding

The paper vectorizes compactly supported continuous data into finite-dimensional CPwL profiles, derives binary branch updates with seam compatibility, and folds each profile together with its reflection.

  • 2.1 Binary vectorization: Finite support and a preserved support window reduce the refinement data to a finite vector space of block coordinates.The vectorization contains all required coordinates because the mask is finitely supported and the support window is preserved.
  • 2.2 Binary cascade identity: Binary refinement acts through two transition matrices selected by the digit of 2x, with a common forcing vector added on both branches.The branch formulas use T0 on the left half and T1 on the right half, with continuous extension at the midpoint.
  • 2.3 Cell-seam compatibility: Every vectorized iterate is CPwL and satisfies a cell-seam identity inherited from continuity of the underlying global function.The identity makes the one-sided branch values agree at the fold.
  • 2.4 Reflection folding: Reflection doubling pairs a profile with its reversal, allowing the reflection-even and reflection-odd cases to be handled in one doubled state.The doubled state carries a fixed swap involution, and the matrix construction is the vector-valued counterpart of the scalar prototype.
  • 2.5 Universal reflection doubling: The doubled recursion replaces branch-dependent transition selection with one fixed block matrix followed by either the identity or the fixed swap involution.The two branch candidates agree at the fold, enabling a continuous folded update.

3 Exact cone switching at the fold

A fixed CPwL cone switch selects the identity or swap branch exactly: seam agreement makes the switch continuous, while Lipschitz estimates control the required cone scales.

  • 3.1 Involution switching: The cone-switch construction replaces the discontinuous choice between two binary branches by a fixed continuous piecewise-linear switch.After reflection doubling, the switch chooses between the identity and a fixed involution rather than between T0 and T1.
  • 3.1 Scalar cone flip: The scalar cone flip is CPwL, odd in its state variable, and acts as the required sign reversal on the appropriate cones.Its coordinatewise extension preserves the swap-odd subspace.
  • 3.2 Involution cone switch: For any finite-dimensional involution, diagonalizing into even and odd subspaces yields a globally CPwL involution cone switch with an exact fixed-width, fixed-depth ReLU realization.The realization weights are bounded by CS(1 + K), with CS depending only on the fixed change of coordinates.
  • 3.3 Fold-cone estimate: The cell-seam identity forces the swap-odd component to vanish at the fold and therefore to grow at most linearly with distance from it.This produces the fold-cone estimate used to choose a valid switching scale.
  • 3.4 Quantitative bounds: The iterate Lipschitz constants grow at most exponentially, and the corresponding cone scales satisfy Km ≤ CKΛm.The constants depend only on the fixed mask, initial profile, and forcing term.

4 Folded replay by the residual memory controller

The residual memory controller provides exact reverse replay of the folded tent-map orbit by lifting circle doubling to an injective CPwL skew product with a recoverable memory coordinate.

  • 4.1 Folded tent dynamics: The binary residual memory controller is interpreted through the reflection quotient of circle doubling, whose global affine readout recovers the tent coordinate.This folded interpretation supplies the chronology needed by the primal recursion.
  • 4.2 Injective memory lift: An injective skew product on a polygonal loop stores the information lost under circle doubling while retaining the folded residual in the loop component.Antipodal separation of inverse branches makes the injective lift possible.
  • 4.2 Injective memory lift: The loop map and memory update are CPwL, and the inverse on the image extends to a global CPwL map.Injectivity ensures compatible affine inverse formulas on the finite polyhedral complex.
  • 4.3 Exact backward folded replay: Iterating the inverse memory map recovers earlier controller states, enabling exact backward replay of the residual orbit.The folded readout ignores the memory coordinate and directly extracts the tent coordinate from the loop.

5 Unit-interval realization of the affine cascade

Combining reflection folding, cone switching, and residual memory yields an exact unit-interval ReLU realization of the affine cascade with fixed width and depth linear in the iterate count.

  • 5.1 Nested folded recursion: The primal folded recursion evaluates the initial profile at the deepest tent-map iterate and applies affine stages in reverse residual order.For fixed x, the update sequence proceeds through xn−1, xn−2, ..., x0.
  • 5.1 Nested folded recursion: Backward folded evaluation recovers the exact cascade states by alternating inverse memory updates, folded residual readouts, affine updates, and cone switches.The backward induction establishes the recursive identity for every intermediate stage.
  • 5.2 Unit-interval realization: The unit-interval realization theorem applies to compactly supported CPwL input and forcing in a preserved support window, with constants independent of n.The construction propagates complete vectorized profiles rather than decomposing them into reference atoms.
  • 5.2 Network assembly: The network uses one forward memory pass followed by n fixed-complexity backward blocks, so width is independent of n and depth is O(n).Stage-indexed cone scales are fixed numerical parameters rather than network variables.
  • 5.2 Quantitative network bounds: Realizing-network weights and biases can be bounded by C2Λn because cone-switch scales grow at most exponentially while the remaining modules are fixed.Exactness follows from the folded update and backward replay results.

6 Global realization and stage-dependent forcing

The section globalizes the exact unit-interval realization through clamped gluing and extends it to stage-dependent forcing from a fixed finite-dimensional family, while retaining fixed width and depth O(n).

  • Global realization: The global theorem combines exact unit-interval realization with clamped gluing to realize compactly supported binary affine iterates.Continuity at cell seams and endpoint conditions make the clamped sum reproduce each unit-cell restriction and vanish outside the support window.
  • Global realization: The exact global realization has fixed width, depth O(n), and weights and biases bounded by C2Λn, with constants depending only on fixed refinement and CPwL data.Globalization evaluates finitely many shifted and clamped copies in parallel, changing width and coefficient bounds only by fixed factors.
  • Affine forcing: No symmetry or seam-separation hypothesis is imposed on the forcing, which is propagated as a complete reflected vectorization in the folded affine recursion.The cone-switch compatibility is supplied by the cell-seam identity preserved by the affine recursion, rather than by decomposing the forcing into translated atoms.
  • Stage-dependent forcing: Stage-dependent forcing from a fixed finite-dimensional family still admits an exact fixed-width, depth-O(n) ReLU realization.The stage profiles are formed from finitely many fixed templates, so each backward block retains fixed width and depth while the residual controller supplies reverse-order replay.
  • Stage-dependent forcing: For stage-dependent forcing, weights and biases can be bounded by C(1 + Mn)Λn, with constants determined only by the mask, support window, initial profile, and forcing templates.The additional factor reflects the bounded template combinations used at each stage.

7 Reflection-equivariant reduction

Reflection-equivariant refinement preserves parity, so the universal doubled cascade can be restricted to an invariant parity sector and realized with half the cascade-fiber dimension.

  • Parity reduction: Reflection equivariance makes the doubled cascade lie in an invariant subspace of half the dimension when the initial profile and forcing share reflection parity.The reduced realization uses a pL-dimensional cascade fiber instead of the universal 2pL-dimensional doubled fiber.
  • Parity reduction: The reduction follows from commutation of the refinement operator with reflection and the transition relation T1J = JT0, which preserve the twisted diagonal parity sector.The folded update leaves the parity subspace invariant after identifying it with the underlying vectorization space.
  • Reduced recursion: The cone-switch estimate remains available on the reduced space because the odd component vanishes at the tent fold and is controlled by its distance from that fold.The reduced recursion therefore uses the same residual memory controller and clamped-gluing procedure as the doubled construction.
  • Lévy–C example: For the Lévy–C dragon mask, reflection equivariance identifies the fiber involution with the similarity involution C and reduces propagation from a four-dimensional doubled fiber to a two-dimensional fiber.The reduction applies because A1 = CA0C and the mask preserves the unit support window.
  • Lévy–C example: The endpoint-extended Lévy–C approximants are recovered in the η = −1 parity sector after adding the fixed anchor profile.The compactly supported anchor defects have negative reflection parity, which is preserved by the affine recursion.

8 Conclusions

The paper proves exact fixed-width, depth-O(n) ReLU realizations for binary vector-valued affine refinement, removes seam separation, supports fixed-span stage-dependent forcing, and identifies parity and arity extensions.

  • Main conclusions: The main result is an exact fixed-width, depth-O(n) ReLU realization theorem for binary vector-valued affine refinement with compactly supported CPwL data.The construction applies to both initial profiles and forcing terms.
  • Main conclusions: Universal reflection doubling removes the earlier seam-separation requirement by pairing each residual profile with its reflection and using cone switching for exact branch selection.The cell-seam identity makes the swap-odd component vanish at the fold, enabling a fixed CPwL switch without a variable selector multiplier.
  • Extensions: The primal folded recursion uses residual memory for reverse-order replay, while homogeneous problems also admit a forward adjoint realization without memory.Reflection-equivariant masks further reduce the doubled cascade to a parity sector.
  • Extensions: The folded-memory architecture extends to every fixed arity through a fixed-arity fan switch, without offset frames or a global symmetry relating all branches.Sharper coefficient bounds, finite-precision stability, and multidimensional affine refinement remain outside the stated extension.

A Adjoint cone switching for homogeneous atoms

The appendix gives a memory-free forward adjoint realization for homogeneous refinement with endpoint-zero unit-interval atoms.

  • For endpoint-zero unit-interval atoms, incorporating the terminal scalar factor into the initial adjoint state allows forward propagation without backward replay or a memory coordinate.

A.1 Adjoint recursion and cone estimate

The adjoint construction encodes reflected orbit states on a polygonal loop and uses cone estimates to select the correct branch exactly at each stage.

  • Reflection doubling: Reflection doubling pairs each profile with its reflected counterpart, allowing branch selection through a fixed swap involution and block-matrix representation.The doubled spaces use a self-adjoint swap involution J, enabling adjoint updates in Euclidean coordinates.
  • Orbit encoding: The loop state is advanced by repeated applications of F, while its projected coordinate follows the tent-map orbit τ^j(x).The construction defines Y_j(x)=F^j(Y_0(x)) and x_j=Π(Y_j(x))=τ^j(x).
  • Adjoint recursion: The adjoint recursion reruns the forward orbit while propagating an adjoint state, then identifies the resulting pairing with the desired cascade.The second pass carries the orbit forward and applies one adjoint update per stage before the final linear pairing.
  • Cone estimate: At the fold x_j=1/2, the terminal factor vanishes, giving Λ_j(x)=0 and supporting exact continuity of the cone switch.The preceding exact updates and projection P− yield the fold estimate used by the cone-switch argument.

A.2 Exact realization and globalization

The unit-interval adjoint recursion has an exact fixed-width ReLU realization with linear depth, and nodal-hat decompositions extend it to general data and fixed higher arity.

  • Architecture: A two-pass architecture computes the residual controller state first, then replays the orbit while applying adjoint updates, so every repeated module has fixed complexity.The first pass has length n+1; the second pass performs one adjoint update at each stage followed by linear pairing.
  • Globalization: General compactly supported vector-valued CPwL seeds are obtained by finite nodal-hat atom sums, translations, parallelization, and clamped gluing.This globalization transfers the unit-interval construction without requiring a new atom-level decomposition of each refinement stage.
  • Fixed-arity extension: For fixed arity M, reflection doubling converts branch dependence into shift–reflection candidates whose fold agreement enables an exact finite fan switch.The fan switch has fixed width and depth because M is fixed, while its scale is controlled by the Lipschitz constant.
  • Fixed-arity extension: The fan-switch scale can be chosen as K≥M Lip(G#), and the resulting switch is exact on every M-branch interval.The proof assembles the branch curves continuously across adjacent folds and inducts over the branch index.

B.3 Realization consequence

The folded realization extends to every fixed arity M and yields exact finite refinement iterates with fixed width and depth O(n), using reverse-order residual replay.

  • Residual controller: The reflection quotient of circle doubling yields an injective CPwL skew product whose residual memory construction provides exact backward replay for fixed M.Nontrivial preimages are uniformly separated on the loop when M is fixed.
  • Theorem scope: For every fixed M≥2, the folded realization handles finitely supported matrix masks with CPwL input and forcing data under the stated support-preservation assumptions.The construction permits constants depending on the fixed refinement and CPwL data, but not on n.
  • Realization consequence: The resulting network has fixed width and depth O(n), with weights and biases bounded by C_2Λ^n, and standard clamped gluing recovers the global iterate.The coefficient bound follows from the Lipschitz estimate and the fixed complexity of the modules.
  • Backward replay: The memory controller supplies x_n,…,x_0 in reverse order, enabling the primal recursion U_j=b^T F_{K_{n−1−j}}(x_j,U_{j+1})+b#(x_j).Backward replay reconstructs the needed residual orbit order for the primal update.
  • Adjoint consequence: The homogeneous adjoint form replaces the binary cone switch by an M-branch fan switch, while endpoint-zero atoms retain vanishing terminal factors at zigzag folds.The same Lipschitz estimate supplies the adjacent cone bounds in the fixed-arity construction.
Loading 2609.11962v1…