PaperlayerView exact revision on arXiv

Source-linked AI summary

Sessa: Selective State Space Attention

Liubomyr Horbatko

arXiv:2604.18580v2cs.LGcs.AIcs.CL

TL;DR

Long-context modeling must preserve and selectively retrieve distant information despite attention dilution and recurrent forgetting. Sessa places input-dependent attention inside a recurrent feedback path, producing many routing paths. Under matched assumptions and experiments, it has slower power-law memory decay, uniquely flexible selective retrieval, and the strongest long-context benchmark performance while remaining competitive on short contexts.

  • Problem

    Diffuse attention can dilute individual-token influence, while recurrent state propagation can lose distant information unless it actively preserves it, limiting long-range evidence integration.

  • Method

    Sessa integrates input-dependent attention into a recurrent feedback path so influence is aggregated across multiple routing depths and temporal paths.

  • Results

    Under matched regimes, Sessa achieves a power-law memory tail O(ℓ^-βtail) with 0 < βtail < 1, uniquely realizes flexible selective retrieval, and leads long-context benchmarks.

  • Takeaways & Limitations

    Sessa provides a theoretical and empirical alternative to one-hop attention and single-chain recurrence for preserving and selectively retrieving long-range information.

  • Takeaways & Limitations

    The theoretical comparisons rely on explicit matched-regime assumptions, including diffuse attention and failed-freeze-time Mamba conditions.

Abstract

from arXiv · show

Modern sequence modeling is dominated by two families: Transformers, whose self-attention can access arbitrary elements of the visible sequence, and structured state-space models, which propagate information through an explicit recurrent state. These mechanisms face different limitations on long contexts: when attention is diffuse, the influence of individual tokens is diluted across the effective support, while recurrent state propagation can lose long-range sensitivity unless information is actively preserved. As a result, both mechanisms face challenges in preserving and selectively retrieving information over long contexts. We propose Sessa, a decoder that places attention inside a recurrent feedback path. This creates many attention-based paths through which past tokens can influence future states, rather than relying on a single attention read or a single recurrent chain. We prove that, under explicit assumptions and matched regimes, Sessa admits power-law memory tails $O(\ell^{-β})$ for $0 < β< 1$, with slower decay than in the corresponding Transformer and Mamba-style baselines. We further give an explicit construction that achieves this power-law rate. Under the same assumptions, Sessa is the only model class among those considered that realizes flexible selective retrieval, including profiles whose influence does not decay with distance. Consistent with this theoretical advantage, across matched experiments, Sessa achieves the strongest performance on long-context benchmarks while remaining competitive with Transformer and Mamba-style baselines on short-context language modeling.

1 Introduction

Sessa addresses complementary long-context weaknesses in attention and recurrent state-space models by placing input-dependent attention inside a recurrent feedback path. Under matched regimes, it yields slower memory decay, flexible selective retrieval, and strong long-context empirical performance.

  • Motivation: Diffuse attention dilutes individual token influence across large effective supports, while stable feedback dynamics can exponentially forget distant information.These complementary effects can hinder reliable integration of far-apart evidence under long contexts.
  • Architecture: Sessa injects input-dependent attention into a recurrent feedback path, combining direct-read routing with stateful aggregation.Its feedback operator aggregates contributions over multiple internal routing depths and, in dense patterns, many temporal paths.
  • Theory: Sessa admits a power-law memory tail O(ℓ^-βtail) with 0 < βtail < 1, attenuating more slowly than exponential-forgetting SSM regimes under stated assumptions.The paper identifies this heavy-tail regime in diffuse, low-separation routing and provides an explicit construction achieving the rate.
  • Theory: Deep Sessa realizes flexible selective-retrieval profiles, including non-decaying ones, whereas diffuse fixed-depth Transformers and failed-freeze-time fixed-depth Mamba do not.This is a qualitative separation in realizable retrieval behavior under the matched theoretical regime.
  • Empirics: Under matched architectures and training budgets, Sessa achieves the strongest long-context benchmark performance while remaining competitive on short-context language modeling.The paper also proves universal approximation for a broad class of causal sequence mappings in Appendix I.

2 Background

The paper distinguishes sequence mixers by whether routing is applied once through a direct read or repeatedly composed through feedback. This framework characterizes attention as one-hop routing, recurrent models as multi-hop routing, and Sessa as multi-hop aggregation over many paths.

  • Conceptual framework: The framework separates how context produces routing coefficients from whether information is accessed by a single read or accumulated through feedback.Routing determines information flow, while the system determines whether routing is applied once or repeatedly composed.
  • Direct-read systems: A direct-read system forms each output through one input-addressed read over a visible set, with bounded support defined by a finite lag window.Variable-support systems allow the visible set to vary by position; bounded-support systems impose a fixed maximum support width.
  • Direct-read systems: Kernel representations alone do not distinguish direct-read systems from feedback systems because any finite-horizon causal linear map has a lower-triangular kernel form.The distinction concerns the computation graph and repeated composition inside the layer.
  • Feedback systems: Feedback systems repeatedly compose routing through an internal state or solve, allowing influence to travel through intermediate temporal nodes.This repeated composition is the paper’s definition of multi-hop routing.
  • Sessa routing: Sessa’s feedback solve sums contributions over all length-k paths and hop counts, enabling heavy-tail influence regimes when path counts grow with lag.Figure 1 contrasts Transformer one-hop routing, Mamba’s single chain, and Sessa’s many-path routing.
  • Model classes: Self-attention is a direct-read system over value vectors, whereas structured SSMs use latent-state recurrences with structured temporal dynamics.Full-prefix, windowed, and sparse attention share the direct-read template through their visible sets.
  • Matched comparison: In matched failure regimes, diffuse attention suffers dilution, failed-freeze-time Mamba exhibits exponential attenuation, and Sessa aggregates multiple hop counts for slower long-range decay.Mamba preserves long-range memory only when it can sustain a long preserve corridor with Δr≈0.

3 Model Architecture

Sessa combines forward causal attention with feedback attention inside a recurrent mixer, so its output is computed through multi-hop routing rather than a single attention read. The feedback path also provides an internally generated absolute positional signal.

  • Block design: Sessa uses a gated-MLP-style block containing a recurrent mixer that combines forward causal attention with feedback mixing of past mixer outputs.The mixer is wrapped within a single block rather than alternating separate attention and MLP blocks.
  • Attention mechanisms: The forward branch performs a one-hop causal mixture of values over the visible prefix, while feedback attention assigns weights over the strict past.Forward attention includes j≤t; feedback attention uses j<t.
  • Feedback solve: The feedback operator forms a scalar strictly lower-triangular routing matrix, and the mixer solves (I−B_fb)s=f independently for each feature dimension.The solve is equivalent to forward substitution and can be implemented with triangular-solve routines, while the dense full-prefix formulation remains quadratic in T.
  • Multi-hop routing: Because B_fb is nilpotent on finite horizons, the solve expands into contributions traversing multiple internal routing steps through the feedback operator.This multi-hop routing can produce the heavy-tail influence regimes analyzed later in the paper.
  • Position information: Sessa applies RoPE in the forward path but omits positional encoding in feedback, whose causal recurrence already supplies an absolute time direction.The feedback output depends on iterated aggregation of the strict past and can generate position-dependent signals from time-constant inputs.
  • Position information: A single Sessa block can produce deterministic position-dependent offsets whose separated token-wise ranges allow recovery of the position index.The construction holds for all inputs in a fixed compact domain and uses pairwise-distinct offset coefficients along a shared direction.

4 Theory

The theory establishes Sessa’s stability, long-range memory, selective retrieval, internal positional encoding, and universal approximation. Under matched assumptions, Sessa has slower-decaying memory and more flexible retrieval profiles than the comparison classes.

  • Stability: Sessa’s feedback solve is BIBO-stable under a bounded causal lower-triangular routing condition.If the feedback operator’s row-sum bound is below 1, the causal solve has a unique bounded solution; the tokenwise gate is also BIBO-stable.
  • Long-range memory: Sessa admits a power-law impulse-influence upper bound O(ℓ^-βtail), with 0 < βtail ≤ 1, under Assumptions 6–7.The bound is horizon-uniform in the stated regime and tends to zero with lag in the subcritical case.
  • Long-range memory: Sessa’s polynomial memory tail decays more slowly than the O(1/ℓ) dilution of diffuse attention and exponential forgetting in stable feedback systems and failed-freeze-time Mamba.An explicit uniform-routing construction achieves Θ_τ(ℓ^-βtail), with βtail = 1 − γ for fixed source position τ.
  • Selective retrieval: Deep Sessa realizes retrieval profiles ν_k(β) = k(1 − β) − 1, including decaying, frozen, and increasing profiles.The realizing network may depend on the finite horizon and source position, while the profile constants remain uniform across those parameters.
  • Selective retrieval: Diffuse fixed-depth Transformers and failed-freeze-time fixed-depth Mamba cannot realize frozen or increasing retrieval profiles in the matched finite-horizon family-over-H regime.Their horizon-uniform end-to-end bounds provide only decaying upper bounds, yielding the comparison-class impossibility result.
  • Additional properties: A Sessa feedback solve can generate separated absolute positional signals without an explicit absolute positional embedding in the feedback branch.The theory also proves universal approximation for continuous causal maps on compact domains using concrete Sessa networks with adapters.

5 Experiments

The experiments compare matched Sessa, self-attention, and Mamba2 mixers on long-context reasoning and short-context language modeling tasks. Sessa is evaluated on SymbolSoup, Diffuse MQAR, and SimpleStories, with additional analysis of its feedback branch.

  • Experimental setup: Three mixers share the same decoder macro-architecture, parameter count, optimizer, training schedule, and optimization steps.
  • Datasets and tasks: Long-context evaluation uses SymbolSoup classification and Diffuse MQAR token-accuracy benchmarks, while SimpleStories measures short-context language modeling.
  • Long-context results: Mamba-2 did not converge on SymbolSoup or Diffuse MQAR under the reported training setup.The authors interpret this as consistent with selective-SSM theory under weakly separable selection signals.
  • Short-context results: Removing Sessa’s feedback branch improved SimpleStories test perplexity from 8.3700 ± 0.0482 to 8.0902 ± 0.0192.The same ablation increased top-1 accuracy from 49.144 ± 0.081% to 49.648 ± 0.026%.
  • Short-context results: The feedback-removed model outperformed full Sessa on SimpleStories, supporting the authors’ view that feedback is less beneficial in this short-context regime.The authors contrast this with Sessa’s stronger results on long-context tasks, where feedback appears more useful.

6 Discussion

The matched comparison isolates memory composition from sharp routing: diffuse attention dilutes, failed-freeze-time Mamba attenuates exponentially, while Sessa aggregates influence across multiple feedback paths. This structure supports slower long-range decay and more flexible retrieval under the same breakdown of sharp retrieval.

  • Matched comparison: Matched regimes compare architectures when sharp retrieval is unavailable, using diffuse routing for attention and failed freeze time for Mamba.These regimes provide a common comparison basis.
  • Memory mechanisms: Diffuse attention remains one-hop and suffers dilution, whereas failed-freeze-time Mamba remains chain-structured and exhibits exponential attenuation.Sessa is also studied under diffuse routing, so the comparison targets memory mechanisms rather than privileged sharp access.
  • Memory mechanisms: Sessa aggregates influence over multiple hop counts and, in dense settings, many temporal paths, producing slower long-range decay.The feedback solve combines direct-read routing with stateful aggregation through the feedback channel.
  • Selective retrieval: Deep Sessa realizes flexible selective-retrieval profiles, including frozen or increasing profiles, unlike diffuse fixed-depth Transformers and failed-freeze-time fixed-depth Mamba.The separation is qualitative in retrieval behavior, not only quantitative in decay rates.
  • Discussion: When sharp retrieval fails, long-context behavior depends on how routing coefficients are composed over time, and Sessa can retain flexible retrieval through multi-hop feedback.This distinction may become increasingly relevant as context length grows and sharp retrieval becomes less available.

B.3 Auxiliary lemmas

The auxiliary lemmas establish bounded mixer states, polynomially decaying inverse-kernel entries, and the resulting polynomial Jacobian tail under diffuse routing. They also provide the softmax derivative and bounded-logit-spread estimates used to control routing sensitivity.

  • State and kernel bounds: Under the stated feedback assumptions, the mixer state is uniformly bounded by S_R = F_R(1 − γ_max)^−1.The bound follows from a standard induction using γ_max < 1.
  • Routing estimates: The softmax Jacobian admits a total-variation bound controlled by twice the α-weighted logit sensitivity.This estimate transfers routing-logit sensitivity bounds to attention-weight derivatives.
  • State and kernel bounds: The inverse feedback kernel satisfies |K_t,k(x)| ≤ C_K(t−k)^−β_tail with unit diagonal.This is the core polynomial-tail estimate for the causal feedback solve.
  • Jacobian tail: Theorem 23 gives a polynomial long-range Jacobian bound, so sensitivity decays at least polynomially in lag up to a logarithmic factor.The result holds on X_R under Assumptions 15–22, 17, 18, and 19.
  • Jacobian tail: The block-output corollary transfers the polynomial Jacobian tail to token-wise gated outputs under a uniform gate bound.The token-wise gate contributes only a bounded multiplicative factor.
  • Routing estimates: Bounded logit spread implies near-uniform softmax weights, with α_j = Θ(1/n) when the spread stays bounded as n grows.This formalizes diffuse routing over a growing support.

C.1 Proof of Lemma 4.3

The proof of Lemma 4.3 uses bounded logit spread to show that diffuse attention weights and their source-token derivatives scale inversely with the visible support.

  • Derivative bound: The softmax Jacobian identity and token-wise query/key locality reduce source-token sensitivity to a support-scaled bound.For a source τ < t, the query depends only on x_t and key dependence is localized to x_τ.
  • Derivative bound: The proof concludes the desired inverse-support derivative estimate for diffuse full-prefix attention.The scalar sum is evaluated after applying the softmax derivative decomposition.
  • Diffuse attention: In the diffuse regime, bounded logit spread makes each attention weight Θ(1/|W_t|) uniformly across visible source positions.For full-prefix attention, |W_t| = t + 1.

C.2 Proof of Proposition 9

The proof compares one-hop Transformer dilution, failed-freeze-time Mamba attenuation, and Sessa’s feedback solve, while establishing stability and boundedness properties for the latter.

  • Transformer comparison: Diffuse full-prefix Transformer attention has per-source influence Θ(1/(t + 1)), yielding inverse-lag decay for fixed old sources.The estimate follows from bounded logit spread and full-prefix visibility.
  • Mamba comparison: Under failed freeze time, stable diagonal state transitions and bounded input/output factors produce exponential-in-lag attenuation for Mamba-style recurrences.The proof controls transition products through accumulated discretization time before converting the bound to lag decay.
  • Sessa comparison: Sessa’s impulse response is governed by the resolvent of its strictly causal feedback matrix and admits a power-law tail under diffuse routing.The exponent is β_tail = 1 − γ_max c_2 under the stated condition γ_max c_2 < 1.
  • Sessa comparison: Uniform routing with constant gain γ gives β_tail = 1 − γ, and the appendix establishes tightness for fixed sources and bounded source families.The explicit construction demonstrates that the polynomial rate is attainable.
  • Mamba comparison: The scalar ZOH recursion is bounded by convexity, yielding a uniform state bound when the driving term is bounded.The componentwise diagonal extension gives the corresponding vector bound.
  • Scope extensions: The same failed-freeze-time conclusion extends to fixed local windows with finite-window slack.The extension applies when the state-space components and inputs depend on a fixed recent window.
  • Stability: The infinite-horizon treatment interprets the lower-triangular solve as a causal linear system and imposes a uniform feedback margin and row-contraction condition.These conditions support BIBO stability and uniform-in-T bounds.

D.4 Proof of Lemma 4.2

The feedback solve is a causal lower-triangular recurrence with a unique bounded solution under a row-sum contraction condition. The resulting state and block outputs admit explicit norm bounds on bounded input balls.

  • Bounded solve: Every bounded input f has a unique bounded solution s obtained by forward substitution.Each s_t depends only on previously defined states, and the recurrence therefore defines a unique sequence.
  • Bounded solve: The solution satisfies ||s||∞,2 ≤ 1/(1−ρ)||f||∞,2.The bound follows by controlling partial maxima through the row-sum estimate.
  • Attention control: Causal attention is ℓ∞-nonexpansive: ||y||∞,2 ≤ ||v||∞,2.The result follows because the attention weights are nonnegative and sum to at most one over prior positions.
  • Block bound: Under bounded feedback, the Sessa solve obeys ||s||∞,2 ≤ F_R/(1−ρ_R), and z=s⊙g obeys ||z||∞,2 ≤ F_R G_R/(1−ρ_R).These bounds hold on inputs with ||x||∞,2 ≤ R when ρ_R<1.
  • Impulse regime: The impulse response has no feedback in the degenerate case η=0, yielding y_0=1 and y_t=0 for all t≥1.The genuine power-law regime therefore requires 0<η<1.

E.4 Polynomial decay theorem

Under bounded attention and feedback assumptions with 0<η<1, the impulse response has a polynomial upper bound in the lag. The same bound holds for impulses at arbitrary source positions and on finite horizons.

  • Polynomial decay: |y_t| ≤ C t^-β_tail with C=ηe^η and β_tail=1−η∈(0,1).The theorem also establishes that y_t tends to zero as t grows.
  • Proof mechanism: The proof derives the rate by bounding a harmonic-growth recursion, which yields the exponent β_tail.The comparison argument uses harmonic-number growth and gives the factor ηe^η t^-β_tail.
  • Finite horizons: The finite-horizon restriction preserves the same constant C=ηe^η for all lags within the horizon.The finite-horizon result is an immediate restriction of the polynomial-decay theorem.
  • Shifted impulses: For an impulse at position j, |y_t| ≤ C(t−j)^-β_tail for all t>j.The constant C=ηe^η is unchanged from the impulse-at-zero result.
  • Tightness: The upper bound alone does not exclude faster decay, so an explicit construction is used to establish an exact polynomial rate.This motivates the tightness analysis under uniform routing.

F.2 Uniform routing yields a Θ(ℓ−𝛽tail) tail

Uniform routing with constant positive feedback provides an explicit Sessa regime whose impulse response has a two-sided Θ(ℓ^-β_tail) heavy tail. The bound is uniform over every bounded source family, but not over all source positions on an unbounded horizon.

  • Construction: Uniform routing with constant positive feedback realizes an explicit heavy-tailed impulse response.This regime is maximally diffuse and supplies the construction used to prove tightness.
  • Two-sided tail: y_t=Θ(t^-β_tail) and y_t=Ω(t^-β_tail), with β_tail=1−γ∈(0,1).The two-sided estimate is given explicitly by γΓ(1+γ)(t+1)^-β_tail and γΓ(1+γ)t^-β_tail.
  • Scope boundary: The source-dependent prefactor decays polynomially with τ, so no positive lower constant is uniform over all source positions on an unbounded horizon.In particular, y_{τ+1} tends to zero as τ grows.
  • Uniformity: On every bounded source family, the uniform-routing regime realizes a uniform two-sided heavy-tail envelope.For 0≤τ≤τ_max, the constants can be chosen uniformly over τ and all lags.
  • Kernel consequence: The explicit impulse response is a column of K=(I−B_fb)^-1, making the polynomial envelope sharp for the influence kernel.Thus the construction attains the theoretical heavy-tailed memory rate.
  • Arbitrary source: An impulse at source position τ has lag-ℓ tail Θ(ℓ^-β_tail).The same computation applies after shifting the impulse to τ, with a source-dependent positive prefactor.

G Heavy-tail convolution estimates

Power-law influence kernels remain stable under discrete convolution and can be propagated through residual depth. Consequently, deep Sessa bounds retain polynomial tails, unlike the exponential decay bound for deep Mamba-style paths.

  • Convolution estimates: The lower convolution bound follows by restricting the sum to a central block containing Θ(n) terms.Each central summand contributes at the same power scale, producing the matching lower rate.
  • Convolution estimates: The k-fold convolution of positive-lag power sequences has exponent kσ−1 up to constant factors.For fixed k, matching upper and lower bounds are established by induction.
  • Residual calculus: Residual composition expands end-to-end Jacobians into ordered products of diagonal and strictly lower-triangular kernel factors.The resulting depth-dependent bound follows from the chain rule and operator-norm submultiplicativity.
  • Deep Mamba: For deep Mamba under failed freeze time, the end-to-end influence is bounded by (1+ℓ)^(N_layer−1)e^(-c*ℓ).The polynomial path-count factor multiplies an exponential temporal decay.
  • Deep Sessa: The Sessa depth bound is obtained because the k-fold heavy-tail convolution contributes the dominant exponent at k=N_layer.For β_tail∈(0,1), this exponent increases with k, up to constants.

H.5 Horizon-uniform bounds

The horizon-uniform analysis gives Transformer, Mamba, and Sessa Jacobian decay bounds whose constants remain independent of the sequence horizon. Under its stated condition, the Sessa bound decays asymptotically.

  • Horizon-uniform residual calculus: Theorem 35 establishes end-to-end residual bounds with constants independent of horizon T.The proof applies the same layerwise constants and kernels at every horizon.
  • Model-specific bounds: Transformer bounds are derived under layerwise envelope assumptions and use a distance factor based on 1 + ℓ.The supplied passages state the assumptions and identify the horizon-uniform Transformer case.
  • Model-specific bounds: Mamba bounds have the form (1 + ℓ)^(Nlayer−1)e^(−c∗ℓ), up to an Nlayer-dependent factor.This is the stated horizon-uniform Mamba decay expression.
  • Model-specific bounds: Sessa uses a common tail exponent βtail ∈ (0, 1), and its bound tends to 0 when Nlayer(1 − βtail) < 1.The asymptotic conclusion is stated explicitly for this exponent condition.

I.2 Architecture and function classes

This section defines Sessa blocks with either identity or LayerNorm preprocessing, their attention-feedback computation, and the comparison RoPE-Transformer class. It also states constructions supporting positional coding and universal approximation with adapters.

  • Sessa architecture: Sessa blocks use tokenwise preprocessing with either Norm = Id or Norm = LNεln, followed by attention feedback and residual computation.The feedback system has a unique solution because its matrix is strictly lower triangular.
  • Function classes: The Sessa function class is formed by finite compositions of width-m concrete blocks with tokenwise input and output adapters.The adapters map between external dimension d_ext and model width m.
  • Function classes: The comparison RoPE-Transformer uses causal self-attention, RoPE logits, a tokenwise FFN, residual connections, and an input absolute positional embedding.Its blocks operate at internal width m and use fixed attention scale σk = d_k^-1/2.
  • Selective positional coding: Softmax sharpening and RoPE diagonalization provide mechanisms for concentrating attention on selected positions.Scaling query and key vectors by a common factor yields arbitrarily sharp rows at fixed attention scale.
  • Approximation constructions: A single Sessa block can generate separated positional codes, and causal RoPE-Transformer and Sessa classes support universal approximation with adapters.The Sessa theorem states that d_k = 2 suffices for its approximation construction.

J.3 Simulating identity-normalized Sessa blocks with pre-norm LN-Sessa blocks

A widened pre-norm LN-Sessa block can simulate an identity-normalized Sessa block on compact sets using a constant scaffold. Layerwise composition then yields universal approximation for pre-norm LN-Sessa networks.

  • Single-block simulation: Lemma J.2 constructs a pre-norm LN-Sessa block that simulates any identity-normalized block on a compact set within εsim.The construction adds an even-dimensional scaffold and chooses a positive scale c.
  • Approximation mechanism: LayerNorm simulation is obtained by approximating its tokenwise preprocessing on compact attainable-token sets.The approximation error can be reduced by increasing the even scaffold width.
  • Single-block simulation: The scaffold coordinates remain constant because the residual increment has zero scaffold coordinates.Dynamic maps read only the dynamic projection, while scaffold rows and columns are otherwise zero.
  • Network-level approximation: Corollary J.3 applies the same common scaffold across layers and controls each layer's simulation error on recursively thickened compact sets.The composition argument propagates layerwise approximations through finite depth.
  • Network-level approximation: The resulting finite-depth pre-norm LN-Sessa network inherits the identity-normalized approximation guarantee.The final error combines the LayerNorm simulation error with the Norm = Id approximation error.

K Proofs for flexible finite-horizon selective retrieval

The proofs construct finite-horizon selective retrieval and signal transport using ordered positional codes, attention focusing, auxiliary channels, and composable transport blocks. These constructions culminate in transparent power-profile and macro-layer transport results.

  • Attention focusing: Ordered scalar codes let feedback attention focus on each token's immediate predecessor while suppressing strict-past non-predecessors.The construction works for pairwise disjoint positive code intervals and any target concentration parameter μ ∈ (0, 1).
  • Attention focusing: RoPE self-focusing realizes analogous concentration in a forward branch using a single actual RoPE plane.The resulting strict-past feedback row is controlled by the ordered positive code ranges.
  • Selection primitives: Symmetrized scaled GELUs provide exact scalar channel reads, while four scaled GELUs approximate plateau windows for selecting a target index.The exact channel read follows from combining two GELU coordinates; the window construction achieves accuracy η.
  • Auxiliary-channel retrieval: An auxiliary channel can be overwritten by the forward average of a target-selective window while positional, signal, and carry channels remain unchanged.The construction bounds off-target contributions and preserves the designated channels exactly.
  • Signal transport: Transport calculus composes these blocks into signal-blind exact scalar transport over control subspaces and bounded signal-fiber saturations.The stated corollary extends exact transport from a control-side compact set to every bounded signal-fiber saturation.
  • Power profiles and macro-layers: The transparent power-profile and macro-layer corollaries establish signal-blind transport with prescribed finite-horizon profiles and preserved control channels.The power-profile construction fixes β ∈ (0, 1) and combines large- and small-lag cases over a finite horizon.
Loading 2604.18580v2…