PaperlayerView exact revision on arXiv

Source-linked AI summary

The Identifiability and Observability of Deep Normalized Attention

Pranav Venkata Konda

arXiv:2610.09620v1cs.LG

TL;DR

The paper studies which parameters of deep attention can be recovered from its input–output function, a question relevant to behavioral safety arguments. It analyzes effective parameters, exceptional fibers, and finite-observation identification. It establishes generic effective identification up to even-normalizer sign ambiguities and studies the loss of sensitivity near collapse.

  • Problem

    For deep attention, which parameter distinctions are determined by the input–output function, and how sensitively can finite observations detect them?

    Source: §1 · p. 2

  • Method

    The paper analyzes effective scores and the combined value map, classifies exceptional fibers, and studies local identification through finite input observations.

    Source: §1 · p. 3

  • Results

    For real-analytic normalizers, the function determines effective scores and the combined value map up to signs induced by even normalizers; finite Taylor observations locally identify parameters at a sharp degree.

    Source: §1 · p. 3, §4 · p. 8

  • Takeaways & Limitations

    Behavioral safety arguments require observations that distinguish parameters when the claimed property differs, while repeated token mixing can make finite-observation recovery increasingly sensitive.

    Source: §1 · pp. 2–3

    Deeper softmax fibers with singular mixed prefixes at three or more tokens remain open, and fixed input sets and parameter paths do not establish weak sensitivity’s prevalence in trained networks.

    Source: §3 · p. 7, §7 · p. 13

Abstract

from arXiv · show

We study which parameters of deep, unmasked, single-head attention are determined by its input--output function. For known positive nonconstant real-analytic normalizers, the function generically determines the effective scores and combined value map up to the signs induced by even normalizers. This proves the real-analytic case of a conjecture of Henry--Marchetti--Kohn, including softmax. We then classify exceptional fibers under explicit normalizer conditions, identifying when collapse makes later scores unobservable, and establish sharp Taylor orders for local identification. Near simultaneous query/key collapse, we compute the complete native Jacobian decay spectrum on separating finite input banks. For common first nonconstant normalizer degree $k$, layer $i$ has contact order $2k3^{i-1}-1$, with exact multiplicities and kernel dimension. High-precision and automatic differentiation calculations illustrate the resulting loss of numerical sensitivity.

1. Introduction

The paper asks which aspects of deep attention are determined by its input–output function, a question relevant to behavioral safety arguments. It establishes effective identification results and studies how repeated token mixing affects recovery sensitivity.

1. Introduction

  • Behavioral safety arguments require observations that distinguish parameters when the claimed property differs.The paper asks which parameter distinctions survive in the function and how sensitively finite observations detect them.Source: §1 · p. 2
  • For known positive, nonconstant real-analytic normalizers, the function determines effective scores and the combined value map up to signs induced by even normalizers.This establishes the real-analytic case of the Henry–Marchetti–Kohn conjecture, including softmax.Source: §1 · p. 3
  • When the combined value map is nonzero and every score has a nonzero symmetric part, the complete effective fiber has only the score-sign ambiguities induced by even normalizers.Competing representations may be degenerate, score matrices may be rank deficient, and the value map may have a nontrivial kernel.Source: §1 · p. 3
  • The paper classifies exceptional fibers under explicit normalizer conditions and gives sharp local-identification Taylor orders and native-Jacobian decay orders near simultaneous query/key collapse.The Jacobian spectrum result includes multiplicities and the exact kernel dimension.Source: §1 · p. 3
  • Near consensus, each layer changes the mean at second order in incoming contrast but transmits contrast at third order, so deeper scores appear at successively higher orders.Subtracting the exact mean of a recovered prefix isolates the next score, connecting identification to declining numerical sensitivity.Source: §1 · p. 3

2. Effective parameters and identification

The paper represents attention using effective score matrices and a combined value map, then asks which such tuples are determined by the realized function. Under its theorem’s conditions, it recovers the effective tuple up to the signs allowed by even normalizers by separating successive scores in perturbation orders.

2. Effective parameters and identification

  • Effective parameters comprise layerwise score matrices in input coordinates and a combined value map, which remain meaningful even for singular or rectangular value maps.Query/key factorizations and invertible changes of factor coordinates can represent the same score and function, so the reduction targets the quantities determined by behavior.Source: §2 · p. 3
  • For known positive, nonconstant real-analytic normalizers, Theorem 1 determines the effective tuple up to independent score signs for even normalizers, when the combined value map and each score’s symmetric part are nonzero.If the final normalizer has nonzero derivative at zero, the final score’s symmetric-part condition is unnecessary.Source: §2 · p. 4
  • Theorem 1 is generic in native and effective parameters: its hypotheses exclude only a proper algebraic subset for every native architecture with positive widths.For injective normalizers, the effective fiber is a singleton, proving the analytic case of the Henry–Marchetti–Kohn conjecture; native factorization freedom adds no effective ambiguity.Source: §2 · p. 4

2.1. Effective fiber determination

  • Consensus inputs recover the combined value map, while perturbations from consensus expose scores through their successive contributions to the token mean.For softmax, score enters the mean at second order and outgoing contrast begins at third order; the order separation extends to the analytic normalizers in Theorem 1 and any token count.Source: §2.1 · pp. 4–5
  • Exact subtraction of the recovered prefix mean isolates the next score’s contribution, and coefficient extraction removes suffix terms.The method identifies scores by analytic cancellation, but does not provide stable numerical division; deeper scores appear at increasingly sensitive orders.Source: §2.1 · pp. 5–6

3. Suffix erasure by a layer

A layer that maps every input to consensus can make all later scores unobservable. Under the stated normalizer conditions, the paper characterizes the resulting effective fibers and gives a counterexample showing why a slope condition matters.

3. Suffix erasure by a layer

  • If a score lies in its layer’s collapse locus, that layer’s entire image is consensus, so changing subsequent scores leaves the realized function unchanged.The paper identifies the free suffix after the first collapse as accounting for the entire effective fiber under the theorem’s conditions.Source: §3 · pp. 6–7
  • When no layer collapses before the last, the fiber is a singleton; for a zero combined value map, the fiber has a distinct form.The supplied passage states the singleton condition and introduces the zero-value-map case without specifying its full fiber.Source: §3 · p. 7
  • At collapse, the score producing consensus can remain identifiable and affect the location of that consensus.The paper gives two-token softmax with a skew score as an example.Source: §3 · p. 7
  • In the two-token zero-slope example, the first layer preserves contrast but the second layer’s averaging erases its score dependence, so the prefix need not be identified through that collapse.The authors state that the nonzero-slope assumption is essential even with two tokens.Source: §3 · p. 7
  • For softmax with three or more tokens, a nonzero skew score can preserve contrast on a noncollinear cloud, and deeper singular mixed prefixes remain unclassified.The paper gives a complete depth-two softmax classification but leaves the general deeper case open.Source: §3 · p. 7

4. Parameter identification by finite input jets

Finite zero-input Taylor jets locally identify effective parameters at a sharp degree under the stated nondegeneracy conditions. The section also characterizes depth-two fibers and how native parameter redundancies change at collapse.

4. Parameter identification by finite input jets

  • For common first nonconstant degree k, degree 2k3^(D−1)+1 locally identifies the effective tuple, and every lower-degree jet is independent of the final score.For softmax, the sharp degrees at depths one through four are 3, 7, 19, and 55; even-degree normalizers can leave distant sign branches indistinguishable.Source: §4 · p. 8
  • Finite Taylor observations are polynomial in local coordinates, although the full softmax function and ordinary input evaluations are generally nonpolynomial.The jet operator retains input derivatives at X=0 through total order N.Source: §4 · pp. 7–8
  • With nonzero readout and suitable nondegenerate score prefixes, a degree 2·3^(D−1)+1 jet identifies the complete effective tuple even for nonzero skew prefixes.The two-plane result also gives an immersion, and permits an arbitrary final score.Source: §4 · p. 8
  • At depth two with nonzero normalizer slopes, a nonzero first score gives a degree-seven identifying jet; if it vanishes, the second score is arbitrary and degree three determines the fiber.With zero readout, every score tuple lies in the same fiber; no lower jet depends on the second score in the nonzero-first-score case.Source: §4 · p. 9

4.1. Native fibers and serial composition

  • Where an observation has injective differential, the local native fiber is a product of marked matrix-multiplication fibers and smooth value-coordinate directions.The proposition notes that its proof and limits at collapse are treated separately.Source: §4.1 · p. 9
  • At first-layer collapse, every later factor remains free; a marked zero score can also enlarge the effective fiber.For depth D≥2, the specified native fiber has 2D irreducible components.Source: §4.1 · p. 9

5. Numerical weakness of exact identification

The paper measures local sensitivity through the native-parameter Jacobian on a finite input bank as query and key factors approach collapse. Its singular-value bands have layer-dependent contact orders, so identifiable directions can nevertheless become weakly observable.

5. Numerical weakness of exact identification

  • The contact order tracks how quickly a positive singular value vanishes as t approaches zero; order zero means it stays bounded away from zero.The Jacobian differentiates observed outputs with respect to all native parameters before restricting to the path.Source: §5 · p. 10
  • On a separating finite input bank, the native Jacobian has d² singular values comparable to 1 and layer-i bands of multiplicity r_i(2d−r_i) comparable to |t|^β_i.The theorem assumes full-column-rank path factors and nonzero symmetric parts of their products.Source: §5 · p. 10
  • For softmax with d=4, D=3, and r_i=2, the 96 native directions comprise 16 order-zero values, twelve each at orders 1, 5, and 17, and 44 exact zeros.The zeros represent query/key factor gauges and intermediate value-coordinate freedom; positive-order bands indicate weakening sensitivity.Source: §5 · pp. 10–11
  • Weakly sensitive identifiable directions can carry arbitrarily little Fisher information near collapse, so small output changes may reflect either redundancy or weak local sensitivity.The stated Fisher-information scales are τ^-2 and τ^-2|t|^(2β_i), with multiplicities matching the Jacobian bands.Source: §5 · p. 11

6. Numerical evidence

Numerical experiments test predicted contact orders, Jacobian rank, and recovery under noise. They find close agreement with predicted softmax orders, while finite precision and noisy observations obscure weakly sensitive parameters.

6. Numerical evidence

  • At depth four, measured softmax slopes were (0.999994, 5.002211, 17.001118, 53.000562), within 0.0023 of predicted (1, 5, 17, 53); the weakest singular value at t=0.02 was 3.86 × 10^-92.The measured orders agree with predictions to maximum absolute discrepancy 0.002212.Source: §6 · p. 12
  • For n=d=4, D=3, and r_i=2, float64 automatic differentiation reports rank 40 versus exact rank 52 at small scales, while 120-digit precision resolves all twelve weak positive values.At t=0.06, the same cutoff still gives rank 40 at high precision because the weak values lie below it.Source: §6 · p. 12
  • As noise standard deviation rises from 1.57 × 10^-10 to 1.57 × 10^-7, final-factor relative RMSE rises from 0.0050 to 1.31 despite unique determination.The experiment estimates the unknown nonnegative final query/key factor from one noisy output mean with the prefix known.Source: §6 · p. 12

7. Related work

The appendix situates effective identification among prior work on attention and neural-network identifiability, then develops analytic signatures for recovering score matrices. It shows that a one-score signature has a sign ambiguity exactly for even normalizers.

7. Related work

  • Attention identifiability work distinguishes fixed-input weight nonuniqueness [5] from whole-function effective identification; this paper extends local normal forms to normalized attention.At collapse, one layer can erase all later scores, so the network fiber need not be a product of individual layer fibers.Source: §7 · pp. 12–13
  • Prior work links cubic contrast contraction and rank collapse, vanishing query/key gradients, and token clustering; this paper quantifies fixed-depth sensitivity loss.The paper's analysis considers query/key factors approaching zero and uses a finite input bank separating divided derivative directions for matching lower bounds.Source: §7 · p. 13

A. Identification

  • The appendix compares native realizations through effective tuples, whose realizable values form the image of the parameter map π.The effective tuple space comprises the combined value map and layerwise score matrices in the original input coordinates.Source: A. Identification · p. 17

A.1. Analytic setting and a one-layer expansion

  • Positive nonconstant real-analytic normalizers make layer maps and finite compositions jointly analytic near zero, allowing parameter differentiation and input-coefficient extraction in either order.The analytic germ argument also permits cancellation of a nonzero germ when comparing exact function identities.Source: §A.1 · p. 17

A.2. A signature determines one score

  • A one-score signature has fiber {M} for non-even normalizers and {M, −M} for even ones; when g(0) ≠ 0, it determines every score.The latter identification includes zero and skew-symmetric scores.Source: §A.2 · pp. 18–19

A.3. Layer isolation and proof of Theorem 1

The proof isolates successive layers by comparing Taylor orders along line probes. Under the symmetric-part condition, the contrast signal remains nonzero, while the zero-readout case leaves all scores unobservable.

A.3. Layer isolation and proof of Theorem 1

  • When sym Mi ≠ 0, each layer's contrast signature and amplitude germ stay nonzero, and layer i's outgoing contrast has order 3^i.The incoming contrast at layer i has order 3^(i−1).Source: §A.3 · pp. 19–20
  • Distinct Taylor orders isolate the next layer's signature after the preceding layers are recovered, completing the layer-by-layer identification induction.The competing next-layer signature is isolated because its later mean increments begin at strictly higher order.Source: §A.3 · p. 20
  • If the final log-slope is nonzero, the final score is identified without requiring a nonzero final contrast; if L = 0, every score tuple realizes the zero function.The zero-readout fiber consists of the zero value map paired with arbitrary score matrices.Source: §A.3 · p. 20

A.4. Native reduction and genericity

The native reduction establishes generic effective identifiability and separates it from exceptional cases. Collapse can make later scores unobservable, whereas under the stated nonzero-slope and curvature regime scores are identified through the first zero.

A.4. Native reduction and genericity

  • For pure serial architectures, positive injective real-analytic normalizers give singleton effective fibers outside a proper real algebraic subset, without requiring invertible value maps for genericity.The native coordinate change in Proposition 4.2 does require square invertible value maps, but the rank-one construction establishes genericity without that condition.Source: §A.4 · pp. 20–21

B. Exceptional fibers

  • After a layer collapses, changing the suffix preserves the realized function because every later layer fixes the consensus subspace.The zero-readout case is more degenerate: every score tuple realizes the zero function when L = 0.Source: B. Exceptional fibers · p. 22

B.1. Nonzero logarithmic slope and curvature

  • In the nonzero logarithmic-slope and curvature regime, each nonzero score transmits a nonzero contrast germ, so scores are identified through the first zero.The collapse locus is {0} in this regime.Source: §B.1 · p. 22

B.2. Two tokens and the exact collapsing condition

This section characterizes collapse in terms of the score matrix, normalizer, and token count, and establishes when finite Taylor jets identify local parameters or complete realization fibers. It also presents adaptive two-token identification results, including global sign discrimination, covariance recovery, and two-plane restrictions under stated conditions on scores, readout, normalizers, or input width.

B.2. Two tokens and the exact collapsing condition

  • A layer collapses exactly when M=0 or M is nonzero skew and h is odd; after the first collapse, all later scores are unobservable.The collapse locus is the space of skew matrices when h is odd and {0} otherwise.Source: §B.2 · p. 23

B.3. Softmax collapse for every token count

  • For softmax, the single-layer image collapses exactly for M=0 when n≥3, or for skew M when n=2; collinear clouds cannot expose skew contrast for n≥3.A three-token construction using two independent directions retains contrast for nonzero skew scores.Source: §B.3 · p. 24

B.4. Why nonzero slope cannot simply be removed

  • With ϕ₁(s)=exp(s²), a nonzero skew first score is noncollapsing yet becomes invisible after a zero-score second layer, showing why the nonzero-slope condition matters.The first score affects contrast but not the mean, and the next layer removes that contrast.Source: §B.4 · p. 24

C. Finite jets

  • Finite-jet analysis distinguishes local coordinates from complete-fiber determination, which must also distinguish distant sign branches; here the whole input is scaled as X=ρY.The passage contrasts this scaling with scaling only the contrast around a fixed mean.Source: C. Finite jets · p. 24

C.1. An order estimate for arbitrary clouds

  • For arbitrary clouds, under L≠0 and nonzero symmetric parts of every layer score, the degree-N_D jet is an immersion, while every lower jet is independent of M_D.For a common first nonconstant normalizer degree k, the passage gives N_D=2k3^D−1+1.Source: §C.1 · p. 25

C.2. Immersion and the sharp local order

  • The local order is attained: the degree-N_D observation has zero Jacobian kernel and a local analytic inverse, whereas lower jets cannot identify the final score.The score-detection argument proceeds layer by layer, with the current score coefficient injective in its matrix variation.Source: §C.2 · p. 26

C.3. Nonzero slope: rational local inversion and uniform determination

  • A finite jet determines the complete effective fiber at every tuple, including zero-score and zero-readout exceptions, and is an injective immersion when L≠0 and all prefix scores are nonzero.The stated degree suffices for arbitrary token count; optimality for noncollinear clouds is not asserted.Source: §C.3 · p. 27

C.4. Exact adaptive order for two tokens

  • For two tokens, the least jet order determining the complete fiber is 2α_{s−1}+1, where s is the first collapsing layer, or D if none collapses.Along the noncollapsing prefix, α_i=3α_{i−1} for nonskew scores and α_i=(ℓ_i+1)α_{i−1}+ℓ_i for nonzero skew scores, with ℓ_i the first nonzero even degree of h_i.Source: §C.4 · p. 28
  • For two tokens, the differential is triangular and injective, and the recurrence gives α_i = 3i + v_i in the nonzero-curvature specialization.Source: §C.4 · p. 29

C.5. Exact global sign discrimination

  • Under L ≠ 0 and sym M_i ≠ 0 at every layer, Nsign is the least order at which the zero-input jet determines the complete effective fiber; signs at even normalizers remain exact symmetries at every order.The competing tuple need not initially be regular.Source: §C.5 · p. 29

C.6. Multidirectional covariance recovery

  • For n ≥ d + 1, nonzero g_i(0), L ≠ 0, and invertible M_i for i < D, the complete effective fiber is a singleton and the jet through 2 · 3^(D−1) + 1 determines it and is an immersion.The final matrix is arbitrary, no lower jet depends on the final score, and invertible skew prefixes and readouts with nontrivial kernels are permitted.Source: §C.6 · pp. 30–31

C.7. Recovery from generic two-plane restrictions

  • Under the stated nonvanishing two-plane restriction condition, the complete effective fiber is a singleton against arbitrary competitors, and the zero-input jet determines it and is an immersion.The conclusions hold when every prefix score is nonzero and skew, in both odd and even dimension; no lower jet depends on the final score.Source: §C.7 · pp. 32–33
  • The two-plane condition includes singular skew prefixes, but rank-one scores with nonzero symmetric part instead fall under Theorem 1, and deeper stacks mixing these types require both kinds of probes.Source: §C.7 · p. 33

C.8. Proof of Corollary 4.1

The corollary proof establishes degree-seven jet identification and injective differentials in its stated cases, while finite evaluations and native contact spectra characterize how input banks preserve and reveal parameter sensitivity.

C.8. Proof of Corollary 4.1

  • Degree seven gives singleton jet fibers and injective differentials in both alternatives for M1 ≠ 0, with arbitrary final matrix.When M1 = 0, consensus makes the second score unobservable, but degree three identifies the first score and the second remains free.Source: §C.8 · p. 33

C.9. Finite ordinary evaluations

  • A finite bank of sufficiently many independent inputs drawn with a density separates a fixed analytic function space almost surely, but consensus-supported or discrete-corpus inputs may not.The construction and conditioning concern the fixed space; optimizing input-bank size and placement remains separate.Source: §C.9 · p. 33

D. Native contact spectra

  • For common first nonconstant normalizer degree k, layer i has native Jacobian contact order 2k3^(i−1) − 1, and the divided derivative frame establishes that these divisibility bounds are attained.Positive Smith-factor powers encode sensitivities tending to zero; exact zero columns encode directions whose differential vanishes identically along the path.Source: D. Native contact spectra · p. 34

D.1. The native source and its value coordinates

Appendix D derives the native Jacobian’s contact spectrum by separating value coordinates from effective scores, then establishes the spectrum under broader prefix conditions. The following appendices clarify finite observations and why the analytic arguments do not extend directly to smooth normalizers.

D.1. The native source and its value coordinates

  • For heterogeneous first nonconstant degrees k_i, layer i has contact order β_i = 2k_i − 1 + 2e_{i−1}, where e_0 = 0 and e_i = 3e_{i−1} + 2k_i; its score group has r_i(2d − r_i) factors.On a fixed separating bank, there are d^2 unit factors and exact source-kernel directions; for common k, β_i = 2k3^{i−1} − 1.Source: §D.1 · p. 34, §D.4 · p. 37

D.2. Homogeneity gives joint analytic divisibility

  • Joint analytic divisibility bounds each layer-i factor derivative by t^β_i, with β_i = N_i − 2 = 2k_i − 1 + 2e_{i−1}, while value-product variations have order zero.The bounds apply to every output component on sufficiently small inputs before choosing a finite bank.Source: §D.2 · p. 35

D.3. The divided frame and exact Smith factors

  • The divided leading derivative groups are jointly independent, and a fixed finite separating bank yields exact Smith orders 0 and β_i, with no hidden higher powers of t.The same bank works for every sufficiently small t.Source: §D.3 · p. 36

D.4. Factor kernels, multiplicities and singular values

  • The Gauss–Newton eigenvalues have orders 0 with multiplicity d^2 and 2β_i with multiplicity m_i; factor-gauge and free prefix-value directions contribute exact kernel directions.The full squared-loss Hessian has additional residual-weighted terms, which vanish at an exact fit.Source: §D.4 · pp. 36–37

D.5. Covariance and two-plane extensions

  • For invertible prefixes, no symmetric-part hypothesis is needed; under the stated full-column-rank tangent-count condition, the finite-bank result gives the full native Jacobian spectrum, including powers, multiplicities, and zeros.For the invertible-prefix corollary, β_i = 2·3^{i−1} − 1; all prefix products may be nonzero skew matrices, even if singular in the ambient space.Source: §D.5 · pp. 37–38

E.1. Germs, finite observations and parameter fibers

  • Taylor truncations give polynomial parameter maps, and whenever the germ differential is injective a finite input bank can also yield an injective finite observation map.The paper distinguishes these finite observations from agreement of realized germs on a neighborhood.Source: §E.1 · pp. 38–39

E.2. The boundary of the analytic argument

  • Smoothness does not support the analytic proof steps: smooth germs can have zero divisors, while an injective smooth normalizer can hide score information from every zero-input Taylor coefficient.The example distinguishes full-function identification from what any finite jet detects.Source: §E.2 · p. 39

E.3. Proof of Proposition 4.2

The local analysis separates effective observations from native factorization fibers and characterizes what remains visible at a first zero score. It then relates Jacobian contact orders to observation sensitivity and describes numerical protocols for examining that sensitivity.

E.3. Proof of Proposition 4.2

  • Fixing the observed function fixes the value product and each effective score, while leaving prefix-value coordinates free; rank-deficient query/key factors can produce singular multiplication fibers.At full-column-rank factors, each score block has multiplication-map rank r_i(2d − r_i).Source: §E.3 · p. 40

E.4. The residual ideal at the first zero score

  • At a first zero score, the visible prefix remains identifiable through that layer, while suffix factors are free under the stated square-invertible-value assumptions.Moving away from the zero score can let that layer transmit contrast again, making the suffix relevant to nearby functions.Source: §E.4 · pp. 41–42

E.5. What the contact spectrum records

  • The contact spectrum records exact source-kernel directions and sensitivity orders |t|^β_i; the d^2 unit factors disappear from the differential cokernel.The corresponding positive singular values have the same powers up to bounded multiplicative constants.Source: §E.5 · p. 43

F. Numerical protocols

  • The numerical protocols evaluate native Jacobians along a fixed parameter path and separately test recovery from one noisy output mean with a known prefix.The computations use fixed weights and inputs without a training procedure; further finite-observation and cotangent-complex comparisons are developed separately.Source: F. Numerical protocols · p. 43

F.1. Scalar depth-four spectrum

  • In the depth-four scalar softmax experiment, fitted singular-value orders 0.999994, 5.002211, 17.001118, and 53.000562 match predictions 1, 5, 17, and 53 within maximum absolute discrepancy 0.002212.At t = 0.02, the weakest positive singular value is 3.86167 × 10^−92.Source: §F.1 · p. 44

F.2. Full native matrix spectrum

The full native matrix experiment separates exact kernel directions from positive singular values and compares their decay with predicted contact orders. A second experiment sets up recovery from one noisy mean with a known prefix.

F.2. Full native matrix spectrum

  • The 4 × 4 scalar query/key spectrum follows powers 1, 5, 17, and 53, as the plotted positive singular values are compared with dashed theoretical guides.Four exact factor-gauge directions are omitted from the figure.Source: Figure 2 · p. 45
  • The matrix-valued experiment has 44 exact kernel directions and Jacobian rank 52, while its weakest positive singular space approaches the final-layer image as t decreases.The largest principal-angle sine decreases from 3.01 × 10^−6 at t = 0.6 to 5.21 × 10^−20 at t = 0.06.Source: §F.2 · pp. 45–46
  • At t = 0.06, the numerical cutoff reports rank 40 versus exact rank 52, omitting twelve identifiable directions in addition to the 44 exact kernel directions.The cutoff is a numerical diagnostic, not an exact-rank test or a threshold calibrated to observation noise; calculations lack interval certification.Source: §F.2 · p. 46

F.3. One noisy observation with a known prefix

  • The noisy-recovery experiment restricts the unknown final balanced scalar factors to q_2 = k_2 = w with true value w* = 0.1, using one mean observation and a known prefix.The supplied passages describe the setup but do not report the resulting estimation summaries.Source: §F.3 · pp. 46–47
Loading 2610.09620v1…