Source-linked AI summary
Feature Priming in Online Linear Regression: Sparse-Regret Lower Bounds and a Tight Univariate Rate
Huibo Xu, Shi Fu, Qixin Zhang, Dacheng Tao
arXiv:2608.17573v1stat.MLcs.LGstat.AP
TL;DR
Feature priming aims to make online regret reflect comparator sparsity, but Warmuth and Amid’s three rules lacked a competitive online guarantee. Using the natural past-only Moore–Penrose protocol, the paper identifies nuisance interpolation as a shared obstruction and shows none of the rules achieves uniform sparse-logarithmic regret.
Problem
Warmuth and Amid asked whether any of three feature-priming rules can guarantee O(k log(ed/k)) regret uniformly over comparators supported on at most k coordinates.
Source: §1 · p. 1Method
Results
None of the three rules admits uniform sparse-logarithmic regret; powered univariate priming has a matched Θ(min{T, d}) worst-case frontier.
Source: §10 · pp. 11–12Takeaways & Limitations
The analysis links costly priming errors to nuisance-selected interpolants and bounds how long they can persist by data rank.
Source: §10 · pp. 11–12Matching frontiers for normalized multivariate and Pearson priming remain open.
Source: §9 · p. 11
Abstract
from arXiv · showhide
In high-dimensional online prediction, the best predictor may depend on only a few features, so regret should scale with sparsity rather than the ambient dimension. Feature priming pursues this goal by estimating feature weights from past data and refitting a minimum-norm predictor on the rescaled design. Warmuth and Amid asked at COLT 2023 whether any of three such rules admits a competitive online regret guarantee. Using the natural Moore--Penrose protocol based only on past data, we give a negative answer to the sparse-logarithmic form of this COLT open problem. Our analysis identifies a common obstruction: cheap nuisance interpolation causes the refit to underweight the truly predictive coordinate. An exact target-mass identity and a two-sign argument turn this effect into clipped prediction loss. Hadamard constructions force $Ω(\min\{T,\sqrt{d}\})$ regret for all three rules against a zero-loss one-sparse comparator, with extensions to fixed prime powers and selectors among the rules. Conversely, regret is controlled by data rank, and a Euclidean-normalized triangular construction matches this dependence for powered univariate priming, even under nonnegative second-stage ridge regularization; a paired ridge construction also covers all three powered rules. Exploratory diagnostics on frozen language-model activations exhibit the same relation among nuisance interpolation, target weight, and loss. The exact multivariate and Pearson frontiers remain open.
1. Introduction
Feature priming scores and rescales features before a second minimum-norm fit to seek sparsity-adaptive online regret. Under the natural past-only protocol, the paper rules out the sparse-logarithmic guarantee while establishing rank-dependent bounds and identifying unresolved frontiers.
Introduction
- Under the natural nonanticipating past-data protocol, the three priming rules do not guarantee O(k log(ed/k)) regret uniformly over k-sparse comparators.This is a clipping-robust negative answer to the open question posed by Warmuth and Amid (2023); it does not rule out dimension-dependent guarantees.Source: §1 · pp. 1–2
- Cheap nuisance-only interpolation can make the refit match past labels while assigning little weight to the true target coordinate.The paper gives an exact identity for learned target weight in terms of transformed target prime and the minimum squared norm of a nuisance-only interpolant; a two-sign argument converts missing target mass into clipped loss.Source: §1 · p. 2
- Hadamard features force Ω(min{T, √d}) regret against a zero-loss one-sparse comparator; paired positive and negative queries make all three powered rules coincide for each fixed α ≥ 1.One fixed sequence defeats all three rules, current-input-dependent switchers, and convex mixtures.Source: §1 · p. 2
- A clipped predictor that interpolates completed history has regret at most 4 rank(X1:T), and powered univariate priming attains Θα(min{T, d}) regret with inputs of norm at most 1.The rational triangular construction matches the rank dependence.Source: §1 · p. 2
- For powered univariate priming, the triangular lower bound persists under every nonnegative ridge schedule on the transformed second-stage coefficient.Weak regularization permits cheap nuisance interpolation, while strong regularization shrinks prediction toward zero; a paired-Hadamard variant gives linear regret for all three powered rules under past-only ridge, with a common distributional lower bound for pre-input switchers and mixtures.Source: §1 · p. 2
- Exploratory frozen-activation tests show that greater nuisance capacity coincides with cheaper nuisance interpolation, lower learned target mass, and higher normalized regret.The diagnostics are not evidence that Qwen2.5 or Qwen3.8 implements feature priming; regularized prime estimation, iterative or distribution-dependent variants, broader randomized algorithms, and the exact multivariate and Pearson frontiers remain open.Source: §1 · p. 2
2. Problem setup
The setup studies nonanticipating online square-loss regression with bounded inputs and predictions refit from past data. Labels follow a fixed sparse comparator, and the lower bounds use the one-sparse comparator e1, which incurs zero loss.
2. Problem setup
- Before each round, the learner uses completed history to compute coefficients, then observes the input, predicts, and receives the label, following Warmuth and Amid (2023).Inputs are coordinatewise bounded in [−1, 1]^d; the lower-bound case uses S = {1}, uS = e1, and yt = xt,1.Source: §2 · p. 2
- Priming rescales the historical design by the multipliers and performs a second minimum-norm fit; signed squaring and literal squaring give prediction-equivalent refits.The sign matrices cancel in the pseudoinverse refit, so only multiplier magnitudes affect prediction.Source: §2 · p. 3
- The empty-history prediction is zero, and clipping predictions to [−1, 1] cannot increase square loss when labels lie in [−1, 1].The clipped prediction is max{−1, min{1, by_t}}.Source: §2 · p. 3
- The setup introduces base primes proposed by Warmuth and Amid (2023, Sec. 2, p. 2), including univariate LLS and empirical Pearson correlation.Univariate LLS uses the past feature column's least-squares coefficient, with value zero for a zero column; Pearson is defined when both empirical variances are positive.Source: §2 · p. 3
3. Multivariate LLS: pt = X†
The section states when Pearson priming is defined and how it is totalized, then specifies the loss, input constraints, and witness quantifiers used for the results.
3. Multivariate LLS: pt = X†
- The formulas agree with the original priming formulas whenever the relevant statistics are defined.Source: §3 · p. 3
- Pearson correlation is undefined for histories of length at most one or zero-variance arguments; finite past-only totalizations preserve defined values, while the rank upper bound requires target preservation.The Pearson lower bounds hold for every finite totalization; target preservation means the filled target prime is nonzero whenever past labels are not all zero.Source: §3 · p. 3
- Because e1 has zero loss on the constructions, regret equals learner loss, and the analysis also uses its clipped version.The clipped loss is Rclip_t = 1( e_y_t − y_t )^2.Source: §3 · p. 3
- Inputs satisfy x_t ∈ [−1, 1]^d, not necessarily ||x_t||_2 ≤ 1; unless stated otherwise, adversarial sequences may depend on the rule and horizon but are fixed before play.Unit-power witnesses are rule-specific; powered separation uses one common witness and holds pathwise for selectors. The triangular ridge result is pointwise in the ridge schedule, while the policy-uniform Hadamard statement is distributional.Source: §3 · p. 3
3. A nuisance-interpolation obstruction
The section isolates a geometric failure: when nuisance features can cheaply interpolate past labels, the minimum-norm refit underweights the true target despite fitting the history. It formalizes the obstruction through the prime-weighted cost of nuisance interpolation and converts the target-weight deficit into clipped loss on one of two next examples.
3. A nuisance-interpolation obstruction
- For an interpolated history, the least-norm coefficient vector obeys z1 = π1J and the rescaled predictor therefore has target weight w1 = π1^2J.J is the norm of the least-norm coefficient vector in the transformed design.Source: §3 · pp. 3–4
- When nuisance columns interpolate y and κ = ∥B†y∥2^2, the refit assigns target weight w1 = π1^2κ/(1 + π1^2κ), governed by π1^2κ rather than π1 alone.Here B is the transformed nuisance design; the relevant quantity is the prime-weighted cost of explaining the history through nuisance features.Source: §3 · p. 4
- For any next nuisance row h, the two legal examples xσ = (σ, h), yσ = σ have average clipped loss at least (1 + π1^2κ)^-2, so one sign incurs at least that loss.The nuisance contribution is the same scalar for both target signs, so it cannot make both predictions accurate after clipping.Source: §3 · p. 4
4. Hadamard instantiations of the obstruction
Hadamard constructions make nuisance features interpolate the labels while prime-weighted refits underweight the predictive coordinate, yielding sparse-regret lower bounds for unit-power and fixed-powered rules.
4. Hadamard instantiations of the obstruction
- Hadamard nuisance columns interpolate labels for every sign sequence, while the powered minimum-norm nuisance cost is at most r^2α/N, with Pearson using r = s after opposite pairs.The Pearson bound is independent of finite totalization.Source: §4 · p. 5
- Hadamard sequences force Ω(min{T, sqrt(d)}) clipped regret for each unit-power rule against a zero-loss one-sparse comparator.The Pearson lower bound applies for T ≥ 4 and is uniform over finite totalizations.Source: §4 · p. 5
- None of the three unit-power rules admits a universal k log(ed/k) sparse-regret bound; the failure already holds at k = 1.For Pearson priming the conclusion is uniform over every finite totalization; the raw predictions fail as well.Source: §4 · pp. 5–6
- For every fixed α ≥ 1, a shared deterministic witness gives Ω(M) clipped regret at T = 2M and d = Θα(T^2α) for all three powered rules and any three-rule selector.An oblivious Rademacher-sign version gives expected clipped regret at least 49M/64, even conditioned on the selector’s random tape.Source: §4 · p. 6
5. Rank controls loss and the univariate frontier
History-interpolating priming rules can incur positive clipped loss only when an input adds a new row-space direction, yielding a rank-dependent upper bound. Powered univariate priming matches this dependence up to constants, while matching worst-case rates for multivariate and Pearson priming remain open.
Rank controls loss and the univariate frontier
- 4r bounds regret when all inputs lie in an r-dimensional subspace, independently of horizon; this is a rank bound, not a sparsity bound.The bound applies to powered univariate and multivariate priming on e1-realizable sequences, and powered Pearson priming under target-preserving totalization.Source: §5 · pp. 6–7
- cα min{d, T} lower-bounds powered-univariate regret on a rational sequence realizable with zero loss by e1; unit power admits the sharper bound 1 + ⌊m/2⌋.A sequence with ∥xt∥2 ≤1 still gives regret at least cαm/4; the triangular construction incurs loss on a constant fraction of final rounds.Source: §5 · p. 7
- Θα(min{T, d}) regret is tight for powered univariate priming, both for coordinatewise bounded inputs and with Euclidean-normalized inputs.The rank upper bound and lower bound hold for sequences realizable with zero loss by e1.Source: §5 · p. 7
- Matching worst-case dependence on (T, d) remains open for multivariate and target-preserving Pearson priming.Although the rank theorem covers both rule families, the matching lower bound established here is for powered univariate priming.Source: §5 · p. 7
6. Ridge changes the failure mode, not the rate
Second-stage ridge changes the failure mode, but does not remove the powered univariate lower bound or the paired-Hadamard separation for the three rules. In the triangular construction, stronger ridge shifts nuisance-driven underprediction toward zero-prediction underfitting.
6. Ridge changes the failure mode, not the rate
- Ridge penalizes the transformed coefficient z, and its nuisance-interpolation analogue cannot restore target mass suppressed by a cheap nuisance interpolant.For λ ≪ λ_c(t), nuisance-driven Moore–Penrose prediction persists; for λ ≫ λ_c(t), predictions approach zero and underfit label one, so the crossover is between two failure modes.Source: §6 · p. 8
- For powered priming, the fixed rational triangular sequence gives regret at least (c_alpha/4) min{T,d}; at unit power it gives 1 + floor(min{T,d}/2).The bounds hold for every finite nonnegative ridge schedule, including schedules randomized or selected after observing x_t; the norm-constrained bound assumes ∥x_t∥2 ≤ 1 and ∥e1∥2 = 1.Source: §6 · p. 8
- For each powered prime rule and deterministic past-only ridge policy, a deterministic realizable paired-Hadamard sequence yields clipped regret at least 32T/81.Under independent Rademacher query signs, the same bound holds in expectation for randomized past-only policies and pre-input switchers or convex mixtures; the Pearson statements are uniform over finite totalizations.Source: §6 · p. 8
- Theorem 10 permits ridge values to depend on the current input, whereas Theorem 11’s common three-rule statement is distributional.The deterministic compiler for the paired-Hadamard result requires the ridge value to be fixed before the current target sign is revealed.Source: §6 · p. 9
7. The obstruction in frozen language-model representations
Exploratory diagnostics on frozen language-model activations reproduce the predicted relation between nuisance interpolation, target mass, and regret. The tests use synthetic coordinate labels and offline input ordering designed to stress the priming maps.
7. The obstruction in frozen language-model representations
- The diagnostic uses post-gating MLP activations from frozen Qwen2.5-7B-Instruct and Qwen3.8-27B layers, with activation coordinates as bounded labels and offline ordering designed to stress priming.Calibration data select six nondegenerate target neurons per layer; each target coordinate has a zero-loss one-sparse comparator. The records are from Alpaca instruction data (Yang et al., 2024), and Qwen3.8 layers are selected at corresponding relative depths.Source: §7 · p. 9
- With 2,047 nuisance neurons, median normalized regret was 1.238 for univariate, 0.303 for multivariate, and 0.587 for Pearson priming, versus 0.029 for each rule on target-only data.Increasing nuisance dimension also lowers target mass and makes cheap nuisance-only certificates common. Independently permuting nuisance coordinates leaves medians at 0.799, 0.306, and 0.555, respectively; Qwen2.5 shows the same mechanism. These exploratory diagnostics do not test whether either model internally implements feature priming.Source: §7 · p. 9
8. Related work
The related work frames the negative result as an answer to the sparse-logarithmic interpretation of Warmuth and Amid’s open problem, while distinguishing prime–refit analysis from other sparse-learning methods and guarantees under different assumptions.
Open problem and scope
- Warmuth and Amid (2023, Sec. 2, p. 2) define univariate, Pearson, and multivariate LLS primes and ask whether any admits competitive regret; this work rules out the uniform sparse-logarithmic interpretation.The open problem does not fix dimension or horizon dependence, and dimension-dependent guarantees remain possible.Source: §8 · p. 10
Sparse regret and adaptive geometry
- Coordinatewise and Euclidean online methods, adaptive guarantees, and restricted-feature algorithms use different updates, regularizers, feedback, or assumptions than the closed-form prime–refit map.Prior approaches include exponentiated-gradient and truncated exponential weighting for sparsity, alongside ridge and least-squares guarantees tied to norms or dimension.Source: §8 · p. 10
Batch sparse recovery
- Batch sparse recovery offers explicit ℓ1 regularization, constrained selectors, greedy recovery, iteratively reweighted least squares, and implicit-bias methods under additional conditions.These are alternatives to online prime–refit, not analyses of the same rule.Source: §8 · p. 10
Reweighting beyond one-stage priming
- Warmuth et al. (2025) and Radhakrishnan et al. (2025) give statistical guarantees for reweighting under prescribed design, noise, or structural assumptions; Theorem 10 rules out ridge repair uniformly over regularization on an adversarial triangular sequence.The linear recursive feature machine alternates reweighting and refitting, unlike the one-stage rules analyzed here.Source: §8 · p. 11
9. Scope and limitations
The section bounds the paper’s claims to three fixed-power one-stage rules and specified regularization, witness, and input conditions. It also identifies unresolved frontiers and clarifies what the frozen-activation experiments do not establish.
9. Scope and limitations
- The analysis covers three one-stage rules at fixed α ≥1 and ridge penalties on the transformed second-stage coefficient; other regularization and priming procedures remain outside scope.Excluded cases include penalties on the transformed-back coefficient, regularized first-stage priming, iterative or alternative-prime procedures, and selectors adding other predictors or corrections.Source: §9 · p. 11
- The shared witness uses d = Θα(T 2α), which is not claimed sharp; powered-univariate regret is Θ_α(min{T, d}), while unit-power multivariate and Pearson frontiers retain a d–d gap.The paired-Hadamard selector guarantee is limited to the three rules at one common fixed power, and its witness is not shared across powers or horizons.Source: §9 · p. 11
- The rank-adaptive guarantee for Pearson follows from target preservation; more generally, it requires interpolation by the transformed historical design.The half-scaled triangular family obeys ∥xt∥2, ∥e1∥2 ≤1, preserving its exact frontier under both Euclidean constraints, while normalized multivariate and Pearson separations remain open.Source: §9 · p. 11
- Frozen-activation experiments test three downstream priming maps on synthetic one-neuron labels in two real representations, not Qwen generation, natural semantic labels, or internal Transformer updates.The triangular theorem is pointwise and permits current-input-dependent regularization; the deterministic Hadamard compiler requires pre-sign ridge regularization, while its policy-uniform version is distributional.Source: §9 · p. 11
10. Conclusion … D.2. Exact equivalence on paired histories
This section summarizes the obstruction to sparse-logarithmic regret for past-only feature priming and the role of rank in controlling costly prediction. Its appendices develop the protocol, minimum-norm and target-mass identities, Hadamard-prefix constructions, and the equivalence of powered refits on completed paired histories used in the lower-bound arguments.
10. Conclusion
- Cheap nuisance interpolation makes all three past-only one-stage rules miss a sparse-logarithmic guarantee; Hadamard features force Ω(min{T, √d}) clipped regret against a zero-loss one-sparse comparator.The target weight is determined by the least transformed cost of interpolating history without the target. On paired histories the three powered refits coincide, also defeating fixed-exponent switchers and convex mixtures.Source: §10 · p. 11
- Every clipped history-interpolating predictor has regret at most 4 rank(X_1:T), and a Euclidean-normalized triangular construction matches this dependence for powered univariate priming.The triangular frontier persists under nonnegative transformed-coordinate ridge; paired Hadamard histories also give linear regret for all three powered rules under past-only ridge policies.Source: §10 · pp. 11–12
A.1. Quantifiers
- For a deterministic rule, choosing the higher-loss of two legal target signs at each decisive round compiles a fixed sequence before online play.The comparator is u = e1 with labels y_t = x_t,1, so it has identically zero cumulative loss.Source: §A.1 · p. 15, §A.7 · p. 17
A.3. Three-rule selectors
- On paired-Hadamard sign trees, the three powered coefficient vectors coincide at every node, enabling one sequence to force linear loss against their switchers and mixtures.The selector extension uses one common fixed exponent and excludes arbitrary current-input corrections.Source: §A.1 · p. 15, §A.3 · pp. 15–16
B.1. Minimum-norm identities used throughout
- The Moore–Penrose refit is equivalently the unique minimum-prime-weighted-norm solution to history interpolation, with zero-prime coordinates fixed at zero.This follows by mapping the transformed minimum-Euclidean-norm solution back to the original coefficients.Source: §B.1 · pp. 17–18
B.2. Hadamard interpolation
- Hadamard prefixes admit an explicit nuisance-only interpolant, a geometric tool used to bound target mass for univariate and Pearson priming.The multivariate LLS analysis likewise studies the first-stage prime and target mass on a Hadamard prefix with target column a and nuisance block H_r.Source: §B.2 · p. 20, §C.2.1 · p. 22
C.1. Rule-specific finite-block bounds
- The finite-block propositions give clipping-robust lower bounds for multivariate LLS, univariate, and Pearson priming against a zero-loss one-sparse comparator.Their sign-valued sequences are offline constructed, and the Pearson witness is independent of any finite totalization.Source: §A.5 · p. 17, §C.1 · pp. 21–22
C.2.1. THE PRIME AND ITS TARGET MASS
- For multivariate priming, the target prime on a Hadamard prefix is p1 = r/(N + r).The target-mass calculation uses the prefix's full-row-rank design.Source: §C.2.1 · p. 22
C.2.2. OFFLINE SIGN COMPILATION
- Once the multivariate rule, N, Hadamard rows, and tie-breaking convention are fixed, the finite deterministic recursion defines the full sign sequence before replay; every label is the first coordinate, so comparator e1 has zero loss.Source: §C.2.2 · pp. 23–24
The corresponding second-stage target coefficient obeys
- On a univariate Hadamard prefix, the compiled sign sequence yields loss at least 9/16 per charged round against the zero-loss comparator e1.The maximizing sign is selected by a finite deterministic recursion before online replay.Source: The corresponding second-stage target… · p. 25
C.4.1. THE PAIRED STATE
- On completed paired histories, Pearson primes are uniquely determined and independent of finite totalization; the target second-stage coefficient is bounded by s^2/N.Every feature and the label has positive empirical variance, so totalization has no freedom on these coordinates.Source: §C.4.1 · p. 26, The second-stage target coefficient… · p. 26
C.4.2. THE TOTALIZATION-UNIFORM CONSTRUCTION
- Each Pearson query incurs clipped loss at least 9/16, each opposite recovery incurs zero loss, and the sequence works for every finite past-only totalization.A two-round seed establishes paired histories; the query–recovery recursion gives the lower bound against comparator e1.Source: The second-stage target coefficient… · pp. 26–27, §C.4.2 · p. 27
C.5. From finite blocks to arbitrary horizons and dimensions
- The arbitrary-horizon construction refutes a uniform sparse-logarithmic regret bound even for k = 1, for clipped and raw predictions.The argument takes dimensions of order T^2; clipping cannot increase square loss.Source: §C.5 · p. 29
Appendix D. Proof of the powered shared-witness theorem
- The powered-rule appendix states individual target-mass bounds for powered univariate and multivariate refits, and a powered Pearson bound independent of finite totalization.It also announces that the three powered refits are exactly the same vector on completed paired histories.Source: Appendix D · p. 29, §D.1 · p. 30
D.1. Individual powered target-mass certificates
- The individual powered target-mass bounds allow rule-dependent adverse signs, while the shared result identifies the entire coefficient vector rather than only its first coordinate.The Pearson calculation also notes that its correlations are defined and the bound is independent of Pearson totalization.Source: §D.1 · p. 31
D.2. Exact equivalence on paired histories
- On every nonempty completed paired history, all three powered second-stage coefficient vectors are identical for a common α ≥ 1; at empty history, all are zero for any finite Pearson totalization.The paired-history calculation is valid when the Pearson correlations are defined.Source: §D.2 · pp. 31–32
- The shared powered target-mass and two-sign loss result applies when 0 ≤ s < M and N ≥ 8M^2α, using the common coefficient before the next query.For a fixed next nuisance row, the argument considers clipped loss for the two possible signs of the current input.Source: §D.2 · p. 32
D.3. Oblivious compilation and selector robustness … Appendix F. Proof of the powered-univariate exact frontier
A single obliviously constructed sequence forces linear clipped regret for every three-rule selector, while realizable-sequence loss for interpolating powered rules is bounded by the design rank. The proofs connect regret to identical selector predictions and to positive-loss rounds spending new row-space directions.
D.3. Oblivious compilation and selector robustness
- A finite greedy recursion fixes one sign string before play, and identical powered coefficient vectors make all selector mixtures incur at least 49/64 on each query.The recursion chooses between the two legal signs on each fixed nuisance row and appends its exact opposite; the bound holds for every random-tape realization and finite Pearson totalization.Source: §D.3 · p. 33
- 49M/64 clipped regret on a length-2M sequence rules out a uniform O(k log(ed/k)) guarantee even for a zero-loss one-sparse comparator.The bound applies to all three-rule selectors; the unsigned and signed power conventions also cover literal squaring at α = 2.Source: §D.3 · pp. 33–34
- Independent uniform query signs retain the same regret inequality for each fixed selector random tape and after averaging over any tape law.The query is paired with its exact opposite after each round.Source: §D.3 · pp. 33–34
Appendix E. Proof of the rank upper bound
- For realizable labels and an interpolating coefficient, cumulative clipped squared loss is at most 4 rank(X1:T), hence at most 4 min{T, d}.Any positive-loss round must add a new row-space direction, and each such round contributes at most four.Source: §E.1 · pp. 34–35
E.1. Rank spending and interpolation
- Powered univariate and multivariate refits interpolate labels y = Xe1 on every history, so they satisfy the rank-bound interpolation condition.A powered Pearson refit also interpolates when its target prime is nonzero for y ≠ 0.Source: §E.1 · p. 35
- For Pearson priming, interpolation holds when the target prime is nonzero; more generally, it is exact when y<t lies in col(X<tDt).The target-preserving convention uses the nonzero-target-prime sufficient condition.Source: §E.1 · p. 35
F.1. A linear-dimensional triangular construction … G.3. Hadamard ridge certificates for all three rules
These sections develop constructions establishing lower bounds for powered feature-priming rules under ridge regularization. They cover rank-matched linear regret for powered univariate priming and Hadamard-based ridge obstructions for all three rules, including paired constructions with tailored witnesses and common bounds.
F.1. A linear-dimensional triangular construction
- The triangular design is nonsingular, adds one row-space direction each active round, and gives powered-univariate priming a normalized regret order of Θ_alpha(min{T, d}).Half-scaling makes every input norm at most one while retaining at least one quarter of each active clipped loss.Source: §F.1 · pp. 36, 38
G.1. A resolvent form of target mass
- The target-mass resolvent gives a nonnegative shrinkage fraction below one for positive ridge and extends continuously to the Moore–Penrose endpoint.The passage also establishes a nuisance certificate and monotonicity for the resolvent quantity.Source: §G.1 · p. 40
G.2. The triangular sequence for every ridge schedule
- For powered-univariate priming, every nonnegative ridge schedule incurs at least (c_alpha/4) min{T, d} clipped loss on the normalized triangular sequence.The bound holds pointwise in the ridge value, including schedules dependent on history, the current input, or a fixed realization of internal randomness.Source: §G.2 · p. 43
G.3. Hadamard ridge certificates for all three rules
- On the completed-pair Hadamard history, each powered univariate, Pearson, and multivariate prime admits a nuisance-only transformed coefficient satisfying a ridge certificate.The Pearson certificate is uniform over every finite totalization.Source: §G.3 · p. 43
- For each powered rule with α ≥ 1 and deterministic history-measurable ridge schedule, a fixed-before-replay paired-Hadamard sequence gives a lower bound against the zero-loss e1 comparator.The construction uses T = 2M and d = N + 1, with 8M^(2α) ≤ N < 16M^(2α); the witness may depend on the rule and ridge policy.Source: §G.3 · pp. 44–45
- For Pearson priming, the charged losses hold for every finite totalization because charged states follow completed opposite pairs with positive active variances.Only the first uncharged recovery can encounter a one-sample undefined correlation.Source: §G.3 · pp. 44–45
- The deterministic witness requires the ridge policy to be fixed before the current target sign and is rule–policy specific, whereas the common result is distributional.The deterministic result does not cover a current-input-dependent ridge choice.Source: §G.3 · p. 45
Appendix H. Frozen-activation experiment details
Frozen-activation experiments test whether nuisance interpolation accompanies declining target weight and rising regret. The Qwen3.8 replication meets the prespecified mechanism criterion on 17 of 18 targets.
Representation and target selection
- The experiment uses post-gating activations from Qwen2.5-7B-Instruct layers 6, 13, and 20, selecting six eligible target neurons per layer from disjoint data pools.Each label is the target activation, so the fixed one-coordinate comparator has zero loss; predictions are strict-past and clipped to [−1, 1].Source: Appendix H · p. 46
Cross-generation replication
- On compiled orders, real nuisance coordinates raised median regret to 1.238, 0.303, and 0.587, versus 0.029 for target-only inputs under each rule.Independently row-permuting nuisance coordinates yielded medians of 0.799, 0.306, and 0.555; exact best-one-sparse ERM selected the target throughout the second half and had median normalized regret 0.058.Source: Appendix H · p. 47
- 17 of 18 Qwen3.8 targets met the full mechanism criterion, with random-order median normalized regrets of 0.805, 0.796, and 0.774 for univariate, multivariate, and Pearson priming.The one failure had normalized regret 1.215 but second-half target weight 0.507, just above the 0.5 cutoff; the one-sparse comparator had zero error.Source: Appendix H · p. 46, Table 1 · p. 47
Fresh dimension–horizon diagnostic
- At T = 48, widening nuisance width from 16 to 2,048 raised median regret by 0.619 and lowered target weight by 0.809 across 18 Qwen2.5 targets.All 18 targets passed the prespecified comparison; at width 2,048, row-permuted and Rademacher controls had median regrets 0.857 and 0.810, versus 0.773 for real activations on the corresponding subset.Source: Appendix H · p. 46, Figure 2 · p. 47