Uncovering Latent Reasoning Strategies in Language Models

Awni Altabaa; John Lafferty

Source-linked AI summary

Uncovering Latent Reasoning Strategies in Language Models

Awni Altabaa, John Lafferty

arXiv:2607.17674v1Jul 20, 2026cs.LGcs.AI

TL;DR

Language models use multiple implicit, entangled reasoning strategies, making them difficult to identify and control. This paper learns a strategy-conditioned router-generator factorization with model-directed reconstruction, recovering strategy-aligned latent codes while preserving the base model’s response distribution.

Problem
Language models’ response distributions entangle multiple reusable reasoning strategies without a variable that identifies or controls which strategy generates each response.
Method
The paper learns a router-generator factorization using base-model-normalized reconstruction and token weighting that concentrates pressure on high-surprisal positions.
Results
The objective recovers latent codes aligned with reference strategies and semantically consistent across inputs while preserving the base model’s response distribution.
Takeaways & Limitations
The factorization makes strategy information transferable across problems while retaining the pretrained model’s observable response behavior.
Takeaways & Limitations
The evidence comes from synthetic algorithmic benchmarks with ground-truth strategies, so extension to open-ended reasoning tasks remains an open question.

Abstract

from arXiv · show

A language model $p_θ(y \mid x)$ trained on reasoning tasks learns to solve problems via multiple distinct strategies, yet these strategies are implicit and entangled within the model's response distribution. We study the problem of decomposing the response distribution of a given pretrained language model into a structured, strategy-conditioned representation. Specifically, we learn a latent-variable factorization $p_θ(y \mid x) \leadsto (r_φ(z \mid x), g_φ(y \mid x,z))$, where a router $r$ maps each input to a distribution over latent strategies $z$ and a generator $g$ produces the response conditioned on that strategy. A key challenge is that the generator, initialized from the base model, already represents $p_θ(y \mid x)$ without using $z$. Standard variational inference therefore gives the model no incentive to route information through $z$ and can yield a severe form of posterior collapse. To address this, we propose a variational objective that measures fractional information gain relative to the base model's response loss and concentrates reconstruction pressure on tokens with high base model surprisal, encouraging $z$ to encode strategy-relevant response variation. We introduce a benchmark of multi-strategy algorithmic tasks and show that this objective recovers latent codes aligned with distinct reference strategies while preserving the base model's response distribution.

1 Introduction

The paper seeks to expose multiple reasoning strategies implicitly entangled in a pretrained language model by learning a strategy-conditioned router-generator factorization. It addresses posterior collapse with model-directed reconstruction pressure that preserves the base response distribution while making latent codes informative and strategy-aligned.

Motivation: Reasoning models can solve the same problem through recurring strategies that remain implicit and entangled in the response distribution.Examples include proof by contradiction, induction, construction, dynamic programming, graph search, and greedy algorithms.
Problem formulation: The proposed representation factorizes pθ(y|x) into a router rϕ(z|x) and generator gϕ(y|x,z), while requiring z to be informative and strategy-aligned.The same latent value should carry the same strategy meaning across related inputs, while varying z changes the generator’s strategy.
Posterior collapse: Because the pretrained generator already models pθ(y|x) without z, standard variational objectives can preserve distributional fidelity while leaving the latent variable inert.Ignoring z is available at initialization and globally optimal for objectives that only fit the observable response distribution.
Methodology: The method uses model-directed reconstruction pressure normalized by the base model’s response loss and concentrated on tokens with high base-model surprisal.This focuses learning on positions where multiple continuations remain plausible, which tend to be strategy branch points.
Benchmark and empirical evaluation: The paper introduces a multi-strategy algorithmic benchmark with withheld ground-truth strategy annotations and studies recovery conditions, ablations, and failure modes.Evaluation covers both pretrained and randomly initialized language models.
Results and broader promise: Experiments show that strategy information exists in base-model hidden states, while model-directed pressure recovers latents aligned with reference strategies and semantically consistent across inputs.The paper also initiates theoretical analysis of conditions for identifying useful factorizations and highlights applications to interpretability, controllability, and exploration.

2 Problem Statement: Latent-Variable Factorization

The paper factorizes a pretrained model’s response distribution into a router over latent strategies and a strategy-conditioned generator. The factorization must preserve observable responses while making the latent informative, reusable, and consistent across inputs.

The router-generator factorization introduces z, with r_φ(z|x) selecting latent strategies and g_φ(y|x,z) generating responses conditioned on them.The base model distribution remains the target observable distribution.
A successful factorization preserves p_φ(y|x) ≈ p_θ(y|x) while making z informative about y beyond x and aligned with reusable response strategies.The latent should expose high-level strategy variation rather than arbitrary predictive surface details.
Matching only the latent marginal is insufficient because multiple factorizations can reproduce the base distribution, including solutions that ignore z or encode surface-form details.Expressive router-generator classes make marginal agreement compatible with uninformative or non-strategic latents.
The latent must support both within-input strategy separation and stable cross-input semantics, so the same latent value or region retains its strategy meaning across related problems.Local variation alone can produce input-specific codes whose interpretations change across inputs.
Controlled experiments with parseable response traces evaluate whether learned latents achieve within-input strategy separation and cross-input semantic consistency.Reference strategy labels are withheld during factorization training and used only for evaluation.

3 Learning Strategy-Aligned Latent-Variable Factorizations

The section develops a lightweight router–generator factorization of a pretrained language model and explains why standard variational training can leave its latent strategy variable unused. It then introduces a base-model-directed reconstruction objective that concentrates learning on high-surprisal response tokens, encouraging strategy-relevant latent information while retaining response fidelity.

Architecture: A router maps each input to a latent-strategy distribution, while a strategy-conditioned generator produces responses using a latent embedding.Both are adapted from a pretrained Transformer; the router and generator share parameters, while a separate posterior is used only during variational training.
Architecture: The factorization uses small additions—Gaussian latent heads, a latent embedding projection, and LoRA-style updates—so the generator can inherit the base model’s response distribution.The inherited distribution also makes a z-ignoring factorization available near initialization.
Failure of standard variational training: Standard variational training is under-specified because matching the marginal response distribution does not require z to explain response variation or align with strategies.This creates latent collapse, with fidelity preserved while posterior-router KL falls and strategy alignment remains low.
Surprisal-directed objective: The proposed objective samples responses from the fixed base model and uses token-level base-model surprisal to direct reconstruction pressure toward positions where z can explain variation beyond the z-free prefix model.Higher-surprisal branch points receive more weight than determined positions, and no strategy annotations are used during training.
Surprisal-directed objective: The reconstruction term is normalized in base-model reference units and interpreted as a weighted fraction of response loss remaining after conditioning on z, balancing reconstruction against the KL penalty.The interpolation parameter α mixes uniform and surprisal-directed pressure, while γ controls concentration of token weighting.

4 Experiments

Experiments use a controlled six-family algorithmic benchmark to test distributional fidelity and strategy alignment, including cross-input semantic consistency. The results show that strategy information exists implicitly in base-model states, but the proposed model-directed reconstruction objective is needed to expose aligned, reusable latents while preserving response behavior.

Benchmark and protocol: The benchmark contains six task families with parseable solution traces that instantiate multiple procedures for the same valid answer, enabling ground-truth strategy-alignment evaluation.The families are list summation, sorting, grid pathfinding, linear equation solving, base conversion, and multidigit addition.
Benchmark and protocol: The two-stage protocol first fits a base model whose response distribution mixes strategies, then trains a router-generator factorization on samples from that fixed distribution.Evaluation separates Distributional Fidelity, which tests preservation of valid base-model behavior, from Strategy Alignment, which tests consistent latent assignments across inputs.
Implicit strategy information: Strategy information is broadly decodable from base-model hidden states, especially where strategy-specific computation appears, but it is not exposed as a reusable latent variable.Linear probes predict reference strategies at each task and token position.
Objective comparison: ELBO-style baselines can preserve Distributional Fidelity while leaving the latent inert or strategy-unaligned because matching responses does not require explaining strategy variation.This demonstrates that implicit strategy information alone does not make standard variational inference identify the desired latent structure.
Objective comparison: The model-directed reconstruction objective and its variants recover strategy-aligned latents while preserving the base model’s response distribution.Strategy Alignment is measured by Analogical Consistency, Pr[strat(Y) = strat(Y′)], after reusing the same latent across related inputs.
Mechanism and ablation: Token-weighted reconstruction loss tracks Strategy Alignment across methods, supporting the mechanism that alignment improves when z explains high-surprisal, strategy-relevant residual variation.An inverse-surprisal ablation that emphasizes low-surprisal tokens weakens Strategy Alignment.

5 Toward a Theory of Strategy-Structure Recovery

This section develops a theoretical framework for when the variational approach recovers a latent variable aligned with a target strategy structure implicit in a pretrained model. Its objective balances information pressure with an effective complexity penalty, yielding population and empirical recovery guarantees.

Motivation: The analysis asks when variational inference can identify a latent variable Z aligned with a target strategy structure S⋆ implicit in the pretrained model.The formal treatment is provided in Appendix F.
Variational objective: The complexity-penalized objective combines reconstruction loss, encoder-to-router KL regularization, and a complexity term C(r, g) capturing architectural, parameterization, and optimization biases.The objective is parameterized by β ∈ [0, 1) and λ ≥ 0.
Variational objective: The objective decomposes into conditional response entropy, information pressure Iq(Y ; Z | X), and an effective approximation-complexity penalty Aβ,λ(q).Information pressure encourages Z to capture information about Y beyond what is predictable from X, while Aβ,λ(q) measures target-distribution fidelity under simplicity constraints.
Recovery guarantees: The primary result gives sufficient conditions ensuring that every population minimizer has recovery discrepancy d(bq, S⋆) < δ from the targeted strategy.The target strategy is defined as S⋆ = f(X, Y), with q⋆(z|x, y) = 1{z = S⋆}.
Recovery guarantees: Under the corresponding empirical-objective condition, every empirical minimizer likewise satisfies d(bqn, S⋆) < δ.The result applies to empirical minimizers of bLβ,λ,n.

6 Conclusion … C Benchmark and Data Generation

The paper frames latent strategy recovery as a factorization problem complicated by posterior collapse, and addresses it with model-directed reconstruction. It evaluates the approach on controlled synthetic benchmarks while emphasizing limitations in generalization and theory.

6 Conclusion: A pretrained model’s response distribution entangles multiple reasoning strategies, so recovering explicit latent variables requires more than fitting observable outputs.The base generator already realizes the response distribution, creating a challenge for latent-variable factorization.
6 Conclusion: Standard variational training can preserve responses while leaving the latent unused; the proposed objective normalizes reconstruction by base-model loss and weights uncertain tokens.This concentrates reconstruction pressure where strategy choices are most consequential.
6 Conclusion: The benchmark supports measurable strategy recovery but limits conclusions because it uses synthetic algorithmic tasks with ground-truth strategies.Extension to open-ended reasoning remains unresolved, and the theoretical connection between oracle selection and training dynamics is preliminary.
A.1 Related Work: The work differs from related latent-variable, mixture-of-experts, and mechanistic-interpretability methods by learning a generative factorization aligned with interpretable strategies.Related approaches commonly target latent variation, capacity, efficiency, or post-hoc internal representations rather than strategy-conditioned generation.
A.1 Related Work: Classical identifiability guarantees generally fail with expressive pretrained generators, making useful factorization selection central among many decompositions preserving the marginal distribution.The paper connects this issue to classical mixture models and nonlinear ICA, which rely on restrictions or auxiliary observations.
A.2 Implications and Future Directions: The proposed factorization may support interpretable strategy comparisons and interventions, but its conclusions remain limited to synthetic benchmarks and preliminary theory.These implications motivate future work in interpretability, directed exploration, and safety.
C Benchmark and Data Generation: The controlled benchmark supplies instances, solution traces, and reference labels where multiple procedures yield the same answer but differ in intermediate traces.This enables direct measurement of whether latent variables correspond to strategy-level variation.

C.1 Task families … D.2 Objective and Normalization Details

The paper evaluates strategy-conditioned factorization on a controlled benchmark spanning multiple algorithmic strategy families, with reference traces and parsers for assessing validity and strategy alignment. Training adapts a router-generator Transformer around a frozen base model and uses surprisal-weighted, base-normalized reconstruction with a KL term.

C.1 Task families: The benchmark covers complementary strategy variation in summation, sorting, grid paths, linear equations, base conversion, and multidigit addition.Each family samples uniformly from listed strategies, which serve as reference labels rather than an exhaustive catalog of procedures.
C.2 Data generation: Examples are generated by sampling a task family, instance, and reference strategy, then rendering a valid solution trace that reaches the correct answer.The reference strategy label is used for evaluation but not training.
C.3 Reference strategy identification: A task parser checks generated traces for validity and identifies the benchmark strategy they express, enabling evaluation of behavioral preservation and latent-strategy alignment.This connects output validity with whether the learned latent variable reflects the benchmark’s reference structure.
C.4 Representative examples: Representative traces demonstrate distinct procedures for each task, including left-to-right versus pairwise summation and bubble versus merge sort.The examples also show right-first versus alternating grid paths, alternative equation-solving orders, repeated division versus decomposition, and two multidigit-addition procedures.
D Details of Training Methodology: Training initializes a router-generator factorization from the base model, freezes pθ as the response distribution and reconstruction reference, and updates adapted parameters while leaving θ fixed.The procedure outputs a router rϕ(z|x) and strategy-conditioned generator gϕ(y|x,z).
D.1 Architecture Parameterization: The router and generator are two roles of one adapted autoregressive Transformer, with a Gaussian router latent projected into a pseudo-token that conditions generation.The posterior reads the full input-response sequence during training, while generation samples latents from the router.
D.1 Architecture Parameterization: Both adapted models start from frozen base parameters and train LoRA-style backbone updates together with Gaussian heads and the latent embedding projection.This keeps the factorization close to the base model while allowing z to receive a nontrivial role.
D.2 Objective and Normalization Details: Surprisal-directed token weights mix uniform and high-surprisal pressure, while the objective combines base-normalized reconstruction Rθ with β-weighted KL divergence between posterior and router.If the generator matches pθ, the normalized reconstruction has expectation 1 and represents the fraction of frozen-base response loss remaining after latent conditioning.

D.3 Inference And Evaluation Interface · E Experimental Details

The posterior is used only during training to support latent reconstruction and bridge the generator and router. At inference, the factorization samples a strategy from the router and generates autoregressively, while strategy annotations are reserved for post-training evaluation.

D.3 Inference And Evaluation Interface: The posterior qξ(z|x, y) is a training-time object and is not needed to sample from the learned factorization.
D.3 Inference And Evaluation Interface: During training, the posterior supplies latent samples for reconstruction.
D.3 Inference And Evaluation Interface: The posterior provides a variational bridge between the generator and router.
D.3 Inference And Evaluation Interface: At inference, the model first samples z from rϕ(·|x).
D.3 Inference And Evaluation Interface: It then generates y autoregressively from gϕ(·|x, z).
D.3 Inference And Evaluation Interface: Strategy annotations enter only after training to evaluate latent informativeness, strategy alignment, within-input separation, and semantic consistency across inputs.

E.1 Evaluation protocol and metrics · E.2 Objective variants and inverse-surprisal ablation

The evaluation tests whether factorized models preserve valid task behavior and assign reusable strategy meaning to latent values across related inputs. Objective variants isolate how base-relative and surprisal-directed reconstruction pressure affects this factorization.

E.1 Evaluation protocol and metrics: Generated traces are parsed against benchmark reference structures to evaluate task behavior and stable latent-strategy meaning; strategy labels are used only for evaluation.This protocol directly assesses whether learned latents preserve benchmark-defined strategy structure without using labels during training.
E.1 Evaluation protocol and metrics: Distributional Fidelity reports the fraction of generated samples that remain strategy-compatible benchmark solutions under router sampling and latent-conditioned generation.It measures preservation of base-model response behavior at the level of valid task solutions.
E.1 Evaluation protocol and metrics: Analogical Consistency reuses a latent sampled for one input on a related target input, testing whether that latent retains the same strategy meaning across inputs.The target input is sampled from the same task family as the source input.
E.1 Evaluation protocol and metrics: Router Strategy Decodability linearly probes reference-strategy information in router-sampled latents, but is weaker than Analogical Consistency because it omits cross-input reuse.Supporting diagnostics also include base-relative reconstruction, token-weighted reconstruction, and posterior-related measures.
E.2 Objective variants and inverse-surprisal ablation: The objective variants share J = R + β KL(qξ(·|x, y) ∥ rϕ(·|x)), differing in reconstruction term while keeping the frozen base model as reference.Standard ELBO uses unnormalized autoregressive reconstruction; proposed variants normalize and direct pressure relative to pθ.
E.2 Objective variants and inverse-surprisal ablation: Uniform base-relative reconstruction uses aθ,t = 1/Ty, whereas surprisal-proportionate weighting concentrates pressure using concentration-corrected frozen-base surprisal bθ,t(x, y).Global+token variants mix uniform and high-surprisal token-weighted terms convexly.
E.2 Objective variants and inverse-surprisal ablation: The inverse-surprisal ablation replaces aθ,t with vθ,t to emphasize well-predicted tokens while preserving z-free reconstruction scale, testing whether gains require high-surprisal regions.Its mechanism-level target is to distinguish strategy-relevant pressure from nonuniform token weighting alone.

E.3 Training settings and hyperparameter sweeps · E.4 Additional diagnostics · F A Theory of Variational Approaches to the Strategy-Structure Recovery Problem

The appendix describes multi-task and diagnostic training settings, benchmark model initializations, and a broad hyperparameter sweep. It also presents additional diagnostics and formalizes the population-ELBO theory underlying the strategy-structure recovery result.

E.3 Training settings and hyperparameter sweeps: The default setting jointly trains one router-generator model across all six benchmark task families.Additional single-task runs support diagnostic sweeps and mechanism plots.
E.3 Training settings and hyperparameter sweeps: The multi-task regime is chosen to better match language models that represent heterogeneous task distributions.The passage contrasts this default with single-task diagnostic runs.
E.3 Training settings and hyperparameter sweeps: Benchmark base models use either pretrained Qwen2.5 checkpoints or random weights before fitting to the benchmark task distribution.The resulting base model p_θ is then frozen and factorized.
E.3 Training settings and hyperparameter sweeps: The randomly initialized diagnostic grid contains 504 complete LoRA adaptation runs across six single-task settings and one multi-task setting.The grid also varies twelve reconstruction-family settings, three KL weights β ∈ {1, 0.1, 0.01}, and two schedul…
E.4 Additional diagnostics: Additional diagnostics examine latent geometry over training, the role of KL weights, metric associations, and phase-space training dynamics.These views support the same claims as the headline pretrained method comparison and compact mechanism diagnostics.
F A Theory of Variational Approaches to the Strategy-Structure Recovery Problem: The appendix provides the formal treatment behind the main-text preview theorem.Its main result studies the population ELBO over independently parameterized expressive encoder, router, and generator classes.

F.1 Setup and the Strategy Structure Recovery Problem · F.2 ELBO Decomposition with Router-Generator Complexity · F.3 Profiled Router-Generator Objective

The paper formalizes strategy recovery through latent encoders, distinguishing input-dependent weak recovery from globally consistent strong recovery. It then decomposes and profiles the ELBO, showing how conditional information, router-generator approximation, and complexity determine encoder selection.

F.1 Setup and the Strategy Structure Recovery Problem: The setup defines finite input and response spaces, a K-state latent variable, and encoder, router, and generator classes linked by an encoder-induced joint law.The joint law is Qq(x, y, z) = µ(x)p(y | x)q(z | x, y).
F.1 Setup and the Strategy Structure Recovery Problem: Weak recovery permits an input-dependent permutation of latent labels, whereas strong recovery requires one global permutation shared across inputs.Null labels are ignored, and approximate recovery is measured with corresponding conditional or unconditional variation-of-information distances.
F.1 Setup and the Strategy Structure Recovery Problem: The recovery distances exactly characterize their targets: dw(q, S⋆) = 0 if and only if weak recovery holds, while ds(q, S⋆) = 0 if and only if strong recovery holds.Strong recovery implies weak recovery because conditioning reduces entropy, but the converse need not hold when latent meanings vary by input.
F.2 ELBO Decomposition with Router-Generator Complexity: The ELBO decomposes into irreducible conditional entropy, a conditional-information term, router and decoder KL gaps, and router-generator complexity.The objective is Lβ,λ(q, r, g) := EQq[−log g(Y | X, Z)] + β EX,Y KL(q(· | X, Y )∥r(· | X)) + λC(r, g).
F.2 ELBO Decomposition with Router-Generator Complexity: The coefficient of Iq(Y ; Z | X) is β −1, so the ELBO seeks information when β < 1, is neutral to it at β = 1, and penalizes it when β > 1.At β = 1, encoder selection can still occur through the profiled cost.
F.3 Profiled Router-Generator Objective: Profiling fixes q and takes the infimum over available routers and generators, producing the encoder-only cost Aβ,λ(q) and profiled population loss Lβ,λ(q).Aβ,λ(q) combines DecGap(q, g), β PriorGap(q, r), and λC(r, g).
F.3 Profiled Router-Generator Objective: The profiled ELBO score is Lβ,λ(q) = H(Y | X) −(1 −β)Iq(Y ; Z | X) + Aβ,λ(q), combining information gain with profiled approximation-complexity cost.Aβ,λ(q) measures whether encoder-induced routing laws and branch conditionals can be approximated by a simple router-generator pair.

F.4 Metric Recovery Margins

The section characterizes strategy recovery through a profiled-score margin: a positive margin forces population ELBO minimizers into the desired recovery neighborhood. It further gives information–complexity conditions for positive margins, explains why strong recovery needs globally aligned labels, and shows when approximate recovery becomes exact.

Population recovery criterion: A positive profiled margin at recovery radius δ ensures every population ELBO minimizer recovers the reference strategy within distance δ.Recovery is controlled by the profiled-score gap against encoders outside the desired neighborhood.
Information–complexity separation: A bad encoder can have up to γI more conditional information, but recovery still follows when its additional profiled cost satisfies γA > (1 −β)γI.The sufficient separation condition makes the tradeoff between conditional information and router-generator complexity explicit.
Weak versus strong recovery: Input-dependent label permutations can preserve weak recovery and conditional information while preventing strong recovery unless the profiled cost favors globally consistent labels.If such a relabeling ties the target’s profiled cost, no positive strong margin can separate it.
Exact recovery: A positive distance gap mρ around the target equivalence class upgrades approximate recovery to exact weak or strong recovery whenever Theorem 1 holds for δ ≤mρ.Every non-recovering encoder lies at distance at least mρ, so a minimizer within δ must be equivalent to the target.
Failure modes: When profiled costs are free, superficial high-information attributes can defeat semantic recovery, and free input-dependent relabelings can defeat strong recovery despite preserving weak recovery.These failure cases occur respectively when λ = 0 with Aβ,λ(q) = 0 and when the objective is invariant to local relabeling.

F.5 Marginal Fidelity and the Small-Complexity Limit · F.6 Empirical Recovery

The paper bounds marginal fidelity by the router and decoder variational gaps, showing that exact factorization is recovered as complexity regularization vanishes under realizability. Empirical profiled-ELBO minimizers recover the target encoder set when uniform estimation error is smaller than the information-complexity margin, with finite-class and bounded-log-loss conditions providing sufficient guarantees.

F.5 Marginal Fidelity and the Small-Complexity Limit: Proposition 3 bounds marginal KL by PriorGap(q, r) plus DecGap(q, g), making the fidelity cost of approximate factorization explicit.The bound follows from data processing for KL divergence and decomposing the joint KL into router and conditional-generation terms.
F.5 Marginal Fidelity and the Small-Complexity Limit: For finite λ, the profiled optimizer may choose a simpler router-generator pair with nonzero variational gaps and imperfect marginal factorization.Complexity can therefore trade off against exact fidelity in the profiled objective.
F.5 Marginal Fidelity and the Small-Complexity Limit: As λ ↓0, exact realizability forces both variational gaps and marginal KL to converge to zero.The exact induced pair has finite complexity and zero prior and decoder gaps, which supplies the comparison needed for the limit.
F.5 Marginal Fidelity and the Small-Complexity Limit: The λ ↓0 limit weakens complexity’s selection role, potentially favoring superficial high-information partitions unless complexity breaks relevant near-ties.The profiled encoder score approaches maximum conditional information in this limit.
F.6 Empirical Recovery: Empirical recovery holds when the empirical profiled ELBO approximates the population profiled ELBO more accurately than the information-complexity loss gap.The theorem is deterministic conditional on a uniform convergence bound; finite samples must preserve the relevant margin in loss units.
F.6 Empirical Recovery: For a finite encoder class, uniform concentration and a union bound imply dρ(bqn, S⋆) < δ for every empirical profiled-ELBO minimizer with probability at least 1 −η.The guarantee applies when the stated sample-size lower bound makes the union-bound failure probability at most η.
F.6 Empirical Recovery: With finite Q, R, G, finite complexity, and support floor τ, bounded log losses yield the same dρ(bqn, S⋆) < δ recovery guarantee with probability at least 1 −η.The data-dependent loss is bounded between 0 and (1 + β) log(1/τ), enabling Hoeffding concentration and a union bound over Q × R × G.
F.6 Empirical Recovery: Support floors are sufficient but not necessary; clipping, tail bounds, direct concentration, or complexity control can instead deliver uniform accuracy at scale (1 −β)∆ρ,prof β,λ (δ).The recovery argument requires the uniform-accuracy event, while support or tail assumptions are only ways to obtain it.