Principled Thoughts for Latent Recursive LLM Systems

1 INTRODUCTION

• Textual chain-of-thought confines reasoning to paths expressible in words, and its long traces raise inference cost and can allocate excessive computation to simple problems.Prior work has explored latent reasoning within a model and latent communication between models, often training only on final-answer CE.Source: §1 · p. 2
• CE-only training can leave thoughts unconstrained once the correct answer retains high probability; valid thoughts should preserve needed information, discard irrelevant information, distinguish distinct inputs, and reflect output distributions.An audit found that every examined latent representation violates at least one of these properties.Source: §1 · p. 2
• REST converts causality, minimality, separability, and stability into differentiable losses added to CE, targeting thoughts as well as decoded answers.It extends latent recursive systems to a single agent recurring on its own hidden states and requires no architectural changes or added inference parameters.Source: §1 · pp. 2–3
• +3.5 points in the multi-agent setting and +3.3 in the single-agent setting are the average accuracy gains over CE-only; best configurations reach +7.5 and +6.5 points, respectively.These comparisons use matched training budgets, the same training data, and the same latent budget.Source: §1 · pp. 2–3

2 BACKGROUND

• A latent thought is produced by mapping an agent’s hidden states through an inner link and then an outer link; the inner link returns hidden states to its own input space, while the outer link maps them to another agent’s input space.The base agents are frozen; the inner link is trained to align hidden states with ground-truth text embeddings, while outer links are trained through final-answer CE.Source: §2 · p. 3
• In the three-agent chain, the planner proposes an approach, the refiner revises it, and the solver answers; each handoff transfers latent thoughts to the next agent.During training, producer hidden states come from a teacher-forced ground-truth output; at inference, the producer takes latent steps from its prompt because that output is unavailable.Source: §2 · p. 3
• After the solver finishes, its thought returns to the planner for another round; only the final solver decodes an answer as text, while earlier hops remain latent.Later agents condition on both their own input context and the thought they receive.Source: §2 · p. 4

2.1 LIMITATIONS OF ANSWER-LEVEL SUPERVISION

• The substitution divergence measures the cost of replacing producer text with a thought, and a divergence reduces the consumer’s correct-target probability by a factor of exp(−nΛ(v)), compounding across hops.For the same divergence per token, longer targets incur a larger penalty; CE-only also falls short of the accuracy achieved with the producer’s oracle text.Source: §2.1 · p. 4

3 REST: REPRESENTATION-SUPERVISED THOUGHT(S)

• REST trains only the outer link Rψ while keeping the agents and inner link frozen, and retains cross-entropy in the objective for legible text answers.Each property loss is added at every transfer alongside the unchanged system cross-entropy.Source: §3 · p. 5, §3.2 · p. 6

3.1 PRINCIPLED THOUGHTS

• Causality preserves the consumer’s predictions across latent and textual transfers, while minimality removes producer-input information irrelevant to its output.The minimality loss uses consumer cross-entropy terms for the producer’s output and input conditional on output and thought.Source: §3.1 · p. 5
• Separability penalizes similarity between pooled thoughts for the current example and preceding training examples, while stability trains a probe to estimate the producer’s mean predictive entropy from the thought.The stability target represents the producer’s output distribution rather than a single sampled sequence.Source: §3.1 · p. 5

3.2 MULTI-AGENT SYSTEMS

• At each transfer, REST adds one property-loss term averaged over transfers while leaving the system’s cross-entropy objective unchanged.Each transfer’s property term receives its own gradient in addition to the final cross-entropy.Source: §3.2 · p. 6

3.3 SINGLE-AGENT SYSTEMS

• REST also applies when a single solver serves as both producer and consumer, recurring on its latent thought without planner or refiner agents.The solver uses the frozen inner link for latent steps and the trained outer link once per round.Source: §3.3 · p. 6

4.1 EXPERIMENTAL SETUP

• The evaluation uses Sequential-Math training data and open-weight Qwen, Llama, and Gemma models, compares against CE-only and auxiliary-loss baselines, and averages results across three seeds.All agent parameters are frozen during training; systems are reported at their best-performing recursion round against CE-only at the same round.Source: §4.1 · pp. 6–7, Table 1 · p. 6

4.2 SINGLE-AGENT EVALUATION

• Every single-agent property term improves accuracy over CE-only, with minimality delivering the largest gain in the single-agent setting.In Light, minimality exceeds the baseline on every task while decoding fewer tokens overall.Source: Table 2 · p. 7, §4.2 · p. 7

4.3 MULTI-AGENT EVALUATION

• Every Light multi-agent setting improves accuracy over CE-only; in Scaled, every setting except separability does so, while separability instead reduces tokens.In Light, causality or minimality matches or surpasses the strongest frozen agent on all three mathematical benchmarks, where CE-only underperforms it.Source: Table 3 · p. 7, §4.3 · pp. 7–8

4.4 ANALYSIS

• Additional recursion rounds increase REST’s gain for Scaled agents but reduce it for Light agents, in both single-agent and multi-agent systems.The analysis interprets this as larger agents using additional latent-exchange rounds, while smaller agents obtain their gain from one round.Source: Figure 5 · p. 8, §4.4 · p. 8
• REST retains an accuracy advantage over CODI and SIM-CoT in both systems, and a token-overhead advantage in the Light system.Unlike those baselines, which align latent states to producer-written text, REST constrains properties of the thought representation.Source: Table 4 · p. 8, §4.4 · p. 8
• REST thoughts encode more of the producer’s plan or refined plan and more useful information for the solver than CE thoughts.CE thoughts contain relatively more input content, which may be irrelevant to the next agent; REST’s advantage grows further along the latent-thought chain.Source: Figure 6 · p. 8, §4.4 · p. 8
• 95% boxed-answer rate for REST versus 73% for CE accompanies REST’s 15.4% average token increase.The analysis attributes the extra tokens to a higher rate of convergence on a final answer.Source: Figure 6 · p. 8, §4.4 · p. 9
• CE thoughts cluster across distinct questions, whereas REST spreads them apart; REST also maintains effective superposition as its loss weight increases.The CE clustering represents collapse without shared content, and lower training CE does not ensure better accuracy.Source: Figure 7 · p. 9, §4.4 · p. 9

5 RELATED WORK

• Latent reasoning uses hidden states within a model, hidden representations or KV caches across agents, and repeated computation across rounds.Most such systems train using cross-entropy on the final decoded answer while leaving latent thoughts unconstrained.Source: §5 · p. 9
• Existing latent-supervision approaches train states against a following answer, explicit chain of thought, or auxiliary decoder alignment, while audits report representation failures.The cited audits identify high accuracy without latent reasoning, collapsed candidate solutions, and unstable trajectories.Source: §5 · p. 9
• REST is positioned as the first latent recursive LLM system to turn theoretically motivated properties of valid thoughts into loss terms.The four-property framework had previously been used to audit existing methods.Source: §5 · p. 10

6 CONCLUSION

• REST adds differentiable supervision for causality, minimality, separability, and stability to CE, without architectural changes or added inference parameters.It is evaluated in both single-agent and multi-agent latent recursive systems.Source: §6 · p. 10
• CE-only thoughts collapse across distinct questions and retain irrelevant information, while REST thoughts contain relatively more producer output than prompt content.The authors suggest this output-focused content may make latent communication easier to audit.Source: §6 · p. 10
• The largest-gain property depends on the consumer: causality leads for cross-agent transfer, while minimality leads for self-recursion.The authors also argue that final-answer accuracy alone gives an incomplete view and should be accompanied by thought inspection.Source: §6 · p. 10
• REST trains only the outer link while freezing the base LLMs and inner link; training the inner link or agent is left to future work.This design isolates the contribution of each property term.Source: §6 · p. 10

Causality

• 2nBε bounds the causal-axiom excess of a surrogate minimizer under bounded disagreement B and reference faithfulness ε.The per-position factorization is exact; the remaining mismatch is weighting dataset prefixes rather than prefixes generated by the consumer.Source: CAUSALITY · pp. 16, 17, 19

Minimality

• 2(Γ + λ2κ) bounds the excess in λ2J for a minimizer of the differentiable minimality surrogate.The cross-entropy objective represents the information-bottleneck axiom for βIB > 1, assuming bounded decoder gap Γ and realization leakage κ.Source: MINIMALITY · pp. 19–21

Separability

• For unit-normalized pooled thoughts, the separability loss bounds the margin against every preceding thought simultaneously, controlling maximum similarity rather than average similarity for disjoint inputs.The axiom’s margin requirement applies to inputs with disjoint semantic supports; the assumption treats distinct training examples as semantically disjoint.Source: SEPARABILITY · p. 22

A.1 FROM AXIOMS TO DIFFERENTIABLE SURROGATES

• Under Assumption A.5, the surrogate threshold suffices for the axiom’s margin, so minimizing eq. (4) maximizes the margin obtainable from H.The corollary establishes the sufficient condition for the axiom’s margin.Source: SEPARABILITY · p. 23
• Exact entropy factorization lets one teacher-forced output provide the required prefixes without enumerating all possible outputs.The per-position entropies are evaluated along a dataset output, and the remaining difference is the origin of that single output.Source: STABILITY · pp. 23–24
• Under producer-reference faithfulness, the stability estimate tracks the producer’s output entropy rate, normalized by sequence length.Dividing by output length makes the target independent of length.Source: STABILITY · p. 24
• At zero stability loss, the probe recovers H(Y)/m up to sampled-output error η(u), whose training-distribution average is bounded by εprod log |V|.The error comes from the one sampled output and does not depend on ψ or ω.Source: STABILITY · p. 25

A.2 EFFECT ON THE CONSUMER’S CROSS-ENTROPY

• When the causality loss is zero, latent transfer recovers textual transfer exactly.The section also states that reducing the substitution cost to zero recovers the textual transfer.Source: §A.2 · pp. 26–27
• The loss in eq. (3) upper-bounds, up to a constant, the cost of substituting a thought for the producer’s text.The section identifies this cost with the consumer’s reconstruction cross-entropy difference.Source: §A.2 · pp. 25, 27
• Under Assumption A.7, the loss bounds the substitution cost and yields a bound on the consumer’s probability for its target.The probability bound reaches the textual-transfer probability when CE(Y | T) and bI(X; T | Y) both vanish.Source: §A.2 · p. 28
• Colliding thoughts for distinct examples force the consumer to use nearly the same answer distribution, imposing a lower bound on average cross-entropy.Under total collapse between two targets, the stated lower bound is log 2; CE does not prevent such collisions through its objective.Source: §A.2 · p. 30
• The latent-transfer advantage over a sampled textual transfer is the sampling variance in the consumer’s target probability.The advantage is strictly positive when the producer’s sample changes that probability.Source: §A.2 · pp. 30–31

B.1 BENCHMARKS •

• Evaluation spans mathematics, science, medicine, and code generation, using seven benchmarks with benchmark-specific question formats and correctness criteria.MATH500, AIME2025/2026, GPQA-Diamond, MedQA, LiveCodeBench-v6, and MBPP+ are described; AIME uses Pass@10, while the other benchmarks use Pass@1.Source: §B.1 · p. 31

B.2 COMPARED BASELINES

• CODI distills hidden states from a teacher pass, while SIM-CoT reconstructs producer text with an auxiliary decoder discarded at inference.Both are adapted as auxiliary losses on the outer-link cross-entropy objective.Source: §B.2 · p. 32

B.3 HYPERPARAMETERS

• Training uses 20,000 steps, batch size 4, and AdamW at 5 × 10−4; property-loss weights are swept, and inference uses 32 latent steps.Thoughts span at most 80 positions in training; decoding uses temperature 0.6 and top-p 0.95, lowered to 0.2 on MBPP+ and LiveCodeBench-v6.Source: §B.3 · p. 32

C EXTENDED RELATED WORK

• REST constrains transferred latent thoughts, unlike prior trace-compression objectives, and adds differentiable cross-entropy terms without changing the architecture or adding inference parameters.Massoli et al. (2026) and Conklin et al. (2026) compress decoded traces; REST applies its terms at agent or round transfer.Source: C EXTENDED RELATED WORK · p. 32

D.1 BASE MODEL ACCURACY

• Frozen base-model accuracy varies substantially across domains, and no single model leads on every benchmark within either agent composition.These are accuracies of the agents before any system is applied.Source: §D.1 · p. 33, Table 6 · p. 33
• The latent system combines agents with complementary strengths.The reported base-model values precede application of the system.Source: §D.1 · p. 33, Table 6 · p. 33
• Table 6 reports each frozen agent’s accuracy before it is used in a system.Source: §D.1 · p. 33, Table 6 · p. 33

D.2 PER-ROUND RESULTS

• Across training seeds, the average standard error is ±0.7 accuracy points and ±2.1% in tokens for Avg. Change.Source: §D.2 · p. 33
• At r = 1, every property term improves over CE-only for Light, while Scaled gains are relatively small; at r = 3, the pattern reverses, with one exception.For Scaled at r = 3, every term except multi-agent separability improves accuracy; the r = 3 comparison uses the authors’ CE-only reproduction.Source: §D.2 · p. 33, Table 7 · p. 34, Table 8 · p. 34

D.3 FULL PROPERTY SWEEP

• Causality and minimality improve over CE-only at every tested weight in both settings, while separability and stability are near or below baseline depending on scale.Separability and stability fall below CE-only in Scaled at every weight, accompanied by substantially fewer decoded tokens.Source: §D.3 · p. 33, Table 11 · p. 35

D.4 AUXILIARY-LOSS BASELINE SWEEPS

• Across the full weight ranges, CODI and SIM-CoT accuracy varies by less than a point on larger benchmarks, supporting the representativeness of the reported operating point.The wider variation on the two AIME sets corresponds to one or two problems; the tables report full weight sweeps.Source: §D.4 · pp. 33–34, Table 12 · p. 36, Table 13 · p. 36

D.5 REPRESENTATION ANALYSIS

• Minimality and causality have higher diversity ratios than CE-only, separability is close to baseline, and stability is lower.The ratio compares mean pairwise distance among thought positions with their mean distance to the consumer vocabulary centroid; 1 matches random vocabulary-token diversity and 0 denotes collapse.Source: Figure 9 · p. 36, §D.5 · p. 36
• Effective superposition measures how many candidate reasoning paths a thought supports, with values of 1 for one supported candidate and M for equal support across M candidates.The metric uses pooled decoded token mass over thought positions to score candidates and derives a posterior over them.Source: §D.5 · pp. 36–37
• Six CE-only thoughts from unrelated questions span 2.8% of the projection, versus 56.4% under separability at β = 3.0.The questions in CE-only clusters have mean pairwise TF-IDF cosine similarity 0.03, compared with 0.02 for random groups of the same sizes.Source: Figure 11 · p. 37, §D.5 · p. 38
• As latent budget grows, CE-only loses accuracy on GPQA-D and LiveCodeBench while REST holds its level; neither objective separates on MATH500.Each budget cell is a single run per arm, so adjacent-budget variation is wider than in the tables.Source: §D.5 · pp. 38–39, Figure 13 · p. 38

• In single-agent samples at least 200 tokens shorter, REST’s saved tokens are repeated text; this held across 4–24-token spans and benchmarks.Source: E CASE STUDY: EFFECT ON GENERATION · p. 39, Table 14 · p. 39
• In the single-agent MATH500 example, CE-only exhausts its token budget rewriting polynomial long division, while REST uses synthetic division and answers correctly in less than half the tokens.Source: E CASE STUDY: EFFECT ON GENERATION · p. 39
• In the GPQA-D example, REST derives a linewidth bound and selects 10−4 eV correctly despite an intermediate arithmetic error; CE-only gives a wrong option without derivation.Source: E CASE STUDY: EFFECT ON GENERATION · p. 40
• In some multi-agent questions, REST uses more tokens for genuine derivation than CE-only, and only REST reaches the correct option.Source: E CASE STUDY: EFFECT ON GENERATION · p. 40