PaperlayerView exact revision on arXiv

Source-linked AI summary

The Depth Ceiling: On the Limits of Large Language Models in Discovering Latent Planning

Yi Xu, Philipp Jettkant, Laura Ruis

arXiv:2604.06427v1cs.LGcs.AIcs.CL

TL;DR

The paper asks how far language models can reason latently without intermediate supervision, a question that matters for evaluating chain-of-thought monitoring. It tests latent strategy discovery and execution with controllable graph path-finding tasks, finding shallow discovery ceilings despite scaling and limited generalization beyond them. The results distinguish discovering a strategy under final-answer supervision from executing an already discovered strategy, while leaving the broader scope of these limits open.

  • Problem

    Little is known about whether language models can discover and execute multi-step reasoning strategies within a single forward pass without supervising intermediate steps.

  • Method

    The paper uses graph path-finding on star graphs with precisely controlled latent planning depth and final-answer-only supervision, probing both performance and attention patterns.

  • Results

    Scaling raises discovered latent planning depth only modestly: small transformers reach 3 steps, fine-tuned GPT-4o and Qwen3-32B reach 5, and GPT-5.4 reaches 7 few-shot, while discovered strategies generalize to 8 steps.

  • Takeaways & Limitations

    The dissociation between strategy discovery and execution suggests that sparse task-success supervision is a bottleneck for discovering deeper latent planning strategies.

  • Takeaways & Limitations

    The study focuses on symmetric star graphs, so whether the same depth ceiling persists in other reasoning domains remains unresolved.

Abstract

from arXiv · show

The viability of chain-of-thought (CoT) monitoring hinges on models being unable to reason effectively in their latent representations. Yet little is known about the limits of such latent reasoning in LLMs. We test these limits by studying whether models can discover multi-step planning strategies without supervision on intermediate steps and execute them latently, within a single forward pass. Using graph path-finding tasks that precisely control the number of required latent planning steps, we uncover a striking limitation unresolved by massive scaling: tiny transformers trained from scratch discover strategies requiring up to three latent steps, fine-tuned GPT-4o and Qwen3-32B reach five, and GPT-5.4 attains seven under few-shot prompting. Although the maximum latent planning depth models can learn during training is five, the discovered strategy generalizes up to eight latent steps at test-time. This reveals a dissociation between the ability to discover a latent strategy under final-answer supervision alone and the ability to execute it once discovered. If similar limits hold more broadly, strategies requiring multiple coordinated latent planning steps may need to be explicitly taught or externalized, lending credence to CoT monitoring.

1 Introduction

The paper asks whether language models can discover and execute multi-step reasoning strategies internally, without intermediate supervision, and uses controllable graph planning to test this question. It finds a persistent ceiling in latent strategy discovery despite scaling, alongside limited test-time generalization beyond the training depth.

  • Motivation and approach: Graph path-finding tasks isolate latent planning by requiring multi-step computation while supervising only final task success.The required planning depth is precisely controlled, and the setting rules out local heuristic shortcuts.
  • Findings: 3 latent steps is the discovery ceiling for a small transformer trained from scratch, with failure beginning abruptly at four steps.Increasing model depth or attention heads does not overcome this limitation.
  • Findings: 3 to 4 or 5 steps is the improvement in discovered depth from the small transformer to fine-tuned 7B–32B models and GPT-4o, while GPT-5.4 reaches 7 few-shot steps.Scaling primarily improves planning breadth rather than depth; GPT-5.4's few-shot result is a lower-bound proxy because its training is unavailable.
  • Findings: 8 steps is the generalization ceiling: models trained on five-step problems generalize to deeper test problems, whereas models trained directly on six-, seven-, or eight-step problems remain at chance.This separates discovering a latent strategy from executing it at greater depth once discovered.
  • Mechanistic interpretation: Attention analysis indicates that successful models learn backtracking along the target-to-source path, while out-of-training-depth errors are predominantly on-path.Models identify the correct branch but often fail to complete all required backtracking iterations.
  • Control comparison: 20 lookahead steps are solved after training with an externalized backtracking strategy, suggesting sparse final-answer supervision—not task difficulty—is the primary bottleneck.The control model converges in only 20 training updates.
  • Limitations: The results do not establish that deeper latent planning is impossible and remain limited to simple graph-search tasks with precisely controlled depth.Whether similar ceilings govern latent strategy discovery beyond planning remains open.

2 Problem Formulation

The paper formulates latent planning as controllable star-graph path-finding that cannot be solved through local cues, then defines metrics for detecting above-chance planning and probes the learned strategy through attention. It progressively trains models at fixed depths to measure discovery ceilings while separating discovery from pretraining effects.

  • 2 Problem Formulation: Latent planning is studied as multi-step lookahead without intermediate planning supervision, with task difficulty controlled by the required depth.The task must rule out local cues and simple heuristics that bypass planning.
  • 2.1 Star Graph: Star graphs provide a controllable topology in which the correct next hop cannot be inferred from local graph structure.The model must reason over nodes several steps away to choose the correct branch.
  • 2.1 Star Graph: The model outputs only the first node on the branch leading from v_source to v_target, avoiding the trivialization that would result from generating the complete path.A branch is sampled uniformly and its endpoint is designated as the target.
  • 2.2 Metrics: Raw accuracy is normalized into an empirical skill score because different branch factors have different random baselines.The score is 1 for perfect performance and 0 for random guessing.
  • 2.2 Metrics: A model has latent planning capability at depth m when its empirical skill exceeds the random-guessing threshold for at least one branch factor in K = {2, 3, 4, 5, 10}.The test uses α = 10^-5 to assess whether performance is statistically distinguishable from chance.
  • 2.2 Metrics: Depth m = 2 is excluded because the correct node can be selected directly from v_target without lookahead.This depth admits shallow pattern-matching shortcuts such as induction heads.
  • 2.2 Metrics: The discovery ceiling is measured progressively by training from base initialization at increasing fixed depths and taking the highest depth with LPC = 1.Each model is retrained on the new depth, preventing bootstrapping across depths.
  • 2.3 Models: The study compares a from-scratch 8-layer, 1-head transformer with 7B–32B Qwen models and GPT-4o/GPT-5.4, disabling GPT-5.4's hidden reasoning trace.The transformer isolates strategies acquired during training, while model-scale comparisons assess latent planning in pretrained systems.

3 Results with Transformers Trained from Scratch

Transformers trained from scratch can discover shallow latent planning strategies, but discovery is sharply limited by planning depth and model capacity for broad branching. Successful configurations exhibit backtracking-like attention patterns, while failed deeper configurations show uniform or unstructured attention.

  • Discovery limits: Depth 3 is the practical discovery limit for the small transformer, with skill rapidly declining as planning depth or branch factor increases.Discovery fails abruptly at four steps and beyond, while increasing model depth or attention heads does not overcome the limitation.
  • Generalization: Models maintain near-perfect skill through the training depth but generalize less reliably as unseen test depth increases.The out-of-distribution evaluation uses unseen graphs at all depths without further training.
  • Discovery limits: At branch factor k = 10, the model fails to learn even the basic neighbor-prediction heuristic, indicating a capacity limitation distinct from depth-related discovery failure.Training typically proceeds through a heuristic first stage and a strategy-discovery second stage; high branching can block the first stage entirely.
  • Mechanism: Successful configurations with k ≥3 show substantially above-baseline backtracking ratios, whereas failed configurations have near-uniform attention over the graph.For k = 2, the topology permits an alternative-neighbor elimination strategy, so successful planning need not rely on backtracking.
  • Mechanism: Successful attention maps concentrate on nodes along the target-to-source path in backward order, while failed configurations show no discernible attention pattern.The pattern becomes clearer as branch factor increases in successful depth-3 configurations.

4 Results with LLMs

Pre-trained and fine-tuned LLMs exhibit shallow latent planning, and scaling improves breadth more than discovery depth. Once a strategy is discovered, however, models can execute it beyond the training horizon, while explicit reasoning supervision can bypass the discovery bottleneck.

  • Pre-trained LLMs: Few-shot prompting extends GPT-5.4's latent planning capacity to 7 steps, but all evaluated models eventually fail as planning depth increases.Zero-shot LPC ranges from 3 for Qwen 3-8B to 5 for GPT-5.4.
  • Fine-tuned LLMs: Qwen 3-32B and GPT-4o solve configurations through m = 5 but fail completely at m = 6, showing that scaling does not resolve the discovery bottleneck.Fine-tuning mainly improves planning breadth: Qwen 2.5-7B solves k = 10 at m = 3 but fails across branch factors at m = 4.
  • Fine-tuned LLMs: Fine-tuned LLM failures occur during strategy discovery rather than the initial heuristic stage, unlike the small transformer's failures at high branch factors.This training-dynamics distinction supports a strategy-discovery bottleneck rather than a mere parameter deficit.
  • Out-of-distribution generalization: Qwen 3-32B generalizes up to m∗+ 3 and GPT-4o up to m∗+ 4 beyond their fine-tuning depths.All models interpolate near-perfectly at m ≤ m∗, and most extrapolate above random at m∗+ 1.
  • Error analysis: On-path errors can comprise 95% of Qwen 2.5-7B's errors, indicating correct-branch selection followed by failure to complete all planning steps.This pattern supports a bounded latent-execution horizon even when the model has internalized the correct strategy.
  • Dense supervision: Explicit backtracking chain-of-thought solves graphs requiring 20 lookahead steps, and ICoT compression reaches perfect latent planning through m = 6 at k = 2.The compressed strategy remains limited by representational capacity on more complex graph configurations.

5 Related Work

Prior work leaves unresolved whether transformers can discover latent planning without intermediate supervision. The paper connects this question to CoT monitoring by arguing that complex multi-step tasks may still require externalized reasoning.

  • Related work: Existing latent-reasoning methods commonly rely on explicit training scaffolding, architectural changes, or auxiliary supervision rather than autonomous strategy discovery.Coconut and ICoT progressively internalize explicit chain-of-thought into latent representations.
  • Implications: The reported depth ceiling suggests that current LLMs fundamentally require externalized reasoning for complex multi-step tasks, supporting a safety margin for CoT monitoring.The implication is stated within the paper's scope of monitoring explicit reasoning traces for malicious or deceptive behavior.

6 Limitations and Future Work

The study’s conclusions are bounded by star-graph tasks, inaccessible closed-model fine-tuning, and fully implicit planning supervised only by final node prediction. The authors therefore call for broader domains, data-scaling analyses, and supervision with varying granularity.

  • Star graphs lack local heuristic cues, making them a clean lower-bound test whose applicability to denser real-world reasoning tasks remains open.The authors note that theorem proving and code generation may contain signals that partially bypass the discovery bottleneck.
  • Closed-weight frontier models could not be fine-tuned, so the strongest-model conclusions rely on few-shot evaluation and remain suggestive rather than definitive.GPT-5.4 did not exceed seven latent planning steps in the few-shot setting.
  • Large-scale fine-tuning of closed-source models is restricted by API costs, and the effect of additional training data on the discovery ceiling remains unresolved.The abrupt failure pattern and open-model data-scaling results suggest that data scaling alone is unlikely to remove the bottleneck, but the authors call for systematic scaling-law studies.
  • The reported boundary applies to fully implicit planning trained solely with a single cross-entropy loss from node prediction.The authors propose testing how supervision at different granularities changes the depth ceiling.
  • The findings suggest that current LLMs struggle to discover deep latent planning strategies, while externalized reasoning may remain important for oversight if this limitation generalizes.The paper connects this possibility to evidence that chain-of-thought can be faithful when required for task performance.

D Detailed Metrics

The paper normalizes accuracy against task-specific random baselines to measure empirical skill, then uses statistically significant above-chance performance to identify latent planning capability at each depth.

  • Accuracy alone is not comparable across branch factors because the random-guessing baseline changes with k.An accuracy of 0.7 is marginal over chance at k = 2 but substantial at k = 10.
  • Empirical skill measures how much accuracy exceeds the random baseline, with skill 1 denoting perfect performance and 0 denoting random guessing.Negative values remain possible and are left unnormalized below the baseline.
  • Latent planning capacity records whether a model shows any statistically significant evidence of planning at a given depth, rather than its exact proficiency.The metric is binary at each depth: it is present when empirical skill exceeds the critical threshold for at least one tested branch factor.

E Strategy Analysis and Attention Probing

The paper distinguishes forward and backtracking strategies for star-graph path finding and probes strategy use through final-token attention. Inputs serialize shuffled graph edges with source and target nodes, while attention over path edges is summarized by a backtracking ratio.

  • Strategy Analysis: Forward breadth-first search propagates from vsource to vtarget while tracking all k branches, requiring m sequential steps and parallel processing across branches.A forward depth-first variant can require O(k · m) steps, except when k = 2 permits elimination after trying one branch.
  • Strategy Analysis: Backtracking propagates from vtarget to vsource along the unique path, requiring exactly m sequential steps without parallel branch tracking or trial and error.The paper treats this as the most efficient strategy for the star-graph task.
  • Input Representation: The graph input is encoded as edge-node tokens followed by source and target nodes, with separator tokens marking graph boundaries and the prediction endpoint.Edges are serialized as (ui, vi) pairs separated by s, while g separates the graph representation from (vsource, vtarget) and marks the input end.
  • Attention Probing: Attention probing uses the final input token because its hidden representation predicts the next token, and extracts its layer-wise attention distribution over all input tokens.Each input token receives a scalar attention weight indicating its influence on the predicted next-hop node.
  • Attention Probing: The backtracking ratio measures the fraction of edge-token attention assigned to tokens on the target-to-source path, distinguishing concentrated backtracking from approximately uniform edge attention.Edge tokens are partitioned into Ton for path edges and Toff for all remaining edges.
  • Dataset Construction: Star graphs use randomly permuted node identifiers and shuffled edge order, forcing models to infer connectivity from topology rather than numerical or positional patterns.The source is the central node and the target is a randomly selected branch terminal; training, validation, and test configurations are mutually exclusive.

F.2 Dataset Statistics

Dataset sizes are scaled by model architecture, using the largest training set for the from-scratch transformer and smaller sets for larger fine-tuned language models.

  • Dataset Statistics: The from-scratch transformer uses 100000 training samples, 2048 validation samples, and 2048 test samples.
  • Dataset Statistics: Fine-tuning uses 10000 training samples for Qwen-2.5 7B and Qwen-3 8B, and 2000 for the larger 32B variants.All Qwen models are evaluated on a fixed test set of 2048 samples.
  • Dataset Statistics: GPT-4o is fine-tuned with a restricted training set of 100 samples.

G Training Details

The experiments train a small GPT-2-style transformer from scratch alongside pretrained open- and closed-source models, using star-graph configurations spanning multiple depths and branch factors.

  • Models: The from-scratch model follows the GPT-2 architecture and uses GELU activation, with a default configuration of 8 layers, 1 attention head, and hidden dimension 128.
  • Models: Open-source evaluations use base Qwen 2.5-7B, Qwen 2.5-32B, Qwen 3-8B, and Qwen 3-32B checkpoints.
  • Models: Closed-source evaluations use gpt-4o-2024-08-06 for evaluation and fine-tuning, and gpt-5.4-2026-03-05 for evaluation only.
  • Training Details: Training uses Adam without weight decay for the from-scratch transformer, default AdamW for open-source fine-tuning, and API-selected optimization settings for GPT-4o.The from-scratch transformer also uses dropout rate 0.1.
  • Experimental Configurations: Training-dynamics analysis examines depths m ∈{3, 4}, while skill tables cover depths m ∈{3, 4, 5, 6} and branch factors k ∈{2, 3, 4, 5, 10}.Additional GPT-5.4 probing evaluates depths m ∈{7, 8}.

H.1 Training Dynamics of Transformer

The from-scratch transformer learns local connectivity before attempting full planning, then either discovers a general strategy abruptly or overfits training graphs without improving validation performance. Architectural scaling does not overcome the depth-four discovery ceiling, while successful attention patterns indicate backtracking.

  • Training Dynamics: The first learning stage predicts any valid neighbor of vsource, reducing loss while raising validation accuracy to the random baseline 1/k.This stage generally takes longer as branch factor k increases, and at k = 10 the model may fail to learn even local connectivity.
  • Training Dynamics: Successful strategy discovery causes validation accuracy to jump abruptly from the random baseline to near-perfect levels, alongside a sharp validation-loss decrease.The transition suggests discovery of the complete planning strategy in a discrete step rather than gradual improvement.
  • Training Dynamics: When discovery fails, training loss continues decreasing while validation loss increases and validation accuracy remains at the random baseline, indicating memorization of training instances.Random node permutations prevent these memorized mappings from transferring to unseen graphs.
  • Ablation Study: Increasing layers, attention heads, or hidden dimension does not overcome the depth-four discovery ceiling: variants succeed at m = 3 but fail at m = 4.Some larger variants also degrade shallower performance, including skill 0.75 at k = 3 for 8 heads and failure at k = 5 for hidden dimension 512.
  • Attention Analysis: Successful m = 3 models develop increasingly clear backtracking attention as branch factor rises, whereas failed configurations show no structured attention pattern.At k = 2, near-uniform BR of 0.50 is consistent with multiple viable strategies in the special topology.

I.1 Training Dynamics of LLM

LLMs exhibit a recurring two-stage training pattern, while latent-planning performance deteriorates sharply with depth. Explicit chain-of-thought supervision instead enables rapid convergence on deep graph-planning tasks, indicating that sparse final-answer supervision creates the main bottleneck.

  • Training dynamics: Fine-tuned LLMs retain the same qualitative two-stage training pattern observed in the from-scratch transformer.The reported training dynamics concern Qwen 2.5 and Qwen 3 models across graph configurations.
  • Zero-shot performance: At depth m = 3, all evaluated LLMs show latent planning ability, but performance deteriorates rapidly as planning depth increases.Qwen 3-8B reaches LPC 3, while GPT-4o reaches LPC 4 with higher skill across branch factors.
  • Zero-shot performance: GPT-5.4 reaches near-perfect skill at m = 3, but its zero-shot LPC is only 5 after failing to reject random guessing at m = 6.Its average skill is 0.96 at m = 3, 0.86 at m = 4, and 0.46 at m = 5.
  • Few-shot performance: Few-shot prompting raises GPT-5.4’s LPC from 5 to 7, yet the model still fails at m = 8.Skill increases from 0.46 to 0.82 at m = 5 and from 0.19 to 0.63 at m = 6.
  • Depth barrier: Graph size does not explain the depth-dependent failures: GPT-4o fails on the smaller G(2,6) while succeeding on the larger G(10,3).The comparison supports planning depth, rather than graph size, as the bottleneck.
  • Explicit supervision: With explicit backtracking supervision, all LLMs converge within approximately 20 training updates across configurations, showing that the task itself is not inherently difficult.This control setting provides the full backtracking trace from vtarget to vsource.

J.2 Mitigating Implicit Planning Limitations with ICoT Training

ICoT training progressively removes externally supplied reasoning tokens so models must reproduce the missing computation internally. This bypasses the sparse-supervision discovery bottleneck on simpler graphs, although the small model remains limited by graph complexity.

  • ICoT method: ICoT gradually removes tokens from the full chain-of-thought response, forcing the model to compute the missing steps in hidden states.The supervision sequence progresses from [z1, ..., zm] to [zm = vground].
  • ICoT results: The 1.6M model achieves perfect latent planning up to m = 6 at k = 2 under ICoT.The result applies to simpler star-graph configurations.
  • Evaluation: LPC evaluates whether normalized skill provides statistically significant evidence of planning beyond random guessing.Skill is normalized so 1 denotes perfect performance and 0 denotes random guessing.
Loading 2604.06427v1…