PaperlayerView exact revision on arXiv

Source-linked AI summary

Reasoning or Just Prompting? Probing the Mechanism of Latent Chemical Reasoning

Kaiyuan Lu, Jiatong Li, Yatao Bian

arXiv:2610.34173v1cs.LGcs.CL

TL;DR

Latent chemical reasoning uses continuous intermediate states, but it remains unclear whether successful predictions depend on the molecule-specific content of those states. The paper intervenes on latent sequences and compares them with shared prefixes and soft prompts, finding that input-specific content offers little aggregate benefit over shared conditioning. The results support viewing the tested sequences more as soft prompts than as hidden reasoning chains.

  • Problem

    Latent states offer continuous intermediate computation for chemistry, but whether their molecule-specific content is causally needed for successful predictions remains unresolved.

    Source: §1 · p. 1

  • Method

    The study intervenes on LatentChem sequences and compares them with shared prefixes, then compares recurrent latents with learned soft prompts in controlled models.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Across the studied chemical models and tasks, input-specific latent sequences provide little aggregate benefit over shared continuous prefixes, and controlled models show no consistent advantage over learned soft prompts.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The authors suggest evaluating latent reasoning by its predictive benefit beyond a shared-prefix baseline.

    Source: §6 · p. 10

    The study covers one released LatentChem checkpoint and one training run per controlled configuration; nonsignificant contrasts are not formal equivalence tests.

    Source: §A.7 · p. 17

Abstract

from arXiv · show

Latent reasoning performs intermediate computation in continuous space rather than through explicit tokens, which is particularly appealing in chemistry, where errors in explicit reasoning can propagate to the final answer. Yet if latent states truly constitute an implicit reasoning chain, they should be causally important. We find otherwise: swapping or moderately perturbing intermediate latent states has little effect on performance, while removing them entirely sometimes causes a substantial score drop. Thus, the latent states matter, but their molecule-specific content appears to matter far less. This points to a simpler interpretation. Rather than encoding a step-by-step reasoning chain, latent states may primarily steer subsequent model computation, much like soft prompts. Indeed, a single fixed prefix shared across molecules reproduces LatentChem's performance, and a learned static soft prompt performs comparably to per-input latent generation in controlled models. The tolerance of latent sequences to substitution and moderate perturbation further supports this connection. We therefore suggest that latent chemical reasoning may be better viewed as soft prompting than as a reasoning chain hidden in continuous space.

1 Introduction

Latent reasoning moves intermediate computation from explicit text into continuous states, but strong task performance alone does not show that those states contribute causally. This study tests whether LatentChem needs molecule-specific latent sequences and finds evidence consistent with shared continuous conditioning.

1 Introduction

  • Latent reasoning aims to move intermediate computation into continuous representations, but performance alone does not establish how those states contribute to an answer.The cited approaches include internalizing textual steps, recurrent latent computation, and self-distillation.Source: §1 · p. 1
  • In chemistry, successful predictions can coexist with incorrect structural claims in explicit reasoning, while LatentChem reports improved generation and reduced textual reasoning.Whether those gains require a molecule-specific latent sequence remains an open mechanistic question.Source: §1 · p. 1
  • Prior work distinguishes shared continuous prompts from input-specific latent states, and separates information recoverable from a representation from information used to answer.The introduction also notes limited sensitivity to latent perturbations in language, chess, and visual tasks, motivating matched controls and causal interventions.Source: §1 · pp. 1–2
  • LatentChem’s sequence substitutions largely preserve task scores, while removing the sequence often lowers scores; a fixed prefix reproduces the shift from explicit steps to direct answers.The study also compares shared-prefix baselines and representation analyses, finding readable molecular information alongside little aggregate benefit from input-specific sequences.Source: §1 · p. 2

2 Related Work

Related work distinguishes input-specific latent computation from shared continuous conditioning, and emphasizes that represented information or visible trajectories do not by themselves establish a reasoning mechanism.

Implicit reasoning and continuous conditioning

  • Implicit reasoning methods internalize textual traces through progressive removal, recurrent continuous states, or self-distillation, whereas prefix and prompt tuning learn vectors shared across inputs.Pause tokens add computation before answer generation.Source: §2 · p. 2

Causal dependence and representation evidence

  • Interventions test whether traces influence predictions, while readouts and geometric patterns address what is represented rather than what determines an answer.Prior latent-state studies report weak steering sensitivity or substitution tolerance, with dependence varying by intermediate supervision in visual tasks.Source: §2 · pp. 2–3

Chemical reasoning and process reliability

  • Chemical reasoning systems use both explicit domain-trained approaches and continuous molecular intermediates, but correct answers can still accompany unreliable structural explanations.Interventions on intermediate molecular drafts can impair generation, so trace correctness and computational usefulness require separate evidence.Source: §2 · p. 3

3 Method

The method distinguishes effects of latent presence from effects of molecule-specific latent content by intervening on sequences while holding the input context fixed. It applies these tests to LatentChem and controlled models, alongside evaluation and representation analyses.

3.1 Preliminaries

  • The study compares direct prediction, textual reasoning, input-generated latent reasoning, and soft prompting, whose continuous prefix is shared across inputs.The decoder retains access to the original input in each route, so these intermediate routes are not exclusive causal paths.Source: §3.1 · p. 3
  • Content interventions replace a recipient’s latent sequence with a same-task donor sequence while holding its input and other decoder context fixed, then assess score changes and donor-directed transfer.Removal and replacement are distinct tests, and shared prefixes test whether generating a sequence for each input is necessary.Source: §3.1 · p. 3

3.2 Latent Interventions

  • Latent interventions compare matched, swapped, and cohort-mean sequences; remove latent positions; test a hard textual prompt; and compare a frozen calibration-set prefix with input-specific sequences.Gaussian sampling and added noise test tolerance to changed latent values; Gaussian Z preserves coordinate-wise moments, not the full joint distribution.Source: §3.2 · p. 4

3.3 Experimental Setup

  • The LatentChem tests use one released checkpoint on ChEBI-20 and six optimization tasks, while controlled models use a Chem-R checkpoint on description and reaction tasks.Controlled training pairs recurrent latent models with jointly trained shared soft prompts; each configuration has one run, and cross-recipe comparisons also differ in training exposure.Source: §3.3 · p. 4
  • Evaluation uses greedy decoding and task-specific scores; paired example-bootstrap intervals quantify score contrasts, while chemical judgments assess factual correctness rather than completeness.Representation analyses probe molecular properties and local structure while controlling for SMILES token count and heavy-atom count, and compare latent variation and attention patterns.Source: §3.3 · pp. 4–5

4 Results

Across LatentChem and controlled models, shared or substituted latent sequences often preserve aggregate performance, while removing the block can reduce scores or reorganize responses. These results support a role for shared conditioning, even though latent states also contain readable molecular information.

4.1 Cross-molecule substitution largely preserves task scores

  • Swapped sequences retain 94.5–103.3% of matched scores across six optimization tasks, while removal lowers scores on six of seven tasks.On ChEBI-20, removal lowers METEOR from 0.1558 to 0.0610; swapped runs show no consistent donor-directed shift, and descriptions can change despite stable aggregate scores.Source: §4.2 · p. 5, §4.1 · p. 5

4.2 Latent removal changes response organization

  • On ChEBI-20, removing latents lowers METEOR from 0.1558 to 0.0610, while zero vectors score 0.0815 versus 0.1564 for Mean Z.Thus, accessible latent positions alone are insufficient; the values at those positions matter, although their molecule-specific identity need not be preserved.Source: §4.2 · p. 5
  • Without latents, descriptions shift structural claims into numbered steps and shorten the answer span scored by METEOR, despite similar total response length.An explicit paragraph instruction closes 57.1% of the score gap, but steps persist and answers remain shorter; the full-response audit finds no clear native-sequence correctness preference.Source: §4.2 · p. 6

4.3 Shared prefixes reproduce performance and the format transition

  • A fixed prefix retains 96.7–101.6% of matched scores across seven tasks and reproduces the shift to step-free responses as prefix length increases.On the description partition, over 92% of responses are step-free at four positions and all are at six; pairing a ten-position fixed prefix with H0 also produces step-free responses on all seven tasks.Source: §4.3 · pp. 6–7
  • ChEBI-20 METEOR stays above 99% of matched through α = 0.25, but falls to 61.4% at α = 1 as step-free responses fall to 57.5%.At α = 0.5, METEOR retains 92.7% and 99.8% of responses are step-free; optimization responses also regain explicit steps at α = 1.Source: §4.3 · p. 7

4.4 Readable molecular information coexists with a shared trajectory

  • Latents improve size-controlled readout of molecular properties, while a shared step component explains 96.23% of centered ChEBI-20 latent variance.For logP, TPSA, and ring count, readout gains are ∆R2 = 0.144, 0.089, and 0.391; the initial molecular context is more informative on every reported measure.Source: §4.4 · p. 7
  • All 800 sequence-level swaps are at or below α = 0.25, a noise level that left description scores essentially unchanged.The median swap amplitudes are α = 0.077 for description and 0.038–0.046 for optimization; controlled models tolerate substitution even with larger molecule-dependent components and swap amplitudes.Source: §4.4 · p. 8, §4.5 · p. 9

4.5 Controlled models show no consistent advantage over soft prompts

  • Description METEOR differs by less than 0.005 between soft prompting and recurrent latent generation, while reaction exact match ranges from 77% to 78%.Paired intervals include zero, though only 40–44% of description answers agree exactly; controlled latent models also tolerate substitution, and removal leaves scores near matched levels.Source: Table 1 · p. 8, §4.5 · p. 9

5 Discussion

Across the studied settings, molecule-specific latent sequences add little aggregate benefit, although LatentChem’s latents affect response organization. The authors distinguish that finding from whether the decoder performs input-dependent chemistry, since it retains the molecular input.

Shared conditioning

  • Predictions benefit little from the current molecule’s latent sequence; a shared prefix can still support molecule-specific computation inside the decoder, which retains the input.The authors frame this as shared conditioning, not as evidence that the model’s chemistry is input-independent.Source: §5 · p. 9

Representation evidence

  • Probes can recover molecule-dependent information in latent deviations, but probe accuracy shows what is available in states, not what determines answers.The decoder may obtain the same information from the retained input context.Source: §5 · p. 9

Interpreting prior findings

  • The fixed-prefix sweep reproduces the return of textual steps, so that pattern need not indicate lost implicit reasoning; its noise results are not directly comparable to completed-sequence replacements.Early-state perturbations propagate through later latent states and molecular updates, unlike the paper’s sequence-replacement intervention.Source: §5 · p. 9

Evaluation implications

  • Latent-reasoning evaluations should separate task quality, response organization, and the benefit of input-specific computation, using complementary controls and auditing full responses when answer-span metrics are used.The authors suggest same-length substitutions, fixed prefixes, and matched-condition learned soft prompts as complementary controls.Source: §5 · p. 9

6 Conclusion

Across the studied chemical models and tasks, input-specific latent sequences provide little aggregate benefit over shared continuous prefixes. The authors recommend evaluating latent reasoning by its predictive benefit beyond a shared-prefix baseline.

Conclusion

  • Across the chemical models and tasks studied, input-specific latent sequences provide little aggregate benefit over shared continuous prefixes.LatentChem’s molecular content can be replaced while largely preserving task scores, while controlled models show no consistent advantage over learned soft prompts.Source: §6 · p. 10
  • LatentChem’s sequences shape response organization, even though their molecular content can be replaced while largely preserving task scores.Source: §6 · p. 10
  • The authors recommend evaluating latent reasoning by its predictive benefit beyond a shared-prefix baseline.Source: §6 · p. 10

A Data, Training, and Evaluation Protocols

The appendix specifies shared experimental protocols and organizes supplementary results around LatentChem interventions, controlled training comparisons, and representation analyses.

Appendix overview

A.1 Data and adaptation

The appendix documents the data, training, prompts, intervention controls, scoring, chemical-correctness judgments, and uncertainty procedures used in the experiments.

A.1 Data and adaptation

  • The released LatentChem checkpoint and controlled models are adapted using curated molecular-description and reaction examples, with one training run per configuration.The adaptation pool contains 4,000 examples; retained rationales meet the stated class-A acceptance criteria, and training exposure differs between recipes.Source: §A.1 · p. 12
  • Evaluation cohorts are compared with task-matched training data for structural proximity, while the audit does not assess overlap with base-model or LatentChem pretraining data.The curated ChEBI validation set is structurally closer than the official-test subset by the reported nearest-neighbor threshold.Source: §A.1 · pp. 12–13, Table 3 · p. 13

A.2 Prompts and decoding boundaries

  • Controlled-model and LatentChem evaluations use different prompts, and the controlled-model scores are not directly comparable to published results under other protocols.The controlled models share user questions and latent boundary markers within each recipe.Source: §A.2 · p. 13

A.3 Intervention interface and calibration

  • Completed-sequence replacements retain the recipient’s other input context, while Fixed Z uses a prefix calibrated on disjoint molecules and shared across evaluation inputs.Mean Z and Gaussian Z use evaluation-cohort statistics as diagnostic controls; Fixed Z uses a disjoint calibration cohort.Source: §A.3 · p. 14

A.4 Scoring and response audits

  • Scoring extracts the first answer span for descriptions, uses canonicalized product SMILES for reaction exact match, and reports explicit-step rates using a specified marker detector.Full-length description runs reach the 512-token generation cap in 9–15.5% of cases, and capped outputs remain in the reported metrics and chemical judgments.Source: §A.4 · pp. 15–16

A.5 Chemical-correctness rubric

  • Chemical-correctness judgments compare complete paired responses for factual correctness, weighting central identity and connectivity errors over minor imprecision and not rewarding length or additional correct detail.Ties therefore do not establish equal coverage, completeness, or absolute correctness.Source: §A.5 · p. 16

A.6 Paired reporting and uncertainty

  • Score differences and other paired contrasts use molecule-level uncertainty procedures, but the intervals do not capture training-run variability and nonsignificant contrasts are not formal equivalence tests.Source: §A.6 · pp. 16–17

A.7 Scope and limitations

The study’s scope is limited to one released LatentChem checkpoint and controlled adaptations with one run per configuration. Supplementary intervention comparisons require paired baselines.

A.7 Scope and limitations

  • The study covers one released LatentChem checkpoint and controlled adaptations with one run per configuration; its intervals do not measure training-run variability.Nonsignificant contrasts are not formal equivalence tests.Source: §A.7 · p. 17

B Supplementary LatentChem Interventions

  • Intervention effects should be judged against each condition’s paired Matched baseline, not across separately executed baselines.The supplementary results cover sequence content, response organization, shared-prefix controls, and perturbation tolerance.Source: B Supplementary LatentChem Interventions · p. 17

B.1 Sequence content and response organization

The supplementary interventions distinguish sequence substitution from removal and separate output organization from factual correctness. Additional tables describe shared-prefix, length, initial-context, and noise controls.

B.1 Sequence content and response organization

  • Table 4 reports ChEBI-20 METEOR and optimization improvement scores, with exact output identity for swaps and zeroing as a removal control.Zero Z retains attended latent positions but sets their vectors to zero.Source: Table 4 · p. 17
  • No swapped optimization output among 600 pairs equals either the donor input or its Matched output after canonicalization.Table 6 frames donor transfer through description similarity to donor references and optimization similarity to donor inputs.Source: §B.1 · p. 17, Table 6 · p. 18
  • Table 5 separates final-answer length and step counts from complete-response preferences and chemical correctness.Its hard-prompt comparison uses follow-up paired baselines, and explicit steps occur in 98% of hard-prompt responses.Source: §B.1 · p. 17, Table 5 · p. 18

B.2 Shared prefixes, length, and initial context

  • Tables 7–9 cover fixed-prefix endpoints, description length and initial-context controls, and optimization length sweeps.Table 8 notes that post-hoc masking after ten native steps differs from native zero-budget decoding; H0 X-only removes latent positions and boundary markers.Source: Table 7 · p. 19, Table 8 · p. 19, Table 9 · p. 20

B.3 Statistical replacements and numerical noise

  • Tables 11–12 report Gaussian-prefix scores and full-coordinate RMS perturbations, including optimization scores, response organization, and answer changes.Table 12’s score-change intervals are paired molecule-bootstrap intervals, not between-draw standard deviations.Source: Table 12 · p. 21, Table 11 · p. 21

C Controlled Training Comparisons

Controlled comparisons report that removing or zeroing continuous blocks can leave scores close to baseline in the controlled models. The section also distinguishes trained soft prompts from checkpoint interventions and describes size-controlled representation analyses.

C Controlled Training Comparisons

  • Training comparisons match exposure within each recipe but use one run per configuration, so their intervals do not measure training-seed variability.Cross-baseline changes should not be interpreted as intervention effects.Source: C Controlled Training Comparisons · p. 22

C.1 Latent reasoning versus learned soft prompts

  • Soft-prompt contrasts change the trained model, whereas Swapped, Mean, and X-only intervene on one checkpoint.Table 13 reports paired contrasts on 200 official test inputs per task; soft prompting is separately trained under the same recipe.Source: §C.1 · p. 22, Table 13 · p. 22

C.2 Does a continuous block remain necessary?

  • Scores remain close to their respective baselines when the block is absent in the controlled models, limiting how generally LatentChem’s prefix dependence explains response organization.The reported conditions retain 100% strict answer-tag onset and have no detected explicit steps.Source: §C.2 · p. 22, Table 14 · p. 22

D Representation Analyses

  • Representation analyses separately report probe controls, latent geometry, and attention because they do not establish the same claims.The probes use scaffold-grouped cross-validation while controlling for SMILES token count and heavy-atom count.Source: D Representation Analyses · p. 23, §D.1 · p. 23

D.1 Size-controlled probes

  • Structural-label probes retain 30 of 85 RDKit labels under a training-subset support rule; deduplicated macro AP is 0.3495 for size alone, 0.7126 for H0, and 0.5690 for Z.The labels have at least five positive and five negative examples in every outer and inner training subset.Source: §D.1 · p. 23

D.2 Variance, dispersion, and swap amplitude

LatentChem’s molecule-to-molecule sequence differences are small relative to the tested noise scale, while the shared step component dominates centered latent variance. The section also presents descriptive attention analyses and qualitative output examples.

D.2 Variance, dispersion, and swap amplitude

  • Across 200 LatentChem description swaps and 600 optimization swaps, α was ≤0.25; median α was 0.0767 for description and 0.0384–0.0464 across optimization tasks.Donor differences retain their native directions.Source: §D.2 · p. 24
  • The centered-variance table identifies the step term as shared across inputs, while static prompts have zero molecule and interaction terms by construction.The molecule and interaction terms sum to 1.151%–1.417% across the six optimization tasks.Source: Table 18 · p. 25

D.3 Attention analysis

  • The attention comparison is descriptive: it contrasts attention to Z at chemical-content queries with matched non-chemical queries, without establishing causal contribution to prediction.The common comparison includes 153 molecules with eligible pairs in all four models.Source: §D.3 · p. 26

Matched Z and Swapped Z (identical complete responses)

X-only

  • The X-only example uses explicit numbered steps to describe the input structure and ends with a short cyclic-urea answer.The example identifies a thioether-linked five-membered ring and a chiral center.Source: X-only · p. 27

X-only + hard prompt

  • The X-only plus hard-prompt example also presents a multistep structural analysis before its answer.It describes a thioether attached to a chiral carbon in a five-membered ring with two carbonyl groups and two amide nitrogens.Source: X-only + hard prompt · p. 27

Related papers

Loading 2610.34173v1…