PaperlayerView exact revision on arXiv

Source-linked AI summary

Lapras: Latent Reasoning for Time Series Language Models

Yuliang Chen, Yu Yvonne Wu, Patrick Langer, Arvind Pillai, Sudarshan Regmi, Martin Maritsch, Juncheng Liu, Robert Jakob, Thomas Kaar, Tess Z. Griffin, Lisa Marsch, Michael V. Heinz, Nicholas C. Jacobson, Andrew Campbell

arXiv:2610.11111v1cs.CLcs.LG

TL;DR

Explicit CoT can lose or misrepresent task-relevant temporal patterns, with errors carrying into later reasoning. Lapras distills a CoT teacher into a student that reasons through continuous thoughts and generates only the final answer as text. Across four backbones and five benchmarks, it improves average F1 by up to 10.79% over CoT while substantially reducing generated tokens.

  • Problem

    Explicit CoT may omit or misrepresent task-relevant temporal patterns, and latent reasoning for TSLMs remains underexplored.

    Source: §1 · p. 2

  • Method

    Lapras uses a CoT-trained teacher to distill reasoning into a student that performs intermediate reasoning through continuous thoughts and generates text only for its final answer.

    Source: §1 · p. 2, §3.2 · pp. 4–5

  • Results

    10.79% maximum average-F1 improvement over CoT across four TSLM backbones and five benchmarks, with substantially fewer generated tokens.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Lapras’s continuous thoughts can be decoded into readable reasoning traces, retaining explanation capability.

    Source: §1 · p. 2

    Lapras relies on explicit CoT annotations for teacher training, restricting use to settings where such data can be obtained or synthetically generated.

    Source: §6 · p. 10

Abstract

from arXiv · show

Time Series Language Models (TSLMs) offer a promising path toward time series understanding by reasoning over temporal signals and producing natural language answers and explanations. A common approach is Chain-of-Thought (CoT), which generates step-by-step rationales linking relevant signal patterns to final answers. Although these models learn from reference CoT traces during post-training, generating faithful descriptions of input time series at inference remains challenging. Expressing high-dimensional, continuous temporal representations in discrete language tokens may cause the model to neglect task-relevant patterns or describe them inaccurately. Because later reasoning steps build on these descriptions, early errors propagate, leading to incorrect answers with plausible explanations that are inconsistent with the input signal. We propose Lapras (Latent Post-trained Reasoning Across Series), a post-training framework that equips TSLMs with latent reasoning. A model trained with Lapras reasons through a sequence of continuous thoughts in the joint time series-language space, producing text only for the final answer. It learns this through teacher-student self-distillation, where a teacher trained on CoT reference traces reasons explicitly through text. The student aligns its hidden states with the teacher's at the answer stage, transferring the teacher's reasoning ability into its latent computation. We evaluate Lapras across four TSLM backbones on five time series question answering benchmarks. Lapras improves average F1 by up to 10.79% over explicit CoT while generating 23.9x fewer tokens. Lapras's continuous thoughts can also be decoded into readable reasoning traces via standard language decoding, preserving textual explanations. Together, these results highlight Lapras as a promising post-training paradigm for efficient, effective, and interpretable TSLM reasoning.

1 Introduction

Explicit CoT can misdescribe task-relevant temporal patterns, and later reasoning may build on those errors. Lapras instead transfers learning from reference CoT into continuous latent reasoning, generating text only for the final answer.

1 Introduction

  • CoT descriptions may omit or misrepresent temporal patterns, allowing early errors to propagate into incorrect answers.The paper terms this the verbalization bottleneck.Source: §1 · p. 2
  • Whether TSLMs reason more effectively in continuous latent space than through discrete tokens remains underexplored.Prior latent reasoning results cited by the paper concern text-based tasks, where inputs and traces share a discrete token space.Source: §1 · p. 2
  • Lapras uses teacher-student self-distillation so a student learns from reference CoT while reasoning through continuous thoughts and producing only the final answer as text.The student matches the teacher’s hidden states at the answer position without generating a textual step for each continuous thought.Source: §1 · pp. 1–2, Figure 1 · p. 3
  • Across four TSLM backbones and five benchmarks, Lapras improves average F1 by up to 10.79% over CoT while substantially reducing generated tokens.Its continuous thoughts can also be decoded into readable reasoning traces.Source: §1 · p. 2

2 Related Work

TSLMs increasingly process signals as continuous representations, but their open-ended reasoning commonly relies on explicit CoT. Latent reasoning avoids intermediate text generation, while time series signals pose distinct interpretation challenges.

2 Related Work

  • TSFMs support forecasting and fixed-label classification, while TSLMs aim to enable open-ended reasoning over sensor signals.TSLM approaches progressed from serializing data as text to encoding signals as continuous representations for LLMs.Source: §2 · p. 3
  • Explicit CoT is elicited through supervised finetuning or reinforcement learning, but intermediate text may increase inference latency and risk hallucination from linguistic priors..Source: §2 · p. 3
  • Latent methods replace decoded tokens with hidden states, but time series may require jointly interpreting local events and global properties that are difficult to express in language.Lapras instead reasons in a joint time series–language space.Source: §2 · p. 3

3 Method

The diagnostic analysis finds that predicted CoT accuracy diverges from reference CoT during signal description and remains lower through later reasoning. Lapras trains a latent-reasoning student using a CoT teacher, aligning their hidden states at the answer stage.

3.1 Diagnosing the Verbalization Bottleneck

  • The early-answering protocol truncates predicted and reference CoT one sentence at a time, prompting for an answer after each step.Sentences are manually assigned to signal-description or reasoning stages to locate when accuracy diverges.Source: §3.1 · p. 4
  • ∆0.22 on TSR and ∆0.17 on Sleep emerge during signal description; the predicted-trace accuracy gap persists through subsequent reasoning.The analysis reports that predicted-trace accuracy plateaus at the final reasoning stage.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 Lapras

  • Lapras trains a student to reason over the time series and question through continuous thoughts while matching a CoT-trained teacher’s hidden states.At inference, it decodes text only for the final answer.Source: §3.2 · p. 4, Figure 3 · p. 5
  • The teacher receives token-level cross-entropy supervision on CoT traces and answers; the student receives answer-only cross-entropy plus hidden-state alignment at the first answer token.The states are aligned across layers, with stop-gradient used in the distillation term.Source: §3.2 · pp. 4–5
  • At each latent step, the student projects its final hidden state back into the input-embedding space and feeds it forward without sampling a token.After K = 6 continuous thoughts, it returns to language generation for the answer.Source: §3.2 · pp. 4–5
  • Training masks 30% of sensor patches, and the reported ablation finds an average performance improvement of 1.80%.The teacher branch is discarded at inference, leaving only the student.Source: §3.2 · p. 5, Figure 3 · p. 5

4 Experiment

Experiments compare reasoning approaches across five time-series question-answering tasks and four TSLM backbones, using reference CoT traces for explicit CoT post-training.

Datasets

  • Evaluation covers ECG, Engine, HAR, Sleep, and TSR, spanning diagnosis, activity recognition, sleep-stage classification, and counterfactual prediction.The five tasks are formulated as time-series question answering benchmarks.Source: §4 · p. 6
  • Reference reasoning traces use templates populated with annotated labels for ECG and Engine, and Qwen3.5-27B-generated traces for HAR, Sleep, and TSR.The generated traces follow Messica et al. (2026).Source: §4 · p. 6

Baselines

  • Baselines include direct-answer No-CoT, reference-trace-trained explicit CoT, and latent methods iCoT and COCONUT.iCoT progressively removes CoT tokens during training, while COCONUT progressively replaces textual steps with continuous hidden states.Source: §4 · p. 6

Backbones and Metrics

  • The study evaluates four backbones from 0.5B to 7B parameters and reports accuracy and macro-F1 from parsed predicted class labels.OpenTSLM-1B refers to its OpenTSLM-Flamingo variant.Source: §4 · p. 6

5 Results and Discussion

Lapras improves benchmark performance over explicit CoT while reducing inference cost, and its latent reasoning can be decoded into readable traces. Analyses examine answer commitment, task-dependent thought counts, and the effects of design choices.

5.1 Main Results

  • 5.53% average F1 gain over CoT across 20 backbone-dataset configurations, reaching 10.79% on ChatTS-7B; Lapras leads on three backbones and exceeds both latent baselines on all four.Its average F1 advantage over COCONUT is 6.78% across the four backbones.Source: §5.1 · p. 7
  • CoT underperforms No-CoT in average F1 on three of four backbones, with a 2.14% average drop on those backbones; all reasoning methods underperform No-CoT on ITFormer-0.5B.The authors suggest that multi-step reasoning may require a minimum model capacity.Source: §5.1 · p. 7

5.2 Qualitative Analysis

  • Decoded continuous thoughts yield readable traces of signal descriptions, inferences, and answers consistent with reference rationales.Standard language decoding with the shared LM head retrieves the traces post hoc.Source: §5.2 · p. 7, Figure 4 · p. 8
  • Lapras keeps multiple answers plausible early and shifts toward its final choice later, while CoT assigns near-total probability to one answer at the first step.The figure plots cumulative top-1 and top-2 answer probabilities across reasoning steps for representative examples.Source: §5.2 · p. 7, Figure 5 · p. 9
  • On a representative TSR example, Lapras assigns 17.35% of first-layer attention to time-series tokens, versus 3.39% for CoT.Lapras assigns a larger share of attention to the time series at all three inspected layers.Source: §5.2 · pp. 7–8

5.3 Quantitative Analysis

  • 6.9 to 23.9× fewer generated tokens and 5.9 to 15.1× faster inference than CoT, with less than 8 ms overhead over No-CoT.Lapras uses K=6 continuous thoughts and generates text only for the final answer.Source: §5.3 · p. 8, Figure 6 · p. 8
  • 68.38% average accuracy with six continuous thoughts, up from 64.82% with one; eight thoughts yield 68.07%.The largest gains are on TSR (+12.46%) and Engine (+4.66%), while ECG, Sleep, and HAR change by at most 1.09%; the authors relate benefits to multi-step inference needs.Source: §5.3 · pp. 8–9, Table 2 · p. 9

5.4 Ablation Studies

  • Removing latent projection lowers average accuracy from 68.38% to 62.23% (−6.15%), removing distillation to 64.34% (−4.04%), and removing masking to 66.58% (−1.80%).The components map thoughts into the next-step input embedding space, supervise thoughts through distillation, and regularize training through masking.Source: §5.4 · p. 9, Table 3 · p. 10

6 Conclusion

The conclusion reports that Lapras improves F1 over explicit CoT while reducing generated tokens, and that its continuous thoughts can yield readable traces. The authors also identify training-data, computation-allocation, and model-capacity boundaries.

6 Conclusion

  • Lapras requires explicit CoT annotations for its teacher, fixes latent reasoning at K=6, and underperforms No-CoT on the smallest backbone, ITFormer-0.5B.The authors identify adaptive computation allocation and the capacity threshold as open areas for future work.Source: §6 · p. 10
  • Across four backbones from 0.5B to 7B parameters, Lapras improves average F1 over CoT by up to 10.79% and reduces generated tokens by up to 95.8%.The authors report that continuous thoughts can be decoded into readable textual traces.Source: §6 · p. 10

AI use statement

  • The authors used generative AI for writing assistance, retrieval and discovery, and synthetic dataset generation, and report reviewing and validating those outputs.They state that AI was not used for research ideation or execution, drafting paper sections, or proving mathematical claims.Source: AI use statement · p. 11

A Dataset Details

The tasks use traces to shape training, but none are emitted at inference. ECG and Engine traces come from certified labels; HAR, Sleep, and TSR traces are teacher-distilled.

A Dataset Details

  • Traces shape training only: ECG and Engine use certified-label traces, while HAR, Sleep, and TSR use teacher-distilled traces.Table 4 reports split sizes, channel counts, series lengths, and answer formats.Source: A Dataset Details · p. 14, Table 4 · p. 14
  • ECG becomes one binary diagnosis question, with a trace concatenating gold stages from criterion selection through the final diagnostic answer.The stages cover criterion selection, finding identification, ECG grounding, and the diagnostic decision.Source: A Dataset Details · p. 14
  • Engine retains samples with certified channel-specific severity labels and builds traces from component roll-call, severity read-off, and maintenance implication.The maintenance implication follows from the component–severity pair.Source: A Dataset Details · p. 14
  • HAR and Sleep traces identify up to three essential contiguous segments, explain them separately from segment selection, and undergo correctness and template checks.Qwen3.5-27B generates explanations conditioned on rendered signal plots, summary statistics, and the gold answer.Source: A Dataset Details · p. 14
  • TSR pairs an original signal with a counterfactual counterpart sharing its question and choices but flipping the correct answer with signal behavior.Its traces are teacher-generated because the dataset supplies no reasoning rationale.Source: A Dataset Details · p. 14

B Backbone Architectures

The four backbones share a language-model, encoder, connector, and fusion template, but differ in patch encoding, sensor-token count, and fusion. Lapras changes the intermediate tokens, not these backbone components.

B Backbone Architectures

  • All four backbones encode a length-L, C-channel series in P-timestep patches, then connect sensor features to a language model through a fusion mechanism.Their design differences span patch encoding, sensor-token count, and fusion.Source: B Backbone Architectures · p. 15, Table 5 · p. 15
  • Encoder choices range from ChatTS’s isolated-patch MLP to ITFormer’s cross-patch attention, OpenTSLM’s 1D convolution, and SLIP’s pretrained sensor-caption encoder.ChatTS and OpenTSLM defer temporal mixing beyond the encoder; ITFormer features carry local temporal context, while SLIP features are aligned to language.Source: B Backbone Architectures · p. 15
  • ChatTS passes every patch as a sensor token, while the other backbones resample to fixed token counts: 25 for ITFormer and 64 for SLIP and OpenTSLM.ITFormer’s resampler is instruction-dependent; SLIP’s and OpenTSLM’s are question-agnostic.Source: B Backbone Architectures · p. 15
  • ChatTS and ITFormer concatenate sensor tokens for self-attention, whereas SLIP and OpenTSLM inject them through gated cross-attention layers.OpenTSLM injects at every layer; SLIP injects at its final four layers.Source: B Backbone Architectures · p. 15
  • Lapras leaves backbone components unchanged and changes only the tokens emitted between prompt and answer; its latent loop applies across all four backbones.The attention analysis instead uses ChatTS-7B because its sensor tokens share a self-attention sequence.Source: B Backbone Architectures · pp. 15–16

C Architecture Variant Discussion

In the controlled OpenTSLM comparison, SoftPrompt has higher average accuracy than Flamingo under each finetuning method, while Lapras narrows their average-accuracy gap. The larger-backbone generality of this pattern remains open.

C Architecture Variant Discussion

  • 3.57% to 0.67%: Lapras narrows the average-accuracy gap between OpenTSLM-SoftPrompt and OpenTSLM-Flamingo from No-CoT to Lapras.Lapras improves accuracy on both variants; the Table 6 caption identifies the comparison as average change over No-CoT.Source: C Architecture Variant Discussion · p. 16, Table 6 · p. 16
  • OpenTSLM-SoftPrompt exceeds OpenTSLM-Flamingo in average accuracy under every finetuning method, although the per-task leader varies without a consistent pattern.The variants share a 1B language model and differ in fusion pathway.Source: C Architecture Variant Discussion · p. 16, Table 6 · p. 16
  • Whether the reduced fusion-pathway sensitivity under Lapras holds for larger language backbones, such as 7B, remains an open question.The reported gains are consistent across the two tested fusion pathways.Source: C Architecture Variant Discussion · p. 16

D Implementation Details

Training fully finetunes each TSLM with a shared configuration and keeps learning rate and weight decay fixed across finetuning methods within each backbone. Lapras adds a distillation weight and uses task- and backbone-adjusted patch sizes.

D Implementation Details

  • All runs use AdamW, cosine scheduling with warmup, bf16, ZeRO-2, gradient clipping, and effective batch size 128; the TSLM is fully finetuned.Lapras uses six continuous thoughts and a 0.3 patch-mask ratio.Source: D Implementation Details · p. 16
  • Within each backbone, learning rate and weight decay stay fixed across methods; Lapras’s distillation weight λ is the only method-specific setting, defaulting to 10.λ is adjusted among 1, 10, and 20 to keep weighted distillation loss on the same order as the student and teacher cross-entropy losses.Source: D Implementation Details · pp. 16–17
  • Patch sizes follow backbone defaults with computational adjustments, including larger ECG and Engine patches for ChatTS and OpenTSLM-SoftPrompt.ChatTS uses patch sizes 32 and 50 on ECG and Engine; OpenTSLM-SoftPrompt uses 100 and 50, respectively.Source: D Implementation Details · p. 17
  • Epoch counts are shared across backbones and methods per dataset, final checkpoints are used without selection, and evaluation uses greedy decoding capped at 512 new tokens.Each run uses four H200 GPUs.Source: D Implementation Details · p. 17

E Per-Token Predictive Entropy via the Logit Lens

The section uses a logit-lens readout to compare token prediction uncertainty during explicit and latent reasoning. Latent thoughts remain distributed across candidate tokens until answer generation, unlike CoT’s sequence of near-deterministic token steps.

E Per-Token Predictive Entropy via the Logit Lens

  • Shannon entropy measures distribution concentration: low entropy places nearly all probability on one token, while high entropy spreads it across candidate continuations.The effective candidate-token count is 2^mean entropy over the continuous thoughts; a value of 1 corresponds to a deterministic prediction.Source: E Per-Token Predictive Entropy via the… · p. 18
  • The logit lens maps hidden states through the language-model output head to next-token distributions, which are diagnostic for latent thoughts because they are never discretized.For explicit decoding, the readout follows an embedded token and its top-probability token is the emitted token.Source: E Per-Token Predictive Entropy via the… · p. 18
  • N_eff ≈22 candidate tokens per latent thought, with commitment only at the first answer token, unlike CoT’s low-entropy 140-step trajectory.The latent readout’s entropy decreases after the first thought, but it does not reach the 0.95 commitment threshold until the answer begins.Source: Figure 7 · p. 19, E Per-Token Predictive Entropy via the… · p. 19

F Attention on the Time Series

The attention analysis compares how No-CoT, explicit CoT, and latent reasoning allocate attention to time-series tokens. It reports different allocation patterns across layers and includes visualizations of the bottleneck and decoded latent thoughts.

F Attention on the Time Series

  • The attention analysis uses ChatTS-7B and one representative TSR example because the cross-attention backbones do not expose the allocation being measured.The analysis averages series attention over reasoning-token queries for each format.Source: F Attention on the Time Series · p. 20
  • Attention maps show latent reasoning keeps the series in view through the layers where computation happens, while CoT’s reasoning span draws attention away from it.The maps use rows for generated tokens, columns for tokens they can attend to, and color for attention relative to the uniform level.Source: F Attention on the Time Series · p. 20, Figure 8 · p. 21
  • Latent reasoning allocates the most attention to the time series in early layers and less later; No-CoT peaks in the middle, while CoT stays flat and below the other formats almost everywhere.The authors interpret the latent pattern as reading the series early before later layers perform high-level reasoning.Source: F Attention on the Time Series · p. 20

G Additional Visualization

  • The additional visualizations show a temperature-series CoT case ending in an incorrect answer, early-answering accuracy comparisons, and six decoded latent thoughts progressing toward an answer.The decoded steps move from a global series description through local segment descriptions and combined inferences to the answer.Source: Figure 9 · p. 22, Figure 10 · p. 23, Figure 11 · p. 24
Loading 2610.11111v1…