PaperlayerView exact revision on arXiv

Source-linked AI summary

Twice Upon a Time: Complementary Representations for Time-Series Reasoning

Hedi Zisling, Liran Nochumsohn, Ilan Naiman, Mihir Parmar, Omri Azencot

arXiv:2610.22327v1cs.LGcs.AIcs.CL

TL;DR

Time-series language models typically use a single representation, although numerical readout and temporal-dynamics reasoning favor different representational properties. This paper combines measurement-preserving line plots with normalized delay embeddings in a shared language model. The dual-view model reaches 53.2% TSRBench accuracy versus 36.7% for the strongest evaluated dedicated TSLM baseline, with gains concentrated beyond basic perception.

  • Problem

    Existing TSLMs typically expose the language model to a single representation, despite the different representational properties favored by numerical readout and temporal-dynamics reasoning.

    Source: §1 · pp. 2–3

  • Method

    The model combines measurement-preserving line plots and normalized delay embeddings through dedicated visual encoders and feature fusion in a shared language model.

    Source: §1 · pp. 2–3

  • Results

    53.2% TSRBench accuracy exceeds the strongest evaluated dedicated TSLM baseline at 36.7%, with gains concentrated beyond basic perception.

    Source: §1 · p. 3, §5.2 · p. 8

  • Takeaways & Limitations

    Complementary representations provide a design principle for general-purpose TSLMs that combines access to temporal structure with the original measurement frame.

    Source: §1 · p. 3, §6 · p. 9

    The preferred secondary representation varies by evaluation: delay embeddings perform better on TSRBench, while STFT offers small advantages on ChatTS numerical alignment and average reasoning accuracy.

    Source: §6 · p. 9

Abstract

from arXiv · show

Time Series Language Models (TSLMs) enable language-based understanding and reasoning over temporal data, yet a fundamental question remains: what information should their time-series representations preserve, and what variation should they factor out? Existing TSLMs typically expose the language model to a single representation, imposing a fixed inductive bias across downstream tasks. We instead introduce complementary representations as a design principle for time-series reasoning. Our model combines two visual views: line plots preserve absolute measurements and the observed coordinate frame, while normalized delay embeddings factor out positive amplitude rescaling and offsets while exposing temporal structure. Dedicated visual encoders integrate both views into a shared language model through visual tokens and multi-level feature fusion. Evaluation-time interventions verify that the two views exhibit distinct invariance profiles, while single-view ablations show that their combination is particularly beneficial beyond basic perception, with the large gains in prediction and reasoning. Built on Qwen3-VL-8B, our dual-view model improves overall TSRBench accuracy from 44.5% with line plots alone to 53.2%, substantially exceeding the strongest evaluated dedicated TSLM baseline at 36.7%. It further achieves 89.9% accuracy on TSExam and 86.8% numerical accuracy on multivariate ChatTS alignment. Together, these results demonstrate that complementary representations provide an effective design principle for general-purpose TSLMs, enabling models to exploit temporal structure without sacrificing access to the original measurement frame.

1 INTRODUCTION

The introduction asks whether complementary representations can improve general-purpose time-series reasoning, proposing views that preserve different information and induce different invariances. It presents a dual-view model and reports stronger results than evaluated dedicated TSLMs.

1 INTRODUCTION

  • The introduction frames representation as a key determinant of time-series reasoning and asks whether complementary views can improve it beyond single-representation approaches.Existing approaches typically expose the language model to a single representation, with absolute measurement scale either retained or provided as auxiliary metadata.Source: §1 · pp. 1–2
  • Complementary representations pair measurement-preserving line plots with normalized delay embeddings that emphasize temporal structure while factoring out amplitude variation.The introduction motivates the pairing because quantitative readout requires the measurement frame, whereas temporal-dynamics reasoning benefits from factoring it out.Source: §1 · pp. 2–3
  • The proposed dual-tower model uses dedicated visual encoders, compact visual tokens, and multi-level feature fusion to combine both views within a shared language model.The line-plot and delay-embedding streams are integrated into a shared language model.Source: §1 · pp. 2–3
  • 53.2% TSRBench accuracy exceeds the strongest evaluated dedicated TSLM baseline at 36.7% and accompanies 89.9% TSExam accuracy and 86.8% multivariate ChatTS numerical alignment.The dual-view model is built on Qwen3-VL-8B.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work has explored both time-series-specific interfaces to language models and image-based representations of temporal data. This paper shifts the focus from choosing one representation to combining views with different information and invariance properties.

2 RELATED WORK

  • Prior image-based time-series work includes Gramian Angular Fields and Markov Transition Fields for classification and imputation, and delay-embedding or STFT images for diffusion-model generation.The passage attributes the first methods to Wang & Oates (2015) and the latter to Naiman et al. (2024).Source: §2 · p. 3
  • Time-series language-model approaches have used serialized values, textual descriptions, learned embeddings, soft prompts, cross-attention, and direct language-model token representations.Examples named include ChatTS (Xie et al., 2025), OpenTSLM (Langer et al., 2026), TimeOmni-1 (Guan et al., 2026), and TS-Reasoner (Yu et al., 2026b).Source: §2 · pp. 3–4
  • Existing TSLMs predominantly expose one primary representation, whereas this work jointly provides views designed to preserve different information and exhibit different invariances.The paper positions its contribution as combining complementary representations rather than selecting one for a particular predictive or generative task.Source: §2 · pp. 3–4

3 METHOD

The method represents each time series through a measurement-preserving line plot and a normalized delay view, then integrates both streams into one language model. Two-stage training first aligns the delay stream and then tunes the model for time-series reasoning.

3.1 PROBLEM FORMULATION AND OVERVIEW

  • A TSLM takes a natural-language query and a collection of time series as input and generates an answer autoregressively.The overview constructs two visual views per series and supplies their tokens to a shared Qwen3-VL-8B backbone.Source: §3.1 · p. 4

3.2 COMPLEMENTARY VISUAL REPRESENTATIONS

  • Line plots retain original values and coordinate ranges, while normalized delay maps encode overlapping temporal windows and remain unchanged under positive affine transformations.Overlapping windows form a visual map of temporal trajectories; periodicity appears as parallel stripes, trends as smooth gradients, and local events as isolated streaks.Source: §3.2 · pp. 4–5

3.3 DUAL-VIEW ARCHITECTURE

  • Dedicated visual encoders feed plot and delay tokens into a shared Qwen3-VL-8B language model, with multi-level features injected into early language-model layers.The plot stream yields 230 tokens per series and the delay stream yields 64; the streams inject features at disjoint visual-token positions.Source: §3.3 · pp. 5–6, Figure 2 · p. 5

3.4 TRAINING

  • Training first aligns the delay stream using captioning data, then freezes the visual pathway and applies language-model LoRA for time-series question answering.In Stage I the Qwen vision encoder and language model remain frozen; Stage II instruction-tunes the language model to reason jointly over both views.Source: §3.4 · p. 6

4 TRAINING DATA

The two-stage training corpus aligns the delay-embedding stream with caption supervision, then instruction-tunes the model across perception, numerical readout, forecasting, and reasoning. Most series and labels are generated procedurally, with evaluation examples kept separate from training.

4 TRAINING DATA

  • Stage I uses 55,696 caption-alignment examples from real-world series and synthetic captions, with its three sources sampled equally.The sources balance real-world diversity, free-form synthetic descriptions, and controlled constructive supervision.Source: §4 · pp. 6–7
  • Stage II contains 182,815 instruction examples spanning question answering, perception, numerical tasks, and broad temporal reasoning.Its 100,000-example in-house reasoning corpus covers numerical, deductive, inductive, abductive, causal, temporal-relation, forecasting, and event-prediction tasks.Source: §4 · p. 7
  • In-house synthetic supervision comes from known generators or exact computations, and evaluation examples from TSRBench, TimeSeriesExam, and ChatTS are excluded.The authors also checked the reasoning corpus for exact series overlap, shared sports-game instances, and exact question–answer matches.Source: §4 · p. 7

5 EXPERIMENTS AND RESULTS

Across TSRBench, controlled comparisons show higher overall accuracy for the dual-view model than for single-view and same-view variants, with gains concentrated beyond perception. Evaluation-time interventions show distinct robustness profiles, and results also extend to TSExam and ChatTS.

5.1 EXPERIMENTAL SETUP

  • The evaluation spans TSRBench’s 15 tasks and compares general-purpose vision-language models with dedicated TSLMs; GPT-5 is included as a larger proprietary reference, not a size-matched competitor.TSExam and ChatTS probe additional aspects including pattern recognition, noise understanding, anomaly detection, numerical alignment, and higher-level reasoning.Source: §5.1 · p. 7, §5.2 · p. 8, Table 3 · p. 8

5.2 DO COMPLEMENTARY REPRESENTATIONS IMPROVE TEMPORAL REASONING?

  • 53.2% overall TSRBench accuracy exceeds the strongest evaluated dedicated TSLM result of 36.7% and GPT-5 with visual input at 52.4%, while approaching GPT-5 with text and visual input at 55.6%.The GPT-5 systems are substantially larger proprietary models, not size-matched comparisons.Source: §5.2 · p. 8, Table 3 · p. 8
  • The dual-view model raises overall accuracy from 45.2% for Plot only to 53.2%, with larger gains in Prediction (38.0% to 61.7%) than in Perception (87.4% to 86.4%).Reasoning rises from 35.8% to 40.1%, while Decision-Making rises from 36.4% to 37.6%; Plot+Plot scores 44.8% overall and Delay+Delay 42.6%.Source: §5.2 · p. 8, Table 3 · p. 8

5.3 HOW DO THE REPRESENTATIONS RESPOND TO DISTRIBUTION SHIFTS?

  • Delay only changes little under affine rescaling and offset shifts, while the combined model remains sensitive to them but has higher absolute accuracy than Plot only across reported interventions.Noise and smoothing degrade both Plot only and the combined model, whereas the combined model changes relatively little with sampling-horizon shifts; the analysis covers 2,020 items and excludes tasks dependent on absolute coordinates or sampling rates.Source: §5.3 · pp. 8–9, Table 4 · p. 9

5.4 CROSS-BENCHMARK GENERALIZATION

  • The model achieves 89.9% TSExam accuracy and 86.8% numerical accuracy on multivariate ChatTS, alongside 80.5% numerical accuracy on univariate ChatTS.The authors report that it exceeds compared baselines on TSExam and that its largest ChatTS advantage is in the multivariate setting.Source: §5.4 · p. 9

6 DISCUSSION

The discussion frames complementary representations as a design principle because numerical readout and temporal-dynamics reasoning favor different representational properties. The preferred secondary representation varies by evaluation.

6 DISCUSSION

  • The dual-view model raises TSRBench accuracy from 45.2% for Plot only to 53.2%, with gains concentrated beyond basic perception and not reproduced by same-view pairs.It also leads evaluated baselines on TSExam and multivariate ChatTS numerical alignment, at 89.9% and 86.8% accuracy, respectively.Source: §6 · p. 9
  • The preferred secondary representation varies across evaluations: delay embeddings perform better on TSRBench, while STFT offers small advantages on ChatTS numerical alignment and average reasoning accuracy.The discussion identifies additional representations, encoder combinations, more than two views, and query-dependent combination as future directions.Source: §6 · p. 9

A.1 VISUALIZATION OF COMPLEMENTARY REPRESENTATIONS

The appendix contrasts a measurement-preserving line plot with a normalized delay map that exposes temporal structure. It also specifies delay-map construction and visual-token details for the two encoder streams.

A.1 VISUALIZATION OF COMPLEMENTARY REPRESENTATIONS

  • The line plot retains absolute amplitudes and axis information, whereas the normalized delay map removes offsets and positive amplitude rescaling while emphasizing temporal structure.Overlapping windows appear as columns, with recurring structure represented by diagonal stripes.Source: §A.1 · p. 13, Figure 3 · p. 13

A.2 DELAY-MAP CONSTRUCTION DETAILS

  • The delay-map window length equals the series length below 48, is fixed at 48 for lengths 48–79, scales as ⌊0.6m⌋ for lengths 80–320, and is capped at 192 for longer series.The map is min–max normalized after within-window interpolation and has fixed dimensions H × W.Source: §A.2 · p. 13, Table 5 · p. 13

B.1 IMPLEMENTATION DETAILS

  • The line-plot stream yields 230 visual tokens per series, while the delay stream yields 64 tokens per variate; multi-level delay features are added at three language-model layers.The delay features use separate projections and are injected at the delay-token positions.Source: §B.1 · p. 14

B.2 TRAINING OBJECTIVE AND HYPERPARAMETERS

The training objective minimizes autoregressive language-modeling loss, with optimization restricted to selected parameters and target answer tokens. The appendix also reports cross-benchmark representation ablations and TSRBench category-level results.

B.2 TRAINING OBJECTIVE AND HYPERPARAMETERS

  • Training minimizes autoregressive language-modeling loss, masking prompts, visual tokens, and padding so that only target answer tokens contribute.Stage I trains DINOv3 LoRA and projection parameters; Stage II trains language-model LoRA parameters.Source: §B.2 · p. 14

C ADDITIONAL EXPERIMENTAL RESULTS

  • Table 6 compares the dual-view model with single-view and duplicated-view controls across ChatTS and TSRBench.The duplicated-view controls use the same representation in both towers while matching the full model's architecture and visual-token budget.Source: C ADDITIONAL EXPERIMENTAL RESULTS · p. 14, Table 6 · p. 15
  • Table 7 aggregates TSRBench results by category, complementing the individual task results reported in Table 3.The appendix also includes detailed ChatTS results, intervention analyses, and an alternative frequency-domain representation.Source: C ADDITIONAL EXPERIMENTAL RESULTS · p. 14, Table 7 · p. 15

C.1 CROSS-BENCHMARK REPRESENTATION ABLATIONS

The cross-benchmark ablations show that the contribution of complementary views varies by evaluation. On TSRBench, the dual-view model improves beyond basic perception, while same-view controls do not reproduce its overall gain.

C.1 CROSS-BENCHMARK REPRESENTATION ABLATIONS

  • Neither duplicated-view control reproduces the complementary pair's overall TSRBench gain despite matching its architecture and visual-token budget.Plot-only is already strong on TSExam and ChatTS; combining views provides modest ChatTS numerical-alignment and reasoning improvements but a larger TSRBench improvement.Source: §C.1 · p. 15

C.2 TSRBENCH CATEGORY-LEVEL ABLATIONS

  • The dual-view model leaves Plot-only perception essentially unchanged while improving aggregate reasoning, prediction, and decision-making results on TSRBench.Same-view controls remain close to their corresponding single-view variants in overall accuracy.Source: §C.2 · p. 15

C.3 DETAILED CHATTS RESULTS

ChatTS results show that no single representation uniformly leads across reasoning dimensions, while the full model has the highest reasoning average. For numerical alignment, the dual representation improves on Plot-only.

C.3 DETAILED CHATTS RESULTS

  • The full model has the highest average across four ChatTS reasoning evaluations, although single-view variants remain stronger on some dimensions.The evaluation reports inductive, deductive, and causal reasoning, plus two-series multiple-choice questions.Source: §C.3 · pp. 15–16, Table 8 · p. 16
  • ChatTS numerical accuracy rises from 0.781 to 0.805 on ChatTS-A and from 0.864 to 0.868 on ChatTS-B with the dual representation.The dual representation preserves Plot-only's strong categorical performance.Source: §C.3 · p. 16

C.4 COMPLETE INTERVENTION RESULTS

The intervention and alternative-representation analyses reveal distinct sensitivity profiles and benchmark-dependent performance. The appendix also documents matched same-view controls, evaluation procedures, and the composition of both training stages.

C.5 ALTERNATIVE SECONDARY REPRESENTATIONS

  • Alternative secondary views have similar TSExam results when combined with plots, but distinct intervention profiles; after full training, delay is stronger on TSRBench and TSExam.STFT is slightly stronger on ChatTS numerical alignment and reasoning average, while the models are effectively tied on ChatTS categorical alignment.Source: §C.5 · pp. 16–17, Table 10 · p. 17, Table 12 · p. 18

C.4 COMPLETE INTERVENTION RESULTS

  • Delay-only shifts by at most 0.8 percentage points under five affine interventions, while dual-view category-average accuracy is virtually unchanged under sampling-horizon changes.The intervention table covers all 15 TSRBench tasks; temporal reversal and sign inversion can change gold answers, so invariance is not always desirable.Source: §C.4 · p. 16, Table 9 · p. 17

C.6 VISUAL-TOKEN BUDGET AND SAME-VIEW CONTROLS

  • Same-view controls match the full model's architecture, trainable parameters, visual-token budget, and training procedure, yet neither reproduces its gain.The controls differ in whether both towers receive identical or distinct views.Source: §C.6 · pp. 17–18

C.7 COMPLETE BENCHMARK RESULTS

  • The complete benchmark results cover TSRBench subtasks, ChatTS alignment and reasoning, and TSExam categories, with specified model-evaluation settings.The Qwen vision-language models were re-evaluated using the same plot-rendering pipeline as the proposed model.Source: §C.7 · pp. 18–19, Table 13 · p. 19, Table 14 · p. 20

D TRAINING DATA DETAILS

  • Training uses two stages: visual-representation alignment with descriptive supervision, followed by instruction tuning for time-series understanding and reasoning.The training mixtures draw on procedurally generated data and a small number of real-world time-series sources.Source: D TRAINING DATA DETAILS · p. 21

D.1 TRAINING CORPUS OVERVIEW

  • Stage I uses 55,696 raw examples and equal-probability sampling across three sources; Stage II draws 182,815 examples per epoch.Stage II retains 100,000 reasoning and 11,000 numerical examples, dividing the remaining draws approximately equally among three sources.Source: §D.1 · p. 21, Table 17 · p. 22

D.2 STAGE I: MODALITY-ALIGNMENT DATA

  • Stage I aligns visual time-series representations with language using descriptive examples rather than downstream reasoning tasks.Its 55,696 examples combine CaTS, ChatTS-style captions, and structural captions, with equal source-level sampling.Source: §D.2 · p. 21

D.3 STAGE II: INSTRUCTION-TUNING DATA

  • Stage II combines instruction-following, perception, numerical, and reasoning data, and excludes evaluation examples from both training stages.Its reasoning corpus covers a broad range of temporal reasoning capabilities and is constructed independently of TSRBench evaluation series, questions, and supervision.Source: §D.3 · p. 22

D.4 REASONING CORPUS SPECIFICATION

The reasoning corpus combines nine task families across 25 templates, with supervision tailored to task type and data drawn mostly from synthetic processes. The numerical corpus supplies exact-statistic questions over generated series.

D.4 REASONING CORPUS SPECIFICATION

  • The reasoning corpus contains 100,000 examples across nine task families and 25 generation templates.Table 18 summarizes family sizes, data-generating processes, channels, answer choices, and supervision formats.Source: §D.4 · p. 22, Table 18 · p. 23
  • Rationale supervision is retained for numerical reasoning, event prediction, and inductive reasoning, while six other families use final-option supervision despite available generated rationales.The rationale-supervised families comprise 36,328 examples; the other six comprise 63,672, and rationales for those families are discarded in the final configuration.Source: §D.4 · p. 22
  • Most reasoning-corpus families use synthetic generative processes; real NBA and NFL play-by-play data appear only in two abductive-reasoning templates, covering 2,156 items (2.2%).Questions and supervision for those real-data examples are constructed independently of the evaluation sets.Source: §D.4 · p. 23

D.5 NUMERICAL CORPUS SPECIFICATION

  • The numerical corpus contains 11,000 examples across three subcategories and 20 templates, each asking for a statistic from a generated series.Each item is a length-256 series from eleven generating recipes with additive observation noise; the exact statistic is computed on the observed series and rounded to two decimals, without answer options. Table 19 summarizes the subcategories and targeted statistics.Source: §D.5 · p. 24, Table 19 · p. 24

Related papers

Loading 2610.22327v1…