PaperlayerView exact revision on arXiv

Source-linked AI summary

LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics

Yueyang Ding, HaoPeng Zhang, Rui Dai, Yi Wang, Tianyu Zong, Kaikui Liu, Xiangxiang Chu

arXiv:2604.17295v1cs.AI

TL;DR

Time-series reasoning lacks a unified taxonomy and comparable benchmarks, limiting rigorous evaluation. LLaTiSA addresses this with the 83k-sample HiTSR dataset and a curriculum-trained dual-view model, consistently outperforming strong baselines across OOD benchmarks and real-world scenarios.

  • Problem

    Time-series reasoning lacks formalized task definitions and comparable benchmarks, making unified assessment across capabilities and difficulty levels difficult.

  • Method

    The paper introduces the 83k-sample HiTSR dataset and trains LLaTiSA with dual-view plots, index-value tables, and a three-stage curriculum aligned with L1-L3 reasoning.

  • Results

    LLaTiSA consistently outperforms strong baselines across diverse out-of-distribution benchmarks and real-world scenarios, while transferring to domain-specific semantic reasoning.

  • Takeaways & Limitations

    HiTSR and LLaTiSA provide a practical path toward more robust unified time-series reasoning models.

  • Takeaways & Limitations

    The study focuses on supervised curriculum fine-tuning and leaves reinforcement-learning fine-tuning on HiTSR for future work because reward design across L1-L4 is challenging.

Abstract

from arXiv · show

Comprehensive understanding of time series remains a significant challenge for Large Language Models (LLMs). Current research is hindered by fragmented task definitions and benchmarks with inherent ambiguities, precluding rigorous evaluation and the development of unified Time Series Reasoning Models(TSRMs). To bridge this gap, we formalize Time Series Reasoning (TSR) via a four-level taxonomy of increasing cognitive complexity. We introduce HiTSR, a hierarchical time series reasoning dataset comprising 83k samples with diverse task combinations and verified Chain-of-Thought (CoT) trajectories. Leveraging HiTSR, we propose LLaTiSA, a strong TSRM that integrates visualized patterns with precision-calibrated numerical tables to enhance the temporal perception of Vision-Language Models (VLMs). Through a multi-stage curriculum fine-tuning strategy, LLaTiSA achieves superior performance and exhibits robust out-of-distribution generalization across diverse TSR tasks and real-world scenarios. Our code is available at https://github.com/RainingNovember/LLaTiSA.

1 Introduction

The paper frames unified time series reasoning as limited by fragmented task definitions and numerical-reasoning weaknesses, then introduces a four-level taxonomy, the HiTSR dataset, and LLaTiSA to address these gaps. LLaTiSA uses numerically enhanced visual reasoning and curriculum training, with experiments showing strong baseline performance and out-of-distribution generalization.

  • Existing unified TSRM development is hindered by the lack of a formalized reasoning-task taxonomy and benchmarks with fragmented or ambiguous task definitions.
  • The proposed taxonomy models time series reasoning as four increasing cognitive levels, progressing from numerical grounding to perception, semantic interpretation, and context-aware generation.
  • LLaTiSA is a VLM-based TSRM that combines qualitative visual perception with quantitative precision and is trained through a three-stage curriculum aligned with L1-L3.
  • HiTSR contains over 83k samples spanning L1-L3, with diverse task settings, unambiguous ground truths, and verified reasoning chains.
  • Extensive experiments show that LLaTiSA consistently outperforms proprietary models and open-source baselines while generalizing strongly to out-of-distribution scenarios.

2 Related Works

Recent multimodal time series research spans question answering, scientific analysis, anomaly detection, forecasting, and multidimensional task taxonomies. Time-series understanding generally uses textual serialization, visual plots, or dedicated time-series encoders.

  • Multimodal Time Series Datasets: Multimodal time series datasets cover question answering, traditional analysis, scientific analysis, anomaly detection with explanatory reasoning, and context-aware forecasting.TSQA and TRQA explore question answering and traditional analysis; SciTS targets scientific domains, RATs40k adds explanatory reasoning to anomaly detection, and TimeMMD addresses context-aware forecasting.
  • Multimodal Time Series Datasets: MMTS-Bench introduces a multidimensional task classification taxonomy but overlooks basic numerical read-out and combining diverse real-world semantics.The taxonomy broadens task classification while leaving numerical read-out and semantic combination unevaluated.
  • Encoding Paradigms: Time-series understanding via language models follows three encoding paradigms: text-token serialization, visual plot rendering for VLMs, and dedicated time-series encoders in TS-MLLMs.Examples include TimeOmni-1 and Time-R1 using textual inputs for context-aware forecasting, while ChatTS and ITFormer are introduced as visual or encoder-based approaches in the supplied passage.

3 Method

The method formalizes time series reasoning as a four-level hierarchy, then builds the verified HITSR dataset and LLATISA framework around progressively harder capabilities. LLATISA combines visual and structured numerical views with a three-stage curriculum to improve grounded reasoning.

  • TSR Taxonomy: A four-level taxonomy organizes TSR from numerical read-out and pattern perception to semantic reasoning and predictive inference.It provides a unified schema for task categorization and a diagnostic lens for model cognitive boundaries.
  • HITSR Dataset: HITSR comprises approximately 83k synthetic and real-world time-series samples spanning L1-L3, with rigorous validity checks and multi-stage annotation and verification.Its subsets contain 30,000 L1, 50,703 L2, and 3,121 L3 samples, each with a human-verified high-fidelity test set.
  • CoT Verification: L2 and L3 tasks use distinct verified Chain-of-Thought annotation pipelines, including numerically aware descriptions, distractor construction, cross-validation, and human auditing.The process removes ambiguous or flawed items and retains only reviewed evaluation examples.
  • LLATISA Framework: LLATISA pairs a standard time-series visualization with an index-value table image, enabling global pattern inspection followed by point-accurate numerical verification.Compared with textual arrays, the framework provides extra index information, mitigating numerical hallucinations and improving numerical-sensitive task performance.
  • Curriculum Fine-Tuning: A three-stage curriculum sequentially fine-tunes LLATISA on HITSR-L1 and HITSR-L2 before training on HITSR-L3 or general-purpose domain-specific TSR benchmarks.The first two stages consolidate numerical read-out precision and pattern perception capabilities.

4 Experiment

The experiment evaluates LLaTiSA on OOD TSR benchmarks, encoding strategies, and real-world ECG interpretation, while testing curriculum learning and verified CoT supervision. Results show broad TSR gains, stronger lead-wise ECG performance, and substantial OOD degradation without CoT or curriculum training.

  • Experimental setup: LLaTiSA is evaluated on OOD TSR datasets across levels L1-L3 against textual, visual, dual-modality, and multiple TSRM baselines.The evaluation uses Qwen3-VL-8B-Instruct as its backbone and includes models such as GPT-4o, Qwen3, Time-R1, Time-MQA, and OpenTSLM.
  • Encoding strategy comparison: LLaTiSA consistently outperforms alternative time-series representation strategies on most OOD tasks across L1-L3 under the same backbone and training configuration.The comparison covers textual arrays, sequence plots, numerical grids, and dual-modality inputs, with all strategies evaluated under standardized training settings.
  • Encoding strategy comparison: Explicit index information substantially improves point-level localization, increasing both accuracy and success rate, while fine-tuning markedly improves L1 success rates for text-based encodings.The success-rate improvement is especially notable for the text-based “w/o index” variant.
  • Real-world generalization: In ID ECG evaluation, LLaTiSA surpasses GEM (LLaVA) by 18.14% in lead assessment coverage and 14.22% in lead assessment accuracy.The lead-wise evaluation reflects adherence to the structured 12-lead diagnostic procedure used by professional clinicians.
  • CoT and curriculum ablations: Removing verified CoT supervision decreases OOD accuracy by 4.31% and 17.91%, while single-stage joint training reduces OOD performance by 1.74% on L2 and 14.93% on L3.Joint training also lowers ID performance by 2.06% on L2 and 3.45% on L3, with the largest degradation occurring on complex OOD L3 tasks.

5 Conclusion

The paper formalizes Time Series Reasoning through a difficulty-stratified four-level taxonomy and introduces HiTSR, a large-scale hierarchical dataset for reliable training and evaluation. Building on HiTSR, LLaTiSA combines visual plots, auxiliary numerical tables, and three-stage curriculum training for TSR.

  • Contributions: The paper formalizes TSR with a four-level taxonomy, L1–L4, characterizing progressively complex reasoning capabilities.The taxonomy provides a difficulty-stratified view of TSR.
  • Contributions: HiTSR contains over 83k high-fidelity TSR questions spanning L1–L3, with unambiguous labels and verifiable reasoning chains.The dataset supports reliable training and evaluation.
  • Contributions: LLaTiSA pairs time series plots with an auxiliary index-value table to connect qualitative perception with precise numerical evidence.The model is trained through a three-stage curriculum aligned with L1–L3.

Limitations

The study focuses on dataset construction and supervised curriculum fine-tuning, leaving reinforcement learning fine-tuning on HiTSR for future work. Applying RFT remains challenging because reward design must jointly supervise numerical precision and semantic logic across the L1–L4 taxonomy.

  • The study primarily focuses on systematic dataset construction and supervised curriculum fine-tuning.
  • Reinforcement Learning Fine-Tuning on HiTSR remains an unexplored future direction.
  • RFT implementation is challenging because reward design must supervise both low-level numerical precision and high-level semantic logic across L1–L4.

Ethics Statement · Appendix · A Findings

The work uses publicly available datasets and language models, with HiTSR derived from established benchmarks and synthetic pipelines and LLaTiSA built on an open-sourced model.

  • Ethics Statement: All datasets used in the work are publicly available.
  • Ethics Statement: All language models used in the work are publicly available.
  • Ethics Statement: HiTSR was constructed from established time series benchmarks.
  • Ethics Statement: HiTSR also incorporates synthetic data generation pipelines.
  • Ethics Statement: The time series benchmarks used to construct HiTSR are publicly accessible.
  • Ethics Statement: LLaTiSA is built on an open-sourced model.

A.1 Model Analysis … C.4 Illustration of HITSR

The paper diagnoses perception-reasoning and benchmark-design gaps in time series reasoning, then constructs and verifies HITSR across numerical, perceptual, and semantic tasks using synthetic and real-world data.

  • A.1 Model Analysis: Existing LLMs, VLMs, and specialized TSRMs struggle with foundational L1-L2 tasks, revealing a fundamental perception-reasoning gap.
  • A.2 Dataset Analysis: Current TSR benchmarks contain ambiguous labels, insufficient context, and limited training-test splits.
  • B Data Source; B.1 Synthetic Time Series: HITSR uses a complete synthetic time series pipeline with merged attribution presets to avoid multiple correct options in local pattern questions.
  • B Data Source; B.2 Real-world Time Series: Real-world datasets support OOD L1 evaluation and semantic subsets, including ETT, Weather, Exchange Rate, Traffic, Electricity, UTSD, Monash, and Time-MMD.
  • B.2 Real-world Time Series: Semantic data are expanded through random cropping and GPT-5-generated scenario-aligned MCQs whose distractors target numerical precision, temporal perception, and contextual consistency.
  • C Details of HITSR; C.1 Task definition: HITSR defines numerical read-out, pattern perception, and semantic understanding tasks within its hierarchical task framework.
  • C.2 Data Verification: Data verification uses rule-based annotations and perturbations, cross-verifies candidate labels, and applies GPT-5 with metadata and plots to generate reasoning-enhanced answers.
  • C.3 Prompt Templates; C.4 Illustration of HITSR: Construction prompts cover L2 annotation and CoT generation plus L3 QA generation and question examination, with examples provided for numerical read-out, pattern perception, and semantic reasoning.

C.5 Comparison to Existed TSR Datesets

HiTSR distinguishes itself from existing TSR datasets through verified reasoning trajectories, numerically grounded CoT, foundational perceptual diagnostics, and a difficulty-stratified taxonomy. Its design supports learning reasoning processes, validating numerical accuracy, and diagnosing failures across reasoning levels.

  • Verified Chain-of-Thought: HiTSR provides 83k samples with verified Chain-of-Thought trajectories, converting raw data into instruction-tuning pairs that teach reasoning processes rather than answer mapping.This contrasts with evaluation-only benchmarks such as BEDTime, TimeMCQ2, and MMTS-Bench.
  • Grounded Reasoning: HiTSR uses unambiguous CoT with a strict Numerical -> Pattern -> Semantic progression, cross-verifying each step against ground-truth data.The approach aims to ground reasoning in numerical reality rather than textual hallucination.
  • Foundational TSR Capabilities: HiTSR introduces L1 precise grounding and localization tasks as a diagnostic anchor before evaluating complex TSR abilities.This addresses existing studies’ emphasis on L2-L4 tasks while overlooking numerical read-out verification, particularly over long horizons.
  • Difficulty-Stratified Taxonomy: HiTSR’s four-level difficulty-stratified taxonomy supports curriculum learning from L1 to L3 and pinpoints where a TSRM’s reasoning chain breaks down.The taxonomy is grounded in Bloom’s Taxonomy and Bertin’s Levels of Reading.

D Evaluation on the HITSR … K.3 Transition to Vision Modalities

The paper evaluates TSRMs across HITSR’s difficulty-stratified tasks, motivates the taxonomy theoretically, distinguishes understanding from generation, and extends evaluation to L4 predictive inference with multimodal inputs. It also documents baselines, prompting, case studies, dataset construction, and training details.

  • D Evaluation on the HITSR: GPT-4o’s L1 accuracy improves from 6.0% to 23.0% when temporal indices are added to textualized numerical arrays, aiding numerical grounding.The HITSR test set contains 100 randomly sampled questions; ChatTS estimates approximate values effectively but struggles with correct mapping.
  • D Evaluation on the HITSR: GPT-4o reaches 93.0% and 94.0% on global pattern differentiation but remains around 55.0% on local differentiation, exposing single-modality limitations.Integrating vision and text raises GPT-4o’s L2-Local accuracy to 69.0%, combining visual global recognition with textual numerical precision.
  • D Evaluation on the HITSR: 78.0% is Qwen3-VL-8B’s state-of-the-art accuracy on L2 Numerical Perception with vision and indexed textual series, whereas Qwen3-8B reaches 36.0% without indices.Subtle numerical perturbations create challenging local-pattern distractors.
  • E Training Details; F.1 Time Series Reasoning Models: The three-stage SFT uses documented training parameter settings and specified training-data proportions, while baseline TSRMs include Time-MQA, Time-R1, ChatTS, and OpenTSLM.Time-MQA applies LoRA to LLM backbones; Time-R1 uses RFT for forecasting; ChatTS uses a 5-MLP encoder; OpenTSLM uses curriculum training transferred to medical TSR tasks.
  • F.2 Instruction Templates: Instruction templates cover text models, LLaTiSA and VLMs with numerical-table images, vision models with textual series, and adapted official prompts for existing TSRMs.The templates are referenced in Figures 7–9.
  • G Case Study: OOD case studies illustrate LLaTiSA responses for L1 Min/Max Grounding, L2 Local and Global Pattern Differentiation, and L3 Semantic Understanding.The cases are organized in Tables 31–33 and an L3 case listing.
  • I Theoretical Underpinning; J Position of Our Work: HITSR’s four levels align with Bloom’s cognitive progression and Bertin’s visual-reading levels, enabling diagnosis of perceptual bottlenecks at L1–L2 versus reasoning deficits at L3–L4.L1–L3 are treated as TSR-Understanding, while L4 generation is a distinct paradigm requiring specialized generative modeling and avoiding confounding generative precision.

K.4 Experiment Results · K.5 Ablation Study · L Expansion of Experiments

LLaTiSA’s fourth-stage curriculum improves out-of-distribution L4 predictive performance on real-world TRQA data, while ablations show that sequential mastery of foundational levels is critical. The results also demonstrate advantages over text-only TSRMs, Qwen3-VL-8B, and joint training.

  • L Expansion of Experiments: The expanded L1→L2→L3→L4 curriculum extends LLaTiSA from diagnostic understanding to L4 predictive inference on real-world data.This fourth-stage expansion directly evaluates whether the model can generalize TSR capability to generative tasks.
  • K.4 Experiment Results: LLaTiSA outperforms close-source and open-source TSRMs using textual time-series inputs.The result supports the effectiveness of its dual-view multimodal paradigm, HiTSR dataset, and multi-stage curriculum fine-tuning.
  • K.4 Experiment Results: LLaTiSA significantly outperforms its backbone model Qwen3-VL-8B.The passage attributes this gap to dual-view representation and multi-stage training on difficulty-stratified TSR datasets.
  • K.4 Experiment Results: 29.1% improvement raises L4 performance from 54.2% with L3 training to 83.3% after L4-specific curriculum training.The L4 stage was trained on 4,993 L4-specific questions for 2 epochs at a learning rate of 1 × 10−5.
  • K.4 Experiment Results: 83.3% accuracy on real-world TRQA data demonstrates robust out-of-distribution generalization beyond synthetic training distributions.The evaluation used real-world data differing significantly from the synthetic training distributions.
  • K.5 Ablation Study: Sequential curriculum training is critical: single-stage training fails on L4 tasks without foundational grounding from L1–L3.The ablation compares single-stage, joint multi-task, and sequential curriculum regimes, with “+” denoting joint training and “→” sequential training.
  • K.5 Ablation Study: 7.9% improvement over joint training (“L1+L2+L3+L4”) confirms the benefit of step-by-step mastery for complex predictive inference.The ablation supports L1–L3 as prerequisites for L4 predictive inference.

L.1 Ablation Study … M.2 Cross-Model Validation

The ablation results establish that LLaTiSA’s three-stage curriculum is necessary for balanced performance across cognitive levels, while baseline evaluations show advantages over ChatTime and comparable VLMs. Data-quality procedures combine deterministic checks, expert audits, and cross-model validation to support annotation reliability.

  • L.1 Ablation Study: The three-stage curriculum is necessary because high-level training alone does not preserve foundational perception, while balanced training across cognitive levels improves TSR performance.“Only L2” causes L1 performance to collapse, whereas the full curriculum achieves the highest reported L3 scores: 87.0% ID and 67.0% OOD.
  • L.1 Ablation Study: 87.0% ID and 67.0% OOD are the full curriculum’s highest L3 scores, exceeding the combined L1 and L2 strategy’s 70.0% ID.The L3 ID score rises from 66.0% to 70.0% when L1 and L2 are combined before reaching 87.0% ID under the full curriculum.
  • L.1 Ablation Study: LLaTiSA consistently outperforms both “L1 →L2” and “L1 + L2,” supporting training on L3 and the multi-stage curriculum regime.The comparison is presented as evidence for the effectiveness of the L3 training stage and staged rather than joint curriculum design.
  • L.2 Baseline Comparison: ChatTime performs poorly across all levels on OOD benchmarks and cannot natively handle multiple series or instruction-defined output formats.Evaluation required manually concatenating multiple series and specifying segment lengths for its Series Comparison task.
  • L.2 Baseline Comparison: LLaTiSA significantly outperforms Qwen3-VL-8B, highlighting the contribution of dual-view representation and multi-stage training on difficulty-stratified TSR data.The passage frames Qwen3-VL-8B as the backbone model and attributes the discrepancy to these design choices.
  • L.3 Expansion of VLM Baselines: LLaTiSA consistently and significantly outperforms comparable VLM-based baselines across visual modalities and numerical-series representation strategies.The additional VLMs perform similarly to Qwen3-VL baselines yet fall short on both L1 numerical grounding and L3 semantic reasoning.
  • M.1 Human Verification: 0.853 Cohen’s Kappa indicates almost perfect agreement between two independent annotators re-verifying 500 random L2 and L3 samples.The protocol checks numerical faithfulness, logical consistency of CoT trajectories, and ground-truth correctness.
Loading 2604.17295v1…