PaperlayerView exact revision on arXiv

Source-linked AI summary

TimeBraid: Unifying Time Series and Language for Understanding and Forecasting

Xinyue Wang, Jiacheng Pang, Kun Zhou, Kexin Zhang, Defu Cao, Fan Feng, Faisal, Songyao Jin, Yan Liu, Biwei Huang

arXiv:2609.29792v1cs.CLcs.AIcs.CE

TL;DR

Time-series data remains largely outside multimodal unification, and connecting it to language is challenging because the modalities have an inherent semantic gap. TimeBraid aligns pretrained language and time-series models through shared attention and a two-stage training recipe. Across understanding and forecasting benchmarks, it reports competitive results, including strong forecasting comparisons and 80.65% on TSAQA for its 6.7B model.

  • Problem

    Time-series data remains largely outside unified multimodal models, and existing add-on approaches can create representational gaps that degrade pretrained capabilities and limit bidirectional coverage.

    Source: §1 · p. 2

  • Method

    TimeBraid connects pretrained language, time-series perception, and forecasting experts through global residual attention, then aligns the modalities on paired data before adapting the model to downstream tasks.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    Across twelve understanding and forecasting benchmarks, TimeBraid-6.7B reaches 80.65% on TSAQA, compared with 85.26% for TSAQA-specific LLaMA3.1-8B SFT and 63.10% for GPT-5.4, while attaining the best average MSE rank on nine Time-MMD domains.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    TimeBraid provides one set of weights for understanding and generating both time series and language, with forecasting at the level of dedicated time-series foundation models and competitive understanding results against far larger general-purpose models.

    Source: §7 · p. 17

    The model has bounded numeric precision and can misread complex waveforms because its inherited patch tokenization compresses raw values and does not fully preserve grounded local detail.

    Source: §6 · p. 16

Abstract

from arXiv · show

We present TimeBraid, a series of unified time-series and language models that align pretrained language models and pretrained time-series foundation models through interleaved global residual attention layers. Each model inherits knowledge, instruction following, and reasoning from one side, continuous-signal perception and zero-shot forecasting from the other, and fuses the two in a shared representation space where both modalities are understood and generated. We study the design choices that make such unified modeling work: where to align the two representation spaces, how to ground language in temporal structure, how to balance understanding with generation, and how to keep joint optimization stable. The resulting recipe combines a unified prompting scheme for diverse time-series and text tasks, stabilized joint training, and supervision from 2.2M curated series--text pairs and 4.9M instruction-tuning samples. Across benchmarks spanning time-series perception, understanding, reasoning, and both context-aided and unimodal forecasting, TimeBraid remains competitive with far larger general-purpose models and task-specific counterparts.

1 University of California San Diego, 2 University of Southern California, 3 Aether AI

The supplied front matter identifies the paper’s research area, notes an internship, and describes Figure 1 as an overview of TimeBraid and its benchmark results.

Acknowledgment

  • Part of the work was completed during an internship at Aether AI.Source: §1 · p. 1

Keywords

  • The listed keywords are unified time-series and language models, time-series analysis, and multimodal forecasting.Source: §1 · p. 1

Figure 1

  • Figure 1 overviews TimeBraid’s unified time-series and language interface, application examples, and benchmark results.The caption names time-series understanding and forecasting benchmarks.Source: Figure 1 · p. 1

1 Introduction

TimeBraid targets the underexplored alignment of pretrained language and time-series representations, using shared attention to unify the modalities. It reports results across understanding and forecasting benchmarks.

1 Introduction

  • Time-series records often lack semantic context for identification and interpretation, while language grounding can connect them with broader world knowledge.Source: §1 · p. 2
  • Existing approaches add specialized components that can introduce representational gaps, degrade pretrained capabilities, and cover only part of the bidirectional interface.Source: §1 · p. 2
  • TimeBraid uses separate pretrained experts for language reasoning, time-series perception, and forecasting, connected through global residual attention in a two-stage alignment and adaptation recipe.The shared chronological context supports cross-modal interaction while substantially preserving pretrained language and forecasting capabilities.Source: §1 · p. 2
  • 80.65% on TSAQA for TimeBraid-6.7B compares with 85.26% for TSAQA-specific LLaMA3.1-8B SFT and 63.10% for GPT-5.4; forecasting results include best average MSE rank on nine Time-MMD domains.The introduction also reports the best CRPS on CAF and comparable zero-shot forecasting to state-of-the-art time-series foundation models on GIFT-Eval.Source: §1 · pp. 2–3

2 Preliminaries

The preliminaries represent time-series and language tasks as ordered sequences of context and target segments, with flexible modality composition.

2 Preliminaries

  • An example is a segment sequence of text or time-series segments; a multivariate series enters as one segment per variable.Source: §2 · p. 3
  • The number and ordering of segments are unconstrained, and each segment is designated as context or target.Source: §2 · p. 3
  • Context segments provide fully observed text or measurements, while target segments are generated responses or time series continuing observed history with future values.Source: §2 · p. 3

3 Method

TimeBraid combines pretrained language and time-series representations to handle varied time-series and text inputs and outputs. Its design pairs a two-stage data curriculum and unified prompting with cross-modal attention, joint training, and adjustable text conditioning at inference.

3.1 Data Recipe

  • TimeBraid trains first on paired understanding and forecasting data, then fine-tunes on a heterogeneous instruction mixture spanning diverse tasks and domains.Stage 1 uses 2.23M examples split 3:7 between understanding and forecasting; Stage 2 has a final one-pass inventory of 4,881,583 samples.Source: §3.1 · pp. 3–4

3.2 Unified Time-Series and Language Modeling

  • Three pretrained experts handle time-series perception, language reasoning, and forecasting, while unified prompts serialize text and time-series segments as chat transcripts.Context segments include length, mean, and standard deviation; targets use history statistics and rolling RevIN normalization to respect causal order.Source: §3.2 · pp. 4–5
  • Interleaved global residual attention projects both streams into a shared attention space, jointly attends to timeline-ordered tokens, and routes outputs back to each stream.Zero-initialized output projections make each global layer start as an identity map; shared global positions place both modalities on one sequential axis.Source: §3.2 · pp. 5–6

3.3 Training and Inference

  • Joint training uses next-token cross-entropy for text and autoregressive Multi-Patch Prediction for time-series targets, combining point regression with quantile loss.The point term describes the central trajectory, while quantile predictions capture the predictive distribution and uncertainty; the total objective is L = LCE + αLTS.Source: §3.3 · pp. 6–7
  • Robust asinh normalization compresses extreme targets, while per-response loss capping limits outlier time-series loss and gradient contributions that can disrupt language-side learning.The paper reports that naive joint training is ineffective and that large losses on strongly non-stationary segments disrupt language learning through shared backbone updates.Source: §3.3 · p. 7
  • At inference, a guidance weight interpolates time-series-only and text-conditioned point forecasts, trading reliance on temporal regularities against sensitivity to text.Small λ leans on global temporal regularities; large λ keeps forecasts sensitive to text within the horizon it describes.Source: §3.3 · p. 7

4 Experiments

The experiments evaluate TimeBraid on time-series understanding and forecasting, then ablate choices in interaction space, optimization, alignment, and text guidance. Across these tasks, the results show competitive performance alongside design trade-offs that vary by benchmark and domain.

4.1 Time-Series Understanding

  • TimeBraid-6.7B scores 80.65% on TSAQA versus 85.26% for benchmark-specific SFT, reaches 63.14% on TimeSeriesExam, and lands within two points of GPT-5.4 on TB-MCQ.On CaTS-Bench, all variants lead other series–text models on alignment metrics, while numeric fidelity is their weakest axis.Source: Table 1 · p. 8, §4.1 · pp. 8–9, Table 2 · p. 9

4.2 Time-Series Forecasting

  • A TimeBraid variant ranks among the top three on eight of nine TimeMMD domains, and TimeBraid achieves the best Macro MSE and Macro MAE on CGTSF.Table 3 covers real-world contextual forecasting; TimeMMD averages per-domain MSE over four horizons, while CGTSF reports standardized MSE across four history settings.Source: §4.2 · pp. 9–10, Table 3 · p. 10
  • TimeBraid-6.7B achieves 43.33% Top-1 accuracy on Ctrl-F against a 33.33% chance level and the lowest pooled CRPS on CAF; TimeBraid-2.5B is comparable to the strongest listed time-series foundation model on CiK.The strongest LLMs remain ahead on Ctrl-F and CiK.Source: §4.2 · p. 10, Table 4 · p. 11
  • On GIFT-Eval, TimeBraid outperforms listed statistical and task-specific supervised baselines, while TimeBraid-2.5B has the second-best average MSE rank and third-best average MAE rank on ETT and Weather.The strongest specialized forecasting foundation models remain ahead on GIFT-Eval.Source: Table 5 · p. 11, §4.2 · p. 11, Table 6 · p. 12

4.3 Ablation Studies

  • The global residual attention shared space avoids the early language-loss plateau of an MLP projector, while dual- and tri-tower variants have nearly identical loss trajectories.The ablation reports that the projector’s language loss plateaus at roughly twice the level reached by residual attention; the two-tower setup shares a time-series model across perception and forecasting.Source: §4.3 · p. 13
  • Robust normalization with loss capping stabilizes time-series loss and language loss; the final recipe uses a 0.5 time-series loss weight and a 3:7 understanding-to-forecasting data ratio.A balanced 5:5 mixture has worse time-series loss than forecasting-heavy mixtures, while language loss is unchanged across the tested ratios.Source: §4.3 · p. 13
  • Aligned initialization stays ahead on understanding and CAF qCRPS during SFT, but ends at nearly the same TimeMMD MSE as direct SFT.The aligned run retains lower qCRPS at every CAF checkpoint; direct SFT does not close the gap.Source: Figure 5 · p. 14, §4.3 · p. 14
  • No single text weight works across domains: text helps more when it matches the target, while Security favors no text and some domains change with history length.The released text is not point-in-time certified, so Figure 6 is a sensitivity analysis; model selection uses a validation-selected λ=0.3 fixed across domains.Source: §4.3 · p. 14, Figure 6 · p. 15

5 Related Work

Prior time-series–language systems typically move one modality into the other’s host, leaving a one-way or lossy interface. TimeBraid instead aligns pretrained time-series and language representations in a shared space while retaining modality-specific processing.

Unified Multimodal Models

  • TimeBraid brings continuous-space alignment to time series and language, avoiding quantization at input and output while giving perception and forecasting different input processing over one time-series backbone.The paper presents this as carrying forward two lessons from unified multimodal modeling: continuous signals lose fidelity when forced through discrete vocabularies, and understanding and generation favor different treatments.Source: §5 · p. 15

Time-Series Foundation Models

  • Time-series foundation models forecast zero-shot across domains, but their numeric interfaces exclude instructions and explanations despite the value of textual context for forecasting.TimeBraid initializes its perception and forecasting experts from pretrained time-series blocks, while adding a language interface and world knowledge.Source: §5 · p. 16

Coupling Time Series with Language

  • TimeBraid keeps each modality in its own pretrained host and fuses them in a shared representation space, covering both directions without forcing either modality into the other’s geometry.The related work contrasts this with systems that host series in an LLM or numeric model, or represent values as discrete tokens or rendered plots.Source: §5 · p. 16
  • Earlier time-series–language approaches commonly attach specialized components to one pretrained host, yielding only one direction of the interface and potentially lossy representation moves.Examples include series encoders or projectors for LLM understanding and serialization, adaptation, or rendered plots for forecasting.Source: §5 · p. 16

6 Limitations

The authors identify limits in training coverage, numeric precision, and control when text guidance conflicts with historical dynamics. They also note that the model was trained without continued pretraining.

Training scope

  • TimeBraid omits continued pretraining because public interleaved series–text corpora at pretraining scale are not yet mature.The authors say large-scale continued pretraining could instill broader world knowledge and improve the model, but frame this as an expectation.Source: §6 · p. 16

Perception precision

  • Perception can misread specific values and complex waveforms, especially long, high-frequency, or low-signal-to-noise cases, partly because patch tokenization compresses raw values and loses local detail.The authors identify training coverage as a contributing factor, but patch tokenization as the main reason.Source: §6 · p. 16

Forecast control

  • Forecast control weakens when text demands abrupt shifts or regime breaks contrary to history, and complex, noisy guidance may not translate into effective signals.The forecasting expert’s pretrained transition dynamics bias it toward historically consistent continuations; the authors report that frontier-language-model reasoning and instruction conversion helps mitigate these issues.Source: §6 · pp. 16–17

7 Conclusion

The conclusion presents TimeBraid as a unified model that aligns pretrained language and time-series representations through tri-expert global residual attention. It reports competitive results across understanding and forecasting while identifying design choices explored by ablation.

7 Conclusion

  • TimeBraid aligns pretrained language and time-series foundation-model representations through tri-expert global residual attention, with one set of weights understanding and generating both modalities.The conclusion describes its recipe as dual-stage training on curated alignment pairs and a broad instruction mixture.Source: §7 · p. 17
  • TimeBraid forecasts at the level of dedicated time-series foundation models while remaining competitive with far larger general-purpose models on understanding and context-aided forecasting.The conclusion frames this as the outcome of the trained unified model, not as a claim that it leads every benchmark.Source: §7 · p. 17
  • The ablations distill design choices about where cross-modal interaction belongs, whether perception and forecasting need separate experts, and how strongly forecasts should draw on text at inference.The conclusion presents these as choices relevant to making this model class work.Source: §7 · p. 17

A Data Curation Pipeline

The curation pipeline creates task-specific series–text supervision and applies shared consistency, style, and format filters. The paper describes distinct routes for univariate understanding, multivariate understanding, and forecasting, alongside the training-data inventories.

A Data Curation Pipeline

  • Each curation route pairs time series with language through capability-specific annotation, followed by shared filtering for factual consistency, style, and format.Figure 7 covers univariate understanding, multivariate understanding, and univariate forecasting.Source: A Data Curation Pipeline · p. 25, Figure 7 · p. 25

A.1 Univariate Understanding

  • Univariate-understanding annotations describe global morphology and local events, using real-series evidence packs, attribute-first synthesis, and rewritten procedural exam prompts.Real-series evidence packs can include deterministic statistics, extrema, trend segments, change points, periodic patterns, and temporal landmarks; a context-rich route adds domain semantics and calendar information.Source: §A.1 · p. 25

A.2 Multivariate Understanding

  • Multivariate supervision combines cleaned ChatTS descriptions with graph-grounded descriptions generated from temporal structural equation models.The active mixture contains 100,000 SCM descriptions: 74,210 positive relation examples and 25,790 comparison controls whose channels are not directly adjacent to the target.Source: §A.2 · pp. 25–26

A.3 Univariate Forecasting

  • Forecasting supervision pairs histories with realized futures or controlled alternatives, where natural-language conditions describe transformations applied to shared-history continuations.The control branch creates a baseline and two variants using up to three transformations per variant; forecasting loss applies only to the held-out suffix.Source: §A.3 · p. 26

B Training Data

  • Stage 1 contains 2,229,500 alignment examples, and Stage 2 has a final inventory of 4,881,583 samples across five composition categories.The Stage-2 inventory includes 400,000 unimodal-corpus samples; Table 8 reports final sample counts and training-time sampling shares.Source: B Training Data · p. 26, Table 8 · p. 27, §B.2 · p. 28

B.1 Alignment Corpus Composition

  • The alignment corpus has 12 families, with understanding and forecasting fixed at a 3:7 split: 668,850 and 1,560,650 rows, respectively.Forecasting branches include base variants with horizons at most 512 and text rewrites with horizons at most 32; control rows share one history across a neutral continuation and two operator-conditioned futures.Source: §B.1 · p. 26, Table 7 · p. 27

B.2 Supervised Fine-Tuning Sources

  • Table 8 reports the final Stage-2 sample inventory and training-time sampling shares.The listed sources include SensorCaps, Time-MMD Open-Ended, and TRACE Captioning, alongside unimodal corpora.Source: §B.2 · pp. 27–28, Table 8 · p. 27

B.3 Overlap with Evaluation Benchmarks

Evaluation examples are kept separate from training at the split level, and the appendix specifies TimeBraid’s parameter accounting and layer geometry.

B.3 Overlap with Evaluation Benchmarks

  • Evaluation examples remain held out at the split level, even when related training data share a benchmark family or data-generating process.Table 9 summarizes the relationship for each benchmark.Source: §B.3 · p. 28, Table 9 · p. 28

C.1 Model Configuration

  • Table 10 allocates parameters across Qwen3 language towers, TimesFM-derived time-series towers, and fusion stacks; shared weights are counted once, while buffers are excluded.Variant names round total parameter counts, including frozen weights.Source: §C.1 · pp. 28–29, Table 10 · p. 29
  • TimeBraid-1.2B and 2.5B interleave 20 time-series and fusion blocks across 28 language layers, while 6.7B pairs 36 blocks with 36 language layers.Time-series segments use 32-value patches, and paired layers exchange information through global residual attention.Source: §C.1 · p. 29

C.2 Training Hyperparameters

Training updates the language, time-series, and fusion parameters in two stages, while the forward pass exchanges information between paired streams. The prompt examples represent tasks as interleaved text and time-series payloads, with context statistics handled differently from targets.

C.2 Training Hyperparameters

  • Both alignment and supervised fine-tuning update every model parameter, sharing optimizer, schedule, batch size, and objective constants while differing in context length, step budget, and data mixture.The training hyperparameter table describes settings for both stages.Source: §C.2 · p. 29, Table 11 · p. 29
  • At paired layers, the streams project into a shared attention width, merge by timeline position, use joint causal attention, and route outputs back as residual updates.The residual output projections are zero-initialized, so fusion starts as an identity map; unpaired language layers proceed alone.Source: §C.2 · p. 30

D Prompt Template Examples

  • Examples interleave text instructions and statistics with bound time-series payloads, covering anomaly identification, multivariate trend analysis, and forecasting from stated scenarios.The transcript uses empty <ts></ts> markers for payloads; context statistics are computed before normalization, while targets receive no statistics block.Source: D Prompt Template Examples · pp. 30–32
  • The prompt examples also show series-description tasks, including a cloud-operations trace summarized through its baseline, sharp V-shaped event, and later drift.The example specifies a five-minute sampling interval and requests a brief written summary.Source: D Prompt Template Examples · p. 31

E Evaluation Protocols

The evaluation protocols use official benchmark splits and task-specific scoring, with separate tests for understanding, context-grounded reasoning, and multiple forecasting settings. The appendix captions specify the scope and aggregation of several reported result tables and figures.

E Evaluation Protocols

  • Public benchmarks use their official splits or released evaluation sets and protocols unless otherwise stated; Ctrl-F uses a frozen evaluation set.The evaluation descriptions identify whether baseline results come from prior studies or were evaluated by the authors.Source: E Evaluation Protocols · p. 33

F.2.2 TemporalBench

  • TemporalBench separates historical interpretation, context-free prediction, context-grounded reasoning, and event-conditioned forecasting across multiple domains.Its results table reports multiple-choice accuracy and forecasting errors separately because forecasting metrics differ across datasets.Source: E Evaluation Protocols · p. 33, Table 16 · pp. 40–41

F.1 Time-Series Understanding

  • TimeSeriesExam tests pattern and noise understanding, anomaly detection, comparative reasoning, and Granger-causality reasoning, while TSAQA spans question formats from anomaly detection to temporal-relation reasoning.TimeSeriesExam uses procedural synthetic series; TSAQA includes puzzling questions scored by correctly ordered successor patches.Source: E Evaluation Protocols · p. 33, Table 12 · p. 36, Table 13 · p. 37

F.2.1 Contextual Reasoning Forecasting

  • CiK evaluates probabilistic forecasts with essential natural-language context, while CAF tests predictive distributions using verified history–scenario–future windows and normalized CRPS.The CAF results table covers 904 correct-context examples and its hard and easy strata.Source: E Evaluation Protocols · p. 34, Table 19 · p. 43

F.2.3 TimeMMD

  • TimeMMD pairs real-world numerical histories with temporally aligned text across nine domains, reporting per-domain MSE and MAE macro-averaged over four horizons.The table notes that multimodal models receive paired text and distinguishes their lookbacks from the fixed 128-step history used by TimeBraid and several other models.Source: E Evaluation Protocols · p. 34, Table 17 · p. 41

F.2.4 CGTSF; F.2.6 Ctrl-F

  • CGTSF pairs three real-world datasets with aligned text and averages standardized errors over dataset and history-length settings; Ctrl-F tests whether forecasts follow stated conditions across three continuations of a shared history.Ctrl-F reports history-normalized errors on 150 original-condition windows and correct-sibling Top-1 accuracy against a 33.33% chance level.Source: E Evaluation Protocols · p. 34, §F.2.4 · p. 42, §F.2.6 · p. 43

F.3 Language Ability Retention

The language-ability evaluation finds a remaining MMLU gap after training, while the qualitative examples illustrate forecasting and understanding tasks, including controlled forecasts and detailed reasoning over candidate answers.

F.3 Language Ability Retention

  • Mixing unimodal data into alignment raises MMLU to 53.4 after an alignment-stage loss of 19.4 points, but the released model remains 9.6 points below its language backbone.After supervised fine-tuning, the variants end within 0.6 points; the fine-tuning mixture already contains an 11.69% unimodal share.Source: §F.3 · p. 45, Table 22 · p. 45

G Success and Failure Cases

  • Matched Time-MMD examples remove only contextual text while holding checkpoint, numerical history, horizon, and inference settings fixed; reported error reductions apply to the selected examples.Other figures show text-conditioned forecasts for constructed scenarios and context-conditioned CGTSF examples without a matched context-removal comparison on that page.Source: G Success and Failure Cases · p. 46, Figure 13 · p. 50, Figure 14 · p. 51
  • The understanding examples span series-property QA, sleep-stage classification, relation analysis, scenario reasoning, captioning, anomalies, activity, bone age, paraphrase consistency, classification, and imputation.Source: Figure 16 · p. 53, Figure 17 · p. 54, Figure 18 · p. 55
  • A reasoning example selects the option whose claimed global maximum, value, and timestamp all match, while alternative options fail on local-pattern or numerical claims.The trace verifies the timestamp as 211 × 0.008 = 1.688 seconds.Source: G Success and Failure Cases · p. 54
Loading 2609.29792v1…