Source-linked AI summary
TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents
Geon Lee, Wenchao Yu, Kijung Shin, Wei Cheng, Haifeng Chen
TL;DR
Real-world time-series event prediction requires contextual information, while prior LLM-based approaches mainly use LLMs as predictors. TimeCAP uses separate contextualizer and predictor agents with a mutually augmenting multi-modal encoder, and it outperforms state-of-the-art methods on seven real-world datasets, including an average 28.75% F1 improvement.
Problem
Real-world time series contain contextual information crucial for comprehensive analysis and effective event prediction, but prior LLM-based approaches primarily use LLMs as predictors.
Method
TimeCAP uses one LLM agent to generate contextual summaries, another to predict events, and a multi-modal encoder to mutually augment inputs and prediction prompts.
Results
TIMECAP outperforms state-of-the-art event-prediction methods on seven real-world datasets, achieving an average improvement of 28.75% in F1 score.
Takeaways & Limitations
TIMECAP demonstrates the effectiveness of combining LLM contextualization, prediction, and multi-modal mutual enhancement for time-series event prediction.
Abstract
from arXiv · showhide
Time series data is essential in various applications, including climate modeling, healthcare monitoring, and financial analytics. Understanding the contextual information associated with real-world time series data is often essential for accurate and reliable event predictions. In this paper, we introduce TimeCAP, a time-series processing framework that creatively employs Large Language Models (LLMs) as contextualizers of time series data, extending their typical usage as predictors. TimeCAP incorporates two independent LLM agents: one generates a textual summary capturing the context of the time series, while the other uses this enriched summary to make more informed predictions. In addition, TimeCAP employs a multi-modal encoder that synergizes with the LLM agents, enhancing predictive performance through mutual augmentation of inputs with in-context examples. Experimental results on real-world datasets demonstrate that TimeCAP outperforms state-of-the-art methods for time series event prediction, including those utilizing LLMs as predictors, achieving an average improvement of 28.75% in F1 score.
1 Introduction
Real-world time series require contextual information for comprehensive analysis, yet prior LLM-based methods mainly use LLMs as predictors. TimeCAP addresses this with contextualization, mutual augmentation, interpretability, and released datasets.
- Real-world time series often include contextual information, such as geographical, climatic, and economic factors, that is crucial for comprehensive analysis and effective modeling.
- Prior LLM-based time-series methods primarily fine-tune or prompt LLMs as predictors, often overlooking contextual understanding.
- 21.98% improvement in F1 scores comes from contextualization alone, while 28.75% improvement follows the addition of augmentation on real-world event-prediction datasets.
- TIMECP uses two independent LLM agents: one generates a contextual text summary, and the other uses it to make event predictions.
- TIMECAP adds a multi-modal encoder that uses time-series data and generated summaries, then retrieves relevant training summaries as in-context examples for prediction prompts.
- TIMECAP supports black-box LMaaS APIs, provides interpretable prediction rationales, and releases seven real-world datasets with generated contextual summaries.
2 Related Work
Related work describes LLM capabilities and their growing integration into time-series analysis. Existing approaches include training models from scratch, fine-tuning pretrained models, and prompt tuning parameterized time-series data.
- LLMs are large models trained on massive text corpora and demonstrate broad performance across language tasks, supported by domain knowledge and zero-shot capabilities.
- In-context learning enhances LLM performance by providing a few input-label pairs as demonstrations.
- LLM applications extend beyond language into computer vision, tabular data analysis, and audio processing.
- LLMs and Time Series: Time-series approaches include training LLMs or smaller language models from scratch, fine-tuning pretrained LLMs, and prompt tuning parameterized time-series inputs.
3 Proposed Method
The method uses LLM agents to contextualize time series before prediction, then combines textual summaries, raw series, and retrieved in-context examples through a multimodal encoder. The resulting predictions support both accuracy improvements and interpretation through rationales or selected examples.
- TIMECP: Contextualize and Predict: TIMECP assigns separate LLM agents to summarize contextual information from a time series and predict future events from that summary.The contextualizer produces sx from x, while the predictor uses sx to estimate the event outcome.
- TIMECAP: Contextualize, Augment, Predict: TIMECAP extends TIMECP with a trainable multimodal encoder that jointly uses raw time series and generated text summaries.The encoder learns representations and event predictions from both modalities, complementing the LLM agents.
- TIMECAP: Contextualize, Augment, Predict: The encoder embeds text and segmented time-series patches, applies multi-head self-attention across modalities, and produces a fused prediction.Text is projected into the encoder space, while each channel is divided into nonoverlapping patches before cross-modal attention.
- TIMECAP: Contextualize, Augment, Predict: TIMECAP retrieves the k nearest training summaries in the encoder embedding space and supplies their text-outcome pairs as in-context examples to the predictor.The retrieved examples augment the prediction prompt so AP can compare the test summary with relevant training cases.
- TIMECAP: Contextualize, Augment, Predict: The framework combines the multimodal encoder prediction with the LLM prediction through a weighted linear fusion.The fusion parameter λ controls the relative contribution of the LLM and multimodal predictions.
- Interpretation: TIMECAP supports implicit rationales and explicit selection of the most relevant retrieved example as forms of prediction interpretation.The LLM can provide a rationale for its prediction or identify a useful example from the in-context set for comparison.
4 Experiments
TIMECAP is evaluated for time-series event prediction across seven real-world datasets and against conventional and LLM-based baselines. Experiments examine accuracy, component contributions, interpretability, data scarcity, and the quality of selected in-context examples.
- Experimental settings: Experiments use seven real-world datasets spanning weather, finance, and healthcare event-prediction tasks.The default setup uses GPT-4 for LLM agents and BERT within the multi-modal encoder.
- Accuracy: TIMECAP ranks first on average across seven datasets using F1 and AUROC, outperforming other methods on most datasets.Table 2 compares TIMECAP and TIMECP with conventional time-series and LLM-based competitors, including zero-shot methods.
- Component effectiveness: Every TIMECAP component contributes to F1 improvement, including contextualization, input augmentation, and prompt augmentation.TIMECP outperforms PromptCast across all datasets; textual summaries complement time-series inputs, while multi-modal selection provides more meaningful demonstrations.
- Interpretability: In-context examples affect both prediction and interpretation, while selected examples support post-hoc comparisons between the input and semantically similar cases.Without examples, the LLM predicts solely from the input and can make incorrect predictions; with examples, it uses prior text-outcome relationships.
- Further analyses: The multi-modal encoder selects more useful in-context examples than PatchTST, as shown by higher average F1 scores across domains.These selected examples lead to more accurate predictions by the prediction agent.
5 Conclusion
The paper presents TIMECAP for time-series event prediction, combining two independent LLM agents with a trainable multi-modal encoder. Results on seven real-world datasets demonstrate the framework’s effectiveness across varied domains.
- Conclusion: TIMECAP uses independent LLM agents for contextualization and prediction, supported by a trainable multi-modal encoder that mutually enhances them.The framework leverages LLM contextual understanding for time-series event prediction.
- Conclusion: Experiments on seven real-world time-series datasets from various domains demonstrate TIMECAP’s effectiveness.The paper makes the datasets available and states that the code is available upon request.