Source-linked AI summary
In-Context Inpainting for Time Series Forecasting
Thang Nguyen, Dung Nguyen, Romero Morais, Truyen Tran
TL;DR
Time-series forecasting often requires specialised architectures and extensive domain-specific training, while applying large vision models remains difficult because temporal signals are not natively visual. ICI-Time converts series into visual prompts, uses a pretrained LVM to inpaint forecasts without fine-tuning or architectural modification, and achieves competitive accuracy across diverse domains with promising low-data adaptability.
Problem
Time-series forecasting remains difficult to generalise because existing methods rely on specialised architectures and extensive training, while temporal signals are not natively visual for LVMs.
Method
ICI-Time transforms time series into visual representations and grid-structured example-query prompts, then uses an off-the-shelf LVM to inpaint and reconstruct forecasts.
Results
ICI-Time matches strong Transformer-based baselines across epidemiology, meteorology, and power systems without training and is markedly more robust in low-data regimes.
Takeaways & Limitations
The findings suggest that visual reasoning models can generalise to temporal tasks when equipped with suitable representations, bridging temporal and visual modelling.
Abstract
from arXiv · showhide
We propose ICI-Time, a novel framework that reframes time series forecasting as a visual inpainting task, leveraging the generalisation power of large vision models (LVMs). Unlike methods that require specialised temporal architectures and extensive domain-specific training, ICI-Time transforms time series into structured visual representations (area charts) and applies visual in-context learning, reformulating forecasting as pattern completion within a grid-structured prompt that pre-trained vision transformers can solve without fine-tuning or architectural modification. Temporal dependencies are represented through spatial layout, with a consistent, invertible mapping between numerical and visual domains. Extensive experiments across epidemiology, meteorology, and power systems demonstrate that ICI-Time performs competitively against deep learning baselines and shows promising adaptability under limited-data settings, introducing a new paradigm that bridges temporal and visual domains.
Time Series Forecasting
The paper lists its authors, institutional location, and keywords describing its focus on time-series forecasting, visual prompting, inpainting, and in-context learning.
- The authors are Thang Nguyen, Dung Nguyen, Romero Morais, and Truyen Tran.
- The listed location is Geelong, Victoria 3216, Australia.
- The keywords are time-series forecasting, visual prompting, inpainting, and in-context learning.
1 Introduction
The introduction frames time-series forecasting as difficult to generalise across domains because existing approaches depend on specialised architectures and extensive training. ICI-Time addresses this by recasting forecasting as visual inpainting with pretrained vision models.
- Forecasting is challenging because temporal dependencies, multi-scale variability, non-stationarity, and labeled-data scarcity complicate modelling.
- Existing deep-learning forecasters typically require domain-specific architectures and extensive training or fine-tuning, limiting scalability and low-data generalisation.
- LVM-based forecasting remains difficult because temporal signals are not natively visual, despite in-context learning’s generalisation capabilities.
- ICI-Time transforms time series into area charts, arranges examples and queries in a grid prompt, and uses an off-the-shelf LVM to inpaint missing forecasts.
- The paper contributes a cross-modal framework, visual in-context forecasting formulation, and bidirectional mapping between temporal and visual domains.
- ICI-Time matches strong Transformer-based baselines across epidemiology, meteorology, and power systems without training and is markedly more robust in low-data regimes.
2 Related Work
Related work shows that foundation-model approaches and image-based representations have expanded time-series forecasting, but many still require adaptation, pretraining, or knowledge transfer.
- Visual in-context learning extends in-context prediction to grid-like images, with Painter and SegGPT demonstrating generalist vision models for visual tasks.
- Time-series foundation models have used tokenisation, visual pretraining, multimodal fusion, or lightweight distillation to transfer predictive priors.
- Unlike these approaches, ICI-Time investigates forecasting with standard line plots, pretrained generalist models, and in-context visual prompting without new weights.
- Line plots preserve continuity and topology that periodicity-based heatmaps can lose, motivating image representations for temporal pattern recognition.
3 Preliminaries
The preliminaries define multivariate forecasting as mapping historical windows to future horizons and introduce in-context learning as prediction from contextual examples without parameter updates.
- A multivariate series X ∈ R^C×L is partitioned into historical and future components, X_I ∈ R^C×T_I and X_P ∈ R^C×T_P.
- Each X_t^j denotes the series value at timestep t and channel j.
- The forecasting objective is a predictor f: R^C×L_I → R^C×L_P mapping a lookback window to a prediction horizon.
- ICI-Time applies this setup through historical example selection, time-series-to-image conversion, visual inpainting, and image-to-series recovery.
- In-context learning supplies similar input-output pairs before a query and produces its output without task-specific training or fine-tuning.
4 Method
ICI-Time forecasts by converting time-series windows into visual prompts and asking a pretrained large vision model to inpaint the missing prediction region. The pipeline selects similar historical cases, normalizes and renders charts, then denoises and inverts the generated image back into numerical forecasts.
- Framework: ICI-Time uses visual case-based selection, image conversion, off-the-shelf LVM inpainting, and numerical back-conversion.These four components form the framework’s forecasting pipeline.
- Case-based selection: Historical series are partitioned into overlapping windows containing lookback inputs of length LI and prediction targets of length LP.Each window is represented as Ws = {xs, ys}, pairing an input segment with its subsequent target segment.
- Case-based selection: The query selects the historical example with minimum Euclidean distance between normalized inputs.The selected example supplies a relevant prior for in-context forecasting.
- Visual representation: Example and query windows are min-max normalized and rendered as separate area-chart figures within a visually consistent prompt.Example images provide prior knowledge, while query images include the unknown target region and share a common image-height range.
- Visual representation: The representation uses fixed 224×224 images and adjusts x-axis limits to accommodate lookback and prediction components.The lookback range is xlimin = [1, LI], while the prediction range is xlimout = [LI + 1, LI + LP ].
- Time-series recovery: Preserved normalization parameters make the visual representation invertible, enabling generated charts to be denoised, converted into boundaries, and denormalized.Denoising binarizes the output and applies morphological opening before boundary-curve extraction recovers the predicted sequence.
5 Experimental Results
Across epidemiology, meteorology, and power systems, ICI-Time is evaluated against Transformer baselines under full-data, ablation, and restricted-data settings. It performs competitively without training, especially on MAE, while visual-scale, temporal-resolution, and smoothing choices improve average errors.
- Full-data Results: The nearest-neighbour sanity check performs poorly against Transformer models and ICI-Time, indicating that performance does not come from simply copying the input example.The baseline reuses the first input example for every future timestep.
- Full-data Results: ICI-Time outperforms baselines in 23 of 24 MAE cases and is second best in the remaining case.The comparison uses MSE and MAE against Informer, Pyraformer, and LogTrans under consistent configurations.
- Analysis of Design Choices: htransfer reduces average MSE by 9.2% to 18.5% versus h1.5 across ETT and Weather, with ILI as the exception.htransfer transfers the height scale from examples to queries, whereas h1.5 applies a fixed factor of 1.5.
- Analysis of Design Choices: wdiff outperforms weven on all six datasets, reducing average MSE by 1.8% to 5.1% on ETT and Weather and by 10.7% on ILI.wdiff preserves distinct native temporal resolutions for input and target images.
- Analysis of Design Choices: Smoothing improves average MSE by 0.7% to 2.2% on all six datasets by blending the final input point into predictions over approximately 10 timesteps.The method applies a Gaussian smoothing decay to address the boundary disconnection problem.
- Few-shot adaptation: With 10% or less training data, ICI-Time often achieves errors that are multiples lower than the baselines, including ETTh2 MSEs of 0.317 versus 4.047 and 4.065 at 10%.At 1% data, ICI-Time records 0.208 on ETTm2 at prediction length 96, versus Informer’s 1.984.
- Few-shot adaptation: On Weather at prediction length 96, ICI-Time’s MAE rises only 3.4% from 10% to 1% data, compared with 32.1% for Informer and 35.3% for Pyraformer.The figure compares MAE across training-data percentages.
6 Conclusion
ICI-Time adapts visual reasoning to complex temporal dynamics by transforming time series into structured images and applying in-context inpainting with pre-trained vision models. The reported approach bypasses specialised temporal architectures and costly training or fine-tuning while achieving competitive accuracy across diverse domains.
- Conclusion: ICI-Time adapts visual reasoning to complex temporal dynamics through structured images and in-context inpainting with pre-trained vision models.The conclusion frames this as an adaptation of visual reasoning to temporal data.
- Conclusion: The approach bypasses specialised temporal architectures and costly training or fine-tuning while achieving competitive forecasting accuracy across diverse domains.The conclusion states this outcome without narrowing it to a single dataset or metric.