In-Context Inpainting for Time Series Forecasting

1 Introduction

• Forecasting remains difficult because of complex temporal dependencies, multi-scale variability, non-stationarity, and scarce labels, while domain-specific architectures and training hinder deep-learning generalisation.Using large vision models for forecasting is an open challenge because temporal signals are not natively visual.Source: §1 · pp. 1–2
• It converts time series into area charts, arranges example pairs and a query in a grid prompt, and uses an unmodified pre-trained vision model to inpaint the missing forecast region.The generated chart is translated back into a numerical time series; the model uses the in-context example without task-specific fine-tuning.Source: §1 · p. 2
• The framework bridges temporal and visual domains, enabling fast adaptation with off-the-shelf vision transformers and exploiting their pattern recognition without architectural changes or additional supervision.Flexible visual prompting is presented as a way to adapt to new forecasting tasks.Source: §1 · p. 2
• A carefully designed bidirectional, invertible mapping represents temporal dependencies through spatial layout, preserving forecasting fidelity across visual and numerical domains.This formulation extends visual in-context learning beyond its NLP roots to solve temporal prediction through visual prompts and inpainting.Source: §1 · p. 3
• ICI-Time matches strong Transformer-based baselines without training across ILI, Weather, and ETT, and is markedly more robust in low-data regimes.The reported domains are epidemiology (ILI), meteorology (Weather), and power systems (ETT).Source: §1 · p. 3

Visual In-context Learning

• ICI-Time applies generalist vision models to time-series forecasting through visual in-context conditioning, addressing a gap in univariate forecasting without parameter updates.Painter and SegGPT demonstrated visual in-context learning across tasks, while WeatherGFM applied grid-based prompts to multimodal weather data; informative example selection had not been studied for temporal dynamics.Source: §2 · p. 3

Foundation Models and Time Series

• Unlike time-series foundation models requiring task-aligned pre-training, adaptation, or distillation, ICI-Time uses standard line plots and a pre-trained generalist model as-is.Chronos and Time-LLM tokenize series; VisionTS and ViTime use visual approaches, while VisionTS++ [22], DMMV, OccamVTS [15], and SVTime [21] involve further pre-training, multi-modal fusion, or distillation.Source: §2 · pp. 3–4

Image-based Representation for Time-series Forecasters

• Line plots preserve data continuity and topology, unlike periodicity-based heatmaps, while connectivity cues can outperform scatter plots or text descriptions.TimeOmni-VL identifies low-loss bidirectional image–series conversion as important for numerically faithful generation, but uses large-scale multi-task training; ICI-Time uses a pre-trained visual prompting model without fine-tuning or external modules.Source: §2 · p. 4

3.1 Time Series Forecasting

• A multivariate series X has C channels and length L, partitioned into historical XI and future XP segments whose lengths sum to L.Source: §3.1 · p. 4
• The forecasting function maps each LI-step lookback window to an LP-step prediction horizon; TI and TP denote available training and evaluation data, with TI ≫ (LI + LP).Source: §3.1 · p. 4

3.2 In-context Learning

• In-context learning predicts a query from preceding similar input–output pairs without task-specific training or fine-tuning, unlike standard supervised learning.Source: §3.2 · p. 5

4 Method

• The framework combines visual case selection, image conversion, off-the-shelf LVM inpainting, and conversion back to the original time series.These four components form the ICI-Time forecasting pipeline.Source: §4 · p. 5

4.1 Visual Case-Based Forecasting

• Overlapping historical windows provide lookback-and-prediction examples; the nearest normalized input window is selected by Euclidean distance for the query.The LVM acts as a nonlinear case interpolator in image space, adapting relevant past examples to the query.Source: §4.1 · pp. 5–6

4.2 Visual Representation of Time Series

• Time series are converted into visually interpretable representations using normalization and boundary visualization to balance information preservation with visual clarity.Source: §4.2 · p. 6

Image Height

• Example and query windows are normalized separately, while each example’s lookback and prediction components share min-max scaling statistics.The image-height procedure sets visual scales for examples and queries to support consistent prompts.Source: Image Height · p. 6

Image Width

• The invertible visual representation preserves parameters needed by its inverse to denormalize the predicted series.Source: Image Width · p. 6

4.3 Visual Prompting and Time Series Recovery

• The LVM inpaints the visual prompt, then denoising and boundary-curve extraction recover the forecast at LP equidistant points.Denoising uses a 3×3 morphological opening; the extracted boundary is interpolated before sampling.Source: §4.3 · p. 7

5.1 Datasets

• The evaluation covers ILI, Weather, and four ETT datasets, comparing MSE and MAE against Informer, Pyraformer, and LogTrans.The study uses a 96-step look-back, dataset-specific prediction horizons, and channel-independent forecasting; training and validation data are merged because ICI-Time requires no training or fine-tuning.Source: §5.1 · p. 7, §5.2 · pp. 7–8

5.3 Forecasting Performance

• A nearest-neighbour method that copies the first input example into all future steps performs poorly versus Transformers and ICI-Time.This sanity check supports the authors’ conclusion that LVM performance comes from generalising over visual patterns rather than simply copying the example.Source: §5.3 · p. 8

5.4 Analysis of Design Choices

• Against fixed scaling h1.5 (factor 1.5 for all inputs), height transfer reduces average MSE by 9.2% to 18.5% on ETT and Weather, largest on ETTh1; on ILI, fixed scaling attains lower average error.Source: §5.4 · p. 8

Full-data Results

• ICI-Time achieves the best MAE in 23 of 24 full-data cases, with the remaining case second best.The authors note that MSE is sensitive to noise and can be improved by averaging predictions from multiple runs using different near-optimal prompting examples.Source: Full-data Results · p. 8, Table 2 · p. 9

5.4 Analysis of Design Choices

• Distinct input and target time resolutions reduce average MSE versus uniform resolution by 1.8% to 5.1% on ETT and Weather and 10.7% on ILI.The distinct-resolution setting preserves the input’s native temporal resolution and outperforms uniform resolution on all six datasets.Source: §5.4 · p. 9
• Gaussian smoothing improves average MSE by 0.7% to 2.2% across all six datasets, with the largest gain on ILI.It addresses the first-step boundary disconnection by blending the last input point into predictions over approximately 10 steps, using σ = window/3.Source: §5.4 · p. 9

Few-shot adaptation

• On Weather at prediction length 96, ICI-Time’s MAE rises only 3.4% from 10% to 1% data, versus 32.1% for Informer and 35.3% for Pyraformer.The corresponding MAE values are 0.234 to 0.242 for ICI-Time, 0.389 to 0.514 for Informer, and 0.360 to 0.487 for Pyraformer.Source: Few-shot adaptation · p. 10, Figure 3 · p. 11
• With 10% data, ICI-Time’s ETTh2 MSE is 0.317 versus 4.047 for Informer and 4.065 for Pyraformer; with 1%, ETTm2’s is 0.208 versus 1.984 for Informer.At 1%, only ETTm1, ETTm2, and Weather provide enough sequence length to train Transformer baselines; at 5% and 1%, only horizons with sufficient baseline training data are reported.Source: Few-shot adaptation · p. 10, Table 4 · p. 11, Table 6 · p. 12

• ICI-Time transforms time series into structured images and applies in-context inpainting with pre-trained vision models, avoiding specialised temporal architectures and costly training or fine-tuning while achieving competitive accuracy across diverse domains.Source: §6 · p. 13
• Carefully designed representations enable pre-trained visual models to generalise far beyond their original modalities, challenging conventional boundaries between temporal and visual modelling.Source: §6 · p. 13
• Future work may explore advanced retrieval strategies and extensions to anomaly detection using forecasting-error scoring and classification by encoding classes as visual objects.Source: §6 · p. 13