Source-linked AI summary
Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting
Bowen Zhang, Hsiu-Wen Cheng, Hongyu Yang, Evie L. Shen, Joleen Vansomphone, Yuna Li, Kerry Zhou, Zitian Qu, Suning Zhao, Xiangning Deng, Hua Zhou, Jin J. Zhou
TL;DR
The paper asks whether time-series foundation models reliably improve CGM forecasting and whether dietary context adds value beyond CGM signals. It benchmarks model classes and adaptation regimes across public CGM datasets, then evaluates residual fusion of meal information on CGMacros. The main findings are that zero-shot foundation models are not consistently superior, fine-tuning is important, and dietary context especially improves postprandial forecasting.
Problem
It remains unclear whether time-series foundation models transfer reliably to CGM forecasting and which dietary signals add value beyond CGM alone.
Method
The study benchmarks classical, deep-learning, and foundation models across eight public CGM datasets, then evaluates a residual-based multimodal framework using CGM, food images, macronutrients, and meal timing on CGMacros.
Results
Zero-shot foundation models did not consistently outperform strong task-specific baselines, while multimodal fusion reduced overall RMSE by approximately 3% and postprandial RMSE by approximately 15% versus CGM-only forecasting.
Takeaways & Limitations
Reliable CGM forecasting with foundation models requires CGM-specific adaptation, while dietary context provides additional signal particularly during postprandial periods.
Takeaways & Limitations
The multimodal analysis uses the modest CGMacros cohort, whose dietary patterns and participant characteristics may not represent broader CGM-using populations.
Abstract
from arXiv · showhide
Continuous glucose monitoring (CGM) provides high-frequency measurements of glucose dynamics and enables short-term glucose forecasting for diabetes management. Although time-series foundation models have shown strong general forecasting ability, their effectiveness for CGM prediction and the added value of multimodal dietary context remain unclear. We conduct a comprehensive empirical study using eight public CGM datasets spanning Type 1 diabetes, Type 2 diabetes, and non-diabetes populations. Under a unified protocol across multiple context lengths and prediction horizons, zero-shot foundation models did not consistently outperform strong task-specific baselines such as Elastic Net and PatchTST. In contrast, lightweight fine-tuning substantially improved forecasting performance. For example, fine-tuned Chronos-Bolt reduced RMSE by 6.5%-18.4% in the T1D cohort and by 8.6%-18.2% in the non-diabetes/T2D cohort, with comparable improvements in both in-distribution and out-of-distribution test settings. We further evaluate multimodal dietary context using CGMacros, which provides temporally aligned CGM signals, food images, and macronutrient records. A residual-based fusion framework reduced overall RMSE by approximately 3% and postprandial RMSE by approximately 15% relative to the CGM-only baseline. Moreover, Chronos-based CGM representations were more strongly correlated with observed postprandial glucose increments than representations from LSTM and CatBoost, even after those models incorporated additional dietary modalities, suggesting that pretrained temporal representations better preserve meal-induced excursion patterns. These findings show that foundation models require CGM-specific adaptation for reliable forecasting and that dietary context provides clinically meaningful signals beyond CGM alone, especially during postprandial periods.
1 Introduction
This study examines whether pretrained time-series representations improve CGM forecasting and whether dietary context adds information beyond CGM signals alone. It uses unified evaluation across public CGM datasets and a multimodal CGMacros setting to address these questions.
- Motivation: Foundation-model benefits for CGM forecasting remain uncertain under limited histories, varying horizons, and heterogeneous populations, particularly for zero-shot deployment versus lightweight fine-tuning.The study targets this gap alongside the broader question of whether pretrained representations transfer effectively to clinically relevant forecasting.
- Motivation: Dietary intake introduces abrupt, heterogeneous glucose perturbations, but the most informative dietary signals and the value of visual representations beyond nutritional variables remain unclear.This motivates evaluating food images together with structured dietary context for postprandial dynamics.
- Study design: The study evaluates classical, deep-learning, and foundation-model forecasters across historical contexts, prediction horizons, and populations using curated public CGM datasets.The benchmark is designed to compare model classes and adaptation regimes under a common protocol.
- Study design: The multimodal analysis focuses on CGMacros, whose temporally aligned CGM, food-image, and macronutrient records enable assessment of dietary context’s incremental forecasting contribution.The dataset is used to isolate the added value of dietary information beyond CGM alone.
- Data preparation: The preprocessing pipeline standardizes temporal sampling, handles short and long missing intervals differently, excludes unstable or adverse-event periods, and supports subject-level external testing.Long gaps are used to split trajectories, while initial segment periods are removed to reduce boundary instability.
4 Experiment
The experiments benchmark forecasting models across multiple context–horizon settings and develop a residual-based multimodal framework that adds meal information to a CGM-only Chronos forecast. Food-image representations, macronutrients, timing, and supervision strategies are compared under controlled ablations.
- 4.1 Experimental protocol: Forecasting is evaluated across multiple historical contexts and horizons using classical baselines, deep-learning models, and pretrained time-series foundation models.Contexts span 4, 12, and 24 hours, while horizons span 30 minutes, 1 hour, and 2 hours.
- 4.1 Experimental protocol: Foundation models are compared in zero-shot and CGM-trained fine-tuning regimes, with validation-selected configurations evaluated on in-distribution and out-of-distribution tests.Each context–horizon combination is treated as an independent forecasting task.
- 4.2 Multimodal framework: The multimodal model uses a fixed Chronos CGM encoder for a baseline forecast, then combines food-image embeddings, macronutrients, and meal timing to estimate a residual correction.The design isolates meal-related information from the CGM baseline and supports controlled modality comparisons.
- 4.2 Multimodal framework: A Weibull-based temporal gate modulates the meal residual according to elapsed time since the latest meal before adding it to the CGM-only forecast.The gate emphasizes meal-related information during postprandial periods and attenuates it at longer delays.
- 4.3 Ablation studies: Ablations compare CGM-only, image-augmented, macronutrient-augmented, and full multimodal configurations while holding the CGM backbone fixed.Additional comparisons test category-supervised versus nutrition-aligned image embeddings under otherwise constant training procedures.
5 Results
Across CGM benchmarks, zero-shot foundation models did not consistently beat strong baselines, whereas lightweight fine-tuning produced substantial gains. In multimodal experiments, dietary information improved forecasting—especially postprandial error—and Chronos representations better tracked meal-induced excursions.
- 5.1 Foundation-model forecasting: Zero-shot foundation models did not consistently outperform Elastic Net and PatchTST across cohorts, contexts, and horizons.Chronos-Bolt variants underperformed these baselines by approximately 7–10% at a 30-minute horizon for T1D with 24-hour context.
- 5.1 Foundation-model forecasting: 6.5–18.4% and 8.6–18.2% RMSE reductions were achieved by fine-tuned Chronos-Bolt models in the T1D and Non-diabetes/T2D cohorts, respectively.These gains were observed across evaluated context–horizon combinations and shifted foundation models toward top-performing results.
- 5.1 Foundation-model forecasting: Fine-tuning produced comparable RMSE improvements on ID and OOD test sets, with differences generally within 1 percentage point.The reported pattern indicates that gains transferred to unseen data rather than being confined to in-distribution evaluation.
- 5.2 Multimodal dietary context: Approximately 3% lower overall RMSE and 15% lower postprandial RMSE were obtained by combining nutrition-aligned food images with macronutrients.The joint configuration achieved the lowest overall RMSE, while meal-related windows showed the larger improvement.
- 5.2 Multimodal dietary context: Chronos-based multimodal forecasting achieved lower overall RMSE than alternative CatBoost and LSTM backbones, while meal-related modalities improved both alternatives.The comparison used alternative CGM encoders under the multimodal forecasting framework.
- 5.2 Multimodal dietary context: Chronos PGI correlation increased from 0.68 with CGM alone to 0.76 with macronutrients and exceeded LSTM and CatBoost across modality configurations.PGI correlation evaluates the relationship between predicted and observed postprandial glucose increments rather than pointwise error alone.
6 Limitations
The study identifies limitations from public CGM data availability, multimodal dataset scope, and the controlled rather than exhaustive optimization of fusion architectures.
- Public CGM datasets limit foundation-model benchmarking because they have fewer subjects and less temporal coverage than pretraining datasets.Fixed clinically motivated context lengths may also restrict use of longer-range dependencies.
- The multimodal analysis uses only CGMacros, whose modest cohort and participant and dietary characteristics may not represent broader CGM-using populations.Macronutrient annotation inaccuracies may add noise and attenuate estimated nutritional effects.
- The multimodal framework prioritizes controlled modality and supervision comparisons over exhaustive fusion-architecture optimization.This supports interpretable ablations but may miss more complex cross-modal interactions.
7 Conclusion
The study systematically evaluates foundation models and dietary multimodality for CGM forecasting under unified clinical settings. It finds that adaptation and dietary context are important for reliable forecasting and postprandial prediction.
- Off-the-shelf foundation models do not consistently beat task-specific baselines, whereas lightweight fine-tuning often improves performance across horizons and populations.
- Dietary context improves postprandial glucose prediction, with macronutrient features providing the largest gains and nutrition-aligned food images offering complementary benefits.
- Foundation models can support CGM forecasting when appropriately adapted, while task-aligned dietary information provides signals beyond CGM alone.
S1 Preprocessing Details
The preprocessing pipeline standardizes CGM data across studies, applies clinically defined cleaning and segmentation rules, and creates subject-level in-distribution and out-of-distribution splits. Multimodal data are additionally aligned chronologically with valid meal events and dietary modalities.
- CGM measurements are aligned to a uniform 5-minute grid, with short gaps interpolated and leakage-free standardized identifiers created.Interpolation is limited to gaps of at most 120 minutes.
- Analyses generally restrict participants to adults and exclude periods affected by closed-loop control or documented adverse events where applicable.
- Only contiguous CGM segments with at least 28 hours are retained, and the first 2 hours are discarded to reduce boundary and sensor-instability effects.The resulting minimum effective segment length is 26 hours.
- An unseen-individuals test set samples 20% of subjects, while the remaining subjects are split chronologically within subject at approximately 2:1:1.
- Multimodal preprocessing retains meal events with valid consumed-item images, rounds meal times to the CGM grid, and splits data chronologically by subject.The multimodal subject-level split uses a 6:2:2 ratio, with exclusions for prolonged gaps or insufficient irregular data.
S2 Hyperparameters
Hyperparameters are optimized separately by model and context–horizon configuration, using simpler procedures for non-deep-learning baselines and automated validation-based optimization for deep models. Foundation-model adaptation varies by model scale and computational constraints.
- Deep models, including PatchTST and fine-tuned foundation models, use automated hyperparameter optimization with validation MSE as the objective.Non-deep-learning baselines use no tuning or simple grid searches.
- AutoGluon trains PatchTST and Chronos without automated ensembling, feature engineering, or model-specific heuristics, varying only listed hyperparameters.
- Chronos-Bolt updates all parameters, whereas Chronos2-Small and TimesFM 2.5 use LoRA with frozen pretrained backbones.
S3 Visual Encoder Fine-tuning
Visual encoders were fine-tuned end-to-end on Food-101, and ConvNeXT-base was selected to generate food-image embeddings for multimodal CGM forecasting. UMAP then reduced the 1024-dimensional embeddings, with eight dimensions chosen using trustworthiness diagnostics.
- S3 Visual Encoder Fine-tuning: ConvNeXT-base was selected after end-to-end Food-101 fine-tuning to extract food-image embeddings for downstream CGM forecasting.All visual backbones were fine-tuned end-to-end before downstream use.
- S3 Visual Encoder Fine-tuning: UMAP reduced the visual encoder’s 1024-dimensional embeddings for multimodal integration.UMAP was applied as a post-processing step.
- S3 Visual Encoder Fine-tuning: Trustworthiness diagnostics selected d = 8 as the embedding dimension used across multimodal forecasting experiments.The selection was based on curves for category-supervised and nutrition-aligned representations.
S4 Computational Resources
Experiments used single-GPU Linux systems for deep learning, while non-neural baselines ran on CPUs. Hyperparameter optimization and repeated runs were executed sequentially.
- S4 Computational Resources: Deep learning models were trained on single NVIDIA T4 GPUs without distributed or multi-node parallelism.Hyperparameter optimization and repeated runs were executed sequentially.
- S4 Computational Resources: Visual fine-tuning and multimodal forecasting used GPUs, whereas non-neural baseline models used CPUs.
S5 Supporting Tables of Model Comparison for Reproducibility
Supporting tables document model comparisons, fine-tuning gains, and stricter multimodal generalization analyses. These analyses show that dietary inputs remain beneficial for unseen subjects, while modality value varies with meal context and data quality.
- S5 Supporting Tables of Model Comparison for Reproducibility: Supporting tables provide full cohort-stratified RMSE and MAE values, plus ID-versus-OOD fine-tuning gains and corresponding RMSE values.These tables are intended to improve completeness, transparency, and reproducibility.
- S5 Supporting Tables of Model Comparison for Reproducibility: RMSE fell from 17.5 for CGM-only Chronos to 16.9–17.0 for image-only, nutrition-only, and combined multimodal models in leave-one-subject-out evaluation.Table S9 reports this stricter subject-level comparison.
- S5 Supporting Tables of Model Comparison for Reproducibility: Multimodal improvements persisted when held-out subjects were completely excluded from training, reducing the likelihood that gains came from memorizing subject-specific glycemic patterns.
- S5 Supporting Tables of Model Comparison for Reproducibility: LOSO multimodal Chronos models remained competitive with, and sometimes stronger than, standard-split non-foundation baselines.
- S5 Supporting Tables of Model Comparison for Reproducibility: The relative usefulness of image and nutrition inputs varied by meal type, with modality reliability and data quality shaping predictive value.Breakfast favored structured nutrition inputs, while lunch supported both modalities when nutrition labels were reliably derived.
S7 A Qualitative Nearest-Neighbor Example of Visual Embeddings
The qualitative nearest-neighbor diagnostic compares category-supervised and nutrition-aligned food-image embeddings. Nutrition-aligned embeddings retrieve visually similar meals with more consistent vegetable, protein, and carbohydrate composition.
- S7 A Qualitative Nearest-Neighbor Example of Visual Embeddings: Figure S3 retrieves the top k = 8 nearest neighbors for a query image under each embedding space and displays associated meal records.
- S7 A Qualitative Nearest-Neighbor Example of Visual Embeddings: Nutrition-aligned embeddings retrieved visually similar neighbors with more consistent vegetable, protein, and carbohydrate composition than category-supervised embeddings.Category-supervised neighbors were visually similar but showed substantial nutritional variability.
- S7 A Qualitative Nearest-Neighbor Example of Visual Embeddings: The nearest-neighbor example serves as a supplementary diagnostic of how the two training objectives structure visual embedding spaces.
- S7 A Qualitative Nearest-Neighbor Example of Visual Embeddings: The experiments used tuned configurations across context windows and prediction horizons, with LoRA applied selectively to Chronos2-small and TimesFM-2.5.The supplied implementation passages describe hyperparameter optimization and LoRA target modules.