PaperlayerView exact revision on arXiv

Source-linked AI summary

Wearable Foundation Models Should Go Beyond Static Encoders

Yu Yvonne Wu, Yuwei Zhang, Hyungjun Yoon, Ting Dang, Dimitris Spathis, Tong Xia, Qiang Yang, Jing Han, Dong Ma, Sung-Ju Lee, Cecilia Mascolo

arXiv:2603.19564v1cs.LG

TL;DR

Wearable foundation models perform well on short-term tasks but remain limited by static encoders that do not reason over evolving personal history, context, and future risk. The paper argues for structurally rich data, longitudinal-aware multimodal modeling, and agentic inference to support continuous, anticipatory health reasoning.

  • Problem

    Most WFMs map short temporal windows to predefined targets, limiting their suitability for longitudinal, personalized health reasoning.

    Source: §1 · pp. 1–3

  • Method

    The paper proposes coordinated shifts toward structurally rich data, longitudinal-aware multimodal modeling, and agentic inference across the WFM pipeline.

    Source: §1 · p. 3, §3.1 · p. 6, §3.2 · p. 7

  • Results

    The paper concludes that moving beyond static encoders requires integrated multimodal trajectories, temporal abstraction, and inference that reasons adaptively over evolving evidence.

    Source: §1 · p. 3, §3.2 · p. 7, §3.3 · p. 7

  • Takeaways & Limitations

    WFMs should be reframed as longitudinal, anticipatory, and human-aligned health-support systems rather than retrospective signal predictors.

    Source: §1 · p. 3, §3.2 · p. 7, §3.3 · p. 8

Abstract

from arXiv · show

Wearable foundation models (WFMs), trained on large volumes of data collected by affordable, always-on devices, have demonstrated strong performance on short-term, well-defined health monitoring tasks, including activity recognition, fitness tracking, and cardiovascular signal assessment. However, most existing WFMs primarily map short temporal windows to predefined labels via static encoders, emphasizing retrospective prediction rather than reasoning over evolving personal history, context, and future risk trajectories. As a result, they are poorly suited for modeling chronic, progressive, or episodic health conditions that unfold over weeks, months or years. Hence, we argue that WFMs must move beyond static encoders and be explicitly designed for longitudinal, anticipatory health reasoning. We identify three foundational shifts required to enable this transition: (1) Structurally rich data, which goes beyond isolated datasets or outcome-conditioned collection to integrated multimodal, long-term personal trajectories, and contextual metadata, ideally supported by open and interoperable data ecosystems; (2) Longitudinal-aware multimodal modeling, which prioritizes long-context inference, temporal abstraction, and personalization over cross-sectional or population-level prediction; and (3) Agentic inference systems, which move beyond static prediction to support planning, decision-making, and clinically grounded intervention under uncertainty. Together, these shifts reframe wearable health monitoring from retrospective signal interpretation toward continuous, anticipatory, and human-aligned health support.

1. Introduction

Existing WFMs largely use static encoders that map short wearable-data windows to predefined targets, limiting longitudinal and personalized health reasoning. The paper calls for coordinated changes in data, modeling, and inference to support anticipatory health reasoning.

  • Static encoders map short physiological and behavioral segments to labels or scalar targets, limiting longitudinal tracking, baseline estimation, and reasoning about gradual change or disease progression.Source: §1 · pp. 1–2
  • Wearable datasets often emphasize short windows, isolated modalities, and outcome-conditioned labels while lacking integrated context and open, interoperable longitudinal data.Source: §1 · p. 2
  • Fixed modalities, temporal resolutions, and task-specific fusion hinder generalization to heterogeneous sensors, missing modalities, variable sequence lengths, and long-term trajectories.Source: §1 · p. 2
  • Discriminative inference produces independent labels or scores rather than accumulating evidence and deciding when sensing, feedback, or intervention is warranted.Source: §1 · pp. 2–3
  • The paper proposes structurally rich data, longitudinal-aware multimodal modeling, and agentic inference as coordinated shifts beyond static encoders.Source: §1 · p. 3

2. Overview of Wearable Foundation Models

The overview frames WFMs as multimodal systems spanning data acquisition, representation learning, and inference. It surveys heterogeneous longitudinal data, multimodal pretraining pipelines, and downstream applications that remain predominantly task-specific predictors.

2.1. Data Acquisition

  • Wearable datasets combine heterogeneous sensors, contextual information, and repeated measurements, shifting modeling toward within-subject dynamics and long-term evolution.Source: §2.1 · p. 3
  • Only a few WFMs reach very large scales, including LSM and LSM-2 with 40 million hours from 165K participants and WBM with 2.5 billion hours from 162K individuals.Source: §2.1 · p. 3

2.2. Modeling

  • Multimodal WFM pipelines tokenize sensor streams, pretrain with reconstruction or contrastive objectives, align modality representations, and increasingly integrate language and context.Source: §2.2 · pp. 4–5
  • Tokenization strategies use temporal, time–frequency, and modality-aware representations, with longitudinal models reducing token resolution through minute- or hour-level features.Source: §2.2 · p. 4

2.3. Inference

  • Despite large-scale pretraining and language alignment, most WFMs are evaluated on fixed classification or regression tasks such as sleep duration, diabetes, pregnancy, or activity prediction.Source: §2.3 · pp. 5–6

3. Call to Action

The call to action argues that WFMs must move beyond static encoders through structurally rich data, longitudinal-aware multimodal modeling, and agentic inference. Together, these shifts target continuous, context-aware, and human-aligned health monitoring.

3.1. Structurally Rich Data

  • Wearable datasets remain fragmented across proprietary access, heterogeneous sensors, short time spans, narrow modalities, and outcome-conditioned collection.These constraints limit reproducibility, cross-dataset evaluation, personalized trajectory learning, and contextual understanding.Source: §3.1 · p. 6
  • Open, structurally rich data should integrate multimodal streams, long-term personal trajectories, and contextual metadata to support longitudinal continuity and cross-dataset reuse.The proposal prioritizes openness and interoperability rather than data volume alone.Source: §3.1 · p. 6

3.2. Multimodal and Longitudinal Modeling

  • Longitudinal-aware multimodal models should use shared evaluation interfaces, contextual grounding, hierarchical temporal representations, and persistent personal memory for heterogeneous, irregular sensing.Proposed mechanisms include trajectory forecasting, event discovery, multi-scale tokens, and selective evidence retrieval inspired by retrieval-augmented generation.Source: §3.2 · pp. 6–7

3.3. The Need for Agentic Inference

  • Agentic inference should iteratively select what to infer, when to infer it, and how to update internal state as multimodal longitudinal evidence evolves.The Bio-Toolset framing treats WFMs as modular perceptual and analytical components that support centralized reasoning rather than end-to-end health decision-makers.Source: §3.3 · pp. 7–8

4. Alternative Views

The paper acknowledges that large unimodal models can provide strong baselines, especially when multimodal data are unavailable or noisy. It nevertheless argues that unimodal scaling is insufficient for comprehensive longitudinal health reasoning, while agentic systems should augment rather than replace clinical judgment.

A sufficiently large single-modality foundation model

  • Large single-modality foundation models can generalize across populations and tasks when trained with sufficient data scale and suitable pretraining objectives.The paper presents unimodal models as strong and often necessary baselines when multimodal data are unavailable or noisy.Source: A sufficiently large single-modality… · p. 8
  • Unimodal models remain limited for states expressed through interactions among physiological, behavioral, and contextual signals, and for cross-signal longitudinal reasoning.They are described as well-suited for detection but insufficient for comprehensive health understanding.Source: A sufficiently large single-modality… · p. 8
  • Agentic WFMs should augment human decision-making by aggregating evidence and supporting longitudinal planning while remaining human-in-the-loop.The paper gives adaptive monitoring as an example, including selecting signals, temporal windows, and comparisons with an individual’s historical baseline.Source: A sufficiently large single-modality… · p. 8

5. Conclusion

The conclusion calls for WFMs to move beyond static encoders toward longitudinal, anticipatory health reasoning. It identifies structurally rich data, longitudinal-aware multimodal modeling, and agentic inference as the foundation for continuous, context-aware, clinically grounded support.

5. Conclusion

  • WFMs should be explicitly designed for longitudinal, anticipatory health reasoning rather than static encoding.The proposed transition encompasses data ecosystems, multimodal modeling, and agentic inference.Source: §5 · p. 8

Related papers

Loading 2603.19564v1…