PaperlayerView exact revision on arXiv

Source-linked AI summary

SOTER: A Generative Time-Series Foundation Model for Wearable Human Physiological Signals

Fangke Chen, Sirry Chen, Wei Chen, Zhongyu Wei

arXiv:2609.16804v1cs.LGcs.AI

TL;DR

Wearable physiological signals combine coupled channels, distinct spectral scales, and irregular observations, while existing time-series models do not jointly accommodate these properties. SOTER unifies channel-aware modeling, PSD-guided expert routing, and continuous-time decoding, and shows transfer across forecasting, classification, and imputation benchmarks.

  • Problem

    Wearable physiological signals have coupled channels, interacting spectral scales, and irregular, incomplete observations, while existing models do not jointly address these properties.

    Source: §1 · p. 2

  • Method

    SOTER combines a spatial feature-aware backbone, PSD-guided expert specialization, and a Neural CDE/ODE decoder in a generative model pretrained on public physiological data.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    Across forecasting, classification, and causal continuous-time imputation, one pretrained SOTER backbone achieves strong and consistent transfer and remains robust under substantial missingness and additive noise.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The findings suggest that wearable-physiology foundation models benefit from jointly modeling channel structure, spectral scale, and continuous-time dynamics.

    Source: §5 · p. 8

    The PSD router uses normalized latent-trajectory frequencies rather than a physical-frequency spectrum, making its statistic a sampling-rate-invariant measure of spectral shape.

    Source: §3.2.3 · p. 4

Abstract

from arXiv · show

Time-series foundation models have demonstrated strong cross-domain transfer, yet their common architectural assumptions remain poorly aligned with wearable physiological signals, which are multichannel, irregularly sampled, noisy, and governed by coupled continuous-time dynamics spanning distinct spectral scales. We present SOTER, a generative foundation model for wearable physiological time series that unifies cross-channel coupling, spectrum-guided expert specialization, and continuous-time latent evolution within a single pre-training framework. SOTER combines a spatial feature-aware backbone that models inter-signal dependencies, a power spectral density (PSD)-guided mixture-of-experts layer that routes representations to experts associated with fixed spectral bands through an inspectable, non-learned rule, and a neural controlled differential equation decoder that supports prediction and imputation at arbitrary timestamps. We pre-train SOTER on 226 billion time points from five public physiological datasets and evaluate the same pre-trained model across out-of-distribution zero-shot forecasting, frozen-encoder linear-probe classification, and continuous-time imputation on wearable benchmarks. SOTER achieves the best RMSE on 4 of 6 datasets and the best MAE on 5 of 6 in zero-shot forecasting, the highest average Macro-AUROC in classification, and the lowest imputation error on all six datasets at 75% missingness. It further remains robust to additive acquisition noise, matching or surpassing baselines evaluated on clean inputs even under the strongest corruption. These results indicate that domain-specialized foundation models for wearable physiology benefit from jointly modeling channel structure, spectral scale, and continuous-time dynamics.

1 Introduction

Wearable physiological signals pose challenges of coupled channels, heterogeneous spectral scales, and irregular, incomplete sampling, while existing time-series models do not jointly address these properties. SOTER combines cross-channel modeling, PSD-guided expert specialization, and continuous-time decoding in a generative model pretrained on public physiological data.

1 Introduction

  • Differences across populations, devices, and institutions, alongside privacy constraints on data aggregation, motivate transfer from de-identified public data.Downstream uses include emotion recognition and seizure detection.Source: §1 · p. 1
  • General time-series models commonly assume channel independence, implicit learned expert routing, or discrete temporal grids, leaving the combined physiological modeling requirements unmet.Existing medical and wearable models address individual aspects rather than jointly modeling channel coupling, spectral specialization, and continuous-time generation.Source: §1 · p. 1
  • Physiological channels provide complementary observations of shared states, dynamics span interacting spectral scales, and irregular incomplete sampling challenges fixed-grid models.These properties motivate cross-channel coupling, spectrum-conditioned capacity allocation, and continuous-time state evolution.Source: §1 · pp. 1–2
  • SOTER jointly models cross-channel coupling, spectral scale, and continuous-time dynamics in a generative foundation model for wearable physiology.Its backbone, PSD-guided expert layer, and Neural CDE/ODE decoder address these properties within a unified generative objective.Source: §1 · p. 2
  • SOTER is pretrained on 226 billion time points from five public physiological datasets and transfers across forecasting, frozen-representation classification, and continuous-time imputation.The paper reports evaluations across six out-of-distribution wearable datasets, including noisy observations, limited supervision, and different pretraining exposures.Source: §1 · p. 2

2 Related Work

Generalist time-series foundation models offer broad transfer but largely retain generic assumptions about temporal grids, channel coupling, and expert routing. Medical and wearable models address some physiological needs, but SOTER combines them in a single pretrained model.

2.1 Generalist Time-Series Foundation Models

  • Generalist time-series foundation models use varied architectures, yet their dominant assumptions do not directly address the combined structural requirements of wearable physiological signals.These include discrete temporal grids, limited physiological channel coupling, and typically implicit learned expert specialization [14] [31] [33].Source: §2.1 · p. 2

2.2 Medical Time-Series Models

  • Medical and wearable models address complementary subsets of the problem, while SOTER unifies cross-channel coupling, spectrum-conditioned specialization, and continuous-time generation.PaPaGei specializes in PPG [43], MIRA incorporates Neural ODEs [27], and NormWear uses a shared liaison token for channel relationships.Source: §2.2 · p. 2

3 Methodology

SOTER predicts values at queried timestamps from wearable recordings using channel-aware temporal representations, spectral-band expert routing, and a continuous-time decoder. Its training alternates between extrapolation and observation-guided decoding on a large public physiological corpus.

3.1 Problem Statement

  • SOTER is pretrained generatively to predict the next queried timestamp from wearable recordings with shared, possibly irregular timestamps.The overview uses channel-independent layers followed by a channel-dependent layer and a continuous-time decoder.Source: §3.1 · pp. 2–3, Figure 1 · p. 3

3.2.1 Continuous-Time Input Encoding.

  • Continuous-time rotary positional encoding rotates queries and keys according to absolute timestamps rather than token positions.Lower layers treat each channel as an independent univariate sequence to avoid premature cross-channel leakage.Source: §3.2.1 · p. 3

3.2.2 Spatial Feature-Aware Backbone.

  • The backbone uses channel-independent temporal attention in its first L−1 layers, then a single high-level layer restores cross-channel interaction.The final channel-dependent layer operates on refined representations; when there is one channel, it falls back to channel-independent temporal attention.Source: §3.2.2 · pp. 3–4, Figure 1 · p. 3

3.2.3 PSD-Guided Mixture-of-Experts.

  • A causal prefix DFT supplies spectral magnitudes for routing to the top two experts in fixed contiguous frequency bands.The normalized latent-frequency statistic is sampling-rate-invariant; routing is deterministic and uses no auxiliary balancing loss.Source: §3.2.3 · p. 4

3.2.4 Neural CDE Extrapolation Decoder.

  • The decoder evolves the last observed latent state to arbitrary query times, using a causal spline control path for within-span queries and reducing to a Neural ODE when control is unavailable.For imputation queries, the spline is fitted only on observations before the prediction origin; beyond the observation span, the control path is zero.Source: Figure 1 · p. 3, §3.2.4 · p. 4

3.3 Model Training

  • SOTER is pretrained for one epoch on approximately 226 billion time points from five public, de-identified datasets using masked Huber loss.Training alternates equally between full-history Neural ODE decoding and a regime masking 15% of historical observations for Neural CDE-style decoding.Source: §3.3 · pp. 4–5

4 Experiments and Analysis

Across zero-shot forecasting, classification, and imputation, SOTER transfers strongly to wearable benchmarks; analyses also examine its component contributions, routing behavior, noise robustness, and pre-training exposure.

4.2 Zero-Shot Future Forecasting

  • SOTER achieves the best RMSE on four of six OOD datasets and the best MAE on five, with nine of twelve dataset–metric combinations best overall.MMASH is the only dataset where it achieves neither best RMSE nor best MAE.Source: §4.2 · pp. 5–6

4.3 Downstream Task Evaluation

  • SOTER has the highest average Macro-AUROC at 80.79, ahead of TF-C (77.08), NormWear (76.34), and MIRA (76.24).It also obtains the best Macro-AUROC on all three datasets, while trailing TF-C in Macro-F1 on MMASH.Source: §4.3 · p. 6, Table 2 · p. 6
  • At 75% missingness, SOTER ranks first on all six datasets for continuous-time imputation under the causal-prefix protocol.It has the lowest RMSE on most datasets at each evaluated missing rate; TiRex outperforms it on MMASH at the two lower rates.Source: §4.3 · pp. 6–7, Figure 2 · p. 6

4.4 Ablation Analysis

  • Removing any of the three core components increases RMSE and MAE; CD-layer removal degrades multi-channel datasets more than single-channel HeartRate.Replacing PSD-MoE with a learned auxiliary-loss router yields higher errors; moving the CD layer downward increases errors on both multi-channel datasets, while HeartRate remains nearly unchanged.Source: §4.4 · p. 7, Table 3 · p. 7

4.5 Corpus-Matched Comparison

  • With the same corpus and training budget, SOTER remains ahead of both corpus-matched MIRA-Large variants on all four forecasting metrics.Against continued-pretrained MIRA-Large, RMSE falls from 1.79 to 1.29 on MIT-BIH and from 3.12 to 2.26 on ScientiSST MOVE; corpus matching improves MIRA but does not account for SOTER’s gains on these datasets.Source: §4.5 · p. 7, Table 4 · p. 7

4.6 Interpretability of PSD-Guided Routing

  • Gate assignments show within-signal similarity and cross-signal separation, with mid-frequency PPG between high-frequency ECG and low-frequency heart rate.This pattern indicates the latent spectral statistic remains associated with underlying signal spectra and provides an inspectable basis for routing.Source: §4.6 · pp. 7–8, Figure 3 · p. 8

4.7 Noise Robustness Analysis

  • Across all six noise levels, both metrics, and three datasets, SOTER has the lowest prediction error; at β=0.5 it matches or beats baselines on clean inputs.Noise is injected into the observed context, and predictions are evaluated against the clean future.Source: §4.7 · p. 8, Figure 4 · p. 8

4.8 Cumulative Pre-training Exposure Analysis

  • Increasing cumulative pre-training exposure consistently reduces RMSE and MAE across all three OOD datasets, with dataset- and metric-dependent saturation.On MIT-BIH, RMSE at 50% exposure is within 2.3% of the final result, while MAE approaches its final value only after 75%; WDD depends most strongly on later-stage exposure, particularly for MAE.Source: §4.8 · p. 8, Table 5 · p. 8

5 Conclusion

SOTER jointly models cross-channel coupling, spectrum-guided expert specialization, and continuous-time dynamics for wearable physiology. It is pre-trained on a large corpus assembled from five public physiological datasets.

5 Conclusion

  • SOTER jointly models cross-channel coupling, spectrum-guided expert specialization, and continuous-time dynamics, with strong transfer across forecasting, classification, and causal continuous-time imputation.It activates 20.29M parameters per inference step and remains robust under substantial missingness and additive acquisition noise.Source: §5 · p. 8

A Datasets and Statistics

  • The approximately 226-billion-time-point pre-training corpus combines five public datasets: MIMIC-III Waveform, Sleep-EDF, PTB-XL, WESAD, and Chapman-ECG.These sources span clinical waveforms, polysomnography, ECG, and multimodal wearable measurements.Source: A Datasets and Statistics · p. 10, §A.1 · p. 10

A.2 Evaluation Datasets

The evaluation datasets span single- and multichannel wearable or ambulatory signals, from short PPG segments to recordings collected during daily activities, exercise, and stress.

A.2 Evaluation Datasets

  • HeartRate pairs short PPG segments with continuous heart-rate targets, while MIT-BIH provides two-channel ambulatory ECG excerpts with cardiologist beat annotations.MIT-BIH contains 48 half-hour excerpts from 47 subjects and approximately 110,000 reference annotations.Source: §A.2 · pp. 10–11
  • ScientiSST MOVE combines chest, arm, and wrist recordings during everyday activities, whereas IEEE PPG pairs wrist PPG and acceleration with chest ECG during exercise.ScientiSST MOVE includes EDA, PPG, ECG, bicep EMG, wrist temperature, and actigraphy; IEEE PPG contains 3,096 sequences recorded at 125 Hz.Source: §A.2 · p. 11
  • MMASH captures 24-hour heart, activity, sleep, and psychological measurements; WDD records wearable signals during acute stress and aerobic or anaerobic exercise.MMASH has the lowest NPE (0.334) and MIT-BIH the highest (0.878); higher NPE indicates less predictable dynamics.Source: §A.2 · p. 11, Table 6 · p. 11

B Pre-training Setup and Logs

SOTER is pre-trained for one epoch on approximately 226 billion time points, with periodic held-out MIT-BIH evaluation showing rapid early improvement followed by stable convergence.

B Pre-training Setup and Logs

  • Pre-training covers approximately 226 billion time points in one epoch, using 2,076,572 optimization steps on eight NVIDIA H200 GPUs.The global batch size is 64, and the maximum sequence length is 512.Source: B Pre-training Setup and Logs · p. 11
  • Optimization uses AdamW with a 3 × 10^-4 learning rate, cosine annealing, and a 20-step warmup; all runs use seed 42.No gradient clipping is applied.Source: B Pre-training Setup and Logs · p. 11
  • The continuous-time decoder uses dopri5 at tolerance 10^-5, and PSD routing uses a parallel, strictly causal prefix DFT.The solver, tolerance, and DFT implementation are examined in a separate sensitivity analysis.Source: B Pre-training Setup and Logs · p. 11
  • RMSE and MAE decrease rapidly in early evaluations, then stabilize at low values for the remainder of the single-epoch run.Figure 5 reports normalized-space errors on 32 held-out MIT-BIH samples, evaluated every 100,000 optimization steps with 128/64 input/output windows.Source: B Pre-training Setup and Logs · p. 11, Figure 5 · p. 11

C Key Sensitivity Analysis

The sensitivity analysis favors a parallel-prefix DFT with dopri5 at tolerance 10^-5 as an accuracy–efficiency trade-off, rather than the lowest-error or fastest configuration.

C Key Sensitivity Analysis

  • The sensitivity runs compare parallel-prefix and loop-based causal prefix DFTs, dopri5 and rk4 solvers, and tolerances from 10^-6 to 10^-3.Each configuration is evaluated after 100,000 optimization steps on MIT-BIH forecasting with 256 samples and 128/64 input/output windows.Source: C Key Sensitivity Analysis · p. 12
  • The adopted parallel-prefix DFT with dopri5 at 10^-5 adds 0.14 seconds per step over the fastest configuration while offering a more favorable accuracy–efficiency trade-off.The lowest errors came from loop-based DFT with dopri5 at 10^-5, but that configuration required approximately four times the per-step latency of its parallel-prefix counterpart.Source: C Key Sensitivity Analysis · p. 12

D From Neural CDE to Neural ODE

SOTER uses one continuous-time decoder for observation-guided imputation and future forecasting: forecasting is obtained by setting the imputation control path to zero. The reduction is an exact identity with shared parameters, not a separate decoder or solver approximation.

D.1 Setup and Notation

  • The setup initializes decoding from the backbone’s terminal hidden state and advances it toward the queried timestamp, with a numerical safeguard for zero-length intervals.For non-degenerate queries, the integration interval equals the timestamp difference.Source: §D.1 · p. 12

D.2 Unified Terminal Dynamics

  • Both tasks solve the same terminal initial-value dynamics over normalized time, with one shared vector field taking time, latent state, and control as inputs.The normalized interval runs from the last observed timestamp to the queried endpoint.Source: §D.1 · pp. 12–13, §D.2 · p. 13

D.3 The Two Decoder Regimes

  • For imputation, observations before the query define a natural-cubic-spline control; for forecasting, the control is zero because no observation-derived path is available beyond context.The zero vector is a valid input to the same vector field.Source: §D.3 · p. 13

D.4 Exact Reduction

  • The reduction is an exact algebraic identity: switching to zero control adds, removes, or updates no parameters.Under the stated continuity and Lipschitz conditions, both formulations have unique solutions.Source: §D.4 · p. 13

D From Neural CDE to Neural ODE

  • SOTER’s controlled dynamics inject the control value as an MLP input, a control-as-input formulation distinct from the bilinear Neural CDE formulation.The zero-control reduction applies specifically to this formulation.Source: §D.4 · p. 14

D.5 Summary

  • Forecasting is the observation-guided decoder evaluated at zero control, so one frozen decoder supports causal imputation and autonomous forecasting with the same parameters.The two regimes differ only in their control path.Source: §D.3 · p. 13, §D.5 · p. 14

E System Overhead Analysis

SOTER’s inference profile trades latency for compactness: it activates 20.29M parameters and is faster than MIRA-Large, but slower than Aurora and both TimeMoE variants. The appendix also reports per-window forecasting measurements in normalized space.

E System Overhead Analysis

  • SOTER takes 6.09 s per sample while activating 20.29M parameters; it is slower than Aurora and both TimeMoE variants, but faster and more compact than MIRA-Large.Adaptive continuous-time integration adds numerical overhead, so latency does not follow parameter count alone.Source: E System Overhead Analysis · p. 14

F Complete Per-Window Zero-Shot Forecasting Results

  • Per-window zero-shot forecasting tables report unscaled normalized-space RMSE and MAE, while Table 1 aggregates full-precision measurements across supported windows.Timer supports only the 96/48 and 128/64 input/output windows.Source: F Complete Per-Window Zero-Shot… · p. 14

G Continuous-Time Imputation across Missing Rates

SOTER’s imputation advantage is strongest at 75% missingness, where it has the lowest error across all 12 dataset–metric combinations. Separate visualizations show example forecasts on two-channel MIT-BIH and single-channel HeartRate signals.

G Continuous-Time Imputation across Missing Rates

H Zero-Shot Forecasting Visualizations

I Label-Efficient Frozen-Representation Evaluation

On ScientiSST MOVE, SOTER’s frozen representations outperform competing results across supervision levels, including with only 1% of labeled data. The paper also states that authors reviewed and verified LLM-assisted code or text.

I Label-Efficient Frozen-Representation Evaluation

  • With 1% of labels on ScientiSST MOVE, SOTER reaches 40.48 Macro-F1 and 89.03 Macro-AUROC, exceeding the strongest competing results by 5.00 and 4.84 percentage points, respectively.SOTER achieves the best Macro-F1 and Macro-AUROC at every evaluated supervision level.Source: I Label-Efficient… · p. 18
  • With 25% of labels, SOTER reaches 48.01 Macro-F1 and 94.89 Macro-AUROC, compared with 50.90 and 95.62 under full supervision.Source: I Label-Efficient… · p. 18

J LLM Usage Declaration

  • Authors say they reviewed, revised, and verified LLM-assisted code or text and take responsibility for the work’s correctness and integrity.Source: J LLM Usage Declaration · p. 18
Loading 2609.16804v1…