Source-linked AI summary
PocketTSLM: Deploying an On-Device Soft-Prompt Time-Series Language Model for Privacy-Preserving Smartphone Reasoning
Max Rosenblattl, Nikhil Krishnaswamy, Leon Nissen, Manuel Milling, Nicolas Zumarraga, Ning Wang, Thomas Kaar, Robert Jakob, Björn W. Schuller, Elgar Fleisch, Patrick Langer, Paul Schmiedmayer
TL;DR
Personal-health systems need interpretations grounded in physiological time-series windows, but mobile context limits and token-only interfaces complicate soft-prompt deployment. PocketTSLM ports OpenTSLM-SP to run locally on a smartphone and evaluates parity and context economics. Soft prompts use approximately 12× fewer time-series tokens than text serialization in the reported benchmark.
Problem
Personal-health systems need descriptions grounded in source time-series windows, while mobile token-only interfaces and limited context constrain feeding soft-prompt embeddings to an on-device model.
Source: §1 · p. 2Method
PocketTSLM ports OpenTSLM-SP to MLX-Swift, adapting on-device generation to accept projected time-series embeddings and evaluating it against reference implementations.
Source: §1 · p. 3Results
Soft prompts use approximately 12× fewer time-series tokens than the text-serialized baseline, reducing prompt-prefill cost and allowing longer windows within the measured on-device budget.
Source: §6 · p. 11Takeaways & Limitations
The evaluated soft-prompt pathway runs physiological time-series inference fully on a consumer smartphone, with health data remaining on-device.
Source: §6 · p. 11The evaluation establishes implementation parity and context economics on fixed Sleep-EDF and ECG-QA samples, not clinical validation or prospective patient HealthKit data.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Consumer phones and wearables increasingly collect physiological time-series data, yet most mobile systems treat these streams as summaries or text rather than as temporal evidence. This representation is poorly matched to personal health systems that need to retrieve, route, or summarize sensor evidence while preserving temporal structure. Cloud-hosted time-series language models avoid smartphone limits, but they repeatedly move sensitive physiology off the device and add latency and per-query compute cost. Mobile model alternatives that aggregate time-series data into statistical features or serialize raw samples as text either compress temporal structure or exhaust the context and memory budgets of small on-device models. We present PocketTSLM, a mobile-native deployment of OpenTSLM SoftPrompt, the soft-prompt variant of the OpenTSLM time-series language model. PocketTSLM runs fully on a consumer smartphone by injecting encoded time-series windows as embeddings instead of tokenizing time series as text. A numerical parity protocol verifies identical greedy-decoded token sequences between the Swift port and a Python reference using Apple's MLX machine-learning framework. Answers also match the PyTorch reference, although rationale wording can differ. An on-device context-economics benchmark compares the soft-prompt path with a text-serialized baseline on the same backbone. In our benchmark format, soft-prompting uses approximately 12 times fewer time-series tokens and reduces prompt-prefill cost. PocketTSLM processes 30-second Sleep-EDF electroencephalography clips for sleep staging and a 1,000-sample (10 s) 12-lead electrocardiogram clip from ECG-QA. In comparison, text-serialized baselines exceed the phone's context or memory budget at longer time-series windows. All inference runs entirely on the device, with no health data transmitted during inference.
1 Introduction
Physiological sensor streams are often reduced to labels, scores, or derived data, while health systems need descriptions grounded in the source time-series window. PocketTSLM addresses the challenge of running a time-series-native language model on a phone, where token-only interfaces and limited context constrain soft-prompt inputs.
1 Introduction
- Personal-health systems often deliver labels or derived-data interpretations, while source-window descriptions can support retrieval, routing, summarization, or deferring when evidence is insufficient.Existing digital biomarkers support screening, monitoring, and study endpoints but do not explain the source time-series window itself.Source: §1 · p. 2
- Remote interpretation moves personal physiological streams or derived evidence off-device and adds network dependence and per-query compute cost; on-device inference can keep raw data local.Mobile deployment also faces memory and efficiency constraints.Source: §1 · p. 2
- Soft-prompt TSLMs require embedding-level input, but mobile generation interfaces accept token identifiers, forcing time series into text and consuming context needed for instructions and generated text.OpenTSLM-SP instead projects time-series windows into soft-prompt embeddings injected into the LLM embedding sequence.Source: §1 · pp. 2–3
- PocketTSLM ports OpenTSLM-SP to MLX-Swift to test on a consumer smartphone whether the soft-prompt pathway preserves reference greedy-decoding behavior.The port preserves the encoder-projector-LLM architecture while adapting generation to accept projected time-series embeddings.Source: §1 · p. 3
2 Related Work
Prior work addresses time-series-to-language modeling, wearable and physiological language systems, and on-device health inference, but does not establish fully on-phone soft-prompt physiological TSLM deployment. PocketTSLM targets that deployment gap while measuring the context cost of soft prompting against text serialization.
Time-series interfaces for language models
- OpenTSLM couples a time-series encoder to a frozen LLM through soft prompts to generate rationales for clinical time-series tasks [11].The paper ports its soft-prompt variant to a smartphone and measures token, memory, and latency costs against text serialization on the same backbone.Source: §2 · p. 3
Wearable and physiological time-series language systems
- Wearable and health language systems often use derived variables or features; SensorLLM and LLaSA focus on motion, while ECG-QA and ECG-Chat address ECG question answering and dialogue [19] [26] [29].These systems do not establish whether soft-prompt physiological TSLM inference can run fully on a consumer smartphone.Source: §2 · p. 3
Deployment settings for health time-series inference
- Prior deployment work leaves privacy, latency, and per-query cost as constraints, while on-device LLM inference is limited by memory and efficiency [11] [14] [15] [19] [29].ODCSIS runs a phone-based language model for activity recognition but serializes statistical features rather than encoding raw time series.Source: §2 · p. 4
3 Methods
PocketTSLM ports OpenTSLM-SP through Python MLX and Swift MLX, adapting the generation path to accept projected time-series embeddings. The methods evaluate parity and compare soft-prompt and text-serialized inputs under a shared backbone and generation policy.
3 Methods
- The soft-prompt architecture encodes fixed temporal patches with a CNN and Transformer, then projects each patch embedding into the frozen Llama-3.2-1B input space for interleaving with text embeddings.Task-specific LoRA adapters select the modality mapping; the decoder must accept precomputed embeddings rather than token identifiers.Source: §3 · p. 4
- Figure 1 shows the on-device pipeline: time-series patches are encoded, projected into LLM embedding space, interleaved with embedded text, and decoded into a rationale.The pipeline runs through MLX-Swift on Apple silicon without health data leaving the phone.Source: Figure 1 · p. 5
- The benchmarks use fixed public de-identified Sleep-EDF and ECG-QA samples for single-channel sleep staging and 12-lead ECG-QA, without retraining the original checkpoints.The demo also runs locally on bundled samples and, on request, the most recent paired-Apple-Watch ECG read from HealthKit.Source: §3 · pp. 5–6
- The evaluation compares the full greedy-decoded token sequence from the Swift port with Python MLX and PyTorch references.A two-stage migration—from PyTorch to Python MLX, then Swift MLX—separates framework conversion from Swift runtime parity.Source: §3 · pp. 4, 6
- The context-economics benchmark compares soft prompts and text serialization using the same raw series, backbone, and generation loop.Soft prompts use ⌈L/4⌉ temporal tokens per channel or lead; the text arm uses three BPE tokens per raw sample per channel or lead in the evaluated zero-padded format.Source: §3 · p. 6
- The sweep reports time-series tokens, prompt feasibility, TTFT, peak memory, and context-window ceiling across series lengths.Theoretical capacity uses an 8192-token context window; measured phone sweeps use a 4,300-token prompt budget, with TTFT and memory averaged over five repetitions.Source: §3 · p. 6
4 Results
PocketTSLM’s soft-prompt path uses substantially fewer time-series tokens and lower prompt-prefill time than text serialization, while remaining within the evaluated on-device budgets. The Swift port matches the Python MLX greedy-decoded sequences, and the study evaluates implementation parity and context economics rather than clinical performance.
Scope
- The reported results cover implementation parity and on-device context economics, not clinical performance or patient-facing deployment.The cited prior clinician review concerns the OpenTSLM model family; this study tests the mobile port and benchmark behavior.Source: §4 · p. 7
Implementation parity
- The Swift and Python MLX implementations produce identical greedy-decoded token sequences; against PyTorch, answers agree although rationale wording can differ under bf16 arithmetic.The four tested components match Python MLX within maximum element-wise tolerance 10−4 (10−3 for rotary embedding).Source: §4 · p. 7
On-device task performance
- On 20 fixed Sleep-EDF samples, PyTorch, Python MLX, and Swift all score 17/20 and make the same three errors, corroborating deployment parity rather than estimating accuracy.The reported 85% has a roughly ±16-point 95% interval; the passage says the 20-sample result corroborates deployment rather than estimating accuracy.Source: §4 · p. 7
Latency and memory
- For 12-lead ECG, text serialization is skipped beyond L=100 to avoid OS memory termination, while soft prompts run the 1,000-sample clip with TTFT ≤11.6 s and peak memory ≤4.2 GiB.Text serialization needs 4,133 prompt tokens and 4.3 GiB at L=100; extensions from L≥250 exceed the 4,300-token on-device budget.Source: §4 · p. 8, Table 2 · p. 10
- For EEG at L=1000, text serialization prefills ∼7× as many tokens and has roughly 6× higher TTFT than soft prompts (14.8 ± 0.8 s vs. 2.4 ± 0.1 s).Soft-prompt TTFT stays ≤3.0 s and peak memory ≤3.9 GiB at every evaluated EEG length; text exceeds the on-device budget at the full 1,500-sample window.Source: §4 · p. 9, Table 1 · p. 9
Context economics
- Approximately 12× fewer time-series tokens across both tasks, with lower soft-prompt TTFT across the evaluated sweep.The comparison is on an iPhone with Llama-3.2-1B; Figure 3 shows the token and TTFT trends.Source: §4 · p. 7, Figure 3 · p. 9
5 Limitations and Future Evaluation
The evaluation establishes implementation parity and on-device context economics, not clinical validation. Its evidence is limited to fixed research samples, a 1B backbone, the evaluated device class, and the soft-prompt pathway.
5 Limitations and Future Evaluation
- Fixed Sleep-EDF and ECG-QA samples do not establish clinical validity; prospective studies using real-world wearable data, clinical review, and predefined safety boundaries are still needed.Wearable noise, variable resolution, missing data, device-specific preprocessing, and population shift were not captured.Source: §5 · p. 10
- The deployment results are limited to Llama-3.2-1B, the evaluated phone class, and the tested soft-prompt pathway; larger backbones and cross-attention were not evaluated.The authors note that larger backbones may exceed current smartphone memory budgets in this architecture.Source: §5 · p. 10
- Local inference reduces exposure from routine cloud inference but does not remove governance obligations, and without central telemetry drift monitoring would need alternative mechanisms.Suggested alternatives include privacy-preserving reporting, local audits, or supervised clinical review.Source: §5 · p. 10
6 Conclusion
PocketTSLM ports OpenTSLM-SP to MLX-Swift for fully local smartphone inference. It uses fewer time-series tokens than text serialization, while the evaluation supports deployment feasibility rather than clinical performance in new settings.
6 Conclusion
- Approximately 12× fewer time-series tokens than text serialization reduces prompt-prefill cost and allows longer physiological windows within the measured on-device budget.The comparison used the same backbone and generation loop on fixed Sleep-EDF and ECG-QA samples.Source: §6 · p. 11
- PocketTSLM executes the encoder, projector, and LLM locally by porting OpenTSLM-SP to MLX-Swift and integrating it into a HealthKit-enabled demo app.The evaluation compares the Swift port with the reference under greedy decoding.Source: §6 · p. 11
- The results are an implementation and deployment contribution, not evidence of clinical performance in new settings.The authors identify prospective evaluation on real-world wearable data with review, safety, monitoring, and governance procedures as the next step.Source: §6 · p. 11
Ethics Statement
The study used public, de-identified research datasets and did not collect new protected health information. No IRB approval was obtained because the evaluation did not involve prospective recruitment, participant interaction, intervention, or identifiable private information.
Ethics Statement
- The study used public, de-identified research datasets and introduced no new protected health information collection.Source: Ethics Statement · p. 11
- No IRB approval was obtained because the study involved no prospective recruitment, participant interaction, intervention, or identifiable private information.Source: Ethics Statement · p. 11
- The reported work was limited to deployment and benchmarking rather than prospective participant research.Source: Ethics Statement · p. 11