Source-linked AI summary
On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health
Ibukunoluwa Soyebo, Alyssa Donawa, Rodrigo Aguilar Barrios, Brice Patchou, Corey E. Baker
TL;DR
Stress prediction in mobile health needs privacy-preserving inference that remains feasible under device constraints, but ODLM evidence for multimodal health time series is limited. This paper evaluates zero-shot ODLM prediction across modalities and temporal representations while measuring accuracy and device efficiency. Objective inputs are generally the most reliable, SS outperforms NLS across modalities and horizons, and compact models can provide competitive accuracy with sub-second latency, although generalization and energy use remain open concerns.
Problem
ODLM feasibility for multimodal health time-series prediction under mobile privacy and resource constraints remains insufficiently understood.
Method
The study evaluates compact, quantized ODLMs with standardized zero-shot prompts across objective, subjective, and combined stress data using NLS and SS representations.
Results
Objective inputs generally provide the most reliable predictions, SS outperforms NLS across modalities and horizons, and Qwen3-0.6B achieves MAE values of 0.51–0.60 under NLS.
Takeaways & Limitations
Compact ODLMs show feasibility for privacy-sensitive mobile stress monitoring by combining competitive accuracy with sub-second latency and low memory requirements.
Takeaways & Limitations
The evaluation uses zero-shot prompting on a single dataset, omits per-device breakdowns, and does not systematically measure energy consumption.
Abstract
from arXiv · showhide
Stress is a pervasive determinant of mental health and a key target for mobile health interventions. On-device language models (ODLMs) offer privacy-preserving inference without cloud dependency, yet their feasibility for health prediction under mobile resource constraints remains underexplored. We evaluate ODLMs for multi-modal stress prediction using zero-shot prompting, measuring predictive accuracy alongside latency and throughput. Our results show that objective sensor features marginally outperform subjective self-reports on average, and that lightweight sub-2B models achieve low latency with predictable resource usage. Our findings highlight both the promise and the practical constraints of ODLMs for mobile mental health.
I. Introduction
Stress prediction matters for mobile health because chronic stress harms health, while ODLMs may enable privacy-preserving, low-latency inference over multimodal data. This work evaluates zero-shot ODLM stress prediction across modalities and temporal representations while measuring accuracy and device-level efficiency.
- The broader motivation is continuous stress monitoring because wearable and smartphone sensing can capture physiological, behavioral, and self-reported signals.
- ODLMs address privacy and latency concerns of cloud-based health inference, but their multimodal reasoning under mobile resource constraints remains insufficiently understood.
- The study evaluates zero-shot ODLM stress prediction using objective sensor-derived and subjective self-reported time-series data.
- The evaluation varies health-data modality and temporal-context serialization rather than relying on fine-tuning.
- Predictive accuracy is measured alongside inference latency and throughput to characterize capability–resource trade-offs on edge devices.
II. Related Works
Prior work spans structured health prediction with general-purpose LLMs, domain-adapted mental-health systems, and mobile deployment infrastructure. These studies motivate evaluating compact ODLMs while distinguishing zero-shot feasibility from gains requiring domain adaptation.
- Prior health-prediction studies compare prompting and fine-tuning strategies, showing that explicit task context can improve predictive accuracy.
- Mental-health applications show that domain adaptation and professionally curated or synthesized dialogue data can improve compact-model performance and response quality.
- The experimental platform, Assuage, supports offline storage and synchronization for ODLM evaluation on iOS and iPadOS devices.
- The deployment setup uses MLXExamples to interface with HuggingFace ODLMs and classifies models by parameter scale.
- Selected models include quantized Qwen3-0.6B and Qwen3-1.7B variants plus the 2-billion-parameter Granite-3.3-2b model.Their on-device footprints are 0.33GB, 0.94GB, and 1.33GB, respectively.
B. Prompting Strategies
The prompting strategy standardizes zero-shot inputs while comparing sequential and aggregate representations of multimodal health time series. Statistical summaries compress each interval into distributional features, whereas NLS preserves ordered values.
- Prompting Strategies: Temporal context prompting represents time-series inputs using Natural Language String or Statistical Summary formats.
- Prompting Strategies: NLS encodes observations as ordered strings and uses NaN for missing entries.
- Prompting Strategies: SS encodes each time series with mean, standard deviation, minimum, and maximum statistics.
- Prompting Strategies: Prompt templates use identical system instructions, temporal formatting, and answer constraints across models.
- Prompting Strategies: The standardized prompts isolate effects of model architecture, scale, and quantization rather than model-specific prompt tuning.
- Dataset: PMData contains five months of objective Fitbit measurements and subjective self-reports from 16 participants.
- Dataset: Objective signals include steps, calories, heart rate, and sleep, while subjective signals include sleep quality, fatigue, mood, and readiness.
IV. Experiment
The experiment decomposes multimodal stress prompts into objective, subjective, and combined variants across multiple retrospective windows. It compares NLS and SS representations while using a reference date to nowcast stress.
- The workflow presents the experiment as a pipeline for constructing prompts, running ODLM inference, and evaluating stress predictions.
- Table I provides an example of the constructed zero-shot prompt designs used in the experiment.
- Three prompt variants isolate objective features, subjective features, and their combination to identify each modality’s contribution.
- Subjective prompts define metric scales relative to upper bounds and label low, normal, and high levels.
- Experiments use 1D, 1W, 2W, and 1M retrospective windows with both NLS and SS prompting strategies.
- Each prompt conditions the model on signals within a reference-date window and asks it to nowcast stress for that date.
V. Results
The evaluation compares conventional supervised baselines and ODLM prompting strategies using MAE across modalities and temporal intervals. Combined features generally improve supervised predictions, while the reported tables organize model performance and prompt-token costs.
- MAE measures disagreement between reported stress values and model predictions, with lower values indicating better agreement.
- RF and SVM consistently achieve the strongest conventional-baseline performance across objective, subjective, and combined feature configurations.
- The results section includes comparisons for traditional baselines, Natural Language String models, Statistical Summary models, and average prompt token counts.
- Combined objective and subjective features generally yield the lowest supervised-model errors across temporal intervals.
B. Model Accuracy: Natural Language String
Under Natural Language String prompting, Qwen3-0.6B is the strongest and most consistent model across modalities and temporal intervals. Objective inputs are generally most reliable, while subjective inputs vary more across models.
- Qwen3-0.6B consistently achieves the lowest MAE across nearly all modality and interval combinations, ranging from 0.51 to 0.60.
- Granite-3.3-2b performs weakest, reaching 0.86 MAE for subjective inputs at 2W.
- Objective inputs generally provide the most reliable predictions, whereas subjective inputs are more variable across models.
- Performance remains relatively stable across temporal intervals despite substantially longer prompts for the 1M interval.
C. Model Accuracy: Statistical Summary
Statistical Summary prompting favors the compact Qwen models, especially for objective features. Fusion performs similarly to the strongest single modality and can reduce errors at longer horizons.
- Qwen3-1.7B with objective features achieves the lowest overall MAE, 0.43 ± 0.24, at the 1D interval.
- Qwen models consistently outperform Granite-3.3-2B across temporal windows, particularly on objective features.
- Objective-plus-subjective fusion yields the lowest errors at 1M for both Qwen models, with MAE of 0.48–0.49.
- The evaluation measures on-device latency and generation throughput to characterize lightweight-model performance under mobile workloads.
- Qwen3-0.6B has the most stable latency profile, while larger models become more sensitive to temporal interval length.
1) Latency:
Throughput decreases with model size, while generation remains relatively consistent across temporal intervals. Qwen3-0.6B therefore combines the highest throughput with the lightest reported model scale.
- Qwen3-0.6B sustains the highest generation rates at 78.0–82.9 tokens/s, followed by Qwen3-1.7B and Granite-3.3-2B.
- Throughput generally decreases with model size, reaching 51.1–56.9 tokens/s for Qwen3-1.7B and 32.0–36.3 tokens/s for Granite-3.3-2B.
- Throughput varies only modestly across temporal intervals, indicating consistent generation performance over time.
A. Traditional ML vs. Zero-Shot ODLMs
Zero-shot ODLMs approach supervised stress-prediction accuracy while avoiding patient-specific training, and Statistical Summary prompting consistently outperforms Natural Language String prompting. Multimodal fusion provides only limited, model-dependent gains.
- A. Traditional ML vs. Zero-Shot ODLMs: 0.51 mean MAE (SS) for Qwen3-0.6B-4bit closely approaches the 0.50 MAE of the best supervised SVM and RF baselines.The ODLM result is zero-shot, whereas supervised models train directly on labeled patient data and may require retraining for new populations or settings.
- A. Traditional ML vs. Zero-Shot ODLMs: Zero-shot ODLM flexibility is especially relevant where labeled mobile-health data are limited, costly, and privacy-constrained.This contrasts with supervised approaches that require labeled patient data and may need retraining for new settings.
- B. Natural Language String vs. Statistical Summary: SS consistently achieves lower MAE than NLS across all modalities and forecast horizons.SS uses a compact, fixed-format representation, while NLS processes increasingly long serialized time-series sequences.
- C. Impact of Multimodal Fusion: Multimodal fusion yields limited and inconsistent improvements, rarely outperforming the strongest unimodal input.The most notable improvement occurs for Granite-3.3-2b at the 1M horizon in the NLS setting.
D. Limitations
The evaluation is limited by zero-shot-only testing, a single dataset, incomplete hardware-specific reporting, and the absence of systematic energy measurements. These constraints restrict conclusions about adaptation, generalization, device variation, and battery-constrained deployment.
- D. Limitations: Zero-shot evaluation without fine-tuning leaves open whether domain-adapted models would reduce the observed accuracy gap relative to supervised baselines.The experiments use no fine-tuning.
- D. Limitations: A single dataset limits evidence about generalization across populations, devices, and stress measurement scales.Results may not transfer across these settings.
- D. Limitations: Unsystematic energy measurement leaves battery-constrained deployment insufficiently evaluated.Device-level results also lack per-device breakdowns, which may obscure hardware-specific variation.