Source-linked AI summary
Large Language Models are Few-Shot Health Learners
Xin Liu, Daniel McDuff, Geza Kovacs, Isaac Galatzer-Levy, Jacob Sunshine, Jiening Zhan, Ming-Zher Poh, Shun Liao, Paolo Di Achille, Shwetak Patel
TL;DR
Health applications require grounding LLMs in physiological and behavioral measurements that are not readily represented as text. The paper evaluates few-shot prompt tuning on wearable and clinical time-series tasks and finds substantial improvements across cardiovascular, activity, metabolic, and mental-health evaluations. However, the evidence is limited to one LLM and small balanced test sets.
Problem
Physiological and behavioral time-series grounding remains relatively unexplored, limiting evidence about LLM performance on measured health data.
Method
The paper constructs numerical question-answer datasets from wearable and clinical sensing data and compares zero-shot, prompt-engineering, prompt-tuning, and supervised approaches.
Results
Across cardiovascular, activity, calorie-estimation, and mental-health tasks, few-shot prompt tuning improved performance over zero-shot and supervised baselines.
Takeaways & Limitations
Appropriately tuned LLMs can handle varied numerical physiological and behavioral health tasks beyond traditional text processing.
Takeaways & Limitations
The study evaluates only one LLM and uses small balanced test sets, limiting extrapolation to other models and settings.
Abstract
from arXiv · showhide
Large language models (LLMs) can capture rich representations of concepts that are useful for real-world tasks. However, language alone is limited. While existing LLMs excel at text-based inferences, health applications require that models be grounded in numerical data (e.g., vital signs, laboratory values in clinical domains; steps, movement in the wellness domain) that is not easily or readily expressed as text in existing training corpus. We demonstrate that with only few-shot tuning, a large language model is capable of grounding various physiological and behavioral time-series data and making meaningful inferences on numerous health tasks for both clinical and wellness contexts. Using data from wearable and medical sensor recordings, we evaluate these capabilities on the tasks of cardiac signal analysis, physical activity recognition, metabolic calculation (e.g., calories burned), and estimation of stress reports and mental health screeners.
1 Introduction
LLMs encode broad knowledge, but health applications require grounding language-based models in physiological and behavioral measurements. This paper investigates whether few-shot tuning can support numerical time-series health inference across clinical and wellness tasks.
- Motivation: Language alone does not capture all information needed to connect model knowledge to physical-world events and actions.The paper motivates grounding LLMs with non-linguistic observations, including physiological and behavioral data.
- Research gap: Physiological and behavioral time-series grounding remains relatively unexplored despite its importance for human health.Existing medical language models perform well on text-based examinations but have been tested less extensively on measured heart-rate and activity data.
- Motivation: Wearables, phones, and low-cost sensors make physiological monitoring commonplace, but traditional task-specific health-data collection is time-consuming and resource-intensive.The paper positions few-shot inference as a possible alternative to collecting substantial data for each task.
- Study aim: The study compiles common consumer-health tasks to test whether LLMs can ground time-series data and act as universal few-shot learners.The dataset spans wellness and clinical tasks such as activity recognition and heart-rhythm classification.
2 Background
The background reviews LLMs’ broad knowledge and few-shot capabilities alongside health-monitoring applications that require processing raw physiological and behavioral sensor data. It also highlights arithmetic and reliability challenges relevant to these tasks.
- Large Language Models: LLMs can encode broad knowledge and perform zero-shot or few-shot inference because they are trained on diverse, large-scale text corpora.Scaling model, dataset, and compute size has been associated with smooth performance improvements across language tasks.
- Large Language Models: Medical-domain language models improve performance on medical benchmarks, but they still exhibit inaccuracies, hallucinations, and limited awareness of their abilities.These weaknesses motivate caution when extending language-model capabilities to health applications.
- Arithmetic: LLMs’ arithmetic performance tends to decline as tasks become more complex and input sequences become longer.This limitation is directly relevant to health tasks involving numerical time series and calculations.
- Health: Health sensing supports monitoring of physical and behavioral states, while raw sensor data remains difficult for people and LLMs to consume.Examples include classifying activities, computing calories, detecting arrhythmia, and responding to stress.
3 Health Tasks
The paper evaluates nine health tasks across cardiovascular, metabolic, activity-recognition, and mental-health domains. These tasks range from simple numerical operations to interpreting physiological and behavioral data in relation to mental states.
- Task Design: The task suite varies from averaging instantaneous heart measurements to interpreting numerical behavior and physiology in relation to mental states.This range lets the evaluation cover arithmetic, sequential-data interpretation, and contextual reasoning.
- Cardiovascular: The cardiovascular domain includes five tasks covering heart-rate computation, interbeat-interval conversion, and classification of atrial fibrillation, bradycardia, and tachycardia.The tasks include both regression and binary classification.
- Metabolic: The metabolic task estimates calories burned from physical activity using the MET formula and inputs for activity, duration, and weight.The task requires recalling activity-specific MET values and applying the calculation.
- Activity Recognition: Activity recognition classifies walking and running using accelerometer data from the PAMAP2 Physical Activity Monitoring Dataset.The study restricts evaluation to these two classes because performance on many-class classification was poor.
- Mental Health: The mental-health tasks predict daily ecological momentary stress and PHQ-8 depression scores or classifications from wearable behavior and physiology.The depression task uses wearable data over a two-week period aligned with the PHQ-8 assessment period.
4 Method & Experiment
The experiments convert quantitative health measurements into textual question-answer prompts and compare zero-shot, prompt-engineering, prompt-tuning, and supervised approaches. Prompt tuning adds a learnable task-specific prefix while keeping the pretrained language-model weights frozen.
- Prompt Construction: Quantitative data for each task are embedded into textual templates to create question-answer pairs for evaluation.The study compares zero-shot evaluation with prompt engineering and prompt tuning.
- Zero-Shot: Zero-shot evaluation uses numerical physiological data and task-specific questions without examples or weight fine-tuning.Inputs include sequences such as interbeat intervals and higher-level sensor measurements.
- Prompt Tuning: Prompt tuning adds a soft learnable prompt embedding that supplies task-specific information to the frozen language model.The learned prefix is intended to help the model interpret numerical and time-series data absent from its original training.
- Prompt Tuning: The study compares context-inclusive prompts, which identify data sources and units, with numerical-only prompts that provide sequences without textual context.These strategies test the effects of contextual information and the model’s ability to recognize patterns from numbers alone.
- Experimental Design: Prompt tuning is evaluated in 3-shot, 10-shot, and 25-shot settings, while prompt engineering uses a 3-shot setting because time-series examples are lengthy.A supervised baseline is included to compare few-shot grounding against conventional learning with small training sets.
5 Result & Discussion
Few-shot prompt tuning substantially improved LLM performance across cardiovascular, activity, calorie-estimation, and mental-health tasks, while context-inclusive prompts generally outperformed numerical-only prompts. Zero-shot performance was often inadequate, and the study cautions that results come from one model and small balanced test sets rather than a deployable system.
- Up to 130% improvement over zero-shot and up to 75% over the supervised 25-shot baseline occurred across five cardiovascular tasks with 25-shot prompt tuning.
- Up to 44% accuracy improvement over zero-shot and up to 30% over the supervised 25-shot baseline occurred in Fitbit-based mental-health tasks.
- Zero-shot performance was poor on several physiological and behavioral tasks, including 139 error in calorie estimation and 0.5 accuracy in atrial fibrillation classification.
- Context-inclusive prompts improved performance across tasks except the very simple task of averaging heart rates, compared with numerical-only prompts.
- The authors evaluated one LLM with small balanced test sets and did not attempt to create a system for practical deployment.
6 Conclusion
The paper concludes that few-shot prompt tuning can ground a 24B language model in physiological and behavioral time-series data. This produced large improvements across cardiac, metabolic, physical, and mental-health tasks.
- Few-shot prompt tuning on a 24B language model grounded time-series data and produced large improvements across cardiac, metabolic, physical, and mental-health tasks.
7 Additional Results of Prompt Engineering & Zero-Shot
Prompt tuning with context-inclusive prompts improves consumer-health performance over zero-shot and prompt-engineered LLMs, while reducing failures on lengthy time-series inputs.
- Table 4 compares zero-shot LLMs with prompt-tuned LLMs using context-inclusive prompts across consumer health tasks.
- Prompt-tuned LLMs outperform zero-shot LLMs across all consumer health tasks except average-heart-rate calculation.
- Prompt-tuned LLMs outperform prompt-engineered LLMs across all tasks, while reducing cardiovascular and activity-recognition output failures from approximately 50% to 0%.
8 Additional Details of Datasets
The study draws on cardiovascular, activity, and mental-health datasets spanning clinical-grade signals, wearable sensor data, and longitudinal survey instruments.
- Cardiovascular tasks use MIMIC-III and MIT-BIH datasets from the PhysioNet repository.
- MIMIC-III contains multimodal ICU biosignals from over 40,000 patients, including ECG traces and automatically derived cardiac metrics.
- The cardiovascular subset targets three IBI-based rhythm classes: Normal Sinus, Sinus Bradycardia, and Sinus Tachycardia.
- MIT-BIH provides approximately 10-hour two-lead ECG recordings from 23 patients with paroxysmal atrial fibrillation and clinician rhythm annotations.
- Interbeat intervals are extracted from adjacent ECG R-wave peaks using peak-detection algorithms from the open-source ECG-kit package.
- Activity recognition uses one-second mean accelerometer magnitudes from PAMAP2, with each prompt containing a five-second window.
- Mental-health tasks use wearable data paired with survey instruments collected during a four-week intensive longitudinal study.