PaperlayerView exact revision on arXiv

Source-linked AI summary

Explainable Prediction from Mobile Sensing Data through LLM-guided Concept Integration

Yuning Wang, Iman Azimi, Amir M. Rahmani, Pasi Liljeberg

arXiv:2609.11995v1cs.LG

TL;DR

Sparse outcome supervision and the need for meaningful explanations challenge prediction from mobile sensing in small cohorts. CIT uses LLM-derived, baseline-aware concept supervision to train a Transformer; it performs strongest on AFFECT and ties for top PHQ-9 class 1 F1, while learned scores expose interpretable patterns.

  • Problem

    Mobile-sensing prediction in small cohorts faces sparse outcome labels, while concept-based interpretation is constrained by the lack of affordable, scalable labels for individual sensing windows.

    Source: §1 · p. 1

  • Method

    CIT uses an LLM to turn baseline-aware statistical summaries into concept abnormality targets with confidence weights, then trains a Transformer to predict outcomes and concepts without manual concept annotations.

    Source: §1 · p. 2

  • Results

    CIT achieved the highest recall, F1-score, and AUC on AFFECT; on PHQ-9, it tied the OCSVM-based variant for highest class 1 F1-score, while that variant had slightly higher AUC.

    Source: §5 · p. 11, §6 · p. 13

  • Takeaways & Limitations

    Learned concept scores provide behavioral and physiological interpretations, with sleep quantity and quality showing the clearest difference between high and low negative affect groups in AFFECT.

    Source: §6 · p. 13

    CIT relies on a predefined five-concept taxonomy that is one possible set rather than universally optimal; other outcomes or sensing modalities may call for alternatives.

    Source: §5 · p. 12

Abstract

from arXiv · show

Mobile sensing enables longitudinal monitoring of behavioral and physiological patterns in everyday settings. However, accurate prediction remains challenging in small-cohort health-sensing studies, where task-specific outcome supervision is limited relative to heterogeneous sensing data. Interpretability is also important, as model outputs should reflect meaningful behavioral and physiological patterns rather than predictive scores alone. We develop a Concept-Integrated Transformer (CIT) with LLM-guided concept supervision for explainable prediction from mobile sensing data. CIT uses a pretrained large language model to generate baseline-aware concept abnormality targets with confidence weights without manual concept annotation. Across two longitudinal datasets, CIT achieves the highest F1 score on AFFECT (0.756) and ties for the highest on a PHQ-9 dataset (0.765). The learned concept scores also reveal interpretable behavioral and physiological patterns; in AFFECT, sleep quantity and quality show the clearest difference between high and low negative affect groups. These findings support LLM-guided concept integration for accurate and interpretable prediction in small-cohort mobile sensing studies.

1 Introduction

Mobile sensing offers longitudinal behavioral and physiological data, but prediction in small cohorts is constrained by sparse outcome labels, while translating sensing patterns into meaningful explanations remains difficult. The paper introduces CIT, which uses LLM-guided concept supervision without manual concept annotation and evaluates it on affect and depression prediction.

1 Introduction

  • Sparse outcome labels make accurate prediction challenging in real-world small-cohort mobile-sensing studies.Repeated self-reports or clinical assessments may be costly, burdensome, or impractical to collect.Source: §1 · p. 1
  • Feature-attribution explanations can be difficult to translate into participant-specific summaries, while concept models need labels that are costly to annotate.The meaning of longitudinal sensing features can depend on a participant’s baseline, temporal trend, and co-occurring signals.Source: §1 · p. 1
  • CIT uses baseline-aware summaries and an LLM to generate concept abnormality targets with confidence weights, avoiding manual concept annotations.The concept targets provide auxiliary supervision and an interpretable intermediate representation; the LLM is not required at inference.Source: §1 · p. 2
  • The study evaluates CIT on two longitudinal multimodal datasets: AFFECT for affect prediction and a PHQ-9 dataset for depression-related prediction.Source: §1 · p. 2
  • Learned concept scores are intended to support class-level, population-level, and individual-level interpretation of model behavior.Source: §1 · p. 2

2 Methodology

CIT uses an LLM to derive concept supervision from baseline-aware summaries, then trains a Transformer to predict health outcomes and concept scores from sensing windows. At test time, the Transformer provides both outputs without querying the LLM.

2 Methodology

  • CIT connects LLM-guided concept abnormality supervision with a concept-supervised Transformer that predicts outcomes and concept scores from longitudinal sensing windows.Source: §2 · p. 2

2.1 Module A: LLM-guided Concept Abnormality Supervision

  • Module A converts each sensing window and participant baseline into LLM-derived abnormality targets and confidence weights used to supervise the concept head during training.These outputs are supervision signals, not final predictions, and are not required at test time.Source: §2 · p. 2, §2.1 · pp. 2–3

2.1.1 Baseline-aware Summary Table Construction

  • Each window is summarized using recent statistics, a preceding 28-day participant baseline, and deviations describing how the current pattern differs from that baseline.Source: §2.1.1 · pp. 3–4

2.1.2 Generating Concept Targets and Confidence Weights

  • The LLM scores five predefined behavioral and physiological concepts using evidence-constrained summaries; repeated-run score and evidence consistency inform confidence weights for available targets.The five concepts cover sleep quantity and quality, daytime activity volume, activity regularity and adherence, cardiac regulation and stress, and context and environment.Source: §2.1.2 · pp. 4–5

2.2 Module B: Concept-supervised Transformer Prediction

  • Module B receives the original sensing window and learns outcome and concept predictions, using ground-truth outcome labels and LLM-derived concept targets during training.Source: §2.2 · p. 5

2.2.1 Architecture

  • An encoder-only Transformer aggregates daily feature vectors into a window representation, which feeds separate classification and concept-prediction heads.Source: §2.2.1 · p. 5

2.2.2 Joint Training Objective

  • Training combines outcome classification with confidence-weighted masked concept MSE, excluding unavailable targets and using λ to control the concept loss’s relative contribution.Source: §2.2.2 · p. 6

2.2.3 Test-time Prediction and Concept Explanations

  • At test time, the trained Transformer outputs the health prediction and five concept abnormality scores without an LLM query; those scores provide structured concept-level interpretations.Source: §2.2.3 · p. 6

3 Experimental Setup

The experiments evaluate CIT on two longitudinal multimodal sensing datasets and compare it with Transformer and OCSVM-based baselines. Participant-wise chronological splits and shared training settings structure the evaluations.

3.1 Datasets

  • The study evaluates negative-affect prediction on AFFECT and weekly depressive-symptom prediction on PHQ-9.PHQ-9 adds a different health-related target and sensing feature space.Source: §3.1 · p. 6

3.1.1 AFFECT dataset

  • AFFECT includes 21 undergraduate students monitored longitudinally in 2020, with wearable, smartphone, and daily affect data.Negative affect was binarized relative to participant-wise medians, excluding the central 20% of values.Source: §3.1.1 · pp. 6–7

3.1.2 PHQ-9 depression dataset

  • PHQ-9 uses weekly assessments and multimodal sensing from undergraduate participants aged 18–22; scores above 4 define the follow-up-needed class.Each weekly assessment is paired with daily features from the preceding seven days.Source: §3.1.2 · p. 7

3.2 Data Preparation

  • AFFECT uses seven-day windows of 43 features and a preceding 28-day participant baseline, with chronological 60/20/20 train-validation-test splits.The final sample counts are 930 training, 391 validation, and 395 test windows.Source: §3.2 · p. 7

3.3 Baseline Methods

  • Comparisons include a classification-only Transformer, a dual-head Transformer with concept loss disabled, and CIT using OCSVM-derived concept targets.The baselines distinguish the effect of concept supervision from adding a concept branch or using an unsupervised statistical alternative.Source: §3.3 · pp. 7–8

3.4 Model Training

  • Transformer-based models share an encoder and input windows, use Adam, and train with binary classification loss; CIT also uses confidence-weighted masked concept supervision.AFFECT selects the checkpoint with best validation F1; PHQ-9 uses participant-wise chronological forward validation for epoch selection.Source: §3.4 · p. 8

4 Results

CIT achieved the highest reported AFFECT F1 score and matched the OCSVM variant’s PHQ-9 F1 score. On AFFECT, concept scores also relate to predicted risk at the population level, while the supplied results describe class-level and individual-level explanation analyses.

4.2 Performance on the PHQ-9 Depression Dataset

  • On PHQ-9, CIT’s F1 (0.765) and recall (0.839) exceeded the Transformer’s (0.758 and 0.807), while its precision was lower (0.703 vs. 0.714).Against the Transformer without concept loss, CIT also had higher F1 (0.765 vs. 0.722).Source: §4.2 · pp. 8–9

4.1 Performance on the AFFECT Dataset; 4.2 Performance on the PHQ-9 Depression Dataset

  • CIT achieved the highest AFFECT F1 (0.756); on PHQ-9, its F1 (0.765) matched OCSVM, whose AUC was higher (0.847 vs. 0.831).The supplied results report class-1 test performance; Figure 2 shows ROC curves with confidence intervals.Source: Table 1 · p. 8, §4.1 · p. 8, §4.2 · pp. 8–9

4.3 Concept-level Explainability

  • Figure 3 compares learned concept-abnormality distributions by ground-truth class, displaying class means, class-1-minus-class-0 differences, and Cohen’s d.The analysis examines whether concept scores separate windows with high negative affect from those without.Source: §4.3 · p. 9, Figure 3 · p. 10
  • On AFFECT, sleep quantity and quality show the clearest positive relationship with predicted class-1 probability, while activity regularity and adherence show a negative relationship.Figure 4 pairs mean concept trajectories over relative time with smoothed concept-score–probability relationships.Source: §4.3 · pp. 9–10, Figure 4 · p. 10
  • The individual-level analysis selects a correctly classified positive AFFECT window and presents its model output alongside five concept-abnormality scores.Higher scores indicate stronger predicted abnormality in the corresponding behavioral or physiological concept.Source: §4.3 · p. 11, Figure 5 · p. 11

5 Discussion

CIT’s gains differ across AFFECT and PHQ-9, while concept analyses highlight sleep as the clearest AFFECT signal and show that other concepts can have more conditional relationships with prediction.

5 Discussion

  • CIT led recall, F1-score, and AUC for high negative affect on AFFECT, while PHQ-9 results were more modest and less uniform.On PHQ-9, CIT slightly improved class 1 F1-score and recall over the Transformer baseline and the model without concept loss, but the OCSVM variant had slightly higher AUC.Source: §5 · p. 11
  • Sleep quantity and quality showed the most consistent positive AFFECT pattern, with higher abnormality in class 1 windows and higher predicted class 1 probability.The authors identify sleep quantity and quality as the concept most strongly aligned with high negative affect in this dataset.Source: §5 · p. 12
  • Activity regularity and adherence showed higher abnormality in non-high-negative-affect windows, so higher abnormality in this concept is not uniformly adverse.Its meaning depends on the behavioral content captured by the corresponding feature group.Source: §5 · p. 12
  • Daytime activity volume, cardiac regulation and stress, and context and environment had weaker or less monotonic relationships with predicted probability, which may reflect interaction-dependent contributions.The authors caution that weaker dependence patterns do not establish that these concepts are irrelevant.Source: §5 · p. 12
  • CIT’s learned concept space supports both aggregate analyses and explanations of individual predictions, with concept abnormality scores produced directly by the model’s concept head.Unlike post-hoc local surrogate methods, the explanation space is learned through concept supervision.Source: §5 · p. 12
  • The study uses one possible predefined concept taxonomy, and generalizability remains to be evaluated across more diverse cohorts, devices, protocols, and outcomes.Future work should also validate learned concept scores against domain knowledge or expert assessment.Source: §5 · p. 12

6 Conclusions

CIT combines annotation-free LLM guidance with Transformer prediction and produces concept-level explanations. Across AFFECT and PHQ-9, it showed competitive predictive performance, with sleep quantity and quality the clearest AFFECT population-level signal.

6 Conclusions

  • CIT transforms baseline-aware statistical summaries into LLM-informed concept abnormality supervision within a Transformer prediction framework.The framework is presented for explainable prediction from mobile health-sensing data without manual concept annotations.Source: §6 · pp. 12–13
  • On AFFECT, CIT achieved the highest recall, F1-score, and AUC for detecting high negative affect; on PHQ-9, it tied the OCSVM-based variant for highest class 1 F1-score.The OCSVM-based variant achieved a slightly higher AUC on PHQ-9.Source: §6 · p. 13
  • In AFFECT, sleep quantity and quality emerged as the clearest population-level signal among learned concept abnormality scores.The analyses also found distinct relationships between learned concept scores and model predictions.Source: §6 · p. 13

Ethics declaration

The study analyzed publicly available, de-identified datasets and did not recruit participants or collect identifiable personal data.

Ethics declaration

  • The study used publicly available, de-identified datasets without new participant recruitment or identifiable personal data collection.Source: Ethics declaration · p. 15
  • The source-study investigators obtained ethical approval and informed consent for the original data collection.Source: Ethics declaration · p. 15
  • No additional ethical approval was required for this study’s secondary analyses.Source: Ethics declaration · p. 15

Confusion Matrix Visualizations

Supplementary figures show test-set confusion matrices for AFFECT and PHQ-9 across four model variants.

Confusion Matrix Visualizations

Supplementary Concept-level Visualizations for the PHQ-9 Experiment

The supplementary PHQ-9 visualizations show predicted concept abnormality scores by binary label and relate those scores to predicted class-1 probability. The trained CIT concept head produces these scores without LLM calls at test time.

  • The PHQ-9 test-set concept scores are inferred by the trained CIT concept head without LLM calls at test time.Source: Supplementary Concept-level… · p. 16
  • Figure S3 reports predicted concept abnormality scores on the held-out PHQ-9 test set, stratified by binary PHQ-9-derived label; class 0 denotes None-minimal and class 1 denotes follow-up needed.Source: Figure S3 · p. 17
  • Figure S4 relates each predicted concept abnormality score to the model-predicted probability of class 1, with smoothed curves averaging probability across score bins.Source: Supplementary Concept-level… · p. 16, Figure S4 · p. 17

Feature-to-Concept Mapping

The supplementary tables document how sensing features are grouped into CIT’s five predefined concepts for both datasets. The mappings include physiological and smartphone-derived context features.

Original Prompt Used for LLM-Guided Concept Abnormality Supervision

The supplementary prompts guide an LLM to score concept abnormality from participant-specific sensing summaries rather than outcome labels. They define five concepts and specify evidence and confidence rules for both datasets.

Original Prompt Used for LLM-Guided Concept Abnormality Supervision

  • For AFFECT, the prompt uses automatically generated summaries for each 7-day window and its corresponding 28-day participant baseline, without supplying the ground-truth negative-affect label.Source: Original Prompt Used for LLM-Guided… · p. 22

Original LLM Prompt for the AFFECT Dataset

Original LLM Prompt for the Depression Dataset

Loading 2609.11995v1…