PaperlayerView exact revision on arXiv

Source-linked AI summary

Affective Agent: On-Device Personalized Intervention Reasoning for Wearable Systems

Reina Mun, Zishen Wan, Vijay Janapa Reddi

arXiv:2609.12322v1cs.AIcs.LG

TL;DR

Wearable systems increasingly infer physiological states, but deciding whether, when, and how to intervene remains challenging. Affective Agent uses a three-layer on-device architecture combining physiological evidence, context, and host-managed user memory. In synthetic evaluation, memory-driven personalization and two-pass reasoning improve intervention decisions, while transfer to real-world outcomes remains for future study.

  • Problem

    Wearable state inference has advanced, but on-device systems still face the challenge of deciding whether, when, and how to intervene in a personalized way.

    Source: §1 · p. 1

  • Method

    Affective Agent combines a compact model with physiological evidence and host-managed structured memory in a three-layer on-device architecture, without per-user retraining.

    Source: §1 · pp. 2–3

  • Results

    Within the synthetic evaluation, memory improves personalized intervention routing without per-user retraining, while two-pass reasoning improves decision quality and confidence calibration in compact models.

    Source: §6 · p. 9, §7 · p. 10

  • Takeaways & Limitations

    The architecture advances wearable computing from passive state inference toward structured, personalized intervention reasoning on-device.

    Source: §7 · p. 10

    The synthetic evaluation does not establish transfer to real-world intervention outcomes, which the authors identify as a question for future field study.

    Source: §4.1 · p. 6, §4.1.2 · p. 6

Abstract

from arXiv · show

Affective computing has advanced wearable state inference, but on-device reasoning about whether, when, and how to intervene remains challenging. We present Affective Agent, a three-layer reference architecture for personalized intervention reasoning under uncertainty on wearable-class hardware. It combines a compact sub-billion-parameter language model with physiological evidence, context, and user history to decide whether, when, and how to intervene, without cloud dependency or per-user retraining. The architecture is organized into three interacting layers (perception, personalization, and reasoning), adapting to individual users through host-managed structured memory evolution rather than per-user weight updates. We instantiate Affective Agent in indoor environmental quality control and evaluate it on held-out, simulator-generated longitudinal scenarios spanning physiological variation, context, signal quality, and intervention history. Results show that memory-driven personalization and two-pass structured reasoning improve intervention decisions within this synthetic evaluation. By moving the decision layer on-device, this work demonstrates a path from wearable state inference toward closed-loop, personalized intervention on wearable-class hardware.

1. Introduction

Wearable systems increasingly infer physiological states, but deciding whether, when, and how to intervene remains challenging because signals alone do not determine an appropriate response. Affective Agent proposes on-device reasoning with structured memory and a compact model, instantiated in indoor environmental quality control.

1. Introduction

  • Physiological state inference alone cannot establish whether an intervention is timely or appropriate, since the same signal may reflect different states and contexts.A prompt during focused work may interrupt concentration, illustrating why timing and intervention form matter.Source: §1 · pp. 1–2
  • Affective Agent combines physiological evidence, context, and user history in an on-device decision layer that can intervene, wait, or abstain.Its three layers are perception, personalization, and reasoning; the design uses local operation and structured memory rather than per-user retraining.Source: §1 · p. 2, Figure 1 · p. 2
  • The architecture is instantiated in occupant-centric indoor environmental quality control, including temperature, lighting, and acoustics.These ambient interventions can operate without active engagement, while their effects may be gradual and feedback delayed or noisy.Source: §1 · p. 2
  • Host-managed structured memory and a staged preference life-cycle improve user-specific intervention routing without per-user retraining.The architecture separates perception, personalization, and intervention reasoning; its compact sub-1B model is trained with synthetic scenarios, a reasoning-complexity curriculum, and explicit two-pass reasoning.Source: §1 · p. 3
  • The authors position the architecture as a reusable framework for on-device intervention reasoning across wearable settings.They cite consumer wearables and emerging applications such as elder care and cognitive-load management.Source: §1 · p. 3

2. Related Work

Related work spans wearable intervention, personalization under resource constraints, and agentic reasoning on edge hardware. The paper frames its contribution as structured on-device intervention reasoning that combines context, physiology, and user memory.

2. Related Work

  • The related work situates Affective Agent across wearable well-being intervention, resource-constrained personalization, and edge agentic reasoning.The paper presents the intersection of these areas as structured on-device intervention reasoning.Source: §2 · p. 3, §2.1 · p. 3, §2.3 · p. 3

2.1. Wearable Intelligence

  • Wearable systems support physiological inference, but many rely on cloud-side inference or rule-based decision logic, while on-device reasoning that integrates physiology, context, and history remains underexplored.Reviews identify limited adaptivity and receptivity in just-in-time adaptive interventions,.Source: §2.1 · p. 3

2.2. Personalization Methods

  • Personalization matters because the same physiological signal can call for different responses across individuals and contexts, yet deployed systems often remain coarse-grained.Affective Agent decouples personalization from model weights through structured on-device memory and a staged preference life-cycle.Source: §2.2 · p. 3

2.3. Agentic Reasoning on Edge Hardware

  • Small edge language models have demonstrated tool routing and API execution, but these systems do not focus on memory-conditioned intervention reasoning.Affective Agent couples a compact on-device model with structured memory for context-sensitive decisions under physiological uncertainty.Source: §2.3 · p. 3

3. The Affective Agent Architecture

Affective Agent organizes on-device intervention reasoning as a closed loop: perception supplies structured evidence, personalization maintains user state, and reasoning selects a bounded action. The design combines host-managed memory with explicit two-pass reasoning.

3. The Affective Agent Architecture

  • Three interacting layers form a fully on-device loop: perception converts signals to evidence, personalization maintains user memory, and a compact model returns intervene, wait, or abstain decisions.Decision outcomes feed back into memory, and no data leaves the device.Source: §3 · p. 4

3.1. Perception Layer

  • The perception layer locally extracts features and signal-quality indicators from each five-minute window, then supplies structured evidence and classifier outputs to reasoning.Classifier outputs are auxiliary evidence, not ground truth; the reasoning layer decides whether to act using context, memory, and environmental state.Source: §3.1 · p. 4

3.2. Personalization Layer

  • Host-managed structured memory carries user-specific state between decisions, allowing the reasoning layer to interpret evidence and select actions using accumulated history.The host retrieves memory, validates proposed updates against write-governance rules, and applies approved updates; the model remains stateless across calls.Source: §3.2 · p. 4

3.2.1. Episodic and Semantic Memory

  • Episodic memory captures recent interventions and their apparent effects, while semantic memory stores slower-moving generalizations learned over time.The episodic channel helps decisions reflect immediate history; semantic patterns situate current observations in a more durable user model.Source: §3.2.1 · p. 4

3.2.2. Preference Model Life-cycle

  • Preference evidence progresses from population defaults to provisional patterns and then learned preferences, with learned status requiring at least 20 feedbacks and 5 days of activity.New conflicting evidence can revise learned patterns, allowing preference drift to be tracked over time.Source: §3.2.2 · pp. 4–5

3.3. Reasoning Layer

  • Two explicit reasoning passes organize evidence into an assessment and then verify it against history, context, preferences, and confidence before action commitment.The second pass can temper personalization and favor safe, small adjustments when preferences remain provisional; the host also enforces safety bounds and runtime constraints.Source: §3.3 · pp. 5–6

4. Affective Agent Workflow

Affective Agent uses structured supervision to compensate for the limited capacity of compact models. Its training workflow combines synthetic longitudinal scenarios, a reasoning-complexity curriculum, and programmatic two-pass supervision.

4. Affective Agent Workflow

  • Structured supervision compensates for compact models’ limited capacity through chronology-preserving scenario generation, a reasoning-complexity curriculum, and programmatic two-pass reasoning.The workflow is designed to expose models to conflicting evidence and multi-step decision logic without scaling model size.Source: §4 · p. 6, Figure 3 · p. 7

4.1. Synthetic Data Generation

  • Synthetic scenarios expose compact models to noisy signals, conflicting evidence, uncertainty, and deferral rather than reproducing a specific participant population.The evaluation tests reasoning over this problem structure; transfer to real-world intervention outcomes remains for future field study.Source: §4.1 · p. 6

4.1.1. Data Hierarchy

  • Each occupancy session preserves memory and environmental carry-over, while each scenario spans five consecutive five-minute windows in a 25-minute episode.This chronology trains reasoning under partial observation early in an episode and judgment updates as evidence accumulates.Source: §4.1.1 · p. 6

4.1.2. Synthetic Data Validation

  • Synthetic physiology targets field-plausible wearable signals with injected artifacts, while supervision uses bounded intervention targets derived from simulator state.Targets reflect signal quality, preference stage, prior outcomes, and safety constraints; source-side audits check consistency, but real-user transfer remains untested.Source: §4.1.2 · p. 6

4.2. Reasoning Complexity Curriculum

  • Six curriculum tiers vary signal clarity, preference maturity, contextual constraints, and prior outcomes to train confident action or calibrated uncertainty.Quota-managed generation preserves proportional coverage, and scenario-specific two-pass reasoning is programmatically derived from known simulator conditions and history.Source: §4.2 · p. 7

5. Evaluation

In held-out simulator-generated scenarios, learned models outperform a rule-based controller on action accuracy, while ablations show contributions from memory and verification. Edge latency and weaker confidence calibration qualify these results.

5.1. Experimental Setup

  • The evaluation uses 1,600 decision points and benchmarks inference on a Raspberry Pi Compute Module 5, with latency excluding sensing and host-side processing.Scenarios vary signal quality, context, intervention history, covariates, and preference state.Source: §5 · p. 7, §5.1 · p. 8

5.2.1. Main QoS Results

  • In the evaluation scenarios, Qwen3 and Qwen2.5 achieve action accuracies of 0.782 and 0.811, versus 0.482 for the rule-based baseline, improvements of 30.0 and 32.9 percentage points.Source: §5.2.1 · p. 8
  • Qwen2.5-0.5B leads in action accuracy and macro-F1, while Qwen3-0.6B leads in modality accuracy; both exceed the baseline on action accuracy and macro-F1.The passage characterizes this as checkpoint-specific variation, not evidence of different backbone specializations.Source: §5.2.1 · p. 8
  • The approximately 900-token two-pass completion accounts for 75.9% of mean latency for Qwen3 and 84.7% for Qwen2.5.Mean latency occupies 27.8% and 17.9% of the five-minute sensing interval, respectively; stale decisions should be revalidated before actuation.Source: §5.2.1 · p. 9
  • Both instantiations achieve 0.993 schema compliance, while agent ECE of 0.255–0.301 exceeds the rule baseline’s 0.097.The host enforces action bounds and confidence-threshold abstention rather than relying on model-reported confidence alone.Source: §5.2.1 · p. 9

5.2.2. Reasoning and Qualitative Analysis

  • The first pass organizes evidence into a provisional interpretation, and the second checks it against history, preference maturity, context, and confidence.Examined outputs show refinement of modality reasoning and confidence, but do not independently establish reasoning faithfulness.Source: §5.2.2 · p. 9

5.2.3. Ablation Results

  • Removing memory reduces macro-F1 from 0.616 to 0.501 and modality accuracy from 0.864 to 0.605, with the larger drop in routing.The results support personalization through host-managed evolving state rather than user-specific weight updates.Source: §5.2.3 · p. 9
  • Removing the verification pass degrades macro-F1 to 0.463 and worsens ECE from 0.301 to 0.377.The passage also reports loss of structured reasoning that makes outputs interpretable and auditable.Source: §5.2.3 · p. 9

6. Discussion

The discussion reports that memory and two-pass reasoning improve intervention decisions within the synthetic evaluation, and presents structured memory and supervised reasoning as a practical path for wearable systems. It also notes that the simulator omits several preference and demand conditions.

6. Discussion

  • Memory improves personalized intervention without per-user retraining, while two-pass reasoning improves reliability and confidence calibration in compact models.These findings come from the synthetic evaluation.Source: §6 · p. 9
  • Structured memory, supervised reasoning, and host-managed state are presented as ways to offset limited model capacity without scaling models or using the cloud.Source: §6 · p. 10
  • The architecture is intended for intervention reasoning beyond IEQ, but the simulator omits mood-driven rejection, situational preference shifts, and competing task and social demands.Source: §6 · p. 10

7. Conclusion

The conclusion reports that memory and two-pass reasoning improve personalized intervention decisions within the synthetic evaluation. It presents the architecture as a reference blueprint and identifies real-user evaluation and deployment improvements as future work.

7. Conclusion

  • Within the synthetic evaluation, episodic and semantic memory improve personalized intervention routing without per-user retraining, while two-pass reasoning improves decision quality and confidence calibration.Source: §7 · p. 10
  • The architecture offers a reference blueprint for personalized intervention across application domains, while its reasoning curriculum and evaluation methodology may inform future tiny-agent benchmarks.Source: §7 · p. 10
  • Future work should test the architecture longitudinally with real users, improve confidence calibration and decoding latency, measure power directly, and investigate quantization and NPU acceleration for wrist-worn deployment.Source: §7 · p. 10
Loading 2609.12322v1…