PaperlayerView exact revision on arXiv

Source-linked AI summary

PIA: A Personal Intelligence Agent Turning Health Conversations into Records and Records into Understanding

Jeonghun Yoon, Dongchan Kim, Hongyeon Yu, Young-Bum Kim, Jaegul Choo

arXiv:2609.31255v1cs.CL

TL;DR

Health-agent questions need typed records and dependable time handling, not only retrieved summaries. PIA combines four memory controls with a health-specific record layer and HU2 to turn conversations into records and records into synthesized understanding. In operation, self-reports were missing not at random, question phrasing shaped synthesis, and rules removed structural noise from candidate causal links.

  • Problem

    General-purpose memory flattens health records, leaves event time to model judgment, and cannot answer longitudinal or concept-linked questions through text similarity alone.

    Source: §1 · p. 1

  • Method

    PIA decides how to write and read each request, using a four-control memory harness and HU2 to turn conversations into typed records and records into synthesized understanding.

    Source: §1 · p. 1

  • Results

    Operational lessons were that self-reported sleep was missing not at random, question phrasing shaped synthesis quality, and rules removed 7 of 23 candidate causal pairs as structural noise.

    Source: §6 · p. 6

  • Takeaways & Limitations

    For health agents, longitudinal understanding is built from typed records and synthesized history rather than summary recall alone.

    Source: §1 · p. 1, §7 · p. 6

    Evaluation used synthetic test users, deterministic probes, and an LLM judge; the three personalization depths were illustrated on scenarios rather than measured.

    Source: Limitations · p. 7

Abstract

from arXiv · show

General-purpose agent memory summarizes conversations: it extracts salient snippets, embeds them, and retrieves the top-k into the prompt. A health agent cannot run on summaries: a dose becomes a sentence, "since last week" is resolved at the model's discretion, and a three-month glucose trend cannot be answered by text similarity. We present PIA, a personal intelligence agent deployed alongside a consumer health agent. PIA receives the agent's natural-language requests, decides for itself whether and how to write or read, and turns conversations into typed clinical records and records into a synthesized understanding of the user. Its memory harness consists of four controls -- extraction, memory, retrieval, and understanding -- each a domain-agnostic mechanism with a pluggable health module: schema, medical alias dictionary, knowledge graph, and temporal rules. We show how the same query receives a different answer as the memory injected into the response context deepens from one-dimensional recall, to a two-dimensional health snapshot, to a three-dimensional trajectory with causality, and report lessons from operation: self-reported health data are missing not at random, question phrasing governs the quality of synthesized understanding, and nearly a third of candidate causal links are structural noise that rules alone remove.

1 Introduction

General-purpose memory can serve preference recall, but health questions need typed records, reliable event times, and links between measurements and clinical concepts. PIA makes write/read decisions for a health agent and combines a four-control memory harness with HU2 and a ladder of understanding.

1 Introduction

  • For health queries, summaries flatten clinical fields, leave event dates to model judgment, and cannot answer longitudinal or concept-linked questions through text similarity alone.Examples include preserving a medication’s drug, frequency, and timing; separating event from mention time; and relating glucose tests to diabetes.Source: §1 · p. 1
  • PIA receives each health-agent request, decides whether to write or read, and returns evidence carrying its source utterance, event time, and mention time.Source: §1 · p. 1
  • The paper contributes a four-control memory harness, HU2’s provenance-bearing answers to seven questions, a ladder of understanding, and a three-dimensional view of personalization.The view traces how a query’s answer changes as recall, a health snapshot, and time enter the context.Source: §1 · p. 1, Figure 1 · p. 2

2 PIA Overview

PIA turns health conversations and documents into evidence-bearing clinical records and makes those records available to an agent as synthesized understanding. It combines a typed system of record with associative memory, while adding health-specific controls around the Hindsight engine.

2 PIA Overview

  • PIA extracts health facts, refines them into clinical-record form, and serves them as evidence the agent can cite under contracts for fabrication, grounding, validation, and inference.The Health User Profile includes static and episodic record types plus derived semantic layers; anything else becomes a typed conversation event.Source: §2 · p. 2
  • Each turn can write to a relational record store and associative memory, while a gate and router select structured retrieval, associative recall, or both.HU2 runs when records change, and its output joins the profile and recent records in each turn’s context pack.Source: §2 · p. 2

2.1 Related Work and What Is Ours

  • PIA uses Hindsight for associative memory, while its own contributions include the typed record layer, temporal resolver, retrieval boundaries, read routing, HU2, and context pack.Unlike general memory systems’ custom entity types, the paper distinguishes these health-specific pipeline mechanisms from the associative engine.Source: §2.1 · pp. 2–3

3 The Memory Harness: Four Controls

PIA’s memory harness separates four domain-agnostic controls from replaceable health-specific components. Together, they extract longitudinal records, govern admission and annotation, retrieve evidence for a judgment, and synthesize a changing user state.

3 The Memory Harness: Four Controls

  • The four-control memory harness constrains what enters memory, what is believed, what is retrieved, and what is synthesized, with health-specific parts replaceable for another domain.Source: §3 · p. 3

3.1 Extraction Control

  • Schema-constrained extraction maps outputs to one of 23 HUP types, while a deterministic resolver converts copied time expressions into event times and intervals.Records can then be assembled into metric trends and related-event episodes; event and mention times remain separate.Source: §3.1 · p. 3

3.2 Memory Control

  • The memory control admits test results only on exact medical-alias matches and annotates admitted records with knowledge-graph concepts, keeping stated facts distinct from read-time synthesis.A near-match is queued for dictionary extension rather than stored; concept tags can connect a user’s diabetes question to relevant tests.Source: §3.2 · p. 3

3.3 Retrieval Control

  • Retrieval is conditioned on the judgment: structured queries handle enumerations, values, and trends, while other requests use associative recall and rehydration from the record store.When a named drug or test is absent, PIA returns no result rather than a similarity fallback.Source: §3.3 · pp. 3–4

3.4 Understanding Control

  • Understanding control composes history into a dynamic state of the user, regenerating six facets from the full history when records change rather than listing retrieved items.The facets map onto seven HU2 fields and include state, trend, risk, goal, constraint, and preference.Source: §3.4 · p. 4

4 HU2: Holistic User Understanding

HU2 builds a synthesized view of the user from consolidated memory in the background, then makes that understanding available to the health agent. The section also distinguishes deployed capabilities from pilots and designs on a five-rung ladder.

4 HU2: Holistic User Understanding

  • HU2 builds understanding in the background by asking seven questions of consolidated memory, so the agent receives a synthesized view rather than a list of records.The output is mounted at request time; retrieval finds evidence, while memory reasoning judges from it.Source: §4 · p. 4
  • The seven questions characterize the user rather than inventory records, and only action_items asks for a recommendation.The question text is part of each field’s identifier, so changing a word re-asks every user.Source: §4 · p. 4
  • For each question, decision-conditioned recall gathers material along semantic, causal, and entity paths, then stores the synthesized answer with provenance.The question-relay approach replaced a single synthesis prompt and validator when question design proved more influential on answer quality than schema enforcement.Source: §4 · p. 4
  • HU2 regenerates when new records arrive, coalesces bursts, republishes unchanged fields, and re-asks changed fields in parallel; action_items is always re-asked.Each output row includes seven fields, data_through, and the memory identifiers used for each field.Source: §4 · p. 4

4.1 The Ladder of Understanding

  • The five-rung ladder marks what memory can know: goal lifecycle is in production, while life-state series and consensus-stratified associations are pilots and rungs 4–5 are designs.The life-state pilot reduced 183 records from one synthetic user to 13 state dimensions; rules excluded seven of 23 candidate causal pairs as structural noise.Source: §4.1 · p. 4
  • Beyond state summary, everything is future work except goal trajectories, so claims above rung 1 describe pilots or designs rather than measured deployed behavior.The ladder bounds claims about the system to the maturity of each rung.Source: §4.1 · p. 4

5 Three Dimensions of Personalization

The section describes personalization as progressively richer context for the same query: question-linked recall, a health snapshot, and then time-aware trends and causal links. The added context changes what the agent leads with, not necessarily the length of its answer.

5 Three Dimensions of Personalization

  • Figure 3 illustrates the same query at increasing depths, with illustrative answer sentences grounded in memory rather than prescribed by the figure.The figure’s dashed edges represent homogeneous transitions across time; solid edges represent heterogeneous causal transitions.Source: Figure 3 · p. 5
  • 1D injects only records the question points to, while 2D adds the user’s static profile and current state regardless of the question.The 2D snapshot supplies a caution; snippet-and-similarity memory provides the 1D level.Source: §5 · p. 5
  • 3D adds time: same-coordinate transitions form trends, while cross-coordinate causal transitions connect a past cause to a present outcome.Which coordinates are active depends on the query, and the answer changes what the agent leads with rather than becoming longer.Source: §5 · p. 6

6 Deployment, Evaluation, and Lessons

PIA runs alongside a consumer health agent, with governance and evaluation mechanisms surrounding its memory operations. Results show fail-closed retention and strong routing and annotation coverage, while operational lessons identify self-report bias, question design, and structural noise in causal candidates.

Deployment and governance

  • PIA is called once per turn; writing, retention, and HU2 regeneration run off the request path, with versioned dictionaries, tiered retention, audit, and user correction and deletion.Personally identifiable information is masked before a record is stored.Source: §6 · p. 6

Evaluation suite

  • Evaluation replays 271 scenarios across 20 synthetic users, using frozen-ground-truth probes and a four-axis pass/fail judge, with repeated independent runs.Three health-domain annotators reviewed ground truth item by item; 381 of 414 rows survived, and run spread was at most 3.4 points for probes and 7.4 for the judge.Source: §6 · p. 6

Results

  • RETAIN achieved 100% precision and 94.9% accuracy; READ recalled 91.0% of turns needing memory, and the router selected the intended tool in 91.0% of 335 decisions.The fail-closed default accounts for misses, with recall of 89.1%.Source: §6 · p. 6
  • Over queries returning rows, MRR was 0.785, recall@k was 0.763, and precision@k was 0.758; the storage judge passed 88.3% of 54 storage scenarios.The storage verdict had a spread of 7.4 over three runs.Source: §6 · p. 6
  • Across 16,122 deployed-store test-result rows, 99.7% carried a standard key and 99.2% a concept tag, but 66.6% had a reference range and 64.4% an abnormality flag.The passage describes these as aggregate row counts, not per-user measures; the dictionary had 144 range rows for 319 tests.Source: §6 · p. 6

Lessons learned

  • In the pilot, 8 of 9 reported sleep nights were bad, motivating report-coverage tracking and treating unreported days as missing; question design shaped synthesis quality more than schema enforcement.Inventory questions yielded lists and characterizing questions prose; the team retired HU2’s structured synthesis prompt and validator.Source: §6 · p. 6
  • Rules alone excluded 7 of 23 candidate causal pairs as self-loops, zero-variance series, or out-of-vocabulary labels, before statistical testing.The authors report that filtering mattered more than finding at rung three.Source: §6 · p. 6

7 Conclusion

PIA turns conversations into typed health records through a four-control memory harness, then uses HU2 to synthesize an understanding of the user. The paper frames progress beyond the first rung of understanding as future work.

Conclusion

  • PIA turns conversations into typed records through four controls: extraction, memory, retrieval, and understanding.The health-specific parts are plug-ins to the harness’s domain-agnostic mechanisms.Source: §7 · p. 6
  • HU2 turns records into an understanding of the user.The conclusion identifies this as the system’s understanding component.Source: §7 · p. 6
  • Climbing the understanding ladder above rung one remains work ahead.The paper’s conclusion presents higher rungs as future work.Source: §7 · p. 6

Limitations

The paper’s higher understanding rungs are mostly designs or pilots, while test-result recall depends on a curated exact-match dictionary. Evaluation uses synthetic users and scenario illustrations rather than reported real-user longitudinal measures.

Maturity of understanding

  • Beyond the first rung, only goal trajectories are in production; life-state series and consensus-stratified associations are pilots, while correction from follow-through and counterfactual simulation remain designs.Claims about these higher rungs are design claims, not measured behavior.Source: Limitations · p. 7

Dictionary coverage

  • Test-result admission requires an exact alias match, so record-layer recall is bounded by curated dictionary coverage; medication handling is still in progress.Only two thirds of stored test results currently receive a reference range.Source: Limitations · p. 7

Evaluation

  • Reported evaluation uses synthetic test users, deterministic probes, and an LLM judge; the three depths of understanding are illustrated on scenarios, not measured.Real-user longitudinal evaluation exists internally but cannot be reported because it involves personal health information; open-source memory comparisons are qualitative.Source: Limitations · p. 7

Self-report bias

  • Conversational health data are missing not at random: users reported sleep when it was bad, so averaging reports describes bad days unless corrected.The pilot had 8 bad reports among 9 reported nights, and this bias propagates from the second rung to the third.Source: Limitations · p. 7

Ethical Considerations

The paper reports synthetic users, records, and scenarios, and limits the memory layer’s role in clinical judgment. Its harness separates general mechanisms from replaceable health-specific components.

Ethical Considerations

  • All users, records, and scenarios are synthetic; deployed-store figures are aggregate annotation-coverage rates without user-level information.PII is masked before storage, and users can correct or delete records under the stated consent and privacy terms.Source: Ethical Considerations · p. 7
  • PIA does not diagnose, adjudicate causality, or recommend treatment; recommendations are confined to one HU2 field, with clinical judgment left to medical knowledge sources and clinicians.The system keeps an audit ledger and versioned dictionaries so stored annotations can be traced to their knowledge source.Source: Ethical Considerations · p. 7

A Health Plug-ins of the Memory Harness

  • Table 3 separates domain-agnostic mechanisms from health-specific parts, which can be replaced to instantiate the harness in another domain; this paper reports only its health instantiation.Source: A Health Plug-ins of the Memory Harness · p. 8

B HUP Schema

The section specifies PIA’s typed health-record schema, HU2 questions, deployed model and prompt contracts, and synthetic personalization scenarios.

B HUP Schema

  • HUP records include representative static, episodic, and semantic fields; planned record types instead remain typed conversation events in the deployed version.Every record carries provenance, including source quotation, source type, event time, and mention time.Source: B HUP Schema · p. 8

C The HU2 Question Contract

D Models

  • All deployed components except the evaluation judge use open-weight models served in-house, and none is fine-tuned.The paper characterizes the harness as the pipeline around the models, rather than the models themselves.Source: D Models · p. 8

E Deployed Prompts (Excerpts)

  • Read prompts route canonical health lookups away from associative recall, while response prompts require provided facts and prohibit added diagnoses, advice, dates, or causal claims.The evaluation judge separately checks storage, retrieval, answer quality, and derived-layer capability.Source: E Deployed Prompts (Excerpts) · pp. 8–9
  • The normalizer copies exact evidence quotes and temporal expressions; it distinguishes one-time medication intake from ongoing prescription changes and rejects assistant-only facts.A deterministic resolver computes sleep duration rather than having the normalizer set it.Source: E Deployed Prompts (Excerpts) · pp. 8–9

F Personalization Scenarios

  • Across synthetic scenarios, the same user snapshot supplies different active coordinates for different queries, and memory depth changes the illustrative answers.The scenarios are synthetic, and the displayed depths are cumulative.Source: F Personalization Scenarios · p. 9

G Temporal Resolver and Dictionary Examples

The section describes verbatim temporal capture followed by deterministic resolution, dictionary-based reference ranges and aliases, and scenario examples in which deeper memory changes advice.

G Temporal Resolver and Dictionary Examples

  • The extractor copies a temporal expression into the record for downstream resolution; without one, event time remains empty under the no-fabrication contract.Source: G Temporal Resolver and Dictionary… · pp. 9–10
  • The deployed dictionary binds standard keys to aliases, reference ranges, and clinical concepts; 11 of 319 tests have sex-specific ranges, and none are age-banded.122 of 144 range rows apply to all users; one alias row is a unit-string data-quality error that the exact-match gate does not admit as a test name.Source: G Temporal Resolver and Dictionary… · p. 10
  • The three-dimensional exercise context changes the answer: postpone the 10 km, suggest the liked brisk walk, and rebuild from 4 km once the knee settles.The context includes a still-painful knee, preceding-week sleep logs at 4 hours, and the user’s positive report about the brisk walk.Source: G Temporal Resolver and Dictionary… · p. 11
  • For the late-night snack scenario, lipid-trend context changes the decision to skip tonight while retaining allergy filtering.The derived trend records triglycerides 135 →232 mg/dL and HDL 55 →31 mg/dL, both flagged abnormal against reference ranges.Source: G Temporal Resolver and Dictionary… · p. 12
Loading 2609.31255v1…