PaperlayerView exact revision on arXiv

Source-linked AI summary

Proactive Memory for Event-Driven LLM Agents

Jinyang Li, Zichuan Fu, Xian Wu, Wenlin Zhang, Yimin Deng, Kaifeng Guo, Guojing Li, Junjia QI, Xiangyu Zhao

arXiv:2610.36967v1cs.AI

TL;DR

Proactive agents need reliable historical evidence to decide when to offer assistance as events unfold. ProactMem combines recent events, source-validated trajectories, and replay-assessed regularities, and ProactiveStream evaluates agents on isolated chronological streams. Across three scopes, ProactMem recovers more opportunities with fewer false interventions than recent-event memory alone.

  • Problem

    Existing approaches offer limited support for validating recurring behavioral regularities and assessing whether they apply to the current event.

    Source: §1 · p. 1

  • Method

    ProactMem combines recent events, source-validated temporal trajectories, and replay-assessed behavioral regularities, matching historical evidence to current conditions.

    Source: §1 · p. 2

  • Results

    Across three scopes, validated hierarchical memory recovers more service opportunities with fewer false interventions than recent-event memory alone.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support maintaining historical evidence at multiple temporal scales for event-driven proactive assistance.

    Source: §7 · p. 8

    The evaluation covers low-risk Sports, Home, and Code settings; recipient benefit, interruption burden, end-to-end cost, and deployment safety require prospective measurement.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Proactive LLM agents anticipate service opportunities from ongoing observations and accumulated history. Existing approaches emphasize recent context or retrieval of related memories, providing limited support for validating recurring behavioral regularities and assessing their applicability to the current event. We propose Proactive Memory (ProactMem), which organizes history into recent events, temporal trajectories, and behavioral regularities. Trajectory facts are checked against source events, and regularities are assessed through historical replay before being matched to current conditions. We also construct ProactiveStream, a longitudinal benchmark covering three application scopes, where agents process isolated event streams chronologically under episode-aware opportunity labels. On ProactiveStream, ProactMem achieves an overall effectiveness (F1) of 52.6%, a 21.4% relative improvement over the strongest of five external baselines. Code is available at: https://anonymous.4open.science/r/ProactMem-5351/.

1 Introduction

Proactive assistance requires agents to use accumulated history as evidence for when to offer a service. ProactMem combines three memory layers, while ProactiveStream evaluates chronological assistance across Sports, Home, and Code.

1 Introduction

  • Prior work addresses service decisions and memory construction, but continuous assistance raises the question of how accumulated history can provide reliable evidence for when to assist.The cited work includes task proposals, contextual service, memory retrieval, and long-term conversational memory.Source: §1 · p. 1
  • ProactiveStream covers Sports, Home, and Code with isolated chronological streams and episode-aware opportunity labels.The benchmark assesses each opportunity’s first actionable window and subsequent changes.Source: §1 · pp. 1–2
  • ProactMem links recent observations, source-grounded temporal activities, and supported conditional behavior in three memory layers.Source: §1 · pp. 1–2
  • Trajectory facts are source-validated, regularities are assessed through historical replay, and applicable evidence is matched to current conditions when judging service value.Source: §1 · p. 2
  • Validated hierarchical memory recovers more service opportunities with fewer false interventions than recent-event memory alone across three scopes.Source: §1 · p. 2

2 Problem formulation

The formulation models proactive assistance as sequential decisions over isolated entity-specific event streams. Each decision uses prior observations and finalized outputs, then commits the event and output to the stream state.

2 Problem formulation

  • Each participant, household, or repository has an isolated chronological public event stream, with only fields available at the event time as input.Source: §2 · p. 2
  • At each event, the policy uses prior observations, finalized outputs, memory, and intervention-control state to produce an intervene-or-abstain decision and optional evidence-grounded message.Source: §2 · p. 2
  • The objective is to detect timely, actionable service opportunities supported by current or prior observations of the same entity.Source: §2 · p. 2

3 Proactive Memory

ProactMem organizes history into recent events, source-checked temporal trajectories, and replay-assessed behavioral regularities. At decision time, it matches retrieved historical evidence with current conditions and controls interventions.

3 Proactive Memory

  • Three complementary layers store recent public events, source-checked temporal trajectories, and replay-assessed conditional behavioral regularities.Construction checks source fidelity and historical support before the decision stage combines current observations with applicable historical evidence.Source: §3 · p. 2

3.1 Memory construction and update

  • Daily trajectory segments contain intervals, typed facts, and source citations, and accepted facts must be supported by their cited events.The validator checks entity, interval, citations, and canonicalized scalar values; source links let the factual basis be inspected and constrain regularity proposals.Source: §3.1 · pp. 2–3
  • Behavioral regularities are replayed against validated daily facts, with missing observations censored, inapplicable days excluded, and each date counted at most once.Confidence measures consistency across observable triggered days, while lift accounts for how common the behavior is under comparable temporal restrictions; activation also requires repeated support across distinct days.Source: §3.1 · p. 3

3.2 Memory retrieval and selection

  • Retrieved regularities are selected by occurrence state, lifecycle, confidence, and temporal distance, excluding fulfilled or insufficiently observed occurrences.A regularity can contribute evidence after its source events leave recent memory when a later event matches its trigger near the usual time.Source: §3.2 · p. 4

3.3 Proactive decision

  • The decision context combines the current event, recent events, trajectories, selected regularities, and observable episode state before intervention control.The controller checks proposed services against previously allowed interventions and converts duplicates to abstentions; finalized decisions precede event commitment and memory updates.Source: §3.3 · p. 4

4 ProactiveStream benchmark

ProactiveStream evaluates service-opportunity detection as agents accumulate history in isolated chronological event streams across Sports, Home, and Code. Its reference protocol labels actionable opportunity windows and replays each stream from empty method state before scoring.

4 ProactiveStream benchmark

  • ProactiveStream evaluates service-opportunity detection in longitudinal streams as agents accumulate history.Its composition and provenance are summarized in Table 1.Source: §4 · p. 4, Table 1 · p. 5

4.1 Scopes and public events

  • Sports, Home, and Code use participant activity, time-bounded home-sensor observations, and repository events, respectively, as scope-specific public inputs.Each participant, home, or repository defines an isolated chronological stream, and agents build memory from contemporaneously available observations.Source: §4.1 · p. 4

4.2 Reference labels and causal replay

  • Reference labels mark the first reliable actionable window for intervention and covered windows for abstention, while allowing later reassessment when an opportunity changes or reopens.An opportunity can be supported by the current event or historical observations, and both require a concrete service opportunity at the current time.Source: §4.2 · p. 5
  • Every event is labeled by annotators assessing its complete prior stream history, with LLM-assisted triage followed by independent labeling, cross-review, and adjudication.Streams are replayed chronologically from empty method state; decisions are finalized before the current event enters memory, and labels are joined only after inference for scoring.Source: §4.2 · p. 5

5 Experimental evaluation

The evaluation compares ProactMem with external methods across Sports, Home, and Code, then examines model choice, memory organization, and validation. ProactMem achieves higher effectiveness and balance than the external baselines across all three scopes, with memory ablations showing distinct effects on opportunity recovery and false interventions.

5 Experimental evaluation

  • The evaluation examines overall effectiveness, model choice, and memory organization and validation, using five external baselines and shared intervention control.Reported measures include relevance, coverage, effectiveness, intrusion, balance, and intervention frequency.Source: §5 · p. 5, §5.1 · pp. 5–6

5.3 Model comparison (RQ2)

  • In event-only comparisons, Luna recovers more Home and Code opportunities than DeepSeek and GLM, while those backbones achieve higher specificity.On Sports, Luna has both higher coverage and higher specificity than DeepSeek.Source: Figure 3 · p. 6, §5.3 · p. 7

5.2 Main results (RQ1)

  • ProactMem achieves higher effectiveness and balance than external baselines across Sports, Home, and Code.Operating trade-offs differ by scope: it is more selective on Sports and recovers more opportunities on Home and Code.Source: Table 2 · p. 6, §5.2 · p. 7
  • Rescoring fixed diagnostic outputs with separate-team references yields similar effectiveness on Sports but lower effectiveness on Home and Code.This indicates that absolute performance depends on how actionable opportunities are labeled.Source: §5.2 · p. 7

5.3 Model comparison (RQ2)

  • Effectiveness increases and intrusion decreases from GPT-5.6 Luna through GPT-5.6 Terra to GPT-5.6 Sol across scopes.Construction and decision LLMs change together in this scaling comparison.Source: Table 3 · p. 6, §5.3 · p. 7

5.4 Ablation and memory analysis (RQ3)

  • Adding trajectory memory raises effectiveness in every scope, with the largest improvement on Code; intrusion is unchanged on Sports and Home and rises slightly on Code.The gain at this stage lies mainly in opportunity recovery.Source: Figure 4 · p. 7, §5.4 · p. 8
  • The full hierarchy further improves effectiveness and lowers intrusion across all three scopes.It retains recurring behavior with triggers and time windows, so earlier observations can inform assessment when corresponding conditions recur.Source: §5.4 · p. 8
  • Removing either trajectory validation or replay reduces relevance and effectiveness, with the larger effectiveness decline when replay is removed.On Code, trajectory validation is associated with fewer false interventions at unchanged coverage; Code suppression removes more false positives than true positives, improving relevance and effectiveness while reducing coverage.Source: Table 4 · p. 7, §5.4 · p. 8

6 Related work

Related work covers agent memory and retrieval, proactive assistance, and evaluation of long-term memory and service opportunities. ProactMem combines source checks for trajectory facts with replay tests of regularities before matching them to current conditions.

Agent memory

  • Agent memory systems organize experience through reflection, updates, or tiered context, while retrieval systems use extracted memories, linked records, or graphs.Graphiti also tracks when event-derived relations are valid and where they originate.Source: §6 · p. 8

Proactive agents and memory

  • Proactive agents study task proposals, contextual service, streaming demand, dialogue reasoning, and memory supplied for long-horizon action; ProactMem checks trajectories against source events and tests regularities through replay.Replay precedes matching regularities to current conditions.Source: §6 · p. 8

Evaluation of proactive service

  • Existing benchmarks assess conversational memory, task proposals, latent-need detection, or memory triggering, whereas ProactiveStream follows successive intervene-or-abstain decisions as memory accumulates.It uses isolated, causally replayed streams and episode-aware references for intervention timing, evaluating both opportunity recovery and intrusion.Source: §6 · p. 8

7 Conclusion

The paper introduces ProactiveStream for evaluating proactive assistance as event-stream history accumulates and ProactMem for organizing historical evidence at multiple temporal scales. Across Sports, Home, and Code, ProactMem achieves higher effectiveness and balance than the evaluated baselines.

7 Conclusion

  • ProactiveStream evaluates proactive assistance over event streams as history accumulates.The benchmark covers Sports, Home, and Code.Source: §7 · p. 8
  • ProactMem combines recent events, source-checked temporal trajectories, and replay-assessed behavioral regularities to bring earlier observations into current decisions.The paper concludes that results support maintaining historical evidence at multiple temporal scales for event-driven proactive assistance.Source: §7 · p. 8
  • Across Sports, Home, and Code, ProactMem achieves higher effectiveness and balance than evaluated baselines; ablations show improved opportunity recovery with trajectories and fewer false interventions with the full hierarchy.Source: §7 · p. 8

Limitations

The evaluation is bounded by the benchmark settings and offline protocol. Broader generalization and deployment consequences require further evidence.

Limitations

  • ProactiveStream covers low-risk service settings in Sports, Home, and Code, under a protocol fixed after joint method and benchmark development.Source: Limitations · p. 9
  • Independent held-out evaluation and cross-family transfer of constructed memory remain future work; broader generalization depends on additional populations, observation conditions, and model families.Source: Limitations · p. 9
  • Offline metrics evaluate opportunity detection, while recipient benefit, interruption burden, end-to-end cost, and deployment safety require prospective measurements.Source: Limitations · p. 9

Ethical considerations

The evaluation is confined to optional, low-risk assistance, with privacy protections and deployment safeguards identified as requirements. The accompanying sections describe stream construction, labeling, memory validation, and intervention control.

Ethical considerations

  • Evaluation covers optional activity support, gentle check-ins, and low-risk repository assistance; higher-risk decisions require separate domain-specific validation and safeguards.Longitudinal events and derived memories can expose sensitive routines, while deployment safeguards include opt-in, user controls, and separate authorization for consequential actions.Source: Ethical considerations · p. 9

A.1 Public events and availability

  • ProactiveStream draws on HeartSteps activity events, CASAS ambient-sensor traces, and public repository activity, with each scope defining its available fields and service boundaries.The Home stream uses closed intervals; the Code stream preserves native event granularity and isolates repository state.Source: §A.1 · p. 11

A.2 Reference labels and causal replay

  • Reference labels are assigned from each event and its complete same-stream history, then streams are replayed chronologically with events committed only after decisions are finalized.Labels and scoring metadata are held out from model inputs; multiple blinded annotators review and adjudicate events.Source: §A.2 · p. 11, §A.1 · p. 11

A.3 Example events and reference labels

  • Figure 5 pairs public event records from Sports, Home, and Code with separate evaluation labels, where yt = 1 means intervene and yt = 0 means abstain.The labels are excluded from agent inputs.Source: §A.3 · p. 11, Figure 5 · p. 12

B.1 Trajectory evidence and replay

  • Daily trajectories retain typed facts and source-event citations; validation checks supported values, field paths, entities, intervals, and resolved citations.Replay treats triggered observable days as fulfilled or violated within a window, censors missing observations, and counts each date at most once.Source: §B.1 · p. 11

B.2 Lifecycle and current matching

  • Regularities move through active, weakening, dormant, and candidate states according to violations and fulfillment; retrieval prioritizes deviated, pending, then upcoming occurrences.Two consecutive violations weaken an active regularity; four make it dormant.Source: §B.2 · p. 13

B.3 Intervention control

  • Intervention control suppresses repeated proposals using scope-specific episode or intervention signatures, while changed states, evidence, or service details can pass the check.Sports and Home keys include episode state, proposed action and target, and available regularity identifiers; Code checks repository and object identity plus prior intervention details.Source: §B.3 · p. 13

B.4 Model and method interfaces

The appendix specifies shared model and scoring interfaces, then describes a chronological, event-only cross-family comparison using common prompts and intervention control.

B.4 Model and method interfaces

  • External methods retain their prompts, retrieval, and budgets, while the shared evaluation reports relevance, coverage, effectiveness, intrusion, and balance from final decisions.Decision limits vary by interface, and immediate or scheduled services count as interventions at the decision event.Source: §B.4 · p. 13
  • Graphiti supplies relation facts directly, with ingestion episodes corresponding to source records.Source: §B.4 · p. 13

B.5 Cross-family event-only configuration

  • Each stream is replayed chronologically from empty state; models share a scope-specific task prompt, intervention control policy, and causal context of the current event plus up to 99 same-stream prior events within 72 hours.Requests use temperature zero; final decisions are scored offline against common reference labels.Source: §B.5 · p. 13

C Additional ablation and intervention control results

Additional analyses report intervention-control and memory-ablation comparisons, test sensitivity to revised references, and diagnose episode matching and follow-up outcomes.

C Additional ablation and intervention control results

D.1 Reference sensitivity

  • Separate-team references yield similar effectiveness on Sports but lower effectiveness on Home and Code, with absolute scores especially label-sensitive on the latter two scopes.Three reviewers independently labeled targets using same-stream prior history, blinded to evaluated system outputs, then reached consensus.Source: §D.1 · p. 14, Table 8 · p. 15

D.2 Episode references and matching

  • Episode coverage is highest on Home and lowest on Code, while redundant-intervention share is highest on Home and lowest on Sports.Matching requires a timely intervention for the goal and object at a listed valid observation; valuable follow-ups require new causal evidence and a changed actionable service or next step.Source: §D.2 · p. 15, Table 9 · p. 15
  • Exploratory episode scores depend on how references are constructed and semantic matching is performed.The diagnostic also defines episode coverage, supported relevance, and redundant-intervention rate from intervention and episode outcomes.Source: §D.2 · p. 15

Related papers

Loading 2610.36967v1…