PaperlayerView exact revision on arXiv

Source-linked AI summary

Making Prospective Memory SLM-Shaped: Typed Intention Stores for Small-Model Agents

Jinqing Zhao, Chengcan Wu

arXiv:2609.01272v1cs.AI

TL;DR

Prospective memory remains sparsely studied in agent-memory work, and the best reported PM-Bench scaffold reaches only 65.1% Set-F1. PIS stores deferred commitments as typed intentions, manages lifecycle logic in code, and leaves scoped semantic grounding to the model without PEFT. PIS scores 82.9% Set-F1 on DeepSeek-Chat, 66.2% on Gemma-E2B (vs. at most 6.6% for seven retrospective baselines), and 70.1% on Qwen3.5-4B (vs. at most 54.4% for seven retrospective memories).

  • Problem

    Prospective memory remains sparsely studied in agent-memory work, and the best reported PM-Bench scaffold reaches only 65.1% Set-F1.

    Source: §1 · pp. 1–2

  • Method

    PIS stores deferred commitments as typed intentions, manages lifecycle logic in code, and leaves scoped semantic grounding to the model without PEFT.

    Source: §1 · p. 2, §3 · p. 3, §3.1 · p. 3

  • Results

    PIS scores 82.9% Set-F1 on DeepSeek-Chat, 66.2% on Gemma-E2B (vs. at most 6.6% for seven retrospective baselines), and 70.1% on Qwen3.5-4B (vs. at most 54.4% for seven retrospective memories).

    Source: §5 · p. 6

  • Takeaways & Limitations

    The results support treating prospective memory as typed state tracking that small models can perform with the PIS scaffold.

    Source: §5 · p. 6

    The evaluation is concentrated on PM-Bench because public prospective-memory benchmarks for agents remain scarce; fine-tuning is not explored.

    Source: §5 · p. 6

Abstract

from arXiv · show

Prospective memory means carrying out a deferred intention at the right future cue while other work continues. Benchmarks now isolate it as an agent skill, yet frontier LLMs still struggle: the best published PM-Bench scaffold reaches only 65.1% Set-F1. We argue that this loop is schema-constrained state tracking rather than open-ended reasoning, and that small models can execute it when the action space is typed. We propose the Prospective Intention Store (PIS) that puts lifecycle logic in code and scoped language work on the model. The scaffold is agentic and training-free: no selector fine-tuning and no trajectory distillation. On PM-Bench, DeepSeek-Chat with PIS reaches 82.9% Set-F1. On Gemma-E2B, Set-F1 is only 4.2% without a store and at most 6.6% under seven retrospective memories, while PIS reaches 66.2%. PIS further reaches 70.1% Set-F1, where retrospective memory methods stay at most 54.4%. PIS sets a new state of the art on this benchmark and enables small models to surpass the published large-model scaffold.

1 Introduction

Prospective memory remains a largely unaddressed agent skill: existing memory systems mainly retrieve the past, while even frontier models struggle with future-cue obligations. The paper proposes PIS, a training-free typed intention store that separates lifecycle logic from scoped language work.

1 Introduction

  • Memory is needed for long-horizon assistance, and many recurring memory operations are narrow and structured, a setting the paper identifies as suitable for small language models.Typed tool interfaces have enabled 1B–7B models to match or exceed cloud-scale APIs in prior work [Erdogan et al., 2024, Chen et al., 2024, Patil et al., 2023].Source: §1 · p. 1
  • Prospective memory is sparsely studied in agent memory, despite its everyday role in carrying out deferred intentions at future cues while other activities continue.Existing evaluations and stores mostly optimize similarity-based retrieval rather than whether a trigger is satisfied and which actions are due.Source: §1 · pp. 1–2
  • The best reported PM-Bench scaffold reaches only 65.1% Set-F1, and prior Memory×SLM systems retrieve or cache the past rather than maintain revisable due sets.The paper frames prospective memory as schema-constrained state tracking involving updates, due-set decisions, and observation.Source: §1 · pp. 1–2
  • PIS casts deferred commitments as typed records and implements their lifecycle in code, leaving scoped grounding to the model without selector fine-tuning or trajectory distillation.The authors present the system as a training-free Form–Decide loop.Source: §1 · p. 2

2 Related Work

Prior agent-memory work largely centers on retrospective storage and retrieval, while prospective memory remains a sparse target. Related SLM systems and prospective-memory benchmarks motivate PIS’s typed prospective object and Form–Decide operators.

SLMs for scoped agent work

Retrospective memory

  • Retrospective agent-memory stacks store and retrieve episodic or semantic notes, and Memory×SLM pipelines typically require selector LoRA or teacher distillation.Their success metrics generally target answer quality or environment reward, and their stored notes are optimized for similarity retrieval rather than revisable intentions.Source: §2 · p. 3

Prospective memory

  • PM-Bench’s best scaffold reaches 65.1% Set-F1, and TriggerBench reports prospective performance far below matched retrospective probes [Zhang et al., 2026b].PIS complements this work with a typed prospective object and Form–Decide operators on a frozen backbone, without selector LoRA or teacher distillation.Source: §2 · p. 3

3 Method

PIS treats conditioned delayed memory as operations over typed intentions, with deterministic lifecycle logic and scoped semantic grounding. Its stepwise operators form commitments, revise and filter them, observe relevant channels, and decide which actions are due.

3 Method

  • PIS separates deterministic, structural intention lifecycle operations from the language model’s scoped semantic grounding, making the factorization usable at inference without PEFT.Intentions are stored as triples of trigger, action, and status, and each step maintains a predicted due set and updated store.Source: §3 · p. 3, §3.1 · p. 3

3.1 Problem formulation

  • At each step, the agent uses observed evidence and candidate actions to maintain pending, done, or canceled intentions and emit a predicted due-action set.The ideal due set contains the actions whose pending intentions have satisfied triggers in the current evidence.Source: §3.1 · p. 3

3.2 Store and decision interface

  • The interface defines the due set by pending intentions whose trigger conditions are satisfied by the evidence; clocks use rules, while event and channel cues use a language judge.Channel-conditioned triggers require replies to be added to the evidence before satisfaction can be evaluated.Source: §3.2 · p. 3, §3.1 · p. 3

3.3 Operators

  • Form extracts commitment spans into typed trigger–action–status rows, while Revise can reschedule triggers, override actions, or cancel pending intentions.A cue’s appearance alone is not cancellation; it is evidence for trigger satisfaction handled later by Filter and Decide.Source: §3.3 · pp. 3–4

3.4 One-step algorithm

  • The one-step algorithm runs Form, Revise, Filter, optional channel queries, and Decide in sequence, with the operators partitioned between the policy π and store update F.The resulting decision is an action set and the next store state.Source: §3.3 · p. 4, §3.4 · p. 4

3.3 Operators

  • Filter builds an eligibility board from pending rows, retaining unqueried channel-conditioned intentions as check targets; Decide may query those channels before selecting due actions and updating status.Filter and the commit operation are structural, while language grounding is used for Form, Revise, the channel judge, and Decide.Source: §3.3 · p. 4

4 Experiments

Experiments evaluate training-free memory scaffolds on a seven-day PM-Bench week with frozen backbones, comparing due-set accuracy and resource use across large and small models.

4.1 Setup

  • The evaluation uses a released seven-day synthetic PM-Bench week, micro Set-F1 as its primary metric, and frozen backbones without selector LoRA, teacher-trajectory distillation, or PEFT.It compares training-free retrospective memories and PIS on DeepSeek-Chat and Gemma-E2B, with Qwen3.5-4B and Qwen3-8B in the scaling evaluation.Source: §4.1 · pp. 4–5

4.2 Main results

  • 82.9% Set-F1 with PIS on DeepSeek-Chat exceeds the 67.7% no-store single baseline; on Gemma-E2B, PIS reaches 66.2% versus 0.0–6.6% for retrospective setups.DeepSeek-Chat PIS also records 22.2% update miss, 0.0% cross-day miss, and 8.8% FA/step; Gemma-E2B's update miss remains 77.8%.Source: Table 1 · p. 5, §4.2 · p. 5

4.4 Computation analysis

  • Several retrospective SLM scaffolds finish quickly with at most about 1M estimated main tokens yet remain near floor Set-F1; fewer check_time/query_state loops explain much of this apparent efficiency.A-Mem and Letta are exceptions, taking tens of minutes and several million tokens as linking and server-side LLM work dominate duration.Source: §4.4 · pp. 5–6

4.3 Scaling to other SLMs

  • PIS leads on both Qwen backbones, reaching 70.1% Set-F1 on Qwen3.5-4B and 57.2% on Qwen3-8B; retrospective methods do not reliably improve over no-store baselines.The 3.5-4B checkpoint is designed for agentic workloads and follows the typed board more reliably than the larger 8B chat model.Source: §4.3 · p. 5, Table 2 · p. 6

4.4 Computation analysis

  • PIS uses more time and main-prompt tokens than the quiet single baseline, but is often cheaper in duration than A-Mem and Letta while attaining the best Set-F1 on every backbone.Estimated main-prompt tokens exclude judge, embedder, and external-memory-server calls, so Tok is a lower bound; duration includes side effects on the critical path.Source: §4.4 · pp. 5–6, Table 3 · p. 6

5 Conclusion

PIS frames prospective memory as schema-constrained state tracking and reports strong PM-Bench results across large and small models. Evaluation remains concentrated on PM-Bench, and fine-tuning is left unexplored.

5 Conclusion

  • DeepSeek-Chat+PIS reaches 82.9% Set-F1, exceeding published scaffolds; Gemma-E2B reaches 66.2% versus at most 6.6% under seven retrospective baselines; Qwen3.5-4B+PIS reaches 70.1% versus at most 54.4%.PIS uses no selector training or student distillation.Source: §5 · p. 6
  • PM-Bench is the main openly available prospective-memory suite, so the evaluation is necessarily concentrated on that dataset.The authors note that fine-tuning is not explored and may further improve performance.Source: §5 · p. 6
  • Future work includes fine-tuning Form–Decide interfaces, isolating operator contributions, and building additional prospective-memory benchmarks.Source: §5 · p. 6

Related papers

Loading 2609.01272v1…