PaperlayerView exact revision on arXiv

Source-linked AI summary

BehaviorMem: Building and Evaluating User Memory from Real-World E-Commerce Behavior Streams

Sizhe Wang, Xinghao Fang, Rui Qiu, Zhenjia Ma, Dagui Chen, Kun Kuang, Hongcheng Guo

arXiv:2610.34790v1cs.CL

TL;DR

Behavioral logs record e-commerce actions without directly explaining users’ preferences or needs, and existing methods and evaluations offer limited coverage of real continual behavior and separate capability tests. This paper introduces a benchmark and memory design for remembering and inference; across two builders and two readers, BehaviorMem leads evaluated agentic memories on both recalls.

  • Problem

    Behavioral logs reveal actions but not users’ underlying preferences or needs, while existing methods and benchmarks provide limited real continual behavior and separate remember-and-infer evaluation.

    Source: §1 · p. 1

  • Method

    BehaviorMemBench evaluates remembering and inference separately, while BehaviorMem maintains explicit behavioral facts and inferred characteristics across long- and short-term scopes.

    Source: §1 · p. 2, §4 · p. 4

  • Results

    Across two builders and two task readers, BehaviorMem leads evaluated agentic memories on both recalls, with method rankings differing across target familiarity.

    Source: §1 · p. 3, §5.4 · p. 7

  • Takeaways & Limitations

    Remembering and inference show different method rankings, supporting their separate evaluation rather than treating memory performance as a single capability.

    Source: §2 · p. 3, §5.4 · p. 7

    The state update and readout use only behavior through time t, excluding downstream queries, candidate pools, and future labels.

    Source: §2 · p. 3

Abstract

from arXiv · show

User memory helps personalized systems build on their understanding of users over time. In e-commerce, users convey their preferences and needs through both explicit language and implicit interactions with products. We study memory built from logs of these interactions, examining how it can \emph{remember} concrete evidence and established interests, and use that evidence to \emph{infer} underlying needs and preferences that inform judgments beyond well-observed product domains. Existing memory methods primarily target language-based interactions, and evaluations offer limited real behavioral data and limited separation of these capabilities. This leaves open how to jointly support and separately evaluate remembering and inference from behavioral logs. We formulate the problem of continual behavioral user memory around these joint requirements and develop a dedicated evaluation framework and memory method to address them. \textbf{BehaviorMemBench} is a benchmark built from real e-commerce logs covering 1{,}194 users and 17{,}462 behavioral batches over more than three years, with automatic future-behavior labels and a task-independent memory interface to separately evaluate remembering and inference under a common protocol. \textbf{BehaviorMem} is a memory design that organizes explicit behavioral facts and inferred user characteristics across long-term and short-term scopes in a two-axis state, with evidence-grounded formation, continual maintenance, and query-independent readout. Experiments across two builders and two task readers show that BehaviorMem leads the evaluated agentic memories on remember and infer, and reveal different method rankings across the two capabilities and competitive statistical baselines. Validation on a public dataset further supports its usefulness for downstream tasks based on behavioral histories.

1 Introduction

E-commerce behavior reveals users’ actions but not their underlying preferences or needs, motivating memory that preserves concrete evidence and infers reusable user knowledge. The paper introduces a benchmark and memory design for jointly supporting and separately evaluating these capabilities.

1 Introduction

  • Behavioral logs show what users did without directly explaining why, so memory must both retain concrete choices and infer preferences beyond remembered information.Inference connects recurring choices and shared product characteristics to judgments in sparsely observed or unvisited, semantically related domains.Source: §1 · pp. 1–2
  • Existing memory methods and benchmarks largely center on language interactions, while behavioral-log benchmarks include synthetic trajectories or lack continual evaluation on real data.The paper also identifies limited common evaluation that separately tests remembering and inference for continual implicit e-commerce behavior.Source: §1 · p. 2
  • BehaviorMem pairs explicit behavioral facts with inferred user characteristics across long- and short-term scopes, using evidence-grounded abstraction and continual maintenance.Its inference process links user characteristics to supporting observations and revises their content and temporal availability as new evidence arrives.Source: §1 · pp. 2–3
  • BehaviorMemBench evaluates remembering and inference separately on real e-commerce logs, using automatically derived future-behavior labels and a common, task-independent protocol.The benchmark covers 1,194 users and 17,462 behavioral batches over more than three years.Source: §1 · p. 2

2 Problem Formulation

The formulation treats behavioral memory as a continually updated user state built from chronological behavior batches, with a query-independent readout. It requires separate support for remembering historical evidence and inferring reusable knowledge.

2 Problem Formulation

  • Each user’s chronological behavior stream consists of batches whose interactions record products, attributes, action types, and timestamps, while preferences and needs remain implicit.The memory system updates its state from the stream and exposes a query-independent readout.Source: §2 · p. 3
  • State updates and readout depend only on behavior through time t, not downstream queries, candidate pools, or future labels.The state may combine records, statistical summaries, and inferred characteristics, and a common reader evaluates its readout.Source: §2 · p. 3
  • Memory must preserve useful historical evidence and established user information while inferring reusable knowledge for less-observed or unvisited domains.The formulation evaluates remembering and inference separately because abstraction and detail preservation can affect them differently.Source: §2 · p. 3

3 BehaviorMemBench

BehaviorMemBench combines real, continual implicit behavior streams with a task-independent interface and separate remember and infer tests. It derives future labels from Taobao histories and evaluates fixed memory readouts against distinct target pools.

3.1 Comparison with Existing Benchmarks

  • BehaviorMemBench combines real, continual implicit behavior streams and automatic future labels with task-independent evaluation that separately tests remembering and inference.Existing benchmarks use language histories, synthetic behavior, real data without continual evaluation, or do not separately test inference beyond well-observed domains.Source: §3.1 · p. 3

3.2 Data Collection and Statistics

  • The Taobao corpus spans February 2023 to June 2026 and contains 913,271 visits, 17,462 batches, and 16,268 history-to-future transitions.The construction retains 1,194 of 2,000 sampled users with at least 50 valid visits; batches close after visit or elapsed-time limits at an inter-visit gap exceeding 15 minutes.Source: §3.2 · p. 3

3.3 Benchmark Framework

  • Prediction targets are normalized L2 categories, and each held-out next batch supplies future labels while memory uses only behavior observed through time t.Favorites, add-to-cart actions, purchases, or qualifying deep browsing produce positives; shallow clicks alone do not.Source: §3.3 · p. 3
  • The remember pool tests visited categories and unseen categories in established domains, whereas infer tests unseen categories in lightly observed or unvisited, semantically connected domains.Both pools include history-conditioned or random distractors, with three distractors per positive.Source: §3.3 · p. 3
  • A frozen task reader evaluates each pool from the same fixed readout, selects as many categories as there are positives, and sees neither method identity nor positive labels.Pool recall measures recovered future positives and distractor exposure measures incorrectly selected eligible distractors; higher recall and lower exposure indicate better performance.Source: §3.3 · p. 3

4 BehaviorMem

BehaviorMem organizes behavioral evidence across explicit and inferred information and long- and short-term scopes. Its formation, maintenance, and readout procedures preserve evidence-grounded claims while exposing temporally scoped memory content.

4 BehaviorMem

  • BehaviorMem targets continual e-commerce logs, preserving concrete behavioral facts while abstracting evidence into reusable user characteristics across long- and short-term scopes.The design is intended to support remembering past choices and inferring user characteristics from implicit product interactions.Source: §4 · p. 4

4.1 Two-Axis Memory State

  • The two-axis state pairs programmatic facts with inferred Claims, while separating accumulated tendencies from recent or stage-specific information.Historical category and brand aggregates and recent item activity occupy the fact state; Active and Dormant Claims track user characteristics across semantic paths.Source: §4.1 · p. 4

4.2 Memory Formation

  • Formation combines programmatic fact accumulation with evidence-grounded abstraction, deriving characteristics through behavior, pattern, and theme steps.Each inference level uses only its predecessor as evidence; candidate Claims retain behavioral references and are placed on semantic paths using the prior registry.Source: §4.2 · p. 4

4.3 Memory Maintenance

  • Maintenance revises claims locally, retiring them only on direct contradiction; missing observations alone do not justify deletion.Continuous paths remain Active without renewed observations, while stage-specific paths can become Dormant and reactivate with new support; Dormant content is preserved with its evidence but hidden from readout.Source: §4.3 · pp. 4–5

4.4 Memory Readout

  • Deterministic projections expose explicit facts and Active Claims with temporal scope, independently of downstream queries, candidates, and labels.Dormant content remains internal, and readout is model-free.Source: §4.4 · p. 5

5 Experiments

BehaviorMem leads the evaluated agentic memories on both recall measures across builder–reader configurations, while ablations and subgroup analyses reveal differences between remembering and inference. Its public-dataset validation shows strong item ranking, though results differ for rating prediction.

5.1 Experimental Setup

  • Experiments evaluate separate remembering and inference on BehaviorMemBench, cross-check downstream tasks on MemoryCD, and compare statistical, general-purpose, and user-modeling baselines.Methods process histories chronologically and expose query-independent readouts to shared task readers.Source: §5.1 · p. 5

5.3 Ablation

  • Removing explicit memory lowers remember-pool recall from 0.612 to 0.529, the largest mask-related decline; masking T3/T4 also matters more than masking T1/T2.The results identify explicit evidence as the main basis for remembering in this readout, with inferred interests and situational needs providing complementary support.Source: Figure 3 · p. 6, §5.3 · p. 6
  • Removing implicit memory lowers infer-pool recall from 0.443 to 0.432, while masking T1/T2 has a greater effect than masking T3/T4.Masking T3/T4 raises new recall from 0.416 to 0.423 despite reducing remember-pool recall, showing that effects differ by objective and subgroup.Source: Figure 3 · p. 6, §5.3 · p. 7

5.4 Discussion

  • By activity, BehaviorMem’s remember-pool recall exceeds A-Mem’s in every group, but infer-pool recall does not improve consistently with activity.For the Qwen3-32B builder, remember-pool recall is highest in the least-active group; A-Mem is slightly ahead on inference in the two most-active groups.Source: Figure 4 · p. 6, §5.4 · p. 7

5.2 Overall Results

  • BehaviorMem achieves the highest remember-pool and infer-pool recall across all four builder–reader configurations.BehaviorMem leads the evaluated agentic memories on both measures, while Long Context and Frequency Profile remain competitive, particularly for remembering.Source: §5.2 · p. 6

5.4 Discussion

  • Frequency Profile is competitive on seen categories but trails on unseen categories within familiar domains; under the Plus reader, Long Context leads on related targets and BehaviorMem on new targets.The four builder–reader configurations are shown in Figure 5.Source: Figure 5 · p. 7, §5.4 · p. 7
  • BehaviorMem averages 2,018 or 1,850 memory tokens across the two builders, using over 70% fewer tokens than Long Context or LightMem.Long Context averages 7,338 tokens; LightMem averages 7,084 and 6,812 for the two builders.Source: §5.4 · pp. 7–8

5.5 Validation on Public Dataset

  • On MemoryCD, BehaviorMem has the highest HR@10 and NDCG@10 across builders and domain settings, but ranks second among agentic memories for MAE and RMSE.Long Context remains best overall for rating prediction; for the Qwen3-32B builder’s cross-domain setting, BehaviorMem scores 0.563 HR@10 and 0.280 NDCG@10 versus Persona Tree’s 0.548 and 0.254.Source: Table 3 · p. 8, §5.5 · p. 8

6 Conclusion

The conclusion frames behavioral user memory as jointly preserving observed choices and inferring preferences beyond well-observed domains. It presents BehaviorMemBench and BehaviorMem, reports leading recall among evaluated agentic memories, and notes complementary components and downstream validation.

6 Conclusion

  • BehaviorMem addresses continual behavioral logs by jointly supporting remembering established information and inference beyond well-observed product domains.The design maintains behavioral facts and revisable user characteristics through a two-axis state, evidence-grounded abstraction, continual maintenance, and query-independent readout.Source: §6 · p. 8
  • BehaviorMemBench separately evaluates remembering and inference using automatically derived future-behavior labels and a common task-independent protocol.The benchmark uses real e-commerce behavioral logs.Source: §6 · p. 8
  • Across two builders and two task readers, BehaviorMem leads evaluated agentic memories on both recalls; public-dataset validation further supports downstream utility.Ablations show complementary roles for observed facts and inferred characteristics, while statistical baselines remain competitive.Source: §6 · p. 8

7 Related Work

Prior work spans language-based memory for LLM agents, behavioral-history user modeling, and evaluations of memory retention and user understanding. BehaviorMem differs by building memory from implicit behavioral records and jointly organizing inferences, supporting events, and temporal revision.

7 Related Work

  • LLM-agent memory research has structured and linked records of experience, generally grounded in explicit interactions such as conversations [14] [20] [21].BehaviorMem instead builds memory from implicit records such as purchases and clicks, which are noisier and less self-descriptive.Source: §7 · p. 8
  • E-commerce user modeling has encoded behavior sequences as latent representations, while LLM approaches describe multiple interests or revise user memories through feedback and collaborative signals.BehaviorMem combines structured organization of inferences and their supporting events with temporal management of revision and availability.Source: §7 · pp. 8–9
  • Existing user-memory evaluations cover interaction recall, incremental streams, synthetic activity profiles, personalized tasks, and interest alignment; BehaviorMemBench uses real implicit behaviors and future transaction labels.Its history-aware candidate pools provide separate operational measures of remembering and interest inference.Source: §7 · p. 9

Reproducibility and Ethics Statement

The statement describes authorization and review, then outlines the Taobao data-processing and evaluation-label pipeline. It reports the dataset’s scale and explains how histories, future labels, and candidate pools are constructed.

Reproducibility and Ethics Statement

  • The behavioral data was authorized by the platform and underwent privacy and ethics review; the benchmark is described as planned for public release after processing and review.Appendices document construction, prompts, baseline adaptations, and evaluation protocols.Source: Reproducibility and Ethics Statement · p. 9

A.1 Data Collection Pipeline

  • Among 2,000 randomly sampled Taobao users, 1,194 had at least 50 valid visits; a batch becomes eligible at 50 visits or 120 days from its first visit and closes at the next inter-visit gap exceeding 15 minutes.Source: §A.1.1 · p. 10

A.1.1 Part I: Data processing.

  • Consecutive same-item events within 30 minutes merge into one visit, while action counters retain entry clicks, browsing, favorites or cart actions, and purchases.Cancellations and refunds are dropped rather than subtracted.Source: §A.1.1 · p. 10

A.1.2 Part II: Evaluation construction.

  • Future-batch behavior supplies positive categories, while history determines remember and infer pools; distractors are relation-matched, exclude future positives, and are frozen across methods and readers.Remember positives include seen categories or unseen categories in frequently visited domains; infer positives include unseen categories in lightly observed or unvisited, semantically linked domains.Source: §A.1.2 · p. 10

A.2 Benchmark Statistics

  • The corpus spans 2,119 L2 categories in 161 L1 domains, with activity varying from 101 users having at most three batches to 501 having at least 16.The normalized taxonomy contains 4,667 L2 categories; median user history comprises 736 visits and 14 batches.Source: §A.2 · p. 10

A.3 Readout and Task-Reader Protocol

BehaviorMem separates explicit behavioral facts from inferred characteristics, then forms, maintains, and reads these memories through a shared state. Its protocol keeps remembering and inference as separate calls over that state.

A.3 Readout and Task-Reader Protocol

  • Reader methods share candidate pools, prompts, and decoding settings; remembering and inference use separate calls on the same state.Fixed readouts fill common description and tree slots, using fixed placeholders for unused slots.Source: §A.3 · p. 11

B.1 Per-Batch Execution

  • Each batch moves through behavior rendering, Claim derivation and placement, state updates, and deterministic readout.T1/T2 use closed slots and T3/T4 use open Claims, which merge into one tree; an append-only visit ledger records F1 facts but does not enter that tree or upstream prompts.Source: §B.1 · p. 11

B.2 State Representation

  • Explicit memory combines recency-decayed categories, brands, attributes, and engagement with recent item activity, while implicit Claims cover four user-information types.The implicit types are demographics, social relations, interests, and situational state; Claim paths are organized by type, domain, and attribute.Source: §B.2 · p. 11

B.3 Formation: The Evidence-Distance Ladder

  • Formation moves from observed behavior through commonalities and attended themes to reusable characteristics or demographic and social attributes.Each abstraction level uses only the immediately preceding level as evidence; T3/T4 stop at free-text characteristics, while T1/T2 continue to values from a closed candidate set.Source: §B.3 · p. 11

B.4 Maintenance

  • T1/T2 replace a conflicting value only when its incoming confidence is strictly higher; T3/T4 retire a Claim only under direct contradiction.Unobserved non-continuous paths may become dormant, while absence alone is insufficient counter-evidence and uncertainty defaults to keeping a path active.Source: §B.4 · p. 11

B.5 Readout

  • F5 emits active Claims and ranks ledger-derived long-term aggregates and recent item activity in a query-independent readout.The readout includes Claim confidence and stable or stage-specific scope, and F5 calls no model.Source: §B.5 · p. 12

C BehaviorMemBench Experimental Details

C.1 Dataset and Model Settings

The evaluation uses chronological, query-independent memory construction and shared readouts, with settings for activity and target familiarity analyses. It also tests downstream tasks on MemoryCD’s review histories.

C.1 Dataset and Model Settings

  • Activity groups preserve batch-count ties, and activity and target-familiarity analyses use common valid transitions within each configuration.Activity analyses use transitions common to five agentic methods and user macro-averaging.Source: §C.1 · p. 12

C.2 Baselines and Adaptation

  • Baselines process histories chronologically into native query-independent representations without candidate access during construction.The evaluated adaptations include long-context, frequency and template profiles, agentic memory systems, and Persona Tree; their readouts and construction settings vary.Source: §C.2 · p. 12

D.1 Dataset and Tasks

  • MemoryCD uses Amazon Reviews 2023 histories in four domains, with the final three interactions reserved for testing.Its cross-domain setting combines Beauty, Books, and Electronics histories to predict the final three Home and Kitchen interactions for a 323-user cohort.Source: §D.1 · p. 12

D.2 Baselines and Implementation

  • MemoryCD methods use shared-readout-token-budget adapters, with Qwen3-32B or Qwen3.6-35B-A3B for agentic builders and readers.Non-parametric baselines do not require an LLM builder.Source: §D.2 · p. 12

D.3 Evaluation Protocol

The evaluation measures item ranking and rating prediction using held-out interactions. It reports results for both readers, with MemoryCD results also tabulated for two memory builders.

D.3 Evaluation Protocol

  • Item ranking uses candidates with one held-out positive and negatives, while rating prediction asks readers to predict scores on a 1–5 scale.HR@10 measures top-ten inclusion and NDCG@10 rewards higher placement; MAE measures average absolute error and RMSE weights larger errors more heavily.Source: §D.3 · p. 12
  • Single-domain results average metrics equally across four domains, and Tables 3 and 5 report both readers.Table 5 reports MemoryCD results under a Qwen3-32B task reader with both memory builders.Source: §D.3 · p. 12, Table 5 · p. 13
  • N@10 denotes NDCG@10 in the reported tables.NDCG@10 rewards higher placement of the held-out positive.Source: §D.3 · p. 12

Related papers

Loading 2610.34790v1…