Source-linked AI summary
ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents
Haohao Qu, Yongcheng Jing, Chun Hin Chan, Shanru Lin, Wenqi Fan, Dacheng Tao
TL;DR
RecAgents face brittle item perception across noisy, heterogeneous pages and inefficient reasoning over long histories and interaction traces. ReMem combines OCR-based multimodal perception with chunk-wise dynamic memory and trains memory updates with a GRPO variant. Across three recommendation-agent tasks, it improves 5.16% over the strongest baseline on average.
Problem
RecAgents face brittle item perception from noisy, heterogeneous pages and inefficient long-context reasoning over extended histories and interaction traces.
Source: §1 · pp. 2–3Method
ReMem combines screenshot-based OCR perception with chunk-wise updates to fixed-size dynamic memory, using a GRPO variant to propagate final-answer advantage to intermediate updates.
Source: §1 · p. 3, §3.2 · p. 6, §3.3 · p. 6Results
ReMem achieves a 5.16% average relative improvement over the strongest baseline across searching, ranking, and judging on three InstructRec datasets.
Source: §1 · p. 3Takeaways & Limitations
ReMem provides a token-space memory that can be inspected, debugged, edited, or constrained by prompts for long-horizon preference modeling.
Source: §3.3.3 · p. 8The current environment primarily models browsing item images, comparing items, and purchasing decisions, leaving actions such as expanding descriptions, searching for information, and checkout for future work.
Source: D Discussion on Limitations and Future… · p. 26
Abstract
from arXiv · showhide
Recent Recommendation Agents (RecAgents) offer a promising alternative by shifting recommendation to an active, user-side paradigm, where generative agents autonomously perceive external platforms, reason over user preferences, and execute decisions. However, existing RecAgents still suffer from two critical limitations: brittle item perception based on noisy and heterogeneous item pages, and inefficient long-context reasoning over extended user histories and multi-step interaction traces. To address these challenges, we propose a novel recommendation agent framework, termed as ReMem, that combines OCR-based multimodal perception with time-evolving dynamic memory. Instead of parsing raw HTML, ReMem observes item pages through screenshots and extracts structured multimodal information via an OCR tool, enabling a more humanoid and platform-agnostic perception mechanism. To support long-horizon preference modeling, ReMem further introduces a chunk-wise sequential memory update strategy, where the agent selectively maintains a fixed-size memory of informative historical interactions while processing arbitrarily long contexts with linear inference complexity and bounded context length. This design allows the agent to preserve evolving user preferences without relying on external memory modules or disrupting the standard autoregressive generation process. To enhance the dynamic memory instruction, we further develop a multi-memory GRPO variant, which propagates the final-answer advantage to all intermediate conversations that contribute to the final response. Extensive experiments on three datasets demonstrate that ReMem consistently outperforms state-of-the-art baselines, achieving an average improvement of 5.16\% across three recommendation agent tasks, namely searching, ranking, and judging.
1 Introduction
ReMem addresses brittle item perception and inefficient long-context reasoning in RecAgents with OCR-based multimodal perception and dynamic memory. Across three InstructRec datasets, it outperforms the strongest baseline by 5.16% on average across searching, ranking, and judging.
1 Introduction
- Existing RecAgents struggle with heterogeneous, noisy HTML pages that omit multimodal details and with histories and interaction traces that exceed finite context windows.Token reduction and external memory approaches can struggle with out-of-distribution generalization and disrupt standard autoregressive generation.Source: §1 · pp. 2–3
- ReMem uses screenshots and OCR to extract structured multimodal item information, avoiding raw HTML parsing.It integrates DeepSeek-OCR-2 as an active tool-use component.Source: §1 · p. 3
- Its chunk-wise memory process selectively updates a fixed-length store of K historical interactions as the LLM reads OCR text sequentially.The memory is intended to track preferences as they evolve over time.Source: §1 · p. 3
- ReMem jointly explores OCR-based perception and RL-enhanced memory for multimodal understanding and time-evolving preference modeling in long-context RecAgents.The paper describes this as the first framework to explore the combination.Source: §1 · p. 3
- 5.16% average relative improvement over the strongest baseline across searching, ranking, and judging on three InstructRec datasets.The experiments compare ReMem with state-of-the-art baselines.Source: §1 · p. 3
2 Pilot Study
The pilot study frames RecAgents as agents that perceive items, reason over user histories, and act on external platforms. Its perception and long-context experiments motivate OCR-based item processing and dynamic memory for recommendation settings.
2.1 Preliminary
- The pilot study evaluates perception on WebWalkerQA and long-context reasoning on HotpotQA, which spans contexts of approximately 7K–3.5M tokens.WebWalkerQA contains 673 examples requiring web access to sites with visual and textual content.Source: §2.1 · p. 4
2.2 Revisiting Agent Perception
- On WebWalkerQA, OCR parsing improves performance over full HTML context and screenshots by capturing multimodal information while filtering noisy content such as advertisements.OCR can also create longer contexts in which retrieving relevant information from the middle may degrade performance.Source: §2.2 · p. 4, Table 1 · p. 4
- OCR enriches agent perception but increases demands on the LLM’s long-context reasoning, especially when recommendation combines multiple item descriptions with user history.The pilot study compares HTML-derived text, screenshot input, and OCR-derived text; evaluations were repeated five times.Source: §2.2 · p. 4
2.3 Rethinking Long-Context Reasoning
- On HotpotQA, dynamic memory shows relative lossless performance extrapolation, while long-context continual pretraining and vectorized memory fail to maintain consistent performance.The study compares long-context models, retrieval of top-5 5K-token chunks, dynamic memory, and a 128K-token truncation baseline.Source: Figure 2 · p. 5, §2.3 · p. 5
- The dynamic-memory findings suggest potential for recommendation agents that must identify time-evolving preferences and manage long-term interaction histories.Source: §2.3 · p. 5
3 Methodology
ReMem combines OCR-based item perception with a fixed-size, evolving memory for long histories, then trains intermediate memories using the final recommendation outcome. Its chunk-wise design keeps the active context bounded and inference linear in history length.
3.1 The Overview of the Proposed Approach; 3.2 Multimodal Perception on Item Pages
- ReMem replaces raw-HTML item parsing with OCR that reads page text, images, and charts and converts results into natural-language item cards for the LLM.The framework pairs this perception module with time-evolving memory and Multi-Memory GRPO.Source: §3.1 · p. 5, §3.2 · p. 6
3.3 Time-Evolving Memory for User Preference; 3.3.1 Workflow
- Time-Evolving Memory processes user histories as sequential chunks and updates a compact, fixed-length state with preference-relevant evidence.The memory is represented as natural-language tokens in the LLM context; after processing all chunks, the final recommendation conditions on the final memory, task input, and candidate information.Source: §3.3 · p. 6, §3.3.1 · p. 7
3.3.2 Derivation
- With fixed chunk and memory sizes, ReMem bounds each update’s context and yields O(N) history-processing complexity, versus O(N^2) attention for feeding the full history to a standard transformer.The derivation describes the memory as a sequence of compact states that propagates preferences without requiring non-standard cache operations or attention changes.Source: §3.3.2 · pp. 7–8
3.3.3 Discussions
- Token-space memory can be inspected, debugged, edited, or prompt-constrained, while supporting long-horizon preference modeling and standard autoregressive generation.The paper gives examples including lightweight-shoe preference, price sensitivity, recent waterproof-product searches, and neutral-color choices.Source: §3.3.3 · p. 8
3.4 Multi-Memory GRPO; 3.4.1 Multi-Memory Generation
- Multi-Memory GRPO assigns the final-answer advantage to every independently generated intermediate memory conversation that contributes to the recommendation.Each query yields sequential memory instances that are aggregated into the final memory used to generate the answer.Source: §3.4 · p. 8, §3.4.1 · p. 8
3.4.2 Final-Answer Reward and Group Advantage; 3.4.4 Multi-Memory GRPO Objective
- Each sampled instance is rewarded using its final recommendation against the ground-truth item, and that advantage also updates intermediate memories so useful memories are reinforced.The objective applies clipped optimization to the answer and the same advantage to memory summaries, with λ controlling memory-token optimization.Source: §3.4.2 · p. 9, §3.4.4 · p. 9
3.4.3 Memory-Level Policy Ratio; 3.4.5 KL regularization
- Memory-token policy ratios condition on the query, current chunk, and that memory’s prior tokens, preventing artificial cross-memory credit assignment.The method also adds a token-level KL penalty over generated memory tokens, with β set to 0.04 following [31].Source: §3.4.3 · p. 9, §3.4.5 · p. 10
4 Experiment
Across three datasets and three agentic recommendation tasks, ReMem outperforms the strongest baseline on average and remains strongest across context lengths. Ablations identify time-evolving memory as especially consequential, while experiments compare model families under specified datasets, settings, and metrics.
4.1.1 Datasets
- MovieTV, Books, and Games datasets are constructed from Amazon Reviews 2023 content, with item snapshots, randomly added ads, and user personas and task instructions.The snapshots include textual item information and product-image URLs.Source: §4.1.1 · p. 10, Table 2 · p. 10
4.1.2 Compared Models
- The comparison spans sequential recommenders, natural-language recommendation systems, recommendation agents, and long-context LLM agents.Named baselines include SASRec, BERT4Rec, P5, TokenRec, ToolRec, iAgent, QwenLong-L1, and Mem0.Source: §4.1.2 · p. 10
4.1.3 Configurations
- Experiments use Qwen3.5-9B, a maximum item-sequence length of 50, and memory length of 5,000 tokens, with chunking window 3 and chunks up to 100,000 tokens.The model typically processes the full context in 3–5 conversational turns.Source: §4.1.3 · pp. 10–11
4.1.4 Evaluation Metrics
- Judging is evaluated with LLM-as-Judge Accuracy, while Searching and Ranking use HR@K and NDCG@K, respectively.Higher values indicate better performance; K values are 1, 3, and 5.Source: §4.1.4 · p. 11
4.2 Recommendation Performance
- 5.16% average improvement over the strongest baseline, with ReMem best in all searching and judging settings and two of three ranking settings.The reported improvements are statistically significant (p < 0.01).Source: Table 3 · p. 11, §4.2 · p. 11
4.3.1 Various-Length Reasoning
- Across short, medium, and long Books contexts, ReMem performs best on all three tasks, while every model declines as context length grows.The groups contain 658 short-, 6,024 medium-, and 694 long-context samples; ReMem in long contexts is comparable to or better than several baselines in medium contexts.Source: §4.3.1 · p. 12, Figure 4 · p. 12
4.3.2 Ablation Study
- Removing any major component degrades searching performance, and removing time-evolving memory causes the largest drop in most cases.The variant without GRPO post-training remains competitive with baselines such as iAgent.Source: §4.3.2 · p. 12, Table 4 · p. 12
5 Conclusion
ReMem combines screenshot-based OCR perception with chunk-wise dynamic memory and reward-guided memory updates. Across recommendation environments, it achieves superior accuracy, efficiency, and cross-platform generalization.
5 Conclusion
- ReMem replaces HTML parsing with OCR-based screenshot perception and uses chunk-wise memory to retain informative user histories with bounded context and linear inference complexity.A GRPO variant assigns final-answer rewards to intermediate conversations contributing to the response.Source: §5 · p. 13
- The framework is designed to improve both item perception and long-context preference modeling.Source: §5 · p. 13
- Experiments across diverse recommendation environments show superior accuracy, efficiency, and cross-platform generalization.Source: §5 · p. 13
Table of Contents: Appendices
The supplied appendix contents list supplements on methods and experiments, while the related-work material reviews LLM agents, OCR perception, and long-context reasoning.
A Related Work ............. 17
- The listed appendices cover related work, research-method supplements, experiment details, and a limitations and future-work discussion.The experiment supplements include dataset construction, baselines, case studies, reasoning-length results, hyperparameters, and latency.Source: A Related Work ............. 17 · p. 17
A.1 LLM Agents
- Recommendation agents use LLMs to interpret user intent, model preferences, interact with tools, and generate recommendations, but most existing methods rely on structured or textual inputs.The passage characterizes the direction as early-stage, with limited multimodal perception and dynamic memory.Source: §A.1 · p. 17
A.2 OCR-based Perception
- OCR can extract text and structure from item-page images and interfaces, but OCR-based perception has rarely been studied in recommendation agents.Source: §A.2 · p. 17
A.3 Long-Context Reasoning
- Long-context recommendation can exceed current LLM context windows; the cited requirements are unbounded-length processing, no performance degradation with scale, and efficient linear-complexity decoding.The passage contrasts external memory systems with learning memorization within the LLM through reinforcement learning.Source: §A.3 · pp. 17–18
A.4 Distinction from Existing Studies
ReMem distinguishes itself from generic memory agents through recommendation-specific memory learning and upstream visual item perception, alongside its autoregressive design.
A.4 Distinction from Existing Studies
- ReMem maintains an explicit, time-evolving representation of user preferences across sequential interaction chunks, rather than only overwriting generic long-document memory.Source: §A.4 · p. 18
- Multi-Memory GRPO sends task-level recommendation outcomes to every intermediate memory update contributing to the final answer.Source: §A.4 · p. 18
- Unlike generic memory agents, ReMem also addresses item-page perception through OCR, while processing arbitrarily long histories with bounded context and linear complexity.Source: §A.4 · p. 18
- Sequential memory updates run within standard autoregressive generation without external retrieval, specialized memory modules, or architectural modifications.Source: §A.4 · p. 18
B Supplements on Research Methods
The supplements specify GRPO’s group-relative policy optimization, task-specific rewards, prompts, and OCR-based item-page processing. Together, these methods define how ReMem trains, evaluates, and supplies item information to its recommendation agent.
B.1 Standard GRPO
- GRPO samples response groups, normalizes their rewards into advantages, and optimizes a clipped policy objective with KL regularization against a reference policy.The appendix presents this standard formulation as the baseline for comparison with Multi-Mem GRPO.Source: §B.1 · pp. 18–19
B.2 Reward Modeling
- Task-specific correctness rewards use HR@K for Searching, NDCG@K for Ranking, and Accuracy for Judging, alongside graded XML-format rewards.The format reward can reach 1.5 points; correctness reward is assigned the corresponding metric value in [0, 1].Source: §B.2 · p. 19
B.3 Prompt Templates
- The prompt templates cover memory updates, final-answer generation, LLM-as-Judge evaluation, and separate Searching, Ranking, and Judging queries.Memory prompts update prior memory with useful information from each section; final prompts answer using memory and candidates.Source: §B.3 · p. 20
B.4 Examples of Item Pages
- DeepSeek-OCR-2 processes screenshots of Amazon-style item pages into Markdown descriptions and cropped images for recommendation reasoning.The examples combine textual context and item images in webpage snapshots.Source: §B.4 · p. 21
C.1 Supplements on Pilot Study
The supplements describe pilot and recommendation data, comparison baselines, a changing-preference case, and evaluations across reasoning lengths. ReMem retains an advantage over QwenLong-L1-32B as reasoning trajectories grow, although performance declines with longer trajectories.
C.1 Supplements on Pilot Study
- WebWalkerQA retains 673 human-verified samples, while the HotpotQA pilot uses article contexts from about 7K to 3.5M tokens with a 1,024-token memory cache and 5,000-token chunks.The WebWalkerQA websites average about 716.41 textual tokens.Source: §C.1 · pp. 21–22
C.2 Recommendation Dataset Construction
- Recommendation data use Amazon reviews filtered to five-core users and items, ratings of at least four, deduplicated interactions, and leave-one-out splitting.The last interaction per user is reserved for evaluation; instructions and user intents follow InstructRec and iAgent procedures.Source: §C.2 · p. 22
C.3 Baselines
- Baselines include ID-only SASRec and BERT4Rec, TokenRec, the tool-using iAgent, QwenLong-L1, and the memory systems Mem0 and MemAgent.SASRec and BERT4Rec use item IDs without textual descriptions or images; QwenLong-L1 is evaluated zero-shot.Source: §C.3 · p. 23
C.4 Case Study
- A case study shows memory shifting from Xbox and VR interests toward Oculus Quest 2 battery accessories, enabling identification of the target head strap with battery.The memory preserves earlier interests while incorporating newly emerging preferences.Source: §C.4 · p. 23, Table 5 · p. 24
C.5 Supplements on Various Length Reasoning
- On long Games trajectories, Searching HR@1 rises from 0.3005 to 0.3563 (+18.6%), and Ranking NDCG@3 rises from 0.3449 to 0.4624 (+34.1%) versus QwenLong-L1-32B.ReMem outperforms that baseline across all three tasks and both Games and MovieTV datasets, while performance declines as reasoning length increases.Source: §C.5 · p. 24
C.6 Hyper-parameter Analysis
Chunk-window analysis finds that the preferred window varies with average user-history length, while inference latency rises with context length and memory-based processing.
C.6 Hyper-parameter Analysis
- Performance generally improves from W = 1 to 3 and then plateaus or slightly declines; Games and MovieTV favor W = 3, while longer-history Books favors W = 4 or 5.A memory turn range of [3, 6] provides robust performance across datasets with diverse history lengths.Source: §C.6 · pp. 24–25, Figure 6 · p. 25
C.7 Inference Latency
- On a single NVIDIA H20 (96 GB) GPU, inference takes 224.60 s per Books sample, 118.53 s per MovieTV sample, and 98.25 s per Games sample.The framework adds latency over vanilla Qwen3.5-9B due to iterative memory construction and multi-step reasoning; time generally rises with input context length.Source: §C.7 · p. 25, Table 7 · p. 25
D Discussion on Limitations and Future Work
The discussion identifies added inference latency as a trade-off and proposes broader user-agent interactions and further evaluation as future work.
D Discussion on Limitations and Future Work
- Iterative memory updates and multi-step reasoning over long histories add inference latency compared with directly applying an LLM.The authors consider this trade-off acceptable for interactive recommendation rather than strict real-time responses.Source: D Discussion on Limitations and Future… · pp. 25–26
- Future environments could add actions such as expanding descriptions, searching for information, and completing checkout beyond browsing, comparing, and purchasing.The authors propose that these actions would provide richer signals for preference modeling.Source: D Discussion on Limitations and Future… · p. 26
- Planned evaluations include more OCR models and platforms, additional chunk sizes, Multi-Mem GRPO comparisons, and further agent and larger-model baselines.The listed chunk sizes are 1K, 2K, 10K, and 20K tokens.Source: D Discussion on Limitations and Future… · p. 26