PaperlayerView exact revision on arXiv

Source-linked AI summary

rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation

Minh Hoang Nguyen, Tung Le, Huy Tien Nguyen

arXiv:2608.18952v1cs.IRcs.AIcs.CL

TL;DR

LLM recommendation reasoning is costly to repeat and difficult to reuse across ranking requests. rEDMRec distills that reasoning into editable experience memory for lightweight retrieval, improving HR@1 across three datasets and ten student backbones, including up to 13.3% improvement over GraphRAG on ML-1M.

  • Problem

    LLM recommenders lack a structured way to reuse explicit reasoning about user preferences, item fit, and hard negatives across future ranking requests.

  • Method

    rEDMRec distills teacher reasoning into four typed, editable memory channels that a lightweight student retrieves for ranking, with debate-based Add/Delete/Modify/Keep optimization.

  • Results

    Across three datasets and ten student backbones, rEDMRec improves HR@1 over zero-shot, few-shot, and RAG on every backbone and over GraphRAG on most, reaching 13.3% improvement over GraphRAG on ML-1M.

  • Takeaways & Limitations

    Channel utility is capacity-dependent: short-term context helps consistently, while long-term preference, item perception, and counterfactual channels vary across student capacities.

  • Takeaways & Limitations

    Performance does not consistently exceed GraphRAG because the approach assumes sufficient student instruction-following and capacity to use retrieved memory.

Abstract

from arXiv · show

Large language models can improve recommendation quality by reasoning explicitly over user history and candidate items - for example, extracting a user's preferences or explaining why one item fits better than another - rather than mapping history directly to a ranked list. This reasoning, however, is expensive to repeat on every ranking request and, once produced, is typically consumed once and discarded, leaving it neither reusable across future requests nor easy to inspect or correct as user tastes drift. Our insight is that reasoning does not need to be regenerated at every call if it can instead be compressed once into a compact, structured memory that a lightweight model retrieves from. We propose rEDMRec, which distills a teacher LLM's reasoning into four typed, editable experience channels - long-term preference, short-term context, item-perception, and counterfactual hard-negative comparisons - maintained by an LLM memory controller that performs Add/Delete/Modify/Keep operations and refines entries via K-agent debate. A lightweight student LLM then ranks candidates purely by retrieving from this memory, without invoking the teacher again, decoupling online inference cost from reasoning depth. Across ML-1M, Amazon Beauty, and Steam and ten student backbones, rEDMRec improves HR@1 over zero-shot, few-shot, and RAG on every backbone, and over GraphRAG on most backbones, with Impv up to 13.3% vs. the second-best baseline on ML-1M. Channel ablations show that short-term context is the only channel that helps consistently across capacity tiers, whereas long-term, item-perception, and counterfactual contributions are capacity-dependent (and can reverse on the strongest students); debate-based memory optimization lowers bank duplication by 7.4 percentage points while raising downstream HR@1 by up to +0.029 over six optimization epochs.

1. Introduction

rEDMRec addresses the cost–depth tension in reasoning-augmented recommendation by distilling teacher reasoning into an editable, four-channel experience memory that students retrieve for ranking. Across three datasets and ten student LLMs, it improves HR@1 over zero-shot, few-shot, and RAG on every backbone and over GraphRAG on most.

  • Motivation: Explicit reasoning can extract preferences, judge item fit, and contrast candidates with hard negatives, addressing opaque and brittle ranking from raw histories or item titles alone.The motivating example distinguishes stable family/animation preferences, short-term context, item perception, and hard-negative contrasts when ranking Frozen against The Dark Knight.
  • Motivation and method: The method separates expensive, infrequent reasoning compression from cheap, frequent retrieval-and-rank inference by using a teacher offline and a frozen student online.This directly targets the tension between reasoning depth and inference cost in existing prompting, RAG, and per-request reasoning approaches.
  • Empirical results: 13.3% Impv over GraphRAG is achieved on Qwen2.5 3B, while rEDMRec improves HR@1 over zero-shot, few-shot, and RAG on every backbone and GraphRAG on most.The GraphRAG exceptions are Llama 3.1 8B and GPT OSS 20B; evaluation covers ML-1M, Amazon Beauty, Steam, and ten 3B–20B student LLMs.
  • Method: rEDMRec stores teacher reasoning in four independently retrievable and editable channels: long-term preference, short-term context, item perception, and counterfactual hard-negative comparisons.Figure 1 depicts offline typed lt/st/ip/cf entries and online student retrieval without calling the teacher.
  • Method: An LLM memory controller revises the experience bank with Add/Delete/Modify/Keep operations, using ranking reward models and optional K-agent debate after student predictions.The controller performs memory optimization across epochs rather than updating the memory through gradient-based changes.

2. Related Work

Prior recommendation models captured multi-scale interests, review semantics, and cold-start signals in parameters or corpora, while recent LLM work adds prompting, reasoning, distillation, retrieval, external memory, and iterative critique. rEDMRec is positioned at their intersection by turning teacher reasoning into persistent, editable experience rather than repeatedly reasoning per case.

  • Neural and signal-based recommendation: Pre-LLM recommenders modeled long- and short-term interests, review-based item semantics, and cold-start behavior, but stored these signals in parameters or corpora instead of editable experience banks.Co-NAML-LSTUR separates durable and recent user representations, RRS incorporates review text, and ViHoRec studies sparse temporal cold-start interactions.
  • LLM recommenders: LLM recommender research spans prompting, tuning, collaborative fusion, generative recommendation, and system-enhancement paradigms, including natural-language ranking and alignment methods.Surveys organize the field into these paradigms, while prompting methods frame ranking as generation or zero-shot ordering.
  • Reasoning, distillation, and retrieval: Reasoning and distillation methods use teacher-generated profiles, rationales, or refined knowledge, but existing approaches remain per-case loops or compress signals without a persistent, editable experience bank.ReasoningRec, R4ec, RDRec, POD, and LEADER represent these reasoning and distillation directions, while RAG and GraphRAG retrieve raw histories or graph summaries without teacher-compressed experience.
  • Memory and iterative refinement: External-memory and iterative-critique research supplies mechanisms for persistent retrieval and Add/Delete/Modify/Keep updates, but recommendation work has mainly refined individual outputs or knowledge artifacts.MemGPT and Generative Agents maintain external memory, Training-Free GRPO uses editable non-parametric libraries, and Self-Refine, debate, and R4ec iteratively critique outputs or knowledge.

3. Method

rEDMRec separates offline teacher reasoning from online recommendation by storing distilled reasoning in an editable, typed experience memory that a frozen student retrieves deterministically. Its design combines four evidence channels, shared candidate–memory encoding, and post-prediction debate-based updates.

  • Overall architecture: rEDMRec factorizes recommendation into offline construction of a typed memory and online teacher-free lookup, so reasoning is paid during writing while the frozen student only retrieves.Memory edits are produced offline by teacher extraction and distillation or online through debate-based optimization after student prediction.
  • Teacher extraction and memory control: A teacher extracts four routed signals, Adapt normalizes them into channel-indexed entries, and the controller commits Add/Delete/Modify/Keep operations to the memory.Preference extraction updates long-term and short-term states over history batches, while optional multi-agent refinement precedes distillation; the full optimization later revises committed entries across all channels.
  • Candidate filtering and retrieval: Candidate filtering uses lightweight recency, popularity, and retrieval scores, while one shared sentence encoder maps candidates, users, and memory entries into the same retrieval space.This avoids a learned projector and a separate alignment step, enabling direct memory queries with candidate or user vectors.
  • Typed experience memory: The memory stores long-term preference, short-term context, item perception, and counterfactual hard-negative evidence as separate channels for retrieving decision-relevant information.The channels represent distinct ranking signals and allow retrieval to select the evidence class needed for a decision.
  • Experience memory optimization: After each student prediction, reward signals and K-agent debate propose revisions that are adapted and applied to the memory, allowing future retrievals to improve while the student remains fixed.This optimization addresses generic or conflicting entries by using prediction–target mismatch as a revision signal.

4. Experimental Setup

The experiments span explicit- and implicit-feedback recommendation datasets, compare rEDMRec with prompting and retrieval baselines, and evaluate frozen student backbones under standardized ranking metrics and full held-out testing.

  • Datasets: Experiments cover ML-1M, sparse Amazon Beauty, and implicit-feedback Steam, spanning explicit ratings, review text, and interaction data.ML-1M uses 1–5-star ratings, Amazon Beauty combines ratings with reviews and weak per-user collaborative signal, while Steam records implicit feedback.
  • Baselines: Baselines include Zero-shot, Few-shot, RAG, and GraphRAG, contrasting prompting, raw-history retrieval, and item-graph retrieval with distilled reasoning.rEDMRec is formalized in Eqs. (2)–(4), while RAG retrieves raw interactions or reviews and GraphRAG uses an item co-occurrence or knowledge graph.
  • Models: A gpt-5.4-mini teacher supports evaluation across ten frozen open student backbones ranging from 3B to 20B parameters.Freezing students isolates cross-backbone gains attributable to the editable experience memory.
  • Metrics: Performance is measured with HR@1, HR@5, HR@10, NDCG@5, NDCG@10, and MRR over fixed 20-candidate evaluation sets.RQ1 additionally reports Impv (%) against each student’s second-best baseline and McNemar p-values for HR@1.
  • Evaluation protocol: Every model–method–dataset cell uses the full chronological held-out test split with 20 candidates per sample and sampling seed 42.The held-out sizes are n_ML−1M=49893, n_Beauty=1460, and n_Steam=1460; significance marks p<0.05 when rEDMRec leads.

5. Results

rEDMRec improves ranking across datasets and student backbones, with the largest gains for weaker students, while channel usefulness depends on student capacity. Better teacher banks and debate-based optimization further improve memory quality and downstream ranking, though gains eventually saturate with additional debating agents.

  • 5.1 Overall Results: rEDMRec improves HR@1 over Zero-shot, Few-shot, and RAG for every evaluated ML-1M student, with a maximum 13.3% gain over GraphRAG for Qwen2.5 3B.Table 4 covers four representative students; the full ten-model comparison appears in Appendix A.
  • 5.1 Overall Results: On Amazon Beauty and Steam, rEDMRec improves HR@1 over Zero-shot for all ten students, with Impv values of 23.6% and 21.5% for Qwen2.5 3B.NDCG@10 and MRR generally rise with HR@1, except beyond top-1 for the weakest Llama 3.1 8B backbone; the largest gains are significant at p<0.05.
  • 5.2 Channel Ablation: Short-term context is consistently useful, whereas long-term preference, item-perception, and counterfactual channels are capacity-dependent and can reverse on the strongest student.Removing long-term preference or item-perception improves HR@1 by +0.03 to +0.04 for the strongest student; removing counterfactuals hurts a mid-capacity student by Δ=−0.04 and has little effect on the saturated 120B student.
  • 5.2 Channel Ablation: As the memory bank grows from B=189 toward 10^5–10^6 entries, reversed channel effects change sign, indicating that bank density moderates redundancy and generic-entry harms.The qualitative before/after analysis further examines how persisted entries become more specific through memory optimization.
  • 5.3 Teacher Quality: Lower bank duplication generally predicts larger downstream gains, but teacher quality is not monotonic because student capacity limits how much additional teacher-bank information can be exploited.gpt-5.4-mini has 12.4% duplication and +0.060 ΔHR@1, while Llama 3.1 8B Instant has 22.8% duplication and +0.015; GPT OSS 120B has 9.8% duplication but only +0.040.
  • 5.4 Debate Optimization: Debate optimization reduces bank duplication from 18.0% to 10.6% and raises experience reward from 0.52 to 0.78, yielding +0.029 HR@1 after six epochs.A no-debate paraphrase control stays flat, while quality rises with debating agents and reaches its quality-per-cost knee at k*=4: +0.022 from k=1 to 4 versus only +0.006 from k=4 to 10.

6. Discussion and Limitations

The discussion identifies limits in operating-point validation, teacher–student coverage, explanation-faithfulness evaluation, backbone-dependent gains, and domain scope. These limitations motivate broader cross-product, faithfulness, operating-point, backbone, and domain testing.

  • Operating-point generalization: Performance should be re-confirmed at the exact production bank size or debate budget because trends across settings do not guarantee identical behavior at each operating point.Experiments span bank sizes up to 10^6 entries, six debate epochs, and ten debating agents.
  • Teacher coverage: Teacher-quality effects remain incompletely characterized because teacher variation was tested with only two students rather than the full teacher × student cross-product.The main results fix the teacher to gpt-5.4-mini, leaving uniformity across all ten students unresolved.
  • Explanation faithfulness: The paper evaluates ranking quality but not whether student-generated explanations are faithful to the retrieved memory, leaving human faithfulness and plausibility evaluation for future work.The student can emit short memory-grounded explanations, but their faithfulness was not measured.
  • Backbone-dependent returns: Returns depend on backbone instruction-following: Llama 3.1 8B has negative Impv vs. GraphRAG, while a near-saturated 20B student slightly trails GraphRAG.The method assumes students can follow prompts combining user context, candidate descriptions, and retrieved memory snippets.
  • Domain scope: The evaluation covers English-language movie, beauty-product, and game recommendation, leaving substantially different domains such as short-video and news recommendation untested.The four-channel schema, especially its counterfactual channel, assumes catalog items have stable, comparable attributes.

7. Conclusion

rEDMRec reuses LLM reasoning for recommendation by distilling teacher reasoning into a four-channel, editable experience memory that a lightweight student retrieves for ranking. Its architecture separates infrequent reasoning compression from frequent recommendation requests.

  • rEDMRec addresses the need to reuse, rather than repeat, LLM reasoning across recommendation requests.
  • The architecture distills teacher reasoning into a four-channel, editable experience memory for recommendation.
  • A lightweight student serves ranking requests by retrieving from the memory, separating frequent serving from infrequent teacher extraction, distillation, and debate-based optimization.

Data and Code Availability

The project provides preprocessing, training, and evaluation code plus experiment matrices for reproducing its tables and figures, while releasing only derived, de-identified artifacts for third-party datasets under their original licenses.

  • Reproducibility: The rEDMRec/ project root organizes preprocessing, training, evaluation code, and JSON experiment matrices used to produce every table and figure.The repository also includes a readme.md quick-start pipeline.
  • Data licensing: ML-1M, Amazon Beauty, and Steam remain under their original third-party licenses; the release contains only derived, de-identified interaction records and memory-bank artifacts.

CRediT authorship contribution statement

The authors contributed across methodology, conceptualization, writing, supervision, project administration, and supporting roles.

  • Minh Hoang Nguyen contributed methodology, conceptualization, and original-draft and revision writing.
  • Tung Le contributed supervision, supporting work, and writing–review and editing.
  • Huy Tien Nguyen contributed supervision, supporting work, conceptualization, and project administration.

A. Full Main-Results Table

The section provides the complete ML-1M main-results matrix for rEDMRec and four comparison methods across ten student models and six ranking metrics. Results use the full held-out test split with 20 candidates per sample and seed 42.

  • A. Full Main-Results Table: Table 12 covers ten student models, five methods, and six ranking metrics in the complete ML-1M main-results matrix.It underlies the main results reported in Table 4 and follows the protocol described in Section 4.4.
  • A. Full Main-Results Table: The evaluation uses the full held-out ML-1M test split, 20 candidates per sample, and seed 42.Best scores are bolded, second-best scores are underlined, and rEDMRec method labels are bolded.
  • A. Full Main-Results Table: Impv (%) measures rEDMRec’s relative HR@1 gain over the second-best baseline on the same student, while p reports the exact McNemar test value.The improvement is defined as (Ours − SecondBest)∕SecondBest × 100, and p compares rEDMRec HR@1 with that baseline.

B. Full Amazon Beauty Results … Reproducibility notes.

The appendix provides complete Amazon Beauty and Steam evaluation matrices, then specifies shared evaluation settings, implementation defaults, system components, and reproducibility protocols for the reported experiments.

  • B. Full Amazon Beauty Results: Table 13 provides the complete Methods × Models matrix for Amazon Beauty on the full held-out test split.The evaluation uses 20 candidates per sample and seed 42; the table defines Impv as rEDMRec’s relative HR@1 gain over the same-student second-best baseline.
  • C. Full Steam Results: Table 14 provides the complete Methods × Models matrix for Steam under the same full-test evaluation protocol.The table uses 20 candidates per sample and seed 42, with Impv defined relative to the same-student second-best baseline.
  • D. Experimental Settings: Unless otherwise stated, reported cells use the full held-out test split with 20 candidates per sample and candidate-sampling seed 42.Each sample contains one positive and 19 negative candidates.
  • D. Experimental Settings: Table 15 records the default hyperparameters for all reported runs, with the expanded implementation stack and knob table provided in Appendix E.The defaults are overridden only when a subsection explicitly states otherwise.
  • E. Implementation Details and Hyperparameters: Appendix E expands the defaults into concrete config.py values, which serves as the single source of truth for reported runs.Table 16 presents the expanded hyperparameters from config.py.
  • Implementation stack.: The system uses OpenAI-compatible chat calls for teacher, controller, and debate modules, while a frozen pretrained student ranks items by retrieving from the experience bank.Dense retrieval uses FAISS FlatIP with all-MiniLM-L6-v2 embeddings of dimension 384; counterfactual edges use Neo4j.
  • Implementation stack.: The implementation stack builds candidate sets offline with one positive and 19 negatives using seed 42, while debate optimization and controller ablations are summarized in Tables 17–19.The listed studies cover debate versus no-debate controls, controller states, and selected agent-count settings.
  • Reproducibility notes.: RQ1 uses chronological train/validation/test splits, while channel and debate studies vary configured memory flags, debate-agent counts, and optimization epochs with the student held frozen.These protocol choices define the reproducibility settings for the ablation and sweep experiments.

F. Debate and Controller Ablations · G. Additional Ablation Figures

The debate/controller ablations show that debate-driven refinement produces most gains by epochs 2–3, while a four-agent configuration offers the best quality–cost trade-off. Additional appendix figures extend channel ablations across MRR, numeric matrices, channel importance, and backbone-specific ΔHR@1.

  • F. Debate and Controller Ablations: The default controller uses k=3 debate agents, one round per epoch, and a single LLM arbiter.The appendix evaluates trajectories and agent-count variants around this configuration.
  • F. Debate and Controller Ablations: The appendix reports controller/debate trajectories with a no-debate control and a one-epoch sweep over the number of agents.These experiments support RQ4 in Section 5.4.
  • F. Debate and Controller Ablations: Full debate combines k-agent critique, an arbiter, and controller Add/Delete/Modify/Keep commits, whereas no-debate paraphrase only refreshes entry wording.The single-agent k=1 variant removes multi-agent disagreement and serves as the Table 9 anchor.
  • F. Debate and Controller Ablations: By epoch 2–3, most Mixtral gains are realized; beyond this point returns diminish, while the no-debate paraphrase control remains nearly flat at +0.007 HR@1.The control isolates the contribution of the critique-and-revise debate loop.
  • F. Debate and Controller Ablations: The agent-count sweep identifies k*=4 as the quality-per-cost knee, with fewer than 0.01 additional HR@1 from six extra LLM calls per case beyond four agents.This sweep is reported in the main-text Table 9.
  • G. Additional Ablation Figures: Additional ablation figures repeat channel comparisons under MRR, provide the full numeric matrix, aggregate channel importance, and decompose ΔHR@1 by ablation-panel backbone.These visuals support Section 5.2 and were omitted from the main text for space.

H. Dataset Statistics … H.3 Capacity-dependent channel reversals.

The appendices define the filtered datasets and evaluation protocol, document teacher-to-memory distillation and ranking traces, and analyze failures, risky edits, and capacity-dependent channel effects. Together, they show that backbone limitations and selective channel retrieval materially qualify rEDMRec’s behavior.

  • H. Dataset Statistics: ML-1M is dense with explicit ratings, Beauty is extremely sparse, and Steam uses implicit positives under chronological 0.8/0.1/0.1 splits with 20-candidate samples.Filtering uses k=20 for ML-1M and k=5 for Beauty and Steam.
  • I. Prompts and Outputs: The teacher prompts extract four structured channels—preference, context, item-perception reasoning, and counterfactuals—for retrieval by the student at ranking time.Prompt documentation highlights role assignments, inputs, chain-of-thought instructions, guardrails, and JSON fields; example outputs preserve ranking-relevant fragments.
  • J. Examples of rEDMRec-generated Predictions: Amazon Beauty and Steam traces show students ranking candidates from chronological histories plus retrieved entries from all four teacher-extraction channels, followed by concise rationales.The examples use public-catalog item titles and abbreviated candidate sets.
  • H.1 Ranking failures vs. GraphRAG.: On ML-1M, rEDMRec trails GraphRAG by 11.1% on Llama 3.1 8B and 3.3% on GPT OSS 20B, with similar near-zero or negative effects on Beauty and Steam.The limitation is attributed to weak instruction-following on concatenated memory prompts rather than an empty bank, making it backbone-dependent.
  • H.2 Qualitative edits: success vs. risk.: Persisted-bank edits include beneficial long-term rewrites, a risky taste flip from sci-fi to noir/crime, short-term compression, and item-grounded perception fixes.The cases distinguish success-like edits, risks or failure modes, and compression behavior.
  • H.3 Capacity-dependent channel reversals.: Removing long-term, item-perception, or counterfactual memory can improve HR@1 for gpt-5-mini, indicating that memory may inject noise when candidates already suffice.This behavior is illustrated by per-backbone channel-ablation analyses.
  • H.3 Capacity-dependent channel reversals.: Short-term context is the only consistently beneficial channel across capacity tiers, making retrieval of all four channels potentially unsuitable for saturated students.The appendix frames this as a practical deployment failure mode alongside backbone-specific reversals.
Loading 2608.18952v1…