PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Effective Structured Context Modeling for Conversational Recommender Systems via Dual-node Monte Carlo Tree Search

Jincheng Zhang, Chen Huang, Wenqiang Lei, See-Kiong Ng, Yang Deng

arXiv:2609.00618v2cs.IRcs.AI

TL;DR

CRSs need better conversational context modeling to track preferences as they evolve and support both elicitation and exploitation. DREAMS uses structured preference-state trees with MCTS-based elicitation and exploitation-based retrieval refinement. On benchmark evaluations, it improves recommendation accuracy and both preference-processing stages, while evaluation scope and backbone limitations remain.

  • Problem

    Existing CRSs often use free-form conversational context, causing information overload for elicitation and retrieval noise for exploitation.

  • Method

    DREAMS jointly models preference elicitation and exploitation over evolving structured conversational states using dual node types and MCTS.

  • Results

    +7.43% average recommendation accuracy, with +9.35% preference elicitation and +4.00% exploitation improvements, is reported on two benchmark datasets.

  • Takeaways & Limitations

    Effective conversational recommendation requires structured representations and structured search over evolving conversational states.

  • Takeaways & Limitations

    Evaluation is limited to GPT-4o-mini and Gemini-2.5-flash backbones, and DREAMS may inherit their social biases and incomplete domain knowledge.

Abstract

from arXiv · show

We investigate the role of conversational context modeling in user preference tracking for Conversational Recommendation Systems (CRSs). In this regard, we propose DREAMS, a novel tree-structured context modeling framework that explicitly captures user preference evolution throughout multi-turn interactions. DREAMS introduces two specialized node types to support the two fundamental objectives of CRSs: preference elicitation and preference exploitation. Specifically, elicitation nodes leverage Monte Carlo Tree Search (MCTS) to strategically explore conversational actions and infer latent user preferences, while exploitation nodes employ LLM-based refinement to transform the tracked preference state into structured retrieval queries for recommendation. Extensive experiments on benchmark datasets demonstrate the effectiveness of DREAMS and its design.

1 Introduction

CRSs must track evolving likes and dislikes to support both preference elicitation and exploitation. DREAMS addresses shortcomings of free-form context modeling with structured conversational states and dual-node search, improving benchmark performance.

  • CRSs use multi-turn conversations to elicit preferences and then recommend relevant items from them.
  • Negative preferences, such as disliking Spike Lee’s style, must persist across turns to prevent repeated recommendations and rejection.
  • Free-form context causes information overload and untargeted questioning during elicitation, while irrelevant content adds retrieval noise during exploitation.
  • Existing JSON trackers miss dependencies among preference states, whereas prior MCTS nodes lack explicit preference semantics and jointly structured elicitation and exploitation.
  • DREAMS represents evolving preference states as JSON-structured tree nodes and uses MCTS elicitation nodes plus exploitation nodes that form structured retrieval queries.
  • +7.43% average recommendation accuracy, with +9.35% elicitation and +4.00% exploitation improvements, is reported on two benchmark datasets.

2 Related Work

Prior LLM-based CRS work addresses preference tracking through dialogue decision-making for elicitation and semantic representations for exploitation, while this area remains challenging.

  • Preference tracking requires maintaining an accurate representation as user preferences are progressively revealed across multi-turn interactions.
  • Prior systems decide when to ask or recommend for elicitation and encode conversational history for downstream retrieval and recommendation.

3 Preliminary Study

The preliminary study evaluates CRS action selection, follow-up questioning, retrieval failures, and overall recommendation success using simulator-based interactions. Its findings show persistent errors across existing representation and planning approaches.

  • Evaluation Setup: LLM-based user simulators interact with CRSs until recommendation acceptance or a maximum turn limit, after which conversation histories are evaluated.
  • Evaluation Setup: CGE2 measures wrong ask-versus-recommend choices, FGE2 measures ineffective follow-up questions, and PE2 measures retrieval failures despite clarified preferences.
  • Experimental Findings: 0.570 CGE2 for InterCRS indicates difficulty deciding when to ask rather than recommend.
  • Experimental Findings: 0.333 FGE2 and 0.461 CGE2 for RA-CRS show that structured JSON contexts alone do not eliminate elicitation errors.
  • Experimental Findings: 0.240 PE2 for InterCRS indicates that explicitly stated preferences are not effectively converted into retrieval cues.
  • Experimental Findings: Neither JSON-style representation nor generic MCTS planning is sufficient; CRS requires state tracking, action selection, and preference-grounded retrieval together.

4 DREAMS

DREAMS models conversational recommendation as search over evolving structured preference states, jointly supporting preference elicitation and exploitation through complementary node types.

  • DREAMS Formalization: DREAMS formulates preference tracking as search over structured conversational states, with nodes encoding evolving preferences and edges representing state transitions.The framework actively searches structured states rather than using them only as static turn-independent memory.
  • ELNode for Preference Elicitation: ELNodes update preference states and search over actions for questioning, failure reflection, or transitioning to recommendation.Their states explicitly represent positive, negative, missing, and corrected preferences in JSON-compatible form.
  • ELNode for Preference Elicitation: MCTS selects actions by balancing accumulated rewards, visit counts, exploration, and LLM-provided action priors.Expansion uses LLM priors to retain promising actions, while simulation estimates long-term effectiveness using attitude, information, and turn-penalty rewards.
  • EXNode for Preference Exploitation: When recommendation is selected, EXNodes convert tracked preference states into retrieval-friendly queries through cumulative-state summarization and iterative refinement.Refinement can remove redundant context, reorder attributes, and convert informal preferences into machine-readable constraints.
  • EXNode for Preference Exploitation: EXNodes retrieve and score items for each refined query, selecting the highest-scoring query for final recommendation generation.This connects evolving preference tracking to exploitation without retrieving directly from noisy dialogue history.

5 Experiments

Experiments evaluate DREAMS against pretrained, LLM-based, and MCTS-based CRS baselines on benchmark datasets, including overall performance, preference tracking, ablations, and query refinement.

  • Main Results: DREAMS achieves the best performance across all datasets and metrics in Table 2, outperforming ChatCRS in average R@1 and SR by 8.57% and 9.07%.It also substantially surpasses SAPIENT-LLM and T-EPL in SR.
  • Main Results: DREAMS yields lower elicitation and exploitation errors than all baselines, reflected in superior CGE2, FGE2, and PE2 scores.Lower CGE2 indicates fewer coarse-grained action errors, while lower FGE2 and PE2 indicate stronger fine-grained modeling and exploitation.
  • Main Results: DREAMS maintains clear preference-tracking and recommendation advantages across Gemini-2.5-Flash and GPT-4o-mini backbones.InterCRS continues to show high elicitation errors despite Gemini’s long-context capability.
  • Ablation Studies: Removing both ELNode and JSON causes the largest degradation, while removing ELNode or replacing MCTS with direct LLM reasoning also worsens preference-tracking performance.The ablations support the roles of structured states and search over evolving conversational states.
  • Query Refinement: EXNode progressively produces queries with higher cosine similarity to ground-truth item embeddings than original queries, whereas random paraphrases are unstable.The comparison highlights the role of structured query refinement in reducing contextual noise during retrieval.
  • Efficiency Analysis: DREAMS(EA) reduces inference latency to 9s while maintaining competitive recommendation performance by retrieving structurally similar successful or failed experiences.The experience-augmented variant uses stored MCTS trajectories to warm-start decisions and bypass expensive online tree search.

6 Conclusion

DREAMS treats structured context modeling as central to conversational recommendation and jointly optimizes preference elicitation and exploitation through dual-node MCTS.

  • DREAMS uses structured representations of evolving conversational states rather than relying only on free-form context.
  • The framework jointly optimizes preference elicitation and preference exploitation within a dual-node MCTS design.
  • The authors argue that effective conversational recommendation requires both structured context and structured search over evolving states.

Limitations

The evaluation is limited to two LLM backbones, while broader CRS assessment relies on potentially biased LLM-based judges or simulators and remains untested across varied domains.

  • Experiments limit all methods to GPT-4o-mini and Gemini-2.5-flash because of computational and budget constraints.The study does not evaluate other state-of-the-art LLMs such as Claude Opus.
  • DREAMS may inherit social biases and incomplete domain knowledge from its LLM backbone.The framework is intended to improve preference tracking and recommendation rather than remove these model-level limitations.
  • Scalable CRS evaluation protocols may inherit biases from LLM-based judges, prompt design, and predefined user-preference ontologies.The paper calls for more human-grounded benchmarks, behaviorally validated simulators, and evaluator-independent protocols.
  • Generalization to domains with different item structures, knowledge requirements, or decision costs remains untested.

A.6.1 Dialogue-level Metrics

The evaluation defines dialogue-level errors for failures in adapting to rejected recommendations and exploiting fully elicited preferences, alongside turn-level action categories and normalized error ratios.

  • Fine-Grained Elicitation Error: FGE2 measures whether the assistant adapts subsequent elicitation or exploitation after explicit negative user feedback.A failure occurs when rejected content or its stated reason does not inform the follow-up action.
  • Preference Exploitation Error: PE2 occurs when genre, actor, and director preferences are fully elicited but the recommended item is not retrieved correctly.The criterion requires all three preference dimensions to be acquired before evaluating retrieval success.
  • Coarse-Grained Elicitation Error: CGE2 evaluates assistant utterances using four intent-relevance classes: Ask-True, Ask-False, Recommend-True, and Recommend-False.The classes distinguish asking versus recommending from whether the associated retrieval result is relevant.
  • Ratio Calculation: Normalized ratios r_x compare the distribution of the four action-relevance classes independently of total utterance counts.This normalization supports comparisons across models or dialogue scenarios with unequal numbers of utterances.
  • Coarse-Grained Elicitation Error: Ask-True represents unnecessary elicitation when retrieval is sufficient, whereas Recommend-False represents premature recommendation when retrieval remains incorrect.Here, True and False describe retrieval relevance rather than action correctness.

A.6.3 Prefix-level Next Action Evaluation

Prefix-level evaluation compares state representation, state-guided decisions, and structured search through next-action accuracy and premature recommendation rate.

  • Prefix-level Next Action Evaluation: DREAMS w/Json Only raises Next-Action Acc. from 0.542 to 0.633 and lowers Premature Rec. Rate from 0.308 to 0.217 versus RA-CRS.The comparison attributes the change to heuristic use of the structured dialogue state.
  • Prefix-level Next Action Evaluation: Full DREAMS further improves all reported metrics by replacing heuristic state use with structured search.The passage frames this as a progression from state representation to state-guided decision making and then structured search.

A.7 Implementation of LLM Evaluation

The LLM evaluation implementation uses documented prompts for the FGE2, PE2, and CGE2 metrics, with the relevant materials provided in the appendices and figures.

  • LLM Evaluation Prompts: Prompts for evaluating FGE2, PE2, and CGE2 are provided in Figures 6 and 7.The passage identifies the prompt locations rather than describing their contents.
  • Evaluation Materials: Table 8 includes a panel reporting annotation reliability alongside prefix-level next-action evaluation.Its caption distinguishes DREAMS without structured search from the annotation-reliability panel.

A.8 Implementation of Human Evaluation and Judge Reliability

Human evaluation uses controlled interactions between crowdworkers and CRS models, while agreement analyses assess the reliability of human labels and the LLM judge.

  • Human Evaluation: 60 crowdworkers with diverse personas compare DREAMS against PC-CRS, ChatCRS, and MACRS on movie recommendation tasks.Annotators interact independently with all four CRS models under the ReDial dataset.
  • Human Evaluation: The evaluation collects Success Rate, Recall@1, FGE2, and PE2, with Fleiss’ Kappa reaching 0.71.The reported agreement supports the reliability of the human verification process.
  • Judge Reliability: Krippendorff’s alpha is 0.74 for Elicitation and 0.68 for Exploitation, indicating substantial annotator agreement.LLM-human consistency is additionally assessed using Spearman’s correlation and utterance-level macro-F1.
  • User Simulation: LLM-based user simulation replaces costly real-user interaction with controlled, reproducible conversations driven by predefined personas and preferences.The simulator performs feeling inference, insight generation, and response generation after each recommendation.
  • User Simulation: Dialogues end when a recommendation matches all preferred genres and at least one preferred actor and director, or when the turn limit is reached.This stopping rule supports systematic evaluation across diverse user profiles and conversational behaviors.
  • Resource Usage: DREAMS adds minimal memory overhead, with the GPT-4o-mini implementation using about 375MB even with additional embeddings.The tree primarily stores lightweight textual metadata rather than dense tensors.

B.1 Practicality and Efficiency Analysis

DREAMS addresses tree-search latency through experience augmentation while preserving recommendation quality and keeping memory use modest. Its efficiency depends on balancing iteration depth against inference time.

  • Experience augmentation: DREAMS(EA) uses an Experience Knowledge Base built from 50 held-out user simulators to warm-start strategic planning.The knowledge base records dialogue contexts, states, actions, rewards, and refined retrieval queries.
  • Experience augmentation: 9s average inference time and Recall@1 of 0.467 show that DREAMS(EA) supports efficient, effective real-time recommendation.DREAMS(EA) uses historical training-phase trees as experience to bypass deployment-time tree search.
  • Hyperparameter and computational trade-offs: Increasing elicitation iterations to 5 and exploitation iterations to at least 3 improves performance, but further increases yield diminishing returns or slight degradation.More iterations and greater simulation depth improve accuracy while reducing time efficiency.
  • Memory efficiency: Approximately 560MB total memory usage indicates that DREAMS and DREAMS(EA) introduce no substantial memory overhead.The systems primarily store lightweight textual metadata, with retrieval embeddings adding limited additional memory use.
Loading 2609.00618v2…