PaperlayerView exact revision on arXiv

Source-linked AI summary

Two-Sided State-Space Models for Sequential Recommendation with Non-Random Multimodal Review Feedback

Ziwen Pan, Zihan Liang, Ruoxuan Xiong

arXiv:2609.00165v1cs.IR

TL;DR

Sequential recommendation has underexplored state-dependent review observation and review-driven item evolution. TS-SSM models both through multimodal observation-aware fusion, dynamic user and item states, local propagation, and asymmetric review carryover, improving Recall@20 over established baselines across reported domains.

  • Problem

    Review generation depends on evolving user and item states, while reviews can reshape item states and propagate through related items, but these dynamics are underexplored.

  • Method

    TS-SSM jointly models dynamic user and item states using modality-missing-not-at-random fusion, temporal and local graph user updates, and asymmetric positive-negative review carryover.

  • Results

    14.8%–18.8%: TS-SSM improves Recall@20 over BSARec across six Amazon categories and exceeds HM4SR by 11.7% on average.

  • Takeaways & Limitations

    Observation patterns, local propagation, and item dynamics each make distinct contributions to TS-SSM's reported performance.

  • Takeaways & Limitations

    Broader validation is needed on platforms with sparser or qualitatively different review behavior, and advantages may diminish with uninformative text, rare multimodal expression, or short histories.

Abstract

from arXiv · show

Two-sided digital platforms are inherently dynamic: user preferences shift, item popularity evolves, and reviews both reflect and drive these changes. Yet most sequential recommendation systems treat reviews as passive signals for updating user states, leaving two aspects underexplored. First, review generation is nonrandom, depending on evolving latent states of both users and items. Second, reviews can reshape item states, induce spillover across related items, and influence future user decisions. To address these gaps, we propose a two-sided state-space model (TS-SSM) for event-conditioned sequential recommendation. TS-SSM consists of three components: (1) a modality-missing-not-at-random fusion module that encodes review content and informative observation patterns; (2) user-state evolution with temporal variation and local graph message passing that uses related item states to refine user preferences; and (3) item-state evolution with asymmetric carryover of positive and negative review feedback. In experiments across six Amazon categories, TS-SSM increases Recall@20 over BSARec by 14.8%--18.8% and exceeds HM4SR by 11.7% on average. On Goodreads Fantasy, Recall@20 improves HM4SR from .5191 to .5847. Ablations highlight distinct contributions of observation patterns, local propagation, and item dynamics.

1 Introduction

TS-SSM addresses two underexplored dynamics in sequential recommendation: review observation depends on evolving user and item states, while reviews also reshape item states and affect related-item information. It models these processes through multimodal fusion, user-state propagation, and asymmetric item-state carryover, outperforming established baselines in reported evaluations.

  • Motivation: Review generation is state-dependent rather than passive, with modality availability and content related to evolving user and item states.Behavioral deviations, such as unusual image uploads or abrupt text-length changes, are associated with rating shifts.
  • Method: TS-SSM integrates review text and informative modality-observation patterns to jointly update dynamic user and item states.Its fusion module uses modality presence or absence and deviations from historical expression patterns.
  • Method: User-state evolution combines temporal variation with local graph message passing so related items inform preference updates.The propagation mechanism carries review-induced preference information through item relationships.
  • Method: Item-state evolution uses separate decay rates for positive and negative reviews, allowing their effects to persist asymmetrically over time.The carryover mechanism represents prior reviews as continuing influences on item states.
  • Results: 14.8%–18.8%: TS-SSM improves Recall@20 over BSARec across six Amazon categories and beats HM4SR by 11.7% on average.On Goodreads Fantasy, it improves over HM4SR by 12.6%.

2 Problem Formulation

The problem formulation represents a two-sided user–item system as evolving latent states driven by review events. It defines state-dependent multimodal observations, event-conditioned transitions, and ranking-based evaluation using pre-event histories and current reviews.

  • System and states: The system is a bipartite user–item system with coarse event-time bins and latent states for users and items.User and item sets are denoted by U and I, while each event time is assigned to a coarse bin.
  • Events: A review event occurs when user u reviews item i and includes multimodal content, observed-modality indicators, ratings, and numeric cues.The multimodal content may include titles, text, and images.
  • Observation model: Review observation can depend on pre-event user and item states through both modality availability and observed review content.This formulation treats observation patterns as state-dependent rather than fixed.
  • State transitions: At each event, user and item states evolve through transition functions using event information and global temporal contexts.Global contexts may represent seasonal user preferences or platform-wide and macroeconomic item shifts.
  • Evaluation: Ranking uses pre-event history and the current review to construct post-event user and candidate-item states, then evaluates the observed item's rank with Recall@K and NDCG@K.Higher Recall@K and NDCG@K indicate better ranking performance.

3 Method

TS-SSM is a two-sided state-space model that encodes review content, modality availability, and historical deviations, then updates and propagates user and item states for query-time ranking. Its method combines dynamic review encoding, within-user evolution, local graph message passing, and time-aligned item scoring.

  • 3.1 Multimodal Review Encoding: Historical-deviation encoding compares each review with user and item histories to capture changes in expression patterns.User and item deviation signals can influence how strongly the current review changes their states.
  • 3.1 Multimodal Review Encoding: TS-SSM encodes review content, ratings, numeric cues, modality availability, and static or temporal context into event representations.The observation pattern itself contributes information beyond multimodal content fusion.
  • 3 Method: Figure 2 organizes the pipeline into encoding, state updates with asymmetric item-side carryover, graph-based user refinement, and query-time candidate ranking.Candidate item states are aligned to the query time before scoring.
  • 3.2 User-State Evolution: The user-state update combines the pre-event user state, current and historical reviews, systematic temporal variation, and history-relative innovation.A bounding operator limits excessively large changes while preserving the proposed update direction.
  • 3.2 Local Graph Message Passing: Local graph message passing propagates information from the user’s most recently reviewed item to related items and back to refine user preferences.The procedure uses the recent item as an anchor and controls both propagation updates with learned gates.

4 Experiments

Experiments evaluate TS-SSM across Amazon and Goodreads settings under common protocols, showing consistent gains over sequential and multimodal baselines. Ablations indicate distinct contributions from item-state updating, message passing, review text, and observation-pattern modeling.

  • Main Results: TS-SSM is best on all twelve reported metrics across Toys, Pets, and Sports, with Recall@20 gains of 17.9%, 17.1%, and 14.8% over BSARec.The comparisons use a shared preprocessing and evaluation protocol across methods.
  • Main Results: Adding current-event modality availability, rating, and time-bin features to BSARec improves it by at most 4.7%, leaving a 9.0%–11.3% gap to TS-SSM.The control tests the explanatory contribution of these metadata fields under the same setting.
  • Main Results: TS-SSM achieves Recall@20 gains of 14.8%–18.8% over BSARec across six Amazon categories and 11.7% over HM4SR on average.On Goodreads Fantasy, it reaches .5847 versus HM4SR’s .5191, a 12.6% relative gain.
  • Ablation Study: Removing item-state updating causes the largest Goodreads ablation drop at 6.81%, followed by message passing at 4.70%.The same component ordering is preserved on Amazon, where item-state update and message passing produce the largest drops.
  • Ablation Study: On Amazon, removing Item State Update, Message Passing, and User State Update lowers Recall@20 by 3.50%, 2.79%, and 2.50%, respectively.Carryover Memory, Deviation Features, and Auxiliary Losses reduce performance by 2.20%, 1.90%, and 1.31%.
  • Ablation Study: Removing dynamic event-level review text lowers Recall@20 by 2.67%, while removing all text lowers it by 6.30%.Replacing MNAR gating with indicators causes a 2.97% decrease, supporting the role of observation-pattern-aware encoding.

5 Related Work

Related work spans sequential recommendation, multimodal modeling, non-random modality availability, and causal recommendation. TS-SSM combines these perspectives by modeling behavior-dependent review expression and two-sided state evolution.

  • Sequential and Multimodal Recommendation: Sequential recommendation includes recurrent, convolutional, Transformer, and foundation-model approaches.These methods provide the sequential modeling context for TS-SSM.
  • Sequential and Multimodal Recommendation: Multimodal methods exploit modality content, while missing-modality methods impute or marginalize absent inputs.TS-SSM instead treats modality availability and expression as behavior-dependent signals.
  • Two-Sided State Modeling: TS-SSM transfers non-random modality-use modeling to a two-sided platform-review setting with jointly evolving user and item states.This extends prior work that used non-random modality availability in clinical multimodal models.
  • Item Dynamics and Bias Correction: Unlike TPNE’s time-dependent excitation, TS-SSM retains signed review shocks in item-side memory with independently learned decay.Its predictive correction is distinguished from causal recommenders and resembles hierarchical decomposition.

6 Conclusion

TS-SSM unifies dynamic review encoding, observation-pattern-aware propagation, joint user–item updates, and asymmetric carryover memory. It improves Recall@20 across Amazon and Goodreads, while ablations support distinct component roles.

  • Conclusion: TS-SSM combines event-level review encoding, local propagation, joint user–item updates, and asymmetric carryover memory in one state-space framework.The framework is designed for event-conditioned sequential recommendation.
  • Conclusion: Recall@20 improves over BSARec by 14.8%–18.8% across six Amazon categories and over HM4SR by 11.7% on average.The Goodreads Fantasy improvement over HM4SR is 12.6%.
  • Conclusion: Sensitivity analyses and ablations support distinct roles for observation patterns, local propagation, item dynamics, and other model components.The conclusion reports component-level support without attributing all gains to a single mechanism.

7 Limitations

TS-SSM’s validation is limited to six Amazon categories and one Goodreads domain, so broader review settings remain untested. Its MNAR formulation is predictive rather than causally identified, and the model adds computation and storage costs.

  • Validation covers six Amazon categories and one Goodreads domain, leaving platforms with sparser or qualitatively different review behavior for future evaluation.
  • TS-SSM’s advantage may diminish when text is uninformative, multimodal expression is rare, or user histories are too short to estimate deviations reliably.
  • The MNAR formulation is predictive rather than causally identified because unobserved factors may also shape review absence.
  • TS-SSM requires more computation, storage, and update costs than lightweight user-only sequential models, although bounded propagation and ANN retrieval can limit online ranking work.
  • The item–item graph uses fixed-window co-occurrence, while auxiliary targets for drift, carryover, and reliability are predictive proxies rather than externally validated latent variables.

Use of Artificial Intelligence

The supplied passages state that language models assisted only with prose polishing. The remaining passages describe datasets, experiments, implementation details, and appendix analyses rather than additional AI use.

  • LLM-based writing assistants were used solely to polish the paper’s prose.
  • The authors state that technical content, experimental design, results, and conclusions were entirely their own work.
  • The other supplied materials concern datasets, preprocessing, model settings, experiments, and supplementary analyses.

B.2 Abstract-to-Implementation Correspondence

The implementation instantiates abstract user and item filtering maps as chronological event updates. It weights historical evidence, computes signed item carryover, aligns candidate states to query time, and retains event-derived scores for later use.

  • Reviews are processed chronologically using lagged statistics, current user corrections, and post-event item states carried into later events.
  • The item filtering map aggregates current and prior event information through gated innovation and carryover memory.
  • Stored event scores retain multimodal content and history deviations, while later queries change only their time-dependent weights.
  • Observation, evidence, reliability, and time factors weight user-history messages, with observation weights learned separately for modality-availability patterns.
  • The user filtering map combines bounded state updates, weighted history aggregation, current-event correction, and local feedback.
  • Positive and negative review feedback use independently learned decay parameters, with the fitted checkpoints retaining negative shocks longer than initialization alone would imply.
  • For ranking, non-target item states are aligned to query time, while the observed target uses its newly updated post-event state.

C Behavioral Deviation Analysis

Behavioral deviations from a user’s usual review expression correlate with rating patterns in the Toys & Games training data. Image uploads and unusually long text are associated with less favorable ratings.

  • Users with unusually rare image uploads had a mean rating of 4.01 versus 4.26 for non-deviating reviews.The difference was statistically significant, with p < 10^-6.
  • Rare image uploads were significantly more likely to accompany ratings of 1 or 2 stars.
  • Longer-than-usual reviews contained fewer 5-star ratings, 64.8% versus 74.8%, and more 1-star ratings, 3.3% versus 1.5%.
  • TS-SSM outperformed BSARec, MACR, and AutoDebias on all reported metrics across three additional Amazon categories.

E Cross-Platform Evaluation on Goodreads Fantasy

TS-SSM generalizes beyond Amazon to Goodreads Fantasy, where it achieves the strongest reported performance across four metrics. Cross-platform robustness analyses also show consistent gains under repeated evaluation, while computational costs remain higher than lightweight sequential baselines.

  • TS-SSM achieves the highest value on all four Goodreads Fantasy metrics.
  • Recall@20 increases from .5191 for HM4SR to .5847, a 12.6% relative improvement.
  • Component ablations: The cross-platform ablation ordering matches Amazon results: removing item-state evolution causes the largest drop, followed by message passing and user-state evolution.
  • Statistical robustness: All 12 Pet Supplies comparisons remain significant after Bonferroni correction across 10 trials, with effect sizes dz from 9.1 to 16.9.
  • Computational cost: TS-SSM is more expensive than SASRec and BSARec, requiring dynamic item states and local message passing, but records 197.3 seconds per epoch and 8.6 ms per query.

J Extended Ablation Study

Extended ablations separate the contributions of asymmetric carryover, learned gating, deviation signals, and encoder choices. Results support persistent item memory, observation-pattern-aware weighting, and stable auxiliary regularization, while larger encoders provide modest gains.

  • Carryover decay: The full asymmetric carryover model reaches .1683 Recall@20, exceeding symmetric carryover at .1657 and no carryover at .1646.The reported recovery percentages are descriptive because the ablations are non-additive.
  • Gating and grouping: Learned gating outperforms every fixed interpolation, while the train-only three-group partition outperforms G = 1 and alternative grouping criteria.
  • Deviation signals: Removing both behavioral-deviation channels produces a smaller drop than summing their marginal drops, indicating partial substitutability.
  • Sensitivity analysis: The largest reduction across fifteen nondefault auxiliary-loss settings is 1.43%, with central settings remaining within 1% of default.
  • Encoder sensitivity: Increasing text-encoder capacity approximately 15× improves Recall@20 by at most 2.14%, while larger image encoders yield gains of 0.71% and 1.27%.
  • MNAR gating: Replacing learned MNAR gating with binary indicators reduces Recall@20 by 2.97%, while removing all observation-pattern signals reduces it further.

O.3 Results

The results show that dynamic, event-level review text contributes materially beyond static item-level representations, while reliability gating and item-state dynamics shape behavior across sparse-history and shock scenarios. Error analysis identifies false deviation as the dominant failure mode.

  • Dynamic review text: Dynamic per-event text achieves .1683 Recall@20, compared with .1638 for static item-level text, a 2.67% reduction relative to the full model.
  • Dynamic review text: Removing all text reduces Recall@20 to .1577, corresponding to a 6.30% decrease.
  • Reliability gating: Mean reliability-gate activation increases from 0.26 for the smallest history bucket to 0.75 for histories of 21–50 events.The smallest bucket includes empty-history first events.
  • History and item sparsity: TS-SSM’s larger relative gain on long-tail items suggests that item-side temporal state and reliability-weighted propagation recover signal when item evidence is sparse.
  • Case analysis: For a high-deviation negative review, the full model moves two same-subcategory items down three rank positions in favor of alternatives with higher standalone quality signals.
  • Error analysis: The dominant error mode is false deviation, in which unusual behavior is over-trusted despite lacking predictive value for the next ranking decision.

T.3 Handling Missing Data in Recommendations

TS-SSM treats review and modality observation patterns as potentially informative rather than uniformly missing, while retaining an associational interpretation. The paper also notes computational, validation, and generalization boundaries for its broader claims.

  • Interpretation: The paper treats modality use and expression patterns as predictive features, without assuming latent preference is their sole cause.
  • Failure modes: The reported error taxonomy identifies false deviation as the dominant failure mode, where unusual behavior is over-trusted despite lacking predictive value.
Loading 2609.00165v1…