Source-linked AI summary
LSF-SR: Latent Semantic Fusion for Sequential Recommendation via Flow-based Conditional Variational Autoencoders
Shih-Hong Chen, Josh Jia-Ching Ying, Vincent S. Tseng
TL;DR
Sparse histories and cold-start items make interaction-derived representations difficult to learn, while aligning collaborative and LLM semantic signals remains a challenge. LSF-SR uses a flow-based conditional variational autoencoder to fuse these signals. Across five benchmark datasets, it consistently outperforms state-of-the-art methods, with gains up to 12.98% in Recall@20 and 14.13% in NDCG@20.
Problem
Sparse interactions and cold-start items leave collaborative representations unreliable, and existing methods struggle to align collaborative signals with LLM-derived semantic knowledge.
Source: §1 · pp. 1–2Method
LSF-SR uses a conditional variational autoencoder with normalizing flows to fuse collaborative item representations conditioned on LLM-derived semantic embeddings.
Source: §1 · p. 2, §4.3 · p. 5Results
Across five benchmark datasets, LSF-SR consistently outperforms state-of-the-art methods, with gains up to 12.98% in Recall@20 and 14.13% in NDCG@20.
Source: §1 · p. 3Takeaways & Limitations
Normalizing flows provide a more expressive posterior than a diagonal Gaussian, while excessive flow depth can moderately reduce recommendation quality.
Source: §4.3 · p. 5, §5.5.3 · p. 9
Abstract
from arXiv · showhide
Sequential recommendation aims to predict users' future interests from their historical interactions. Although Large Language Models (LLMs) capture rich item semantics, existing methods often struggle to align collaborative signals with textual semantic knowledge. As a result, the learned item representations fail to capture the complementary strengths of both signals, leading to suboptimal recommendation quality. To address this limitation, we propose Latent Semantic Fusion for Sequential Recommendation via Flow-based Conditional Variational Autoencoders (LSF-SR), a novel framework that uses a Conditional Variational Autoencoder (CVAE) with Normalizing Flows to fuse item ID embeddings and LLM-generated semantic signals. At the core of LSF-SR is a conditional fusion module augmented with planar or radial flows. This module learns a flexible latent space that encourages items with similar semantic profiles to cluster together within the latent manifold. Through extensive experiments on five public benchmark datasets, we demonstrate that LSF-SR consistently outperforms state-of-the-art baselines, achieving gains of up to 12.98% and 14.13% in Recall@20 and NDCG@20, respectively.
1 Introduction
Sequential recommenders rely on interaction-derived item representations, which become unreliable with sparse histories and cold-start items. LSF-SR addresses the mismatch between collaborative and LLM-derived semantic representations through probabilistic fusion, and reports consistent gains over state-of-the-art methods on five benchmarks.
1 Introduction
- Sparse interaction data weakens learned item embeddings, while new or rarely interacted items lack enough behavioral signals for reliable collaborative representations.These conditions make item representation difficult in extensive, long-tail catalogs.Source: §1 · p. 1
- Side-information fusion and contrastive learning can improve behavioral representations, but neither addresses the lack of semantic item knowledge in sparse collaborative signals.Metadata may also be fragmented or missing, limiting methods that assume complete attributes.Source: §1 · pp. 1–2
- LLM semantic embeddings and collaborative embeddings have disjoint neighborhood structures, so their top-K neighbors often differ and late fusion may fail to align them.Figure 1 analyzes cosine-similarity distributions on Amazon Beauty and reports severe misalignment.Source: §1 · p. 2, Figure 1 · p. 2
- LSF-SR uses a CVAE conditioned on LLM-derived semantic embeddings to probabilistically transform collaborative item representations into a shared latent space.The formulation accounts for uncertainty when aligning heterogeneous representation spaces.Source: §1 · p. 2
- On five benchmark datasets, LSF-SR consistently outperforms state-of-the-art methods, with gains up to 12.98% in Recall@20 and 14.13% in NDCG@20.The framework restricts LLM use to item-level offline processing, avoiding real-time user-level profiling overhead.Source: §1 · p. 3
2 Related Work
The related work covers behavioral sequential recommendation and LLM-augmented recommendation. LSF-SR differs from prior strategies by treating behavioral and textual fusion as probabilistic inference and reshaping the posterior with normalizing flows.
2 Related Work
- The review frames LSF-SR against two threads: behavioral sequential recommendation and LLM-augmented recommendation.The former underpins behavioral modeling; the latter motivates semantic fusion.Source: §2 · p. 3
2.1.1 Attention Mechanisms and Frequency-Domain Enhancements.
- Attention models weight historical interactions, while frequency-domain methods filter short-term variations to preserve preference signals.These approaches address sequence modeling and behavioral noise.Source: §2.1.1 · p. 3
2.1.2 Side-Information Integrated Methods.
- Side-information methods use attributes such as categories, brands, or ratings to alleviate sparsity and cold-start issues, combining them with behavioral signals at input or intermediate layers.The cited approaches include concatenation or gating and later intermediate-layer operations.Source: §2.1.2 · p. 3
- Contrastive recommenders augment supervision through alternative sequence views or hierarchical intent and sequence groupings.The review describes heuristic augmentation and coarse-to-fine construction of contrastive pairs.Source: §2.1.3 · p. 3
2.2 LLM-Augmented Recommendation
- LLMs support recommendation as direct generators or semantic extractors; LSF-SR instead frames behavioral-textual integration as probabilistic inference and reshapes the posterior with normalizing flows.Figure 2 depicts offline item prose and semantic extraction followed by fusion and a Transformer backbone optimized end-to-end.Source: §2.2 · p. 3, Figure 2 · p. 4
3 Preliminaries
The preliminaries define next-item prediction from a user's interaction history and describe a Transformer-based sequential recommendation backbone. The backbone processes fused item representations while preserving sequence order and produces a sequence representation for recommendation.
3.1 Problem Formulation
- Next-item prediction estimates the probability of each candidate item given a user's historical sequence and returns a ranked list.Source: §3.1 · p. 4
3.2 Sequential Recommendation Backbone
- The sequential backbone passes enhanced item embeddings through self-attention blocks and a point-wise feed-forward network while preserving temporal order.The attention blocks aggregate sequence information using queries, keys, and values.Source: §3.2 · p. 4
- The backbone outputs a sequence representation that captures the user's latent intent and serves as the basis for recommendation.Source: §3.2 · p. 4
4 Our Proposed LSF-SR
LSF-SR generates descriptive item text and extracts semantic signals, then fuses them with collaborative representations using normalizing flows. The enriched representations feed a Transformer backbone that models dynamic preferences end-to-end.
4 Our Proposed LSF-SR
- LLMs synthesize descriptive item prose and extract semantic signals from text for the framework.Source: §4 · p. 4
- The conditional semantic fusion module uses normalizing flows to reshape the latent posterior and generate condition-aware item representations.This step bridges collaborative signals and textual semantic conditions.Source: §4 · p. 4
- Enriched item representations enter a Transformer-based sequential backbone to model dynamic user preferences end-to-end.Source: §4 · p. 4
4.1 Item Semantic Acquisition
LSF-SR derives item semantics through LLM-generated descriptions and maps their encoded vectors into the recommender’s hidden space to condition fusion.
4.1 Item Semantic Acquisition
- LSF-SR uses an LLM’s reasoning capabilities to profile item semantics, providing a foundation for representation learning.Source: §4.1 · p. 4
4.1.1 LLM-based Profiling.
- LLM profiling combines item attributes with representative historical interaction sequences to generate descriptions of item characteristics and hypothesized user intent.The descriptions turn sparse or incomplete metadata into a coherent, context-aware input for downstream components.Source: §4.1.1 · pp. 4–5
4.1.2 Semantic Representation.
- A pre-trained semantic encoder converts each item description into a dense vector, which a learnable linear projection maps into the recommender’s hidden space.The projected vector captures high-level semantic signals and conditions the subsequent fusion module.Source: §4.1.2 · p. 5
4.2 Conditional Semantic Fusion
LSF-SR treats fusion of collaborative and textual semantic representations as conditional stochastic inference. The CVAE produces enhanced item embeddings for next-item prediction.
4.2 Conditional Semantic Fusion
- The conditional fusion module frames alignment as stochastic inference to capture nonlinear interactions between collaborative and textual semantic spaces.Source: §4.2 · p. 5
4.2.1 Probabilistic Fusion Framework.
- The CVAE uses semantic embeddings to condition collaborative embeddings, accounting for alignment uncertainty and generating enhanced embeddings for next-item prediction rather than reconstructing the input.Source: §4.2.1 · p. 5
4.2.2 Latent Encoding and Reparameterization.
- The conditional encoder processes concatenated collaborative and semantic embeddings, then uses reparameterized stochastic sampling to form a differentiable base latent code.Source: §4.2.2 · p. 5
4.2.3 Condition-aware Embedding Generation. 𝑖.
- A conditional decoder combines the transformed latent code with the semantic condition to produce enhanced item embeddings within a unified latent manifold.When flow depth K is zero, the latent code remains z0; conditioning both encoder and decoder is described as reducing mismatch from deterministic fusion.Source: §4.2.3 · p. 5
4.3 Expressive Posterior with Normalizing Flows
Normalizing flows make the CVAE posterior more expressive for complex, nonlinear collaborative-semantic relationships. Planar and radial flow layers provide alternative transformations for refining the latent posterior.
4.3 Expressive Posterior with Normalizing Flows
- Normalizing flows transform a simple initial density through invertible mappings, giving the latent posterior more flexibility for complex, non-Gaussian item characteristics.Source: §4.3 · p. 5
4.3.1 Probability Transformation via Flows.
- A chain of K invertible transformations maps the base latent variable to the final latent, with Jacobian log-determinants accounting for each transformation’s volume change.This warping is used to refine the latent space for underlying item semantics, especially with non-Gaussian feature distributions.Source: §4.3.1 · p. 5
4.3.2 Planar and Radial Flow Stacks.
- Planar flows transform along hyperplanes to capture nonlinear cluster boundaries, while radial flows center transformations on a reference point to model localized clusters.Stacking flow layers adaptively refines the latent posterior for fusion of collaborative signals and semantic conditions.Source: §4.3.2 · pp. 5–6
4.4 Training Objectives and KL Annealing
LSF-SR jointly optimizes next-item recommendation and regularity of the fused latent space. Dynamic KL annealing is used to address posterior collapse during training.
4.4 Training Objectives and KL Annealing
- The joint training objective balances recommendation accuracy with fused-latent regularity, and dynamic KL annealing is used to address posterior collapse.Source: §4.4 · p. 6
4.4.1 Recommendation Loss.
- The recommendation loss is the negative log-likelihood of the ground-truth next item, while full-catalog logits use a deterministic inference path and KL regularization uses stochastic fusion.Source: §4.4.1 · p. 6
4.4.2 KL Divergence in Conditional Fusion Module.
- The KL term regularizes the flow-transformed approximate posterior toward a standard isotropic Gaussian prior while preserving informative fused representations.Source: §4.4.2 · p. 6
4.4.3 Dynamic KL Annealing Strategies.
- The total objective weights recommendation and KL losses with a dynamic coefficient β, which can follow linear, sigmoid, or cyclical annealing schedules.Source: §4.4.3 · p. 6
5 Experiments
Across five benchmark datasets, LSF-SR significantly outperforms the compared behavior-centric and LLM-augmented baselines. Ablations and cold-scenario analyses examine the roles of semantic profiling, flow-based fusion, KL scheduling, and sparse-interaction settings.
5.1 Datasets
- Experiments use five filtered public datasets, leave-one-out full ranking with Recall and NDCG at 10 and 20, and behavior-centric plus LLM-augmented baselines.The model uses a two-layer self-attention backbone; LLM-augmented baselines share a standardized text-processing pipeline.Source: §5.1 · p. 6, §5.2 · pp. 6–7, §5.3 · p. 7
5.5.1 Comparison with SOTA.
- LSF-SR significantly outperforms all baselines across five datasets and metrics under full-ranking evaluation.Experiments used five random seeds and paired t-tests (p<0.05); Table 2 presents the overall comparison.Source: §5.5.1 · pp. 7–8, Table 2 · p. 8
5.5.2 Ablation Study.
- Skipping LLM-generated profiles weakens results across datasets, with the smallest drop on Yelp, whose metadata provide fewer detailed textual features.Raw, fragmented metadata provide less effective semantic guidance than LLM-synthesized profiles; the gains depend on contextual richness in the input metadata.Source: §5.5.2 · p. 8
- Removing flows reduces performance, while removing KL regularization causes the largest ablation drop.The flow-removal reduction is relatively small on Sports; on Yelp, Recall@20 drops slightly while NDCG@20 is unchanged.Source: §5.5.2 · p. 8
- Cyclical KL annealing achieves the best or near-best NDCG@20 on all five datasets and the best or comparable Recall@20 on four.A fixed penalty is suboptimal; cyclical annealing periodically relaxes and tightens KL constraints.Source: §5.5.2 · pp. 8–9, Table 3 · p. 9
5.5.3 Hyperparameter Study.
- Performance initially rises and then declines slightly as flow depth and βend increase; the reported optimal settings use K=2 or 3 and βend=0.1–0.2.Excessive flow depth complicates optimization, while excessive βend can suppress behavior-centric signals.Source: §5.5.3 · p. 9, Figure 4 · p. 9
5.6 Case Study: Cold Scenarios
- On short histories (length ≤5), LSF-SR outperforms representative baselines; Yelp models perform better on short than long histories, unlike the stable-preference pattern described for e-commerce.The analysis uses NDCG@20; long Yelp histories can span location and intent changes, making next-item prediction noisier.Source: §5.6 · pp. 9–10, Figure 5 · p. 10
- For tail items appearing at most 9 times, LSF-SR increases NDCG@20 by 22.88% on Beauty and 18.50% on Yelp over the strongest LLM-augmented baseline, SRA-CL.Source: §5.6 · p. 10, Figure 6 · p. 10
6 Conclusion
LSF-SR fuses collaborative item representations and LLM-derived semantics through probabilistic inference, and outperforms state-of-the-art baselines across five benchmark datasets. The conclusion also identifies scope for future work in adaptive flows and temporally aware semantic conditioning.
6 Conclusion
- LSF-SR consistently outperforms state-of-the-art baselines across five datasets, with gains up to 12.98% in Recall@20 and 14.13% in NDCG@20.For tail items, it improves NDCG@20 over the strongest LLM-augmented baseline by 22.88% on Beauty and 18.50% on Yelp.Source: §6 · p. 10
- LSF-SR uses a Conditional Variational Autoencoder to condition collaborative item representations on LLM-derived semantics in a shared latent space.Stacked planar or radial normalizing flows model non-Gaussian geometry beyond a diagonal Gaussian posterior.Source: §6 · p. 10
- Dynamic KL annealing stabilizes training and prevents the latent fusion path from collapsing into an unregularized deterministic mapping.Source: §6 · p. 10
- Case studies under extreme data sparsity and cold-start scenarios support the framework’s practical robustness.Source: §6 · p. 10
- Future work will explore adaptive flow architectures for domain-specific data and temporally aware semantic conditioning for evolving user preferences.The authors note that sparser collaborative signals yield comparatively less benefit from deep flow stacks, while Yelp case studies show intent drift patterns.Source: §6 · p. 10
GenAI Usage Disclosure
The authors disclose that generative AI tools were used only for language editing and grammatical clarity. They report no use in experimental design, implementation, data analysis, or methodology development, and take responsibility for the final manuscript.
GenAI Usage Disclosure
- The authors used generative AI tools solely for language editing and improving grammatical clarity.Source: GenAI Usage Disclosure · p. 10
- The authors report no generative AI use for experimental design, code implementation, data analysis, or developing the core algorithms and methodology.Source: GenAI Usage Disclosure · p. 10
- The authors reviewed and edited the final text and take responsibility for the paper’s academic integrity and content.Source: GenAI Usage Disclosure · p. 10