Source-linked AI summary
Preference Flow Matching with Spectral Factorization for Micro-video Recommendation
Xinxin Dong, Haokai Ma, Fei Hu, YuZe Zheng, Bin Wu, Yonghui Yang, Xiaodong Wang
TL;DR
Micro-video recommenders often entangle stable and dynamic video cues while generating preferences from coarse behavioral context. PrismRec factorizes frame-level content with SSF and guides conditional preference flow matching with CPM. Across four datasets, it consistently outperforms representative baselines while maintaining robustness and efficiency.
Problem
Existing methods entangle static semantics and dynamic variations in holistic representations, while flow-based recommenders leave video temporal structure outside preference formation.
Method
PrismRec uses SSF to derive complementary static and dynamic factors, then CPM injects them as structured conditions into conditional preference flow matching.
Results
PrismRec achieves the best performance on all sixteen dataset-metric pairs with p < 0.01 and improves over FMRec.
Takeaways & Limitations
The framework treats factorized video content as an intrinsic driver of preference matching rather than auxiliary side information.
Abstract
from arXiv · showhide
Micro-video recommendation aims to infer user preferences from historical interactions and multimodal video content, thereby identifying the next video of interest. However, prevailing methods compress frame sequences into a single holistic representation, entangling the stable visual semantics and the evolving dynamics that jointly shape user preferences. Meanwhile, diffusion- and flow matching-based recommenders condition their generation process solely on coarse behavioral context, leaving its internal temporal structure outside preference formation. We therefore propose PrismRec, a Preference Flow Matching framework with Spectral Factorization for Micro-video Recommendation. Analogous to a prism that disperses white light into its constituent spectrum, PrismRec devises Spectral Semantic Factorization (SSF) to derive complementary static semantic and dynamic factors from frame-level representations via a prior-guided learnable frequency mask in the temporal frequency domain. Then, it proposes Context-Calibrated Preference Matching (CPM) to weigh them with each user's specific sensitivity and inject the calibrated context as a structured condition to steer the matching trajectory toward the target representation, making video content as an intrinsic driver of preference formation rather than auxiliary side information. Experiments on four datasets from two platforms show that PrismRec surpasses the SOTA baseline by up to 22.65%, with the lowest inference cost and peak memory among the compared methods.
Introduction
Micro-video recommendation research has separately modeled interactions and video content, but holistic representations entangle temporal cues and flow-based generation remains behavior-conditioned. PrismRec integrates spectral factorization with preference flow matching to make structured video content intrinsic to preference generation.
- Interaction-centric methods capture observed behavioral regularities, whereas content-centric methods enhance multimodal semantics and local interests.
- Spectral analysis separates stable visual semantics from motion and transition dynamics, while flow matching learns continuous transport from a prior to target representations.
- Existing pipelines aggregate static semantics and dynamic variations into holistic representations, while FM recommenders condition generation on coarse behavioral context.
- PrismRec uses SSF to derive complementary static and dynamic factors with a learnable frequency mask and CPM to inject structured conditions into preference transport.
- Experiments on four real-world datasets show that PrismRec consistently outperforms representative baselines with strong robustness and efficiency.
Related Work
Related work has advanced interaction modeling, multimodal video representation, spectral recommendation, and flow-based generation, but frame-level spectral representations remain weakly integrated with preference generation.
- Micro-video recommendation shifted from ID-based collaborative filtering toward joint modeling of video content and fine-grained behavioral signals.
- Content-driven methods use multimodal side information, raw content, local semantics, and user–clip relations to improve video representation and interest modeling.
- Frequency-domain recommenders project interaction signals into spectral space for noise suppression and multi-scale preference modeling.
- Flow matching learns continuous transport trajectories from a prior to a target distribution and has been adopted in recommendation settings.
- Integrating frame-level spectral representations with flow-based preference generation remains underexplored.
Preliminaries and Empirical Analysis
Flow matching models continuous transport between source and target distributions, while empirical analysis measures video temporal variation to motivate frequency-aware preference modeling.
- Flow Matching: Flow matching regresses a time-dependent vector field along a prescribed probability path, avoiding stochastic diffusion simulation.
- Flow Matching: Conditional flow matching constructs an affine path between samples x0 ∼ p0 and x1 ∼ p1 using differentiable schedules.
- Flow Matching: At inference, the learned ordinary differential equation is integrated from t = 0 to t = 1, producing a terminal state that approximates a sample from p1.
- Flow Matching: Flow matching directly models continuous transport trajectories and supports auxiliary signals guiding the entire transport process.
- Empirical Analysis: The dynamic energy ratio measures high-frequency energy after temporal FFT, with larger values indicating stronger temporal variation.
- Empirical Analysis: The dynamic energy ratio is highly non-uniform across videos, revealing diverse temporal patterns that holistic modeling may overlook.
Methodology
PrismRec combines spectral decomposition of frame-level video features with conditional flow matching to model personalized preference evolution from structured multimodal content.
- PrismRec contains SSF for complementary static and dynamic factors and CPM for personalized preference evolution through conditional flow matching.
- The recommendation task predicts the next video from each user’s chronological interaction sequence and associated multimodal content.
- Spectral Semantic Factorization: SSF addresses heterogeneous frame signals by combining prior-guided spectral decomposition, branch-specific frequency refinement, and asymmetric dual-track fusion.
- Spectral Semantic Factorization: SSF applies temporal real FFT to frame-level representations and combines a low-/high-frequency prior with video-specific residuals to produce complementary soft masks.
Context-Calibrated Preference Matching (CPM)
CPM calibrates static and dynamic video factors to each user’s context, then conditions a preference-flow transport process on this context and temporal information to predict the next-item representation.
- Spectral factor fusion: Because the global video representation already retains appearance information, SSF treats the static factor as a residual correction and the dynamic factor as an independent temporal track.
- Spectral factor processing: SSF produces complementary static- and dynamic-oriented content signals for subsequent preference modeling.The factors are refined through branch-specific gates and multi-scale depthwise convolutions before fusion.
- Preference flow: CPM models preference formation as a context-calibrated transport process from a Gaussian initial state toward the next-interacted item representation.The rectified-flow path linearly interpolates between the initial state and target representation while injecting the current flow state, flow-time embedding, and temporal interval into historical preference.
- Context calibration: CPM derives an intent gate from the sequential preference state to calibrate static and dynamic video factors according to users’ differing sensitivities.It also aggregates the calibrated factors over the interaction history and constructs textual and temporal-interval contexts.
- Prediction and optimization: The context-calibrated prediction network directly estimates the target endpoint, and the trajectory-aware preference, calibrated video context, and textual context jointly predict it.At inference, the predicted endpoint determines Euler transport updates before combination with the sequential preference representation; training uses flow-matching and full-softmax recommendation objectives.
Experiments
Experiments across four real-world datasets evaluate PrismRec against fourteen baselines, its component contributions, backbone generality, robustness, sampling sensitivity, and efficiency. PrismRec consistently achieves the strongest recommendation performance while maintaining robustness and favorable computational cost.
- Experimental Setup: Experiments use four real-world benchmarks from two platforms with multimodal videos, timestamped interactions, chronological sequences, and leave-one-out evaluation.The datasets include MicroLens-Small, MicroLens-Big, Shortvideo-Small, and Shortvideo-Big, processed with 4-core filtering.
- Overall Performance: PrismRec achieves the best performance on all sixteen dataset-metric pairs with p < 0.01, with gains largest where interaction-derived item embeddings are least reliable.The comparison covers fourteen baselines spanning collaborative filtering, sequential, multimodal sequential, and video recommendation methods.
- Ablation Study: PrismRec-SSF underperforms the full model, showing that holistic video representations lose complementary static and dynamic cues across frames.The ablation removes spectral semantic factorization by replacing it with a holistic video representation.
- Ablation Study: PrismRec-CPM degrades on all datasets, whereas CPM refines preference transitions step by step under content and temporal conditions.SSF acts on item-side content representations, while CPM calibrates the user-side preference trajectory; their relative contributions vary across datasets.
- Universality Analysis: Equipping SASRec or CL4SRec with SSF or CPM improves average H@10 and N@10 by 16.60% and 21.01%, while combining both modules performs best across settings.The results indicate that PrismRec’s improvements do not depend on a specific sequential encoder.
- Robustness Analysis: PrismRec retains the best H@10 at every tested noise level, and its margin over DMMD4SR widens from 7.13% to 12.87% as behavior noise increases.Under representation-level noise, the corresponding drops are 13.41%, 23.21%, and 21.02%; the model also remains competitive across frame-sampling configurations.
- Efficiency Analysis: PrismRec achieves the best H@20 on both efficiency datasets while requiring the least GPU memory and evaluation time.It trains fastest on MicroLens-Small and remains comparable to FMRec on MicroLens-Big.
Conclusion
PrismRec is a multimodal flow-matching framework for micro-video sequential recommendation that combines spectral factorization with context-calibrated preference transitions. Experiments on four datasets, alongside robustness and efficiency analyses, demonstrate its effectiveness and practical utility.
- PrismRec combines Spectral Semantic Factorization and Context-Calibrated Preference Matching for micro-video sequential recommendation.SSF separates frame-level representations into static- and dynamic-oriented factors, while CPM models preference transitions through a context-calibrated transport process.
- Experiments on four datasets demonstrate the effectiveness of PrismRec.
- Robustness and efficiency analyses confirm PrismRec's practical utility.
- Future work will pursue a more efficient and unified framework for representing heterogeneous video information.