Source-linked AI summary
Vista: A Visually, Socially, and Temporally-aware Model for Artistic Recommendation
Ruining He, Chen Fang, Zhaowen Wang, Julian McAuley
TL;DR
Artistic recommendation must account for complex visual preferences alongside temporal behavior, social dynamics, and large sparse datasets. Vista combines visual, sequential, and creator-focused modeling on Behance, with the components jointly improving recommendation and gains reaching 30% AUC for newly uploaded art. The paper also identifies higher-order sequential dependence and cold-item modeling as important scope boundaries for the formulations discussed.
Problem
Artistic recommendation involves complex visual semantics, large sparse interaction data, temporal consistency, and preferences for artists that are independent of their art.
Method
Vista combines visual appearance modeling, sequential Markov dynamics, and social modeling of preferences for artists and art styles on Behance.
Results
Vista’s social, temporal, and visual components jointly outperform models using each factor in isolation, with improvements of up to 30% AUC for newly uploaded art.
Takeaways & Limitations
Modeling visual, temporal, and social signals together is important for recommendation in online art communities.
Takeaways & Limitations
The first-order formulation assumes the next action is independent of earlier actions given the most recent one, motivating higher-order Markov chains.
Abstract
from arXiv · showhide
Understanding users' interactions with highly subjective content---like artistic images---is challenging due to the complex semantics that guide our preferences. On the one hand one has to overcome `standard' recommender systems challenges, such as dealing with large, sparse, and long-tailed datasets. On the other, several new challenges present themselves, such as the need to model content in terms of its visual appearance, or even social dynamics, such as a preference toward a particular artist that is independent of the art they create. In this paper we build large-scale recommender systems to model the dynamics of a vibrant digital art community, Behance, consisting of tens of millions of interactions (clicks and `appreciates') of users toward digital art. Methodologically, our main contributions are to model (a) rich content, especially in terms of its visual appearance; (b) temporal dynamics, in terms of how users prefer `visually consistent' content within and across sessions; and (c) social dynamics, in terms of how users exhibit preferences both towards certain art styles, as well as the artists themselves.
1. INTRODUCTION
Vista addresses artistic recommendation by jointly modeling visual content, temporal consistency, and social preferences in large-scale Behance data. Experiments on clicks and appreciates show that combining these signals improves performance, especially for newly uploaded art.
- Challenges: Behance recommendation must model high-dimensional visual content, temporal consistency within and across sessions, social preferences for artists and styles, and large sparse interaction data.The dataset involves millions of users and items and tens of millions of interactions.
- Method: Vista combines visual appearance modeling with short-term sequential dynamics to capture users’ preferences for art styles and visually consistent successive interactions.The approach builds on VBPR and FPMC-style methods.
- Method: Vista jointly models preferences toward both artistic content and the artists who create it.Behance users are simultaneously content creators and evaluators, so creator identity can matter independently of visual style.
- Results: Up to 30% AUC improvements occur for newly uploaded art when content or artist identity is modeled.The experiments use both implicit click data and explicit appreciate data.
- Dataset: The paper introduces Behance as a benchmark and plans to release complete appreciate data for benchmarking and evaluation.Click data is proprietary, whereas appreciate data is intended for release.
- Results: The experiments find that social, temporal, and visual components are each critical and outperform models using the factors in isolation.The model also supports visualization of users’ preferred art styles.
2. RELATED WORK
Related work spans visually aware recommendation, temporal and sequential modeling, and social recommendation. Vista differs by combining these signals for large-scale artistic preference modeling.
- Visually-aware recommender systems: Visually aware recommender systems learn user preferences from visual signals, extending earlier work on retrieving or parsing visually similar clothing images.The cited visual recommendation work targets recommendation-related link prediction tasks involving alternative or complementary products.
- Sequential recommendation: Earlier temporal recommendation work uses methods such as matrix factorization, while sequential recommendation commonly uses Markov chains to model transitions between states.These traditions motivate modeling temporal dynamics in recommendation.
- Sequential recommendation: Probabilistic Markov embeddings have been applied to sequential domains such as playlists and points of interest, but differ from Vista in modeled signals and model types.The passage also states that comparable scale to Behance had not been demonstrated to the authors’ knowledge.
- Social recommendation: Social recommendation has often modeled social networks to mitigate cold-start problems, whereas Behance requires modeling preferences between evaluators and content creators.This creator–evaluator relation differs from traditional friendship or trust relations among evaluators.
3. THE MARKOV CHAIN MODEL
The model predicts users’ next artistic actions by combining long-term user–item preferences, short-term sequential consistency, creator affinities, and item content. It extends personalized Markov chains to higher-order histories, incorporates visual features for sparse items, and supports creator-based cold-start signals.
- Problem setup: The task is to recommend each user’s next item from sequential action histories and item features such as visual representations.Each item has a creator, and users’ histories contain clicks or appreciates.
- Personalized Markov chains: The first-order model combines long-term user–item preference with short-term similarity between the candidate item and the previous action.Creator affinities are modeled alongside item-level interactions in the corresponding latent spaces.
- Creator and cold-start modeling: Creator representations provide social signals that can rank items from similar creators for cold users and help model cold items through active and passive creator interactions.For jointly created items, the model averages the associated creator vectors.
- Higher-order dynamics: Higher-order personalized Markov chains weight similarities to multiple previous items with user-specific decaying weights.The decay emphasizes recent actions while allowing users to differ in their short-term browsing behavior.
- Content-aware modeling: Content-based item features augment latent item representations with embeddings and residual components, supplying auxiliary information when collaborative observations are sparse.The formulation projects explicit item features into the model’s latent spaces.
- Learning objective: Pairwise ranking learns model parameters by comparing the predicted likelihood of each positive item with sampled negative items.The ranking probability uses a sigmoid over the difference between the two Markov predictions.
4. EXPERIMENTS
The experiments evaluate Vista and comparison methods on large Behance click and appreciate datasets using held-out next-action prediction, transition-specific settings, AUC, and qualitative visualizations. Results indicate that combining visual, temporal, and social signals improves recommendation, especially for sparse, cold-start, and socially or sequentially distinct cases.
- Experimental Setup: Experiments use timestamped Behance clicks and appreciates, with separate datasets containing millions of users, items, and interactions.The appreciate corpus has 373,771 users, 982,002 items, and 11,807,103 appreciates; the click corpus has 381,376 users, 972,181 items, and 48,118,748 clicks.
- Experimental Setup: Held-out validation and test actions evaluate how highly each method ranks the ground-truth next item using AUC.The most recent two actions per user form validation and test examples, while earlier actions are used for training.
- Comparison Methods: The comparison spans popularity, matrix-factorization, visual, Markov-chain, personalized sequential, and Vista models with different visual, temporal, and social capabilities.Vista and Vista+ extend visual and sequential recommendation with ownership signals and higher-order personalized Markov chains.
- Performance Analysis: Transition-specific evaluation separates owner changes from same-owner actions and session changes from same-session actions.These four settings test model behavior across different combinations of social and temporal transitions.
- Performance Analysis: Vista beats FPMC significantly in all settings and improves cold-item recommendation by as much as 47.66% on average.The authors attribute the advantage to ownership signals and fully personalized higher-order Markov chains.
- Qualitative Analysis: The learned latent space places content- and style-similar items near one another, while user sessions show long-term regional preferences and short-term click consistency.Sessions from users with larger long-term weighting show greater within-session consistency, whereas users with smaller weighting show more variance while retaining long-term preferences.
5. CONCLUSION
The paper addresses large-scale artistic recommendation by modeling visual, social, and sequential signals together. Its methods outperform state-of-the-art baselines on both explicit and implicit artistic-preference tasks using Behance datasets.
- The proposed methods significantly outperform state-of-the-art baselines on both explicit and implicit artistic-preference tasks.Evaluation uses large-scale datasets collected from Behance.
- The approach builds visually and socially aware Markov chains that model visual appearance and social dynamics simultaneously.
- Figure 3 contrasts sessions with strong versus weak short-term consistency and shows Vista+ recommendations for those sessions.Sessions above the dashed line have the largest w_u; those below have the least.
A. SCALABILITY ANALYSIS
The scalability analysis expresses K-order Vista+ prediction cost in terms of latent dimensions, visual-feature computation, and the number of previous items considered.
- For each sampled training triple, K-order Vista+ prediction requires O(D1 ∗ F + K ∗ D2 + K ∗ D3 ∗ F) time.The calculation combines user-item interactions with interactions between the item and K previous items.