Source-linked AI summary
From Zero-Shot Learning to Cold-Start Recommendation
Jingjing Li, Mengmeng Jing, Ke Lu, Lei Zhu, Yang Yang, Zi Huang
TL;DR
Cold-start recommendation and zero-shot learning are separately studied problems that share a framework for predicting unseen cases from complementary spaces. The paper formulates CSR as ZSL and proposes LLAE, whose bidirectional design performs effectively across both tasks and supports a win-win connection between them.
Problem
CSR lacks past behavior for new users, while its shared attribute and behavior spaces make it a special case of ZSL that had not previously been studied from that perspective.
Method
LLAE uses a low-rank encoder to map user behavior to attributes and a symmetric decoder to reconstruct behavior from attributes.
Results
Experiments on ZSL and CSR show that CSR can be addressed by ZSL models and that reconstruction constraints from CSR can benefit ZSL.
Takeaways & Limitations
Linking CSR and ZSL provides a win-win formulation in which advances from the two communities can be shared.
Abstract
from arXiv · showhide
Zero-shot learning (ZSL) and cold-start recommendation (CSR) are two challenging problems in computer vision and recommender system, respectively. In general, they are independently investigated in different communities. This paper, however, reveals that ZSL and CSR are two extensions of the same intension. Both of them, for instance, attempt to predict unseen classes and involve two spaces, one for direct feature representation and the other for supplementary description. Yet there is no existing approach which addresses CSR from the ZSL perspective. This work, for the first time, formulates CSR as a ZSL problem, and a tailor-made ZSL method is proposed to handle CSR. Specifically, we propose a Low-rank Linear Auto-Encoder (LLAE), which challenges three cruxes, i.e., domain shift, spurious correlations and computing efficiency, in this paper. LLAE consists of two parts, a low-rank encoder maps user behavior into user attributes and a symmetric decoder reconstructs user behavior from user attributes. Extensive experiments on both ZSL and CSR tasks verify that the proposed method is a win-win formulation, i.e., not only can CSR be handled by ZSL models with a significant performance improvement compared with several conventional state-of-the-art methods, but the consideration of CSR can benefit ZSL as well.
Introduction
The paper connects cold-start recommendation (CSR) with zero-shot learning (ZSL), formulating CSR as a ZSL problem and introducing LLAE to address its central challenges. Experiments show benefits in both directions across the two tasks.
- CSR addresses recommendation for new users without available past behavior, a setting where collaborative-filtering systems can become ineffective.
- CSR and ZSL share two spaces—direct feature representation and supplementary description—and both predict unseen cases using shared descriptive information.
- The paper is the first to investigate CSR from the ZSL perspective and formulate CSR as a ZSL problem.
- LLAE uses an encoder to map user behavior into user attributes and a decoder to reconstruct behavior, targeting domain shift, sparse behavior, and efficiency.
- Experiments on ZSL and CSR show that ZSL models can improve CSR, while considering CSR can also benefit ZSL.
Related Work
Related work covers zero-shot recognition and two main cold-start recommendation model families: matrix factorization and feature mapping. These approaches use side information to connect unseen cases with learned representations.
- Zero-shot learning: Zero-shot learning seeks to recognize unseen objects from semantic descriptions when collecting training samples for new or rare objects is difficult.
- Matrix factorization models learn latent representations by factorizing a relationship matrix and can share a latent representation learned from side information for cold-start prediction.
- Feature mapping models learn mappings between side information and latent representations, unlike matrix factorization models that jointly learn a shared representation from relationships and side information.
Problem Formulation
The paper formulates LLAE as a low-rank, denoising autoencoder that maps behavior into attributes and reconstructs behavior for zero-shot classification and cold-start recommendation. Its optimization uses tied weights, explicit low-rank regularization, alternating updates, and complexity that depends on dimensionality rather than sample count.
- Notations: The notation section represents vectors with bold lowercase letters, matrices with bold uppercase letters, and matrix magnitude through the Frobenius norm and trace operator.The Frobenius norm is defined through the matrix singular values, while trace notation is used in the optimization formulation.
- Linear Low-rank Denoising Autoencoder: LLAE learns an encoder W from behavior X into latent attributes S and a decoder M that reconstructs X from S by minimizing reconstruction error.The latent space has a direct semantic interpretation: semantic space in ZSL and user side information in CSR.
- Linear Low-rank Denoising Autoencoder: Tied weights set the decoder to M = W⊤, reducing parameters while allowing attributes to reconstruct behavior and enabling warm and cold users to be compared in attribute space.For new users, reconstructed behavior is used to generate recommendation lists.
- Linear Low-rank Denoising Autoencoder: The low-rank constraint on W targets sparse, high-dimensional behavior data by filtering spurious correlations and highlighting attributes shared across users.The paper gives a shared attribute such as being a basketball fan as an example of a common factor revealed by the constraint.
- Linear Low-rank Denoising Autoencoder: LLAE combines reconstruction, low-rank regularization, and denoising by zeroing 10% of X before enforcing the encoder constraint W bX = S.Reconstruction is especially important when behavior and attribute spaces are weakly correlated because it mitigates domain shift between warm and cold users.
- Problem Optimization: Because direct rank optimization is NP-hard, the method uses an explicit singular-value formulation and relaxes the constraint into a convex optimization problem with global optimal solution.The resulting optimization is rewritten into a form solved by alternating updates of W and V.
- Problem Optimization: The warm-up algorithm alternates eigen-decomposition for V with Sylvester-based optimization of W and its decoder, then predicts item probabilities and recommends the top-k items.The stated computational cost is O(d^3) generally, reducible to O(r^2d) for updating V, and independent of the number of samples.
- Zero-shot Classification: For ZSL, new samples are mapped to semantic space and classified by distances to projected prototypes; for CSR, new-user attributes reconstruct potential user-item relationships for recommendation.CSR uses user behavior such as purchase, browse, and share data as X and attributes such as preferences, personal information, and social data as S.
Experiments
Experiments evaluate LLAE on zero-shot recognition and cold-start recommendation, comparing it with established baselines and examining its efficiency, convergence, and constraints. Across both tasks, the reported results support the bilateral formulation and the roles of low-rank and reconstruction constraints.
- Zero-shot Learning: LLAE is evaluated on four zero-shot recognition benchmarks against five state-of-the-art baselines, including DAP, ESZSL, SSE, JLSE, and LESD.The reported ZSL results state that LLAE performs much better than the compared methods.
- Cross-task formulation: LLAE's bilateral formulation projects behavior or features into attributes and reconstructs them back, benefiting both zero-shot recognition and cold-start recommendation.The reconstruction component was introduced for CSR but is reported to benefit ZSL as well.
- Cold-start Recommendation: CSR experiments use four recommendation datasets, with 10% of users held out as new users and precision@k, recall@k, and mAP@100 as evaluation measures.Training uses the remaining users, cross-validation, and ten random training-test folds.
- Cold-start Recommendation: Low-rank representation outperforms baselines in most evaluations by filtering spurious connections and handling extremely sparse social-network observations.The comparison includes LoCo, which also uses low-rank representation, and conventional matrix-factorization-based baselines.
- Model Analysis: LLAE is a linear algorithm that runs faster than most previous work, requiring about 1/1000 of SSE's training time on AwA and converging very fast.Figure 3 examines parameter sensitivity and convergence on AwA.
- Model Analysis: LLAE's low-rank and reconstruction constraints both contribute to performance, with low rank supporting shared semantics and reconstruction contributing substantially across evaluations.Figure 4 reports CSR Precision@5 results across datasets and ZSL results on CUB.
Conclusion
The paper reframes cold-start recommendation as a zero-shot learning problem and proposes LLAE as a shared solution framework. Experiments support benefits in both directions: ZSL models address CSR, while CSR-inspired reconstruction benefits ZSL.
- CSR and ZSL are presented as two extensions of the same underlying problem, despite having been studied independently.
- The paper formulates CSR as a ZSL problem for the first time and proposes a tailor-made ZSL model for it.
- LLAE uses a low-rank encoder to map user behavior to attributes and a symmetric decoder to reconstruct behavior from attributes.
- Extensive experiments on eight CSR and ZSL datasets verify that ZSL models can address CSR and that CSR-related reconstruction constraints can benefit ZSL.
- The authors characterize this cross-task benefit as a win-win formulation that may encourage collaboration between the two research communities.