PaperlayerView exact revision on arXiv

Source-linked AI summary

Task-adaptive Neural Process for User Cold-Start Recommendation

Xixun Lin, Jia Wu, Chuan Zhou, Shirui Pan, Yanan Cao, Bin Wang

arXiv:2103.06137v1cs.IR

TL;DR

Cold-start recommendation must work with only a few interactions, and shared meta-learning knowledge may not adapt well to users with different purchase intentions. TaNP maps user interactions to predictive distributions and adapts decoder parameters using task relevance. It consistently outperforms several state-of-the-art meta-learning recommenders across benchmarks, including a reported 4.2% P@5 improvement on MovieLens-1M.

  • Problem

    Cold-start recommenders have few user interactions to exploit, and shared meta-learning knowledge may not fit users with differing purchase intentions.

    Source: §1 · pp. 1–2

  • Method

    TaNP treats each user as a stochastic-process task, maps observed interactions to a predictive distribution, and uses learned task relevance to customize decoder parameters.

    Source: §1 · p. 2

  • Results

    TaNP consistently yields the best performances on all datasets and reports a 4.2% improvement in P@5 on MovieLens-1M versus state-of-the-art meta-learning recommenders.

    Source: §5.3 · p. 9

  • Takeaways & Limitations

    The reported results support TaNP as a neural-process framework for user cold-start recommendation that incorporates task relevance into decoder adaptation.

    Source: §1 · p. 2, §6 · p. 10

Abstract

from arXiv · show

User cold-start recommendation is a long-standing challenge for recommender systems due to the fact that only a few interactions of cold-start users can be exploited. Recent studies seek to address this challenge from the perspective of meta learning, and most of them follow a manner of parameter initialization, where the model parameters can be learned by a few steps of gradient updates. While these gradient-based meta-learning models achieve promising performances to some extent, a fundamental problem of them is how to adapt the global knowledge learned from previous tasks for the recommendations of cold-start users more effectively. In this paper, we develop a novel meta-learning recommender called task-adaptive neural process (TaNP). TaNP is a new member of the neural process family, where making recommendations for each user is associated with a corresponding stochastic process. TaNP directly maps the observed interactions of each user to a predictive distribution, sidestepping some training issues in gradient-based meta-learning models. More importantly, to balance the trade-off between model capacity and adaptation reliability, we introduce a novel task-adaptive mechanism. It enables our model to learn the relevance of different tasks and customize the global knowledge to the task-related decoder parameters for estimating user preferences. We validate TaNP on multiple benchmark datasets in different experimental settings. Empirical results demonstrate that TaNP yields consistent improvements over several state-of-the-art meta-learning recommenders.

1 INTRODUCTION

User cold-start recommendation is difficult because new users provide few interactions, while shared meta-learning knowledge may not fit users with different purchase intentions. TaNP maps user interactions to predictive distributions and adapts decoder parameters using learned task relevance; experiments report consistent improvements over state-of-the-art meta-learning recommenders.

Motivation

  • Sparse interactions make recommending items to cold-start users a major recommender-systems challenge.Traditional matrix-factorization and deep-learning models typically work well when adequate user interactions are available.Source: §1 · p. 1
  • Shared MAML-style parameter initialization may poorly fit users whose purchase intentions differ, making task relevance important for adapting global knowledge.The paper illustrates this contrast with similar tasks a and c and a largely different task b.Source: §1 · p. 2, Figure 1 · p. 2

TaNP

  • TaNP treats each user task as a stochastic process and maps observed interactions directly to a predictive distribution, using amortized variational inference.This approach is presented as relieving training issues such as model sensitivity and getting stuck in a local optimum in MAML-based recommenders.Source: §1 · p. 2
  • Its customized module derives task relevance from deterministic task embeddings and a global pool, then uses that relevance to generate task-related decoder parameters.The mechanism is intended to balance model capacity and adaptation reliability.Source: §1 · p. 2

Contributions

  • For user cold-start recommendation, TaNP uses a neural-process formulation, and experiments on benchmark datasets show that it consistently outperforms several state-of-the-art meta-learning recommenders1.Source: §1 · p. 2

2 RELATED WORK

The related work reviews meta-learning and cold-start recommendation methods before positioning TaNP within neural processes. It distinguishes TaNP by applying the neural-process principle to discrete user-item interactions and incorporating task adaptation.

2.1 Meta Learning

  • Meta-learning approaches include memory-, optimization-, metric-, and gradient-based methods; MAML is a popular gradient-based framework that learns a shared initialization.Some MAML-based methods consider task distributions to vary and explore task-specific adaptations.Source: §2.1 · p. 2

2.2 User Cold-Start Recommendation

  • Cold-start recommenders include side-information approaches and MAML-based methods such as MeLU, MetaCS, MetaHIN, and MAMO, each using different personalization or adaptation strategies.The surveyed gradient-based methods may retain MAML training issues, while some innovations depend closely on side-information and thus have limited application scenarios.Source: §2.2 · pp. 2–3

2.3 Neural Process

  • Neural processes model predictive distributions conditioned on permutation-invariant context representations, and TaNP applies this principle to discrete user-item relations for cold-start recommendation.The paper describes this as the first use of the neural-process principle for this recommendation problem and adds task adaptation to balance model capacity and adaptation reliability.Source: §2.3 · p. 3

3 PRELIMINARIES

The paper frames each user’s personalized recommendation as a task with a small support set and a query set. It describes meta-learning across training users and evaluating recommendations for disjoint cold-start users using their support interactions and learned global knowledge.

Task Definition

  • A task corresponds to personalized recommendation for one user, with available user-item interactions split into a small support set and remaining query set.Each interaction pairs a user-item tuple with the user's actual item rating; the support-set size is typically small.Source: §3 · p. 3

Meta-learning Setup

  • Meta-learning trains on task support sets and updates across multiple query sets to derive global knowledge that can adapt to a new task.The paper describes this as a learning procedure followed by a learning-to-learn procedure.Source: §3 · p. 3

Problem Formulation

  • Training and test users are disjoint, and test-time recommendations use a cold-start user's small support set together with knowledge learned from training tasks.Depending on the interaction values, recommendation is framed as binary engagement classification or rating regression.Source: §3 · p. 3

4 TASK-ADAPTIVE NEURAL PROCESS

TaNP frames each user’s observed interactions as an instantiation of a stochastic process and uses a neural-process model to predict interactions. Because shared encoder and decoder parameters can be inflexible across tasks, it adds a task-adaptive mechanism.

4 TASK-ADAPTIVE NEURAL PROCESS

  • The model combines an encoder, customization module, and adaptive decoder to estimate variational prior and posterior distributions, identify task relevance, and generate task-related decoder parameters.Embedding strategies and prediction losses can be selected for different recommendation scenarios.Source: §4 · p. 4

4.1 Overview

  • Each user’s interactions are modeled as draws from a task-specific stochastic process, approximated using a latent vector and learnable neural functions.A sample of the latent vector is treated as a realization of that process.Source: §4.1 · p. 4
  • TaNP uses amortized variational inference because the true posterior is intractable, with a posterior conditioned on the task interactions and a prior conditioned on the support set.The prior approximation supplies a KL-divergence regularization term for stochastic-process consistency.Source: §4.1 · p. 4
  • Shared encoder and decoder parameters can underfit complex tasks and overfit easy ones when task relevance is ignored, motivating task adaptation.The mechanism is intended to balance model capacity against adaptation reliability.Source: §4.1 · p. 4

4.2 Embedding Layers

TaNP can initialize user and item embeddings from categorical side information or, when user contents are unavailable, from a user identifier. Its encoder aggregates interaction representations into a permutation-invariant task representation.

4.2 Embedding Layers

  • When categorical user contents are available, TaNP embeds each content and concatenates the embeddings; without them, it obtains the user embedding from the user identifier.Item embeddings follow a similar process.Source: §4.2 · p. 4

4.3 Encoder

  • The encoder maps each interaction in the support set or task set to an embedding, then aggregates the vectors into a permutation-invariant representation.The aggregation uses a commutative operation, with a mean used for efficiency.Source: §4.3 · pp. 4–5
  • The encoder uses the support-set and task-set representations to form variational approximations, with the latent variable sampled using Gaussian noise and a reparameterization operation.The framework then combines the sampled latent variable with the user-item input for prediction.Source: §4.3 · pp. 4–5, Figure 2 · p. 5

4.4 Customization Module

The customization module uses support-set interactions and a shared global pool to estimate task relevance. Soft cluster assignments contribute to a final task embedding used to customize decoder parameters.

4.4 Customization Module

  • A task identity network encodes support-set interactions into a temporary task embedding, which is aggregated before interacting with the global pool.The pool contains soft cluster centroids, and its size is a hyperparameter.Source: §4.4 · p. 5
  • The module measures similarity between the task embedding and pool centroids to derive soft cluster assignments, then uses those assignments to form the final task embedding.The final embedding incorporates the learned task relevance.Source: §4.4 · p. 5
  • Training uses an unsupervised clustering loss defined as KL divergence between the assignment matrix and an auxiliary target distribution.The stated clustering objective improves cluster purity and emphasizes high-confidence assignments.Source: §4.4 · pp. 5–6

4.5 Adaptive Decoder

TaNP replaces a decoder shared across tasks with an adaptive decoder whose parameters are generated from each task embedding. It describes FiLM and Gating-FiLM as alternative modulation strategies.

4.5 Adaptive Decoder

  • The adaptive decoder uses the final task embedding to generate task-specific parameters, unlike the original decoder that predicts with parameters shared across tasks.The paper presents FiLM and Gating-FiLM as candidate modulation strategies.Source: §4.5 · p. 6

4.5.1 FiLM.

  • FiLM adaptively influences prediction by applying a feature-wise affine transformation to intermediate features, and here scales and shifts each decoder layer’s features via two generated parameters.The tanh nonlinearity restricts the modulation output to [−1, 1].Source: §4.5.1 · p. 6

4.5.2 Gating-FiLM.

  • Because FiLM cannot filter information with opposite effects on learning, the paper introduces Gating-FiLM as an alternative modulation strategy.FiLM and Gating-FiLM can be used alternatively in the adaptive decoder.Source: §4.5.2 · p. 6

4.6 Loss Function

TaNP reformulates its likelihood as a regression-based loss, with binary cross-entropy for implicit feedback. Its training objective includes a KL consistency regularizer, and test-time predictions use only the available support set.

4.6 Loss Function

  • TaNP reformulates the likelihood as a regression-based loss function.For implicit feedback, the per-task loss is binary cross-entropy.Source: §4.6 · p. 6
  • The training objective includes a KL consistency regularizer weighted by λ, and the model is optimized end-to-end with Adam.The regularizer compares q(z_i|τ_i) with q(z_i|S_i).Source: §4.6 · p. 6
  • At test time, TaNP samples z_i and obtains the task embedding from the available support set, then predicts query outcomes.The query labels are unavailable during testing.Source: §4.6 · p. 6

4.7 Time Complexity

TaNP uses fully connected DNNs for its encoder, customization module, and adaptive decoder, with component costs governed by layer depth and hidden size.

4.7 Time Complexity

  • Each fully connected DNN component has approximate time complexity O(l d^3), where l is layer count and d denotes hidden size.Hidden size may vary across layers.Source: §4.7 · p. 7
  • Computing the final task embedding costs O(k d^2), with k denoting the number of global-pool centroids.The passage notes that d can vary among layers.Source: §4.7 · p. 7
  • FiLM and Gating-FiLM are computationally efficient adaptations, requiring two and four weight matrices per adaptive-decoder layer, respectively.The model can also be optimized batch-wise for parallel computation.Source: §4.7 · p. 7

5 EXPERIMENTS

Experiments evaluate TaNP across three recommendation datasets and four research questions. TaNP leads the reported benchmark comparisons, while additional analyses examine reduced support sets, task assignments, and hyperparameter sensitivity.

5 EXPERIMENTS; 5.2.1 Evaluation Metrics.; 5.2.2 Compared Models.

  • The experiments ask whether TaNP outperforms cold-start baselines, adapts with fewer support interactions, captures task relevance, and generalizes across hyperparameter settings.Performance is averaged over test users using Precision@N, NDCG@N, and MAP@N for N = 5, 7, and 10; baselines include classic, content-based, and meta-learning models.Source: §5 · p. 7, §5.2.1 · p. 7, §5.2.2 · pp. 7–8

5.1 Datasets; 5.1.1 Data Preprocessing.

  • The evaluation uses MovieLens-1M, Last.FM, and Gowalla, with explicit ratings for MovieLens-1M and implicit feedback for the other two datasets.Users with histories of 40–200 interactions are retained; support sets contain 10, 15, or 20 interactions, with remaining interactions assigned to queries.Source: §5.1 · p. 7, §5.1.1 · p. 7

5.1.1 Data Preprocessing.; 5.2.3 Implementation Details.

  • Dataset comparisons use a 7:1:2 train-validation-test split and fixed embedding dimensions of 32, while other hyperparameters are selected on validation data.All models are trained for 150 epochs; TaNP uses three stacked layers, learning rate 5e-5, and α = 1.0.Source: §5.1.1 · p. 7, §5.2.3 · p. 9

5.3 Performance Comparison (RQ1)

  • 4.2% P@5 improvement on MovieLens-1M, 3.6% P@5 lift on Last.FM, and 5.4% MAP@5 improvement on Gowalla accompany TaNP's best performance across all datasets.The study reports that Gating-FiLM is slightly better than FiLM and that both task-adaptive variants significantly improve over the ablation without the task-adaptive mechanism.Source: Table 3 · p. 8, Table 4 · p. 8, Table 5 · p. 8

5.4 Impact of 𝑁𝑆𝑖(RQ2)

  • At reduced support sizes, TaNP (Gating-FiLM) remains better than other meta-learning models across datasets and outperforms the task-mechanism ablation in all reported settings.The analysis uses P@10 with support sizes of 10 and 15; the authors suggest TaNP is less affected by fewer interactions than MetaNLBA and MetaCS.Source: Figure 3 · p. 9, §5.4 · p. 10

5.5 Visual Analysis (RQ3)

  • The task-assignment visualization indicates that TaNP distinguishes both similar and dissimilar tasks, including shared high cluster scores for tasks τ5 and τ7.The authors also report that these users share gender and two support-set movie items, and conclude that learned task relevance informs decoder modulation.Source: Figure 4 · p. 10, §5.5 · p. 10

5.6 Hyper-parameter Analysis (RQ4)

  • On Last.FM, TaNP (FiLM) performs best at λ = 0.1 and k = 10, while TaNP (Gating-FiLM) performs best at λ = 0.1 and k = 20.Both variants are reported as robust to changes in λ and k, and remain better than Table 4 baselines even in their worst cases.Source: Figure 5 · p. 10, §5.6 · p. 10

6 CONCLUSION

The conclusion presents TaNP as a meta-learning model for user cold-start recommendation that maps observed interactions to predictive distributions. Its task-adaptive mechanism learns task relevance and modulates decoder parameters, and the reported results consistently outperform several state-of-the-art meta-learning recommenders.

6 CONCLUSION

  • TaNP maps observed user interactions to predictive distributions conditioned on a learned conditional prior.The conclusion describes the framework as generic.Source: §6 · p. 10
  • Its task-adaptive mechanism learns relevance across tasks and uses it to modulate decoder parameters.Source: §6 · p. 10
  • The reported extensive results show consistent improvements over several state-of-the-art meta-learning recommenders.Source: §6 · p. 10

Related papers

Loading 2103.06137v1…