PaperlayerView exact revision on arXiv

Source-linked AI summary

MAMO: Memory-Augmented Meta-Optimization for Cold-start Recommendation

Manqing Dong, Feng Yuan, Lina Yao, Xiwei Xu, Liming Zhu

arXiv:2007.03183v1cs.IRcs.LGstat.ML

TL;DR

Cold-start recommenders lack interaction data for new users and items, while shared meta-learning initialization can lead to local optima for users with distinct preferences. MAMO uses feature-specific memories for personalized initialization and task-specific memories for fast preference prediction. Experiments across two datasets and four cold-start situations show the proposed methods are effective.

  • Problem

    Cold-start recommendation lacks user-item interactions for new users and items, and a shared initialization may lead models into local optima for users with distinct preference patterns.

  • Method

    MAMO uses feature-specific memories for personalized parameter initialization and task-specific memories that provide fast weights for personalized recommendation.

  • Results

    MAMO shows stable performance across different scenarios and outperforms comparison methods in cold scenarios, especially for cold users.

  • Takeaways & Limitations

    The results support memory-augmented meta-optimization as an effective approach for cold-start recommendation through personalized initialization and fast preference adaptation.

  • Takeaways & Limitations

    MAMO becomes inefficient when auxiliary user or item profile information is unavailable, because the method assumes sufficient essential profile details.

Abstract

from arXiv · show

A common challenge for most current recommender systems is the cold-start problem. Due to the lack of user-item interactions, the fine-tuned recommender systems are unable to handle situations with new users or new items. Recently, some works introduce the meta-optimization idea into the recommendation scenarios, i.e. predicting the user preference by only a few of past interacted items. The core idea is learning a global sharing initialization parameter for all users and then learning the local parameters for each user separately. However, most meta-learning based recommendation approaches adopt model-agnostic meta-learning for parameter initialization, where the global sharing parameter may lead the model into local optima for some users. In this paper, we design two memory matrices that can store task-specific memories and feature-specific memories. Specifically, the feature-specific memories are used to guide the model with personalized parameter initialization, while the task-specific memories are used to guide the model fast predicting the user preference. And we adopt a meta-optimization approach for optimizing the proposed method. We test the model on two widely used recommendation datasets and consider four cold-start situations. The experimental results show the effectiveness of the proposed methods.

1 INTRODUCTION

Cold-start recommendation is difficult because new users and items lack interaction histories. Meta-learning addresses this with few-shot preference prediction, but shared initialization can cause instability, slow convergence, weak generalization, and local optima for users with distinct preferences.

  • Cold-start recommendation methods struggle with new users or items because they lack user-item interactions.
  • Auxiliary information, including item descriptions and cross-domain latent mappings, is a traditional way to address cold-start recommendation.
  • Meta-learning frames recommendation for each user as a task and predicts preferences from only a few past interacted items.
  • Most meta-learning recommenders learn a global initialization parameter and then locally update parameters for each user.
  • MAML-based approaches can suffer instability, slow convergence, weak generalization, and gradient degradation leading to local optima for atypical users.

2 PROPOSED APPROACH

MAMO treats user recommendation as a task, builds embeddings from user and item profiles, and locally updates personalized model parameters. Its memory-augmented meta-optimizer replaces single global initialization with personalized initialization and memory-derived fast weights.

  • MAMO treats recommendation for a user as a task with support and query sets, aiming to predict query-item ratings from limited interactions.
  • MAMO addresses inadequate single initialization by learning multi-level personalized parameters instead of relying only on one global parameter.
  • The recommender predicts scores from user and item embeddings, whose parameters are locally updated for personalized recommendation.
  • User and item profiles are transformed into embeddings through fully connected layers, with embedding size and profile dimensions defining the representations.
  • Feature-specific memories retrieve personalized bias terms for initializing user-embedding parameters, while task-specific memory supplies fast weights for recommendation.

2.3 Memory-Augmented Meta-Optimization

MAMO uses feature-specific and task-specific memories within meta-optimization to personalize initialization and accelerate user-preference prediction. Its training alternates local adaptation on support data with memory and global-parameter updates guided by query-set loss.

  • Feature-specific Memory: Feature-specific memories MP and MU generate a personalized bias bu that adjusts the global user-embedding initialization for each user.Profile memory retrieves attention values from user profiles, while user embedding memory stores fast gradients or bias terms; θu is initialized as ϕu − τbu.
  • Memory Updates: The two memory matrices are randomly initialized before training and updated during training using attention-weighted profile information and gradient information.Profile memory uses an attention mask with update coefficient α, while user embedding memory incorporates gradients with retention coefficient β.
  • Task-specific Memory: Task-specific memory MU,I retrieves a user-preference matrix Mu,I that serves as fast weights or a transform matrix for recommendation.The retrieved matrix is locally updated after support-set learning and written back to task-specific memory; γ controls how much new preference information is added.
  • Local Training: For each training user, MAMO initializes local recommender parameters and task-specific memory, predicts recommendations, and performs local updates using the support set.The local parameters θu, θi, and θr are updated by minimizing single-user prediction loss, and the preference matrix is also updated through back-propagation.
  • Global Update: MAMO updates feature-specific memories, task-specific memory, and global parameters using query-set loss after local support-set training.The meta-gradient is computed by back-propagation, and the procedure follows the one-step meta-optimization idea to avoid expensive higher-order derivatives.

3 EXPERIMENTS

Experiments evaluate MAMO on two public datasets across cold-start settings, parameter choices, and recommender-model comparisons. Results indicate robust performance, benefits from personalized memories, and a trade-off between guidance granularity and computation.

  • Experimental Setup: Experiments use MovieLens 1M and Book-Crossing, splitting users 80:20 and limiting each user's history to 20 records.The default support set contains the first 15 items, with remaining items forming the query set.
  • Experimental Setup: Evaluation covers warm users and items, warm users with cold items, cold users with warm items, and cold users with cold items.MovieLens defines cold items as having fewer than 10 ratings and identifies warm or cold users by first-comment time.
  • Evaluation Metrics: MAE measures rating-prediction accuracy with lower values better, while NDCG@N measures preference-ordering quality with higher values better.NDCG@N compares the ordering of observed query-set preferences.
  • Parameter Studies: Larger training and support sets improve performance, although acceptable predictions remain possible with half the users or smaller support sets.Support-set comparisons use sizes from 5 to 15 while retaining the last 5 samples as queries.
  • Parameter Studies: Moderate embedding depth and size provide effective predictions, while very shallow or deep networks can perform poorly and larger structures may increase training demands.The number of embedding layers has slight effects overall, whereas moderate embedding size reduces training-time requirements.
  • Comparison and Memory Studies: MAMO performs stably across scenarios, and K values around 2-4 achieve the best performance while larger K values increase computation cost.Each preference-memory type stores parameters matching the user-embedding network.

4 RELATED WORK

Related work addresses cold-start recommendation through auxiliary information, meta-learning, and memory networks. Meta-optimization commonly adapts a global initialization to each user, while memory-based methods retrieve and update task- or feature-specific information.

  • Meta-learning: Meta-learning applies few-shot adaptation to recommendation by treating each user’s recommendation as a learning task.Optimization-based methods learn across tasks with few samples and adapt to new users using parameter initialization.
  • Meta-learning: Most meta-optimization recommenders learn global parameters and locally update model parameters for personalized recommendations.Examples differ in recommender architecture and in how local and global parameters are updated.
  • Memory networks: Memory-augmented recommendation methods explicitly retrieve information from memory matrices using learned attention or read mechanisms.Neural Turing Machine-style systems read weighted memory rows and update them with learned writing weights.
  • Memory networks: RUM enhances user embeddings with memory retrieved according to item embeddings, using item-level or feature-level memories.Its recommendation score is computed from user and item embeddings, with the memory embedding incorporated into the user representation.
  • MAMO: MAMO uses feature-specific memory for personalized parameter extraction and task-specific memory as fast weights for user preference prediction.Feature-specific memory matches user profiles to stored preference patterns, while task-specific memory stores preference information for rapid recommendation.

5 CONCLUSION

The paper targets cold-start recommendation, where limited interactions make new users and items difficult to handle. It replaces uniform initialization with feature- and task-specific memories and reports effectiveness on two public datasets.

  • Problem: Cold-start recommendation is difficult because new users or items lack sufficient user-item interactions.The paper considers both user cold-start and item cold-start settings.
  • Limitation: Existing meta-learning recommenders may use one global initialization for all users, leading to local optima for users with distinct preference patterns.The paper identifies this as a limitation of the common MAML-based approach.
  • Method: MAMO uses feature-specific memory for personalized initialization and task-specific memory to guide recommendation.The two memories store different kinds of information for adapting and predicting user preferences.
  • Evaluation: The proposed method is evaluated on two publicly available datasets, and the experimental results show its effectiveness.The conclusion reports the evaluation scope without specifying a numerical result.

A.1 Dataset Details

The dataset details describe user and item profile fields for MovieLens and preprocessing choices for Book-Crossing. Book-Crossing requires filtering implausible ages and retaining country-level location information.

  • MovieLens: MovieLens user profiles include gender, age group, and occupation, while item profiles include release year, genres, directors, and rating.The raw MovieLens data gives each user at least 20 rating histories.
  • MovieLens: MovieLens ratings have a mean value of 3.58 and span comment dates from 2000-04-26 to 2003-03-01.These values characterize the dataset used in the experiments.
  • Book-Crossing: Book-Crossing user information includes age and location, but the preprocessing restricts ages to 5–110 and retains only country information.The original age range extends from 0 to 237, and the filtered data covers 65 countries.
  • Book-Crossing: Book-Crossing item information includes publication year and is processed because the raw data contains missing and misleading values.The passage specifically motivates filtering and simplification of the user information.

A.2 Compared methods

The comparison uses representative recommendation methods spanning meta-optimization and memory-network approaches. Implementations combine authors’ code, modified baselines, and an independent PyTorch implementation with specified parameter settings.

  • Implementation: The authors use published code for MeLU and s2 Meta, modify MeLU for MetaCS-DNN, and implement RUM in PyTorch because its code is unavailable.These implementation choices adapt all baselines to the experimental settings.
  • Parameter settings: RUM’s SGD learning rate is selected from five candidate values, and its memory-slot count K is set to 20.The candidate learning rates are [1, 0.1, 0.01, 0.001, 0.0001].

A.2.2 Parameter configuration.

The evaluation defines cold-start cases by whether users and rated items are warm or cold, covering four combinations.

  • Users are categorized as either warm or cold for evaluation.
  • Rated items are likewise categorized as warm or cold, producing four labeled rating scenarios.
  • The four scenarios are W-W, W-C, C-W, and C-C.

A.2.3 The evaluation metrics in cold-scenarios.

Performance in the four cold-start scenarios is evaluated with MAE and NDCG@N, using scenario-specific aggregation procedures.

  • The evaluation metrics are MAE and NDCG@N.
  • MAE is calculated as the mean rating value separately for W-W, W-C, C-W, and C-C scenarios.
  • For NDCG@N, scenario results are concatenated, divided into clips, scored per clip, and averaged.

A.3 Parameter settings.

The implementation uses PyTorch on Linux with specified model, optimization, and testing settings; testing computes biases, embeddings, predictions, and local updates.

  • The code uses PyTorch 1.4.0, Python 3.7, Linux, and an NVIDIA TITAN X.
  • Movielens uses embedding dimension 100, two layers, local learning rate ρ=0.01, and LeakyRelu as the default activation.
  • One epoch over all users takes about half an hour, so global parameters are updated after batches of training users.
  • Algorithm 2 describes MAMO's testing process and outputs predicted preferences for testing users and query items.
  • Testing calculates a bias term from the user profile and memory matrices before obtaining embeddings and predictions.
  • The testing procedure includes a local update of M_U,I.
  • The prediction pipeline obtains user and item embeddings using Eq. (1), then computes predicted y-hat_u,i using Eq. (2).
Loading 2007.03183v1…